When I was at CUE, I had the opportunity to see Vince Cerf do the opening keynote presentation. He gave a very interesting keynote, and one thing really stuck with me, and it was about what constituted a contribution in a networked environment.

Mr. Cerf suggested that we would never publish a single word as a book, as a journal article, or magazine article. He continued by saying that in a Web 2.0 world that included Wikipedia, you could and would publish a single word, and most importantly, it could be a significant contribution.

So, last night in Dean Shareski’s uStream session, many of us had a very interesting discussion about online contribution, and levels of contribution. Clarence Fisher was the guest and he was talking about his ideas relative to the classroom as a studio as well as what it meant to improve information. Some interesting ideas from the chat:

Jeff Utecht: How would you assess a student who changed a single word?
Ryan Bretag: Think about contributing one word from a poetry standpoint, how critical is one word? Writing in a hypertext society makes that one world critical.
EdtechVision: (paraphrasing here). How would peer assessment enter into this?

Personally, I think you can begin by taking a look at Darren Kuropatwa’s framework for his classroom wiki, where he defines what a significant contribution is and what a constructive modification means.

Back in the chat, after much discussion, a single question was distilled:

How do you assess contribution in a networked classroom?

Ok, so what does it look like? What’s new, what’s different, what’s the same? Your ideas?

9 Responses to “A Single Word”
  1. Mathew says:

    Certainly you have to look at how that one word contributes to the end result rather than quantifying it as just one word.

    Being in school at the moment myself, one thing that bothers me is when instructors only take into account the final product and do not consider the contributions made to the group’s thinking along the way. One comment early in the process can affect the whole direction of the group but how instructors keep this in mind I’m not sure.

    Things like wikis and blogs are cool because it does give you a public record of thinking over time.

  2. Great question.

    I don’t have an answer but have been thinking about the same issue in terms of our teachers, and posted on my blog about this recently. If we are trying to tell teachers that we value them working together in professional learning communities, how are we assessing their “group” contributions and creating an environment that rewards their working together.

    Similarly, I think for students, we have to be sure that our assessments reward the networked contributions. I’ve heard students at our own campus express concern that sometimes grading rubrics are almost penalizing them for group work if one member of the group isn’t contributing strongly. So I think in network assessments, it’s important to account for the contributions in a way that assesses both the networking but also the contributions of the individual to the whole. But a method that also rewards or recognizes the value of networking is important too.

    How does one “measure” the contributions that happen in a networked environment? Is it quantity? quality? effort? value-added? I look forward to hearing everyone’s ideas.

  3. It brings up whether the assessment is on “process” or “product”. Darren Kuropatwa’s framework is about product. But as Ryan and Jeff point out, in other contexts or subjects, critical changes could be something not reflected in the final product. For example, could you assess a student who contributed zero words to the poem, but asked such important questions that it affected the whole group’s thinking and therefore significantly improved the final product? Teachers assessing group work have used peer review and observation in this context.

    I think it depends on how much you expect the group to do together, and whether the teacher has the capability to “watch” the process to any extent. I’m sure in Darren’s class the wiki is not the only thing being assessed. Otherwise, a student who looked over another student’s shoulder and said, “hey, you have a mistake there” would not be considered to have made a significant contribution because the interaction occurred outside of the electronic process. If that weren’t considered, I think you would force students to not collaborate in the real world because they wouldn’t get credit for it, but rather wait until someone posts something wrong, and then go to the wiki to fix it and get credit. Seems like that would unintentionally discourage peer to peer collaborative and teachable moments.

    So from the “edtechie vision” – I think you have to ask how much of the real process can you capture, can you hold peer reviews, and what can you observe as a teacher in a networked classroom? It seems like it would be a combination of new and the same – while you can’t rely completely on electronic tracking, you can add that to how you assess any group project.

  4. Cathy Nelson says:

    I like to think of the 2.0 tools I utilize as my networked classroom, the place where I have learned significant amounts, for myself as well as application in a k12 classroom. I can see definite failures and successes in each one, but the beauty is my network (or most of them anyway) are very forgiving and always encourage me to keep trying, to continue learning. Yes, some turn their backs and have little patience with me, but that’s okay, I’ll just have to learn from other more generous people in my network. What’s different? We are allowed to play, fail, try again, play some more, have successes, and then try for even more complexity. IN our failures we are not made to feel like we cannot succeed or that we’ve failed others. Instead we are motivated to continue striving for success. Don’t we wish the real k12 world were like this? Funny that I am also thinking this is the way we assess our PLNs too.

  5. In looking at Darren’s wiki assessment, I like how he distinguished between a significant contribution and a constructive modification. I think that’s a good and helpful distinction for students and fleshes it out for them.

    I also agree with Sylvia that it seems significant to catch the “process” in action as well–the verbal conversations and interactions.

    You ask what is different, and I’m thinking that the tools ability for you to “capture” a picture of who is editing what, and making what contribution is different, because it’s like cloning the teacher who can now be observing the process in ways that they couldn’t before when working with many different groups in one class. (not to mention the process and group contributions that would be happening outside of class).

    And I wonder how the group dynamic changes with the use of the technology–I tend to find it brings out contributions from some of the quieter students or students who need more reflection time than the typical class period allows, but sometimes it may stifle or limit some students who are more uncomfortable with the technology. One of our teachers recently had his students do a questionaire on a wiki and discussion board software they were using, and the responses bore that out. Some students liked being able to go back and reflect, and others felt like they couldn’t adequately defend their ideas the same way that they could in class, because they “weren’t there” when someone made a comment.

    Anyway, my point being that the comfort level is something “different” that maybe needs to enter into the evaluation process as well?

    I also like Cathy’s comments about trying, failing, exploring–and I wonder if the asynchronous and collaborative nature of using wikis allows students to do more of that explorative process, and if so, how can that be entered into the assessment equation, or should it be?

  6. How do you assess contribution in a networked classroom?

    Ok, so what does it look like? What’s new, what’s different, what’s the same? Your ideas?

    I must say that this question still requires some distillation. If you are talking about the mechanics of how to do assessment in these networked environments the answer is RSS subscriptions. I try to encourage teachers in my school to utilize web 2.0 tools such as wikis and blogs and the one overwhelming concern is how do I manage this information overload and how do I know when someone posts? The answer is to aggregate responses into a single reader so you are notified when someone posts among various places.

    The flip side of this question, and the side that is most important and I assume is what is really being asked here has to do with how we define the rubric. I think it is dangerous for us to assume there is one or could ever be one set rubric for assessing contribution in a networked environment or any other learning environment. This is going to depend highly on what it is you want students to achieve academically. Ultimately, whether we use pen and paper or web 2.0 tools, we have to assess the learning according to the desired outcomes as they relate to the content. Otherwise we are just teaching students how to use the tools. Why do I need to learn this? When will I ever need to know this? These questions will flourish if we approach web 2.0 tools as an end and not simply a means. It is like teaching students how to build a house but giving them new tools to do it. In the end, whether they have hammers, handsaws, and screwdrivers, or power drills, table saws, and air hammers the house still needs to be measured by the same standards and benchmarks.

  7. David Jakes says:

    To all the commenter’s:

    Thanks for responding, your insights are very helpful and extend my thinking. Mathew is bothered by teachers assessing the end product only and I agree-I think assessment has to be more global in nature, and be capable of monitoring formative contributions as well as the final summative product. That’s really not new, but I think many teachers lose sight of that from time to time. Matthew makes an interesting point, about how a single comment, while not part of an electronic process as Sylvia suggests, may be critical to the learning process. I agree with Sylvia that observation of the activities of students need to be present to factor into such evaluations of student performance.

    Both Carolyn and Carl mention rubrics-I’m not sold on rubrics at all. I think they turn assessment, which I consider to be something elegant if done correctly, into a “cookie-cutter” format. I know many teachers have used them successfully, but I want to move past them. I’m wondering when rubrics became the de facto standard in assessment. What’s beyond the rubric?

    I strongly believe there is a place for more peer assessment in what we do. This, if done correctly, could prove instrumental in capturing the subtle contributions of all during the process-perhaps that should be the goal. In any case, I would think that we would have to look at the assessment from multiple angles using multiple strategies-again, not a new idea. And we certainly need to focus as much on process as we do the mastery of content.

    Sylvia poses an interesting question about receiving credit for contribution. Networks are about contributing, and at what point would we expect students to contribute to networked learning without the expectation that that contribution would not be for a grade? I would think that is ultimately what we are striving for.

    Like Carolyn, I also find that different technologies “level the playing field” for student contribution…our use of discussion boards in my school district have provided direct evidence of greater student participation, greater reflection, and increased dialogue within and between classes. I also believe that we need to assess not only the students, but the technology itself and it adds value to the learning experience.

  8. Reggie says:

    We’ve been using both Drupal and Moodle to facilitate student writing for two years now. I chosen to assess student contributions a couple of ways; first to allow student to assess postings themselves through a few mechanisms- kind of a rating system which I monitor. I also do an assessment based on a modeled criteria that attempts to measure contributions within regard to what others are contributing. This seems to foster scaffolding of learning. The four types of contributions are:

    Triggering:
    Recognizing the problem
    Sense of puzzlement

    Exploration:
    Divergence within online community
    Divergence within single message
    Information exchange
    Suggestions for consideration
    Brainstorming
    Leaps to conclusions

    Integration:
    Convergence among group members
    Convergence within a single message
    Connecting ideas, synthesis
    Creating solutions

    Solution:
    Vicarious application to real world
    Testing solutions
    Defending solutions

    These four general areas are derived from Garrison’s “Critical thinking, cognitive presence, and computer conferencing in distance education” (2001 I think).

    I go over these areas with students prior to using it.

  9. I am going to post this comment also on Clarence Fisher’s blog and my own since this conversation seems to be taking place in multiple places and connectivity is essential to readers outside the network:

    I agree that rubrics should not be the end all be all of assessment in a classroom. However, like Clarence said, “How can they hit a moving target without knowing what they are expected to do?” I believe the inclusion of rubrics or checklists for assessment are necessary for dealing with what could arguably be a damaging necessity in our classroom: grades. Most of us are mandated to give our students some kind of mark at the completion of a learning activity or unit. I don’t believe grades are always healthy for learning as they are by nature extrinsic forces and only really work when the student doesn’t find any intrinsic value in the learning activity. Grades can do more harm than good, especially when dealing with something as engaging and intrinsically rewarding as a PLN because they diminish the intrinsic motivation by replacing it with an extrinsic reward or consequence. However, if we are mandated to use these extrinsic devices we have to make it fair. There has to be some way to clearly lay out for students how they can achieve a certain grade. The nice thing about rubrics is they can also work as teaching tools, not just assessment tools, because they can be instructive. They can also be written in such away that they lessen the extrinsic consequence factor by laying out exactly what needs to be accomplished leaving room for the intrinsic motivators to take over where the rubric leaves off.

    But, assessment should be about more than just grades. Assessment should be about giving useful feedback that the student can take with them and either reinforce good behaviors or products or redirect misinterpretations or misunderstandings. Assessment should show students what they need to do to take their work to the next level. Whenever I have my online students participate in any learning activity I always give them a rubric and make it known that this is the measuring tool that I will use to give them their grade and ask students to do a self assessment using the rubric when they hand in their work. When I assess the work I additionally give feedback beyond the rubric that is individualized. This feedback I would argue is the valuable part of assessment for true learning learning. In a networked learning environment this can be brought to peer assessment or even opened to public assessment. The student’s peers and any member of the the public who would contribute to the work’s assessment is unlikely to use a rubric. Instead their assessment will come in the form of comments. Comments that are meaningful because they are from real people and represent real opinions and not sterile placement on a rubric. But, the rubric still determines the grade.

  10.  
Leave a Reply