How can I effectively manage peer evaluation among my students?

Question

My students do quite a lot of their work in projects and much of that work is outside my view. While I can just give everyone on a team the same grade (or not) it is useful to me to know who is contributing to the work and how they are contributing. I don't require everyone contribute either the same "amount" (whatever that means) or in the same way?

How can I manage peer evaluation so that I get useful information for grading, for advisement, and for course improvement?

I want my students to have a way to comment on the work of their peers on group projects. They can also comment on their own contributions, perhaps, as well. This is especially valuable, I think, when students do much of their work outside my direct view.

I need the peer evaluation to not seem threatening to the students, and I don't want it to generate ill feeling among them.

---

To make the problem a bit harder, I don't want to divide up the work among the team members myself as I think that would inhibit their learning to be a team. The team itself is responsible for managing its work. Ideally, however, I hold each responsible for the team effort.

Answer 1

Something like the following has worked for me over several years. In any group assignment, each member of the group also submits a peer review sheet. For a group of two (e.g. pair programming) I ask each participant to detail the most important contribution of their partner AND their own most important contribution. I don't ask who worked harder or ask them to make an evaluation. I ask them for the contributions.

In a group of about five, I will ask each member to name the (say) three most important members of their group and to mention the key contribution of each. I tell them that they can include themself in the list of three. I also ask, separately for their own most important contribution.

I let them know at the beginning of the course that this will happen and its parameters, so it isn't a surprise.

Students see that they are supposed to contribute to a group and are not asked to speak ill of anyone. Keep the questions entirely positive. But you also learn a lot from what is not said and who is not included in the contributor list.

If the project happens outside your view, you also learn things about individuals that you can not otherwise know. I've had students I thought were slacking who were praised by teammates for making the team work properly.

These peer evaluations are not optional.

This same idea was also discussed here: https://cseducators.stackexchange.com/a/2578/1293
It is also, essentially identical to part of this: https://cseducators.stackexchange.com/a/3402/1293

---

Let me note that there may be cultural limitations here. In some cultures it may be odd to name oneself as a top contributor. In others it might be very natural. You can modify the scheme, of course, either requiring or forbidding naming oneself as one of the top people. Use your judgement here.

Note also that this solution is based on the Peer Feedback pattern in Pedagogical Patterns: Advice for Educators

Answer 2

How can I manage peer evaluation so that I get useful information for grading, for advisement, and for course improvement [re] ...on group projects.

I use peer evaluation forms as a part of group projects as discussed by @Buffy above, and I also split the marks between group and individual elements. Taking the question in a wider context, I'll just speak from my own experience of peer evaluation, and the opportunities and difficulties I have found with it.

My first experience of peer evaluation was on the Rice University MOOC, An Introduction to Interactive Programming with Python. I liked it from the start, and thought it was a great way to grade work, provide feedback and reduce teacher marking time for formative assessments. We do not use it for summative assessment.

1) Sample exam questions: theory

An exam paper based on the theoretical elements of the course was designed, and a grading rubric with sample answers prepared.

I assigned each learner an ID number known only by myself.

Each learner received a copy of the exam paper with their own number on it

The exam paper had space for answers- no separate sheets.

At least one question has some ambiguity- this is not told to the class, and they cannot ask questions during the exam. This is to allow discussion of question design afterwards.

At the end of the exam, all papers were collected and each photocopied 3 times.

Each learner is given a copy of a grading rubric and three random answer papers (not including their own)

They are given time to grade each paper and make a note where marks not were achieved or partially deducted- this provides each learner with feedback. All feedback must be positive- indicate what would have improved the answer, and why marks were deducted.

Collecting all the papers and re-ordering to give each learner back a copy is tedious work.

After the exercise, we had an open discussion on each of the questions, if they could have been phrased more effectively.

Expansion option: Given a specific Learning outcome, how might they create an appropriate question for an assessment.

At the end of class, I provided them with sample answers to the exam so they could compare with what they had themselves.

Outcomes:

It was interesting that many found it difficult to award a grade, and did not realise how much time it can take to correct a paper.

There was some variation in the scoring of identical questions- this proved a useful talking point.

Each learner got useful feedback on their own work.

Learners have a greater appreciation of the creation and marking of assessments.

It was quite time consuming, but a useful exercise.

2) Coding mini-assignment- peer grading on Moodle

Given that photocopying code is a non-runner, I set up a peer-graded Assignment on Moodle. Each learner had to work on a mini-programming assignment and had two days to complete it and submit online.

Firstly, getting this to work the first time on Moodle was very time consuming. Even on the day, there were several delays in getting started.

The grading rubric identified many elements such as use of "appropriate" names for variables and functions, header info, docstrings (Python), clear user prompts/instructions, working code, use of comments and so on.

When peer-grading, it is possible to place a comment in each section to explain why mark(s) were lost, and each marker can place a final comment/general observations.

Benefits of Moodle Peer assessment

Learners are assigned the work of others to mark randomly

Anonymity is an option in the setup

Each learner was forced to really look at the grading scheme, which was provided from the outset. Many lost marks for simple things that they should have included (e.g. header information). This is important- that they realise they can get some marks even if the code doesn't work completely or partially.

All got to see examples of other learners approaches, structure, working and non-working code. Seeing mistakes others make should help them identify those that they make themselves.

Teacher can review all more easily without looking through a lot of paper, i.e. the grades given and received by each learner, and the comments given and received.

Difficulties

I have to say, from a practical perspective, it ran about as smooth as a dragons tail (i.e. not at all smoothly) but this was likely due to my inexperience with using this in Moodle. That said, it was better than paper.

Some students failed to provide any feedback even when it was a requirement of the exercise.

There are some relevant/complimentary points also raised here.