How should students be encouraged to reflect and improve after summative assessment?

Question

Computer Science lends itself to a variety of different assessment methods - from the more practical assignment based programming problems, to the more formal final exams that cover the theoretical content. More formative assessments give students and educators the ability to improve learning and teaching whilst it is ongoing, but there is usually some concept of a "high-stakes" summative exam in most courses. I'm sure we can all agree that portions of one module (e.g. on data structures) will be exceptionally useful in later modules, and so it makes sense to do as much as possible to help students master these concepts.

Often (from my own experience as a student), the "feedback" from the summative exam is a simple number or grade, which isn't broken down by assessment component. Further, there is usually no way to see the exam script or any form of mark scheme or question breakdown, which makes it difficult to improve in the general context of the university degree. The summative exam does nothing other than provide a coarse measure of the candidate's performance during the exam.

Is this sort of status quo widespread throughout undergraduate level CS, and what can be done to give students the opportunity to learn from and reflect on these sort of assessments? Should a more formative approach be taken for the majority of each module?

Answer 1

First, I dispute your statement that "the summative exam does nothing other than provide a coarse measure of the candidate's performance during the exam." Assessments are not merely chances for us to discover how the students are doing, they are also useful in the learning process itself. They provide a focus for study, and they provide vital motivators for memory retreival, and they provide the stakes that allow strong enough emotions to trigger memory.

Now, as for your question itself, I have also particularly struggled with summative assessments, and I ultimately concluded that I did not really believe in the model, and could not administer strict summative assessments in good conscience.

You've pinpointed the problem perfectly: once you've given a summative assessment, where do you go from there? Instead, I utilize a test-retest model for all assessments provided throughout the year. Every retest is provided with a lower maximum attainable grade.

At my current institution, I have calibrated this at 6 points per makeup. (At a previous institution, I used 8 points.) This is not a deduction, it is a cap. So, earning a 65 on a first makeup will net you a 65, but earning a 98 will net you a 94. My goal is to encourage targeted study, so I do not allow for further makeup penalties. This means that the highest grade always wins, so you cannot lose points by taking a makeup.

Now, I also create a practice quiz or test for every assessment. This approach certainly involves a lot of work on my part (as I am always making 3 versions for practice-test-retest, and sometimes have to create 4, or even 5, assessments), but I love the net results I get in my class. This system has virtually eliminated students falling seriously behind.

I do have a rule that students who fall below a certain grade (a) must take a makeup, and (b) must check in with me or a pre-approved student (roughly a TA) before they are allowed to take it. This prevents the problem that some students have of simply taking makeup after makeup, with no appreciable improvement from one exam to the next.

Answer 2

The Pedagogical Patterns community has collected and published quite a lot of work on Feedback. The book referenced describes practices that have been vetted by the community. There are many more as well. Not all are related to exam grading, of course. Some that apply here are:

• Kinds of Exam
• Self Test
• Own Words
• Reduce Risk
• Differentiated Feedback
• Mock Exam
• Grade It Again, Sam
• Fair Grading

Other idea, that I don't think have yet been captured as patterns are thing like having students submit questions for an upcoming exam, probably with some reward for good questions.

The pedagogical patterns community has continued the work also. Especially in the assessment area. A couple of workshops have been held in Europe, for example. Christian Köppe at the Han University in The Netherlands has been one of the drivers of the recent work. Newcomers welcome, of course.

I agree, however, that a completely summative assessment has little value and gives you little guidance for improvement.

But to the specifics of the question. If your class isn't too big, you could meet individually with those who need to improve. You could let students form small discussion groups (say 3 students each) to discuss answers on the exam. They need not reveal how they did, but could discuss how they approached the questions. Or you could meet with a small group rather than individuals to discuss the future and strategies for improvement.

Of course, if the assessment vehicle is poor you need to work to improve that. It is possible for questions to be so bad that students are misled by them. I've written one or two of those in the past (alas).

---

Some of the recent work can be found here: http://hillside.net/eduplop/patterns-publications/