Is there a better way to validate student learning?

Question

I'm a student in an undergraduate Computer Science program with several years of experience before coming to university. As I began working through the coursework here, a large problem stuck out to me.

At my university, students are thoroughly prohibited from collaborating in any way on assignments, except for certain ones with permission. This includes even to the point of hosting review sessions where students talk through the intricacies of the best way to complete a project, without sharing any code. This is understandable. Instructors I've spoken to explain that it's important to verify student learning by forcing students to work individually. Unfortunately, this goes against everything I've been taught about how to learn.

I was fortunate enough to be mentored in programming by more experienced software engineers before I came to university, so I attribute all of my learning to being able to work in teams and ask questions of my peers/collegues. Here, that learning style isn't just discouraged, but actively punished. The instructors have a valid point however: it's important to ensure students are capable of working on their own when called to do so.

As Computer Science Educators, what techniques have you used to ensure students understand the material while continuing to allow extensive collaboration during practical programming assignments?

I may not be an educator myself (yet) but I have a passion for improving how people learn. I hope this question fits here.

Answer 1

You seem to already realize that this is a subtle question. When I taught Mathematics early in my career, I also forbade students to work together. Later on, teaching Computer Science, I found myself at the opposite pole. I normally forced students to work together on nearly every task. In some ways I changed, but it was more that the subjects are different. So, I learned to (try at least to) match the student activities to the kind of thing they needed to learn and understand.

In some courses personal insight is essential and hard-gained. Much of math is like that. The student needs to grok the ideas deeply so that they can be integrated into a whole so that the frontier of knowledge can be pushed forward. If students work together in such courses, it is inevitable that one of them will have the key insight first. A-Ha. From that point the collaborators are really just watching someone else work and will be denied the experience of overcoming the block themselves. Since those blocks are frequent in math, overcoming them is an important skill in its own right. Some CS topics also have this same nature, but most do not.

As you note, most work in industry is done in teams and team members share insights with one another. In fact, Pair Programming is specifically designed to force insight sharing. But as you also note, you sometimes need to work alone and so you need the skill to do that. But Pair Programming also fosters this skill as the Driver and Navigator roles swap frequently. One way to bring a newbie up to speed is to have him/her pair with a veteran. The experienced programmer could probably build everything alone but surprisingly often benefits from the fresh insight of the new hire. Since team-work is a learned skill, I think the university should help the students learn it.

In graduate education, things may be a bit different, but again, it depends on the course. More work is likely to be individual at this level as it is more likely to be insight-rich. Certainly by the dissertation level the student needs a lot of experience working through their own tasks. But even here, discussions between dissertation-level students working on different problems can be valuable in seeing how some things can be approached.

I worry mostly that forbidding collaboration in the classroom raises the stress/tension level in students. I hope there aren't many who see this as a good thing. Students have enough stress anyway and I think we faculty should work to reduce, not increase, it.

The faculty that forbids collaboration of any kind is probably a bit naive about what really happens outside their line of sight. Students no longer live in monastic cells. They meet. They talk. It seems better, in my view, to recognize this and to guide it.

One top US college at which I taught has an honor code in which students can collaborate on projects without limit as long as they share no code. Students can ask others for clarifications and advice. Students formally sign this code as a condition of acceptance to the program and it is kept on file. There are a few violations, of course, and the college treats them as a really big deal, with expulsion as a possible punishment. It isn't automatic, however, and review committees are permitted to show mercy as appropriate.

I've taught important courses in which the first task is to break up the students in to teams of 4-5. All of their work, with one exception, was done in collaboration and we (there were two professors) provided collaboration tools so that the students could easily meet in virtual space as needed. The one exception was an individual project that was done (take-home) over the period of one week. This project might have been related to the overall team project, or not, but it had a similar "flavor" in any case. The teams were encouraged to bring their less experienced members up to speed prior to this task and the teams took pride in the success of their members. Many of the "newbies" struggled with this, but a few had really dramatic improvements.

In other courses in which most work was collaborative and most of the grading depended on group project work, there were one or two fairly low pressure exams that counted for a small part of the grade. In total it was about enough to raise/lower the marks by one level, A <-> B, say. But one full level change would be extraordinary unless the student simply skipped the exam. Collaborative exam prep was neither encouraged nor discouraged. I assume that the best students were more likely to collaborate in prep than others, but have no data.

One advantage of collaborative project work in the classroom is that students can work on larger projects than would otherwise be feasible. Time becomes less of a constraint. Seeing large projects with many parts is a good thing, even if you primarily work on only one part of it.

One of the issues that some faculty have with collaboration is that it can be difficult to determine who the chief contributors are so as to provide appropriate rewards (and sanctions, of course). This needs to be planned for. There is a concept (and Pedagogical Patterns) about Peer Evaluation that are worth exploring. It is quite easy to do and can be structured so that it is a positive experience for students. Students can be encouraged to reward their best and most helpful team mates. I once also learned that a particular student who appeared to be disengaged in the class was actually the chief contributor to a team and that I'd likely misjudged him.

---

Please don't weaponize these comments against your faculty. It won't do you much good to be seen as a contrarian.

Answer 2

I look at it as two separate tasks.

Learning: The goal here is to learn new things and practice what you've learned. For my classes, these are mostly small lab style assignments. I want them to work with their neighbors. I switch seats every two weeks or so that they have new neighbors to work with.

I've gotten pretty good at explaining concepts, but sometimes another student can explain it in a way that I've never seen and makes perfect sense. Best example is from a couple of years ago. We were going over indexOf and the starts at X, up to but not including Y. A student raised her hand and asked if we could just do Y-X to figure out how many characters to keep. It works. I just never saw it.

What I tell them first day is that the point of labs is to learn, and if they understand the labs when they're done it doesn't matter how they got there. Some like to work alone. Some work well in pairs or small groups. Some find almost solutions on Stack Overflow and tweak until it works. Some bang their head on their desk until something clicks. The guideline is that I should be able to give them a similar problem on paper the day after a lab is due and they should be able to solve it.

Yes, there has to be a grade assigned here because there are some students that won't do the work if it doesn't directly affect their grade. And, in high school at least, students don't seem to get the connection between learning how on labs and doing well on exams. Fortunately for me, our district has a very low percentage (10%) assigned to daily work.

Assessment: At some point though we have to figure out where the kids are. They have to do an individual assessment.

For me these are always in class either as a multiple choice test when it can be assessed that way or writing code on paper. The goal is to figure out what that student is capable of when they're working alone.

And in most cases I'll let them go back and correct if it's not a grade they're happy with. I don't really care as much about what they know on the test day as what they know when they walk out of the room on the last day of school.

Mixed: Projects are a mix for me. They're multi day assignments where students work together. But only part of their grade is based on the actual project. This year 70% of the points came from the project and the other 30% came from a written assessment that's similar to what they should have written for the project. Next year I'm probably going to 50/50.

Answer 3

As an instructor, this is certainly a sticky problem. There are two questions at play here:

1. What can I help you to understand during our time together?
2. How do I know that you actually know it?

There is an understandable impulse to work only on the first question. After all, what is the purpose of education if not to help the students progress as fast and far as they can? However, the two questions are not really in opposition. The first goal would be hobbled if the second question were not properly answered. For one thing, I can provide special resources (time, extra practice, reading materials, recommended tutors, etc.) to a student who is having trouble. For another, in assessing my students, I also discover where I need to improve as an instructor. Making these self-improvements is how I obtain long-term gains for future students. Thus, cutting out the second question would hurt students, both in the short- and long-run.

If you look in a grade book at the end of a semester, you will usually have anywhere from 4 to 10 touch points, and even these cannot always be assessed in great depth. These are paltry number, and it is already hard to characterize what students know from such assessments. This is even more pronounced when we allow for group projects and free interaction! We certainly help students learn by doing so, but we lose out on almost any form of assessment.

Separating everything out isn't necessarily better for the students, either, as this means increasing the number of grade book items, and increasing the stress and workload of the students (not to mention the faculty!) considerably. Thus, instructors often want these grade book items to pull double-duty, and attempt to answer both questions at once.

For myself, I have nevertheless decided to sequester my labs purely for learning. My tests are now how I assess that learning. This has allowed me to open the flood-gates to paired programming, full use of SO and other resources, and nearly unhindered access to peer and instructor help.

But it also means that there is more work for the students. I must now include nearly twice as many grade book items, and the sorts of labs that optimize learning are rich and deep, and these involve more time and investment from everyone. As a professional who believes in limiting stress for my students and aspires to reduce homework whenever I can, I find that this course structure severely limits my ability to do so. And, of course, all of this extra work is also taxing on me.

I don't have any cut and dried answers here because I don't think that there are any. There are competing goals, all of them important, and any approach the instructor takes will ultimately be the result of a push-and-pull between these interests. I have decided to heavily utilize collaboration as a learning tool, (and standards-based assessments with nearly unlimited retesting to help ensure student learning), but these techniques comes with real costs to everyone involved.

Answer 4

I believe that the best tool is communication. Letting students know, beforehand, why their progress is being assessed in a particular way and how they are allowed to collaborate is a very important part of teaching. For instance, MIT has a very interesting page on what type of collaboration when is allowed writing code for assignments.

Regarding the no code shared policy, this is only for assignments that count towards grading. And, since your grade should reflect your competence in the domain of the discipline, it seems fair that only what you did counts towards your final grade.

That does not mean that you should learn all alone. There is nothing that forbids students (and instructors) to do code reviewing or pair programming-like activities during class. In fact, there is (somewhat weak) evidence ([1][2]) that indicates that self-assessment and peer assessment activities are not only about grading but may contribute to learning as well.

[1] Falchikov, Nancy, and Judy Goldfinch. "Student peer assessment in higher education: A meta-analysis comparing peer and teacher marks." Review of educational research 70.3 (2000): 287-322. (available here)

[2] Sadler, Philip M., and Eddie Good. "The impact of self-and peer-grading on student learning." Educational assessment 11.1 (2006): 1-31. (available here)

Answer 5

I think that, when code is shared, one of the central questions is: After it has been seen, does the student understand what they have seen and will they be able to use this knowledge independently in the future. That is: Did they actually learn something?

There certainly is a population of students for whom this works. None of this is really about them.

There is also a population of students which seem to fool themselves that they are learning this way. I hesitate to psychoanalyze them too deeply, but they seem to have a very narrow per-assignment focus; just scraping by. If you let them, they will cobble together their assignments solely using social connections with negligible (or negative) value added.

Code sharing restrictions are the least bad method (that I have seen to date) of preventing students from joining this second population.

Answer 6

Background: As a senior in my IT company, I am often teaching fresh colleagues on-the-job, so I know where you came from. On the other hand, as teamlead, getting people to work together (i.e., tough things like being able to ask questions and actually listen to answers...), is very important for me, as there's only so much of myself (in the first role) to go around.

As Computer Science Educators, what techniques have you used to ensure students understand the material while continuing to allow extensive collaboration during practical programming assignments?

That long question mixes up a lot of things:

• People need to learn to collaborate.
• People need to "grok" the material.
• (The university needs to give marks - I'm not interested in this aspect, for this answer.)

Those two topics are quite at odds, in my experience:

• First of all, you can learn both separately from each other. (For example, if you have years of IT experience, you have probably witnessed SCRUM or Kanban teams - those are mechanisms for working together which do not care about what the team is actually doing.)
• Both are incredibly important on their own.
• Both can hinder each other, in any combination (i.e., one who knows nothing may feel so inferiour that he cannot really collaborate, i.e., won't ever ask questions, won't contribute etc.; but also someone who knows a lot may not be able to collaborate because they feel constantly superiour etc.).

So, my answer to your question would be:

• Make sure individuals really understand the material on a deep, intuitive level. This you can only achieve by actually making them think long and hard about them, actively working with techniques you teach, discovering stuff for their onw, and so on. It is your challenge as educator to find ways for them to do this! For some, you can simply pose them a vague problem and Google or Stackoverflow (or old-fashioned books) do the rest. For others, you might need to give them specific, linear information, stacked in a very structured manner, and very tightly guide them to the goal.
• Separately, make them work together and find out how that works. One way to make sure it's not just one individual doing all the work, is to make the problem so big that it's just not possible for one person to do it. Then they have to find out how to distribute the work, and how to re-integrate it. This is exactly what's happening in the business world as well (or should, anyways). In my opinion, the point of having CS students work together is not that they somehow all do the same at the same time (that's just offloading homework to the fastest person...).

How you do it exactly depends a lot on the topic and on your own preferences. Be it the occasional large 2-week project; or a "background project" that runs through the whole semester, or whatever.

I would certainly stay clear from encouraging the students to do every single bit of homework together. Having "heureka" moments is incredibly important. At the end of the day, much of the content in CS is incredibly complex and difficult, but can readily be chopped into pieces which can be handled individually. Students really need to be able to do that; but to do that, they really need to know the building blocks themselves, intuitively, and get into the habit of flexing their brain muscles, a lot.

Answer 7

"Group work" is hard to handle. One of the first courses I taught here was structured around a bunch of homework programming tasks, to be done in two-person teams. Around midterm one of the students showed up complaining he had done his half of the homeworks (his mate didn't participate), and now it was the mate's turn... and the mate had left the class.

The problem is that as long as you want to grade the work (and you, much less your students, rarely have the time for "non-graded work") you have to assess what each one did. A much worse problem is that in the later "real life" group work is a must, and you have to teach that somehow...