8
$\begingroup$

As you may know, schools in many parts of the world will remain closed for a long time due to the coronavirus outbreak.

In Italy, the government has invited schools to provide remote teaching, and since many schools are subscribers of G Suite or Office 365, teachers are giving online lessons using web conferencing services (Google Hangouts Meet or Microsoft Teams).

Now, the main problem is with tests.

Regarding computer science, if we know that "it is spectacularly easy to cheat on computer science labs" (and in my tests students have to write code on paper), we can just imagine how simple is to cheat in an online test, whether it is an assignment to be submitted after several days, or a few minute quiz.

A different assignment for each student is not a solution, since lazy students could ask clever students to solve their assignment, so I think I am forced to abandon tests which require students to write their own code.

Maybe a better idea could be a quiz with automatically generated questions, like a code snippet and the request to determine the output for a given input, with different input values and variable names for each student.

Do you know any tool to create such questions? Or do you have any other suggestions?

$\endgroup$
  • $\begingroup$ How many students is this for, and do you have TA support for marking? $\endgroup$ – kaya3 Mar 10 at 12:17
  • 2
    $\begingroup$ This is a very timely question, and highly deserves my upvote. It's also something I needed to research this week anyway. Thanks for bringing it up! $\endgroup$ – Ben I. Mar 10 at 12:43
  • 1
    $\begingroup$ The SIGCSE mailing list currently has a long thread about teaching online. It is more directed at delivering lecture, though. $\endgroup$ – Buffy Mar 10 at 13:01
  • $\begingroup$ @kaya3: 20-28 students per class. No assistant. $\endgroup$ – totera Mar 11 at 1:08
12
$\begingroup$

I don't think there are any published tools to generate programming assignments, though there are articles that describe such tools (such as this one from a CUNY, or this one from Croatia, so one option is to try to contact those authors and find out whether they would share their results with you.)

I will state categorically that there is no way of guaranteeing that students will not cheat. As the old saying goes, "locks are to keep honest people honest." Putting road blocks in the way of cheating will help dissuade students from trying, both because it becomes more of a bother, and because it gives them a chance to remind themselves of the ethics involved.

The bit about ethics is important, actually, and it can be remarkably effective all on its own. Dan Ariely did an excellent set of studies about this, which he summarized hilariously in this TED talk. If you don't want to watch that, then you can read this blurb from his blog, though the talk is well worth the time and much more interesting.

"We used an honor code in an experiment at MIT in 2007. We asked a few hundred undergraduates to do some simple math problems but didn’t give them enough time to finish. We then asked them to score their own tests and tell us how well they had done, with payment of $1 for each question they reported getting right. The students were asked to shred their papers afterward—but our shredder didn’t really work, so we could see how they had done. Many cheated. We had a second group of students follow the same procedure but only after they first signed the honor code. Signing the honor code before the test eliminated cheating altogether.

The effectiveness of such a code doesn’t stem from signing it, though. It comes from being reminded about moral issues. If your students eventually stop thinking about the ethics code as they sign it, it will lose its power. If they keep reading and reflecting on the code, its effectiveness might increase.

I ask my own students to write their own version of an ethics code—not because I’m especially interested in their interpretations but because it helps to ensure thought."

Dan Ariely

So, one way of working around the COVID-19 problem is to combine both ethical thinking and a few roadblocks to help gently dissuade cheating:

  1. Remind students about ethics up front, at the top of your exam. (I make them copy out, by hand, an honor statement. On a computerized exam, I would provide an image of the statement and make them type it.)
  2. Schedule the exam in advance. Give the exam at a specific time, and a narrow window (say, 10 minutes) to begin the exam.
  3. There is no need to make different exercises for each student. Make 3 programming problems, and assign each student to different problems using a rule. This decreases the chances that they will have a friend in the class with the same problem.
  4. Require that the exam be finished within a reasonable time from the start time.

It's not foolproof. They could, for instance, have their parent sitting next to them, telling them the answers. But you can't do anything about such a scheme. What you can do is set up roadblocks and set the stage for ethical action on the part of your students. If Ariely's studies are any guide, you should get the large majority of students to submit honest work this way, and that is probably the best you can hope for.

| improve this answer | |
$\endgroup$
  • $\begingroup$ The honor code at Dartmouth has worked pretty well, I understand. It was put in place about 1968 and has been revised a couple of times. It works well enough that when violations occur you hear about them. $\endgroup$ – Buffy Mar 10 at 12:57
  • $\begingroup$ But I think that one reason for the success at Dartmouth is the long history of it. Also that it puts responsibility on the students themselves to deal with violations. It is pretty well thought out. $\endgroup$ – Buffy Mar 10 at 13:36
  • $\begingroup$ @Buffy Did you watch the video? In the study, students agree to abide by the MIT Honor Code, and this turns out to remove almost all of the cheating. One of the wonderful punchlines is that MIT doesn't actually have an honor code at all. It was simply the reminder to act ethically that seemed to have the effect. Even more interesting, there was the same effect from students trying to remember as many of the ten commandments as possible, even though neither "don't cheat" nor "don't lie" is in there. Also, it turns out that people are almost hilariously bad at remembering them. $\endgroup$ – Ben I. Mar 10 at 16:39
  • $\begingroup$ @ben don't use link text of "this" or "here" is can cause accessibility problems. If there is one article you can use the text "article that describes such tools", or you can use the name of the article. OR ... $\endgroup$ – ctrl-alt-delor Mar 15 at 9:59
  • 1
    $\begingroup$ @ctrl-alt-delor I never even considered that, but it makes total sense. I've fixed it on this one, and I will try to avoid that practice in the future. Thanks! $\endgroup$ – Ben I. Mar 15 at 19:36
5
$\begingroup$

I agree with Ben I. and Buffy. I'm planning to cancel the remaining exams in the semester (assuming my university ends up moving to online instruction) and replace them with projects. Many studies have shown that project-based learning is more effective for most learners, and especially for learners from underrepresented groups.

That said, if I needed to proctor an exam remotely, I'd write an exam that's taken online (e.g. using Jupyter Notebooks) and then have all my students share their screens with me. I might also ask them to keep their mics on and their volume off, so I could hear if there was talking going on, but students wouldn't be bothered by the noise coming from each others' settings. You could even just record the whole hour long thing as a video on your machine, and students would know that if they cheated there would be a recording that the professor could dig through in the future to catch them. I think that would be a strong enough disincentive to cheat. It doesn't help with silent texting, but it stops students from actually talking to anyone else who is physically present, or from googling when they should be working on the exam.

EDIT: I just learned about a software called "respondus lockdown browser" for exams. You get a two month free trial, which should get you through the end of the semester if deployed at the right time. It prevents students from opening any window other than the one you give them (e.g. to google) and it also films them using their webcam (e.g. to prevent checking the book or texting).

| improve this answer | |
$\endgroup$
2
$\begingroup$

This isn't really an answer and requires long term thinking, but perhaps this is an opportunity to start to rethink how we evaluate students. I'm not a big fan of testing. Especially high stakes testing and testing dependent on memorization.

Perhaps there is just a better way to determine whether students have mastered the material. I depended much more on paired and small group projects for evaluation, combined with peer evaluation. This was toward the end of my teaching career after I'd learned some things about teaching and learning.

But pairs can work over the internet as can groups of up to about five or so students if you provide a rich set of communication tools. Also, pairs meeting face to face isn't as dangerous in the spread of disease as large groups (citation needed, I suppose). But people can also choose how to meet, on-line or in person, as long as you don't force them to meet face to face.

But, my base position is that if we can learn how to evaluate students without traditional testing it would be a step forward. The Pedagogical Patterns Project has some thoughts on all of that, of course.

| improve this answer | |
$\endgroup$
1
$\begingroup$

At my University, we had the chance to modify the syllabus, if needed, to remove or decrease the weight of "traditional" written tests.

While in most programming courses eliminating tests is not a problem, as most of us were already relying mostly on assignments, group works, oral presentations, and so on, there are some courses, were more traditional tests may eventually still make sense.

Most of them are assuming that these are open-book tests, so the questions involve some degree of understanding of the topics and not simply memorization.

Some of my colleagues are using a tool called Exam.net that is capable of recording the student's face and the exams are made in full screen, and whenever the student comes out full screen, the test may automatically be blocked, and the student has to ask the teacher to unlock it. Even Moodle has a similar feature that allows you to have a locked full-screen window, which can be used along with other tools such as Zoom to record the student's face. Some colleagues are also using Google Forms and Microsoft Forms along with Zoom for the same purpose. Some of these tools also have features to control the student's IP address.

The problem I see with these approaches is related to the fact that you are only seeing the student's face. You cannot see who else is in the room, who is typing, if the student has multiple screens, if the student is using TeamViewer of any other tool and someone else is answering on their behalf. So although it is possible to have some more control, in the end, these measures end up working more like some kind of a way to make us feel that we are doing the right thing to make the tests valid. In the end, I believe that there is not a 100% cheating-prof testing solution. If anyone knows of one, I certainly would also like to use it. :)

| improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.