Can sharing learning stats be beneficial to guide beginner students?

Question

Context

The main consensus in most compsci programs I have been at is that a sizable amount of students is inherently unable to learn how to code. An old entry on Jeff Atwood's blog summarizing the issue cites this 2006 article by Saeed Dehnadi and Richard Bornat, claiming:

Formal logical proofs, and therefore programs – formal logical proofs that particular computations are possible, expressed in a formal system called a programming language – are utterly meaningless. To write a computer program you have to come to terms with this, to accept that whatever you might want the program to mean, the machine will blindly follow its meaningless rules and come to some meaningless conclusion. In the test the consistent group showed a pre-acceptance of this fact: they are capable of seeing mathematical calculation problems in terms of rules, and can follow those rules wheresoever they may lead. The inconsistent group, on the other hand, looks for meaning where it is not. The blank group knows that it is looking at meaninglessness, and refuses to deal with it.

It even goes on as to propose some of the prospective students to be advised away from compsci courses:

7 Conclusion

There is a test for programming aptitude, or at least for success in a first programming course. We have speculated on the reasons for its success, but in truth we don’t understand how it works any more than you do.

Today, I found a retraction from Richard Bonat, claiming his conclusions were affected by his mental health issues:

2 How it happened

Though it’s embarrassing, I feel it’s necessary to explain how and why I came to write “The camel has two humps” and its part-retraction in (Bornat et al., 2008). It’s in part a mental health story.

In autumn 2005 I became clinically depressed. My physician put me on the then-standard treatment for depression, an SSRI. But she wasn’t aware that for some people an SSRI doesn’t gently treat depression, it puts them on the ceiling. I took the SSRI for three months, by which time I was grandiose, extremely self-righteous and very combative – myself turned up to one hundred and eleven. I did a number of very silly things whilst on the SSRI and some more in the immediate aftermath, amongst them writing “The camel has two humps”. I’m fairly sure that I believed, at the time, that there were people who couldn’t learn to program and that Dehnadi had proved it. The paper doesn’t exactly make that claim, but it comes pretty close. Perhaps I wanted to believe it because it would explain why I’d so often failed to teach them. It was an absurd claim because I didn’t have the extraordinary evidence needed to support it. I no longer believe it’s true.

Question

I've been a TA for compsci courses in three colleges and, in all of them, transparency was a core value: past exams, classes, and grades were made available from the very beginning. Failure rates were never lower than 30%, often beating failure rates for subjects widely regarded as more difficult, such as physics or calculus.

On the one hand, some students might feel discouraged to put in the effort to learn knowing that they have a high likelihood of failing. On the other hand, it might be beneficial for them to know hiccups are expected and perhaps compsci might not be the way for them.

My questions are: do students benefit from knowing most of them will likely fail the subject? Should they be informed that there might be a high chance they won't advance past CS101?

Answer 1

We had a question recently that also touched on some of these same issues.

I am a fan of expressing that the material can be difficult, as it sets the tone and prepares students to work hard in a course. However, if failure rates are as high as you suggest, then that suggests something is going seriously wrong.

For context, I teach at a high school where kids apply during 8th grade into 4-year majors. They are tested in math and literacy, and we interview them to see if they are generally engaged kids. We do not evaluate them on programming or CS background, because most of the 8th graders have none.

If accepted, they cannot change their major at any point. That means that our CS kids will have a 4 year CS curriculum that builds upon itself, and it gives us teachers a very powerful motivation not to lose kids at the beginning. It simply isn't a viable option. A freshman who gets lost will probably remain lost for their next 3 years in the school.

Not every kid remains enthusiastic about CS. That's okay! They applied to the school when they were quite young, and many move on to other interests. However, over the last 3 years, we have had exactly one student not pass the AP test as a sophomore, and that student was dealing with significant mental health challenges that also brought about high absenteeism.

Now, a university program is not under the same constraints, and not every student at every institution can be classified as "gifted" (whatever that actually means), so I can absolutely accept that our results might be an outlier.

However, virtually everybody can learn to code. It comes down to what work the student puts in, and how well the instructor breaks down the concepts and designs the course materials.

My general advice to achieve high passing rates is to create good supporting materials to use with kids who struggle, and to break things down (and make connections explicit) far beyond what you would think you would need to. I know that that advice is quite vague!

And if you are ever unsure about how to break down an idea really clearly, ask around. (Heck, I could tell you about this one website devoted completely to getting answers to the problems of CS educators ... ;-) But please don't accept a 30% failure rate as the norm, or even as acceptable. So much more is possible!

Answer 2

30% is really troubling. Not to say that it's your fault, or the students', but there is some sort of impedance mismatch going on. In some learning communities you have the challenges of hard material and a student body that is not prepped for success and is easily discouraged. I see this a lot in bodies that have to (eg) hold down a job to survive or are the first in their family to go the school. There's something akin to survivors guilt going on I sometimes feel. Anyway, try looking into different more collaborative models like pairing everyone up with a study buddy and offering collaboration based problem sets instead of lone student homework challenges. Good luck!

Answer 3

I'm so glad you found the retraction. I am often surprised at how far that (non-peer reviewed) paper has spread, and how few people know about its retraction. I imagine you've also seen the more recent work calling bimodal distributions in CS grades into question.

On the topic of motivation, I found the relevant chapter in How Learning Works to be very useful (search "Thermodynamics" in "Look Inside"). There is a case study of a professor faced with a similar dilemma about sharing poor past grades, and then a breakdown of the various factors that influence students' motivation. In short, a supportive environment, student self-efficacy, and student value in learning the material are all needed in order to produce motivation. Making students feel that they may fail even if they put in effort will almost certainly lead to a lack of motivation, leading to poor learning. Luckily, it's possible to address all three factors in the classroom. Personally, my tool of choice has been active, collaborative learning activities like Peer-Led Team Learning and Peer Instruction---always from high-quality sources.

I believe NCWIT at one point recommended that professors make the overall grade distributions public during the course, so that students could have a realistic view of where they stand. This is for students who may think they are not cut out for CS, when in fact they are doing well in relation to their classmates. Think of the classic case where a female student gets a B and thinks she is terrible at CS, while a male student is pulling a C and still maintains his enthusiasm and drive.

Answer 4

do students benefit from knowing most of them will likely fail the subject?

Let's see here. I do this with all my students. I explain with hard data (evaluation, the number of hours they spend studying, the quality of their assignments, the discipline they exhibit) that there is a good chance they might fail a given subject.

Most, will flat out refuse to believe me. I would say, 8 out of 10 would disagree strongly with me. However, 2 out of 10, will come back to me, schedule one on one sessions. They will work on where they are weakest, and then, these 2 actually clear the subjects they are weak in, while the remaining 8, will end up failing anyway, thereby proving my prediction about them.

Short answer, yes, as a educator, we must share them what we know about their chances of failing at a subject. They should know.

Should they be informed that there might be a high chance they won't advance past CS101?

As with the earlier answer, they should be informed. That way, at least those that are willing to rectify their learning style, can improve their chances of success.

the key thing, I spend a considerable amount of time collating data about the students, before telling them that they might fail. its not really required of my job description but I do it anyway.