6
$\begingroup$

I am currently teaching the pretty much normal second year Data Structures and Algorithms course. In this course there are four programming projects, with the first project completed the automated testing and grading yesterday. However, out of the 200+ students taking the course, only 72 managed to complete this project so that they passed my automated tester for the pseudorandom seed that I used. As my first question to this forum, I would like to get some second and third opinions about this issue before I decide how deep I dig my heels into the ground about it.

This project (full specification) asked them to create a Java class named IntervalUnion whose objects represent unions of integer intervals, such as [1-5,9-12] or [4-8,11-15], with an arbitrary number of individual intervals allowed inside each such object. This class should then have the public methods union and intersection to create new objects, with the union of these two objects being [1-15] and intersection being [4-5,11-12], along with toString, equals and hashCode as usual.

I simply cannot believe that this project would have been unreasonably difficult for second year computer science majors who have already taken the standard two first year courses of introduction to programming and computer science using Java. And yet for some reason it seemed to be, especially using my automated tester that, based on the given pseudorandom seed, generates random test cases and computes a checksum of the results returned by the implemented methods, this checksum then being expected to be equal to the checksum produced from my private model solution.

For a test run that generated one million objects, easily automated with a Bash shell script, the passing projects took between one and three seconds to finish, whereas some solutions had to be terminated after minutes of waiting and thus rejected. The project spec specified a thirty second time limit that I would have assumed to be more than enough for them even to dilly dally.

Of course I might be completely mistaken in my view about this, so I would like to hear some opinions of other teachers who have taught this same second year intro to algorithms, before I make any announcements to the students.

Since my course uses the three best projects out of four in determining the total course grade, I am currently of the opinion that I am going to just tell the students to wake the hell up, realize that this is serious, and do better in the three remaining projects. The second programming project consists of writing method to remove all nodes with given key from the given linked list, and to sort the given linked list, using the sorting algorithm of their choice. (As with all these projects, the grading is 5 points for passing the automated tester, and 5 points for comparative running time rankings among all submissions.) There is just no way that that task could be considered too hard for second year CS majors.

The last two programming projects ("Packing words in bins" and "Word filling") due at the end of the course should be easy to at least pass in that within the simple rules of both word search puzzles, there are really no wrong answers, but the difficulty of both projects is in quickly finding a good solution that scores well.

(As an aside, I am actually pretty proud of that Word filling problem, as its idea came to me last summer during a nice mellow walk, and the solution sketch the next day during a ride home followed by about an hour to implement after dinner, and I wonder if I could be the inventor of such a simple but tricksy puzzle that I simply can't recall ever seeing anywhere before this. Another more interesting variation might be to maximize the scrabble score of the words, since that doesn't actually change the solution algorithm.)

$\endgroup$
  • 2
    $\begingroup$ Do you have any logs from the autotester? If they all decided to start the project the night before it was due, that's going to end poorly. I find that to be one of the principle sources of low marks for my students. They do not allocate enough time to debug, etc. It may have worked in the past, but now they are expected to write more complicated programs. $\endgroup$ – Adam Oct 9 '18 at 15:14
  • 2
    $\begingroup$ Have the automated tester and your model solution been well validated? Have you inspected any of the failing submissions by hand to verify that they should fail, and why? $\endgroup$ – Gypsy Spellweaver Oct 9 '18 at 16:09
  • 1
    $\begingroup$ Sorry for the delay, I was away for a while. The TA's had instructions not to help with the code. The automated tester accepts successfully all top ten submissions no matter which random seed I tried, and each method call has only one correct answer at all times. I think the problem may have been late starting and then not realizing that the project was not easy as they thought it would be. $\endgroup$ – Ilkka Kokkarinen Oct 9 '18 at 20:44
  • 2
    $\begingroup$ I found the requirement description was difficult to understand - it seemed unnecessarily convoluted and artificial to me. It is possible that the students simply couldn't figure out what was expected, since the final class should have been relatively simple. $\endgroup$ – pojo-guy Oct 9 '18 at 23:21
  • 1
    $\begingroup$ Did you include any source material about Allen's Interval Algebra? Maybe that was the problem. $\endgroup$ – Scott Rowe Oct 10 '18 at 22:45
4
$\begingroup$

Sorry, but digging in your heels or telling the students to wake the hell up is going to get you exactly nowhere. When a small percentage of your students fail to successfully complete a project it is likely their own lack of background or application. But when only about a third are able to successfully complete it, it is a problem with the course, not with them.

Some possible reasons for the problem. Perhaps they started late on it and weren't able to do a good job as user Adam asked? If so, why was that? Are they overworked generally in your course and elsewhere? I doubt that so many are just late starters. Was it a problem of insufficient testing? Was it too many tried a brute force approach that doesn't scale? How much does the course stress that early on - both testing and scaling? It seems like they weren't able to ask questions (of the TAs at least) as the project progressed. If they get stuck, how do they get unstuck? Just on their own?

I think you have a problem and I can't diagnose it, but you can. You can, for example, have your TAs do a postmortem with small groups. I hope you have more than one or two for 200+ students. The first question I would want the failed students to answer was "Were you surprised by the failure?" That can lead to a lot of discussion about the nature of it. I would ask about how much testing they did, especially on large sets. Were they able to use the tester effectively, providing appropriate inputs, or were they misled by it. Another question I'd want an answer to is how many students now actually know why they failed. What was it in the code or its creation that led them to failure. If they don't know that you have a serious problem going forward. I think that for these and similar questions your TAs are better able to get you usable answers. The students are less likely to feel intimidated by the questions than if you ask them directly.

On another level, I'd ask how many independent runs did you do with each student's work? Just one? The data was random, remember. Maybe naive inputs to the tester gave them poor feedback while doing their own testing.

Maybe a five week project with little guidance or feedback is sub-optimal. Especially if you treat it as an exam. It seems like a lot of time, until it isn't. Why not shorter projects? Why not more feedback? Why not, to be honest, continuous feedback?

I wonder how many students who failed did so only for run time and how many weren't even able to put together an algorithm at all. Has anyone given them feedback on their code and approach other than "failed"?

Are your expectations about what they "should have learned" in the earlier courses realistic? Are you making assumptions not warranted by the facts?

Don't just stamp your feet. Find out why this happened and change your teaching structure accordingly. Treat it like a scientific problem with an engineering solution.

$\endgroup$
  • $\begingroup$ That postmortem is a great idea. I will definitely instruct my TA's to do that in the next week's lab. $\endgroup$ – Ilkka Kokkarinen Oct 9 '18 at 20:50
  • $\begingroup$ I saw situations like this in a college course I took in 1984: many students befuddled and unable to develop a working algorithm. We had no automated testing system, and limited access to computer terminals (mainframe environment). As you say, the info about why submissions failed is vital to figuring out the OPs problem. Coaching the students in incremental build and test would be valuable. $\endgroup$ – Scott Rowe Oct 9 '18 at 21:21
  • $\begingroup$ One other potential issue I would add to this answer, somewhat related to the five week issue, is that they just can't fathom the steps necessary to break this problem down. I see this all the time in my CS1 and CS2 courses, even for less complex problems. I would either draw out a suggested timeline (e.g., "complete X method by Friday, Y method by Monday"), or have a check in at some point in time. Depending on the number of TAs you have, you could set up gitlab or something similar for students to check in their code periodically, and the TAs to assess whether they are making progress. $\endgroup$ – cryptic_star Oct 11 '18 at 15:15
1
$\begingroup$

Your students do not know how to engineer software. They have not been taught. Teach them. (Suggest you find out what their syllabus has been & what they actually learned.) You do not know how to acquire software. You have been burned. The solution is the same: software engineering. You must be the engineer of your desired software. A software client must be a software manager. You just didn't realize that you assigned yourself a software engineering project.

The Emperor's Old Clothes:

At last, there breezed into my office the most senior manager of all, a general manager of our parent company, Andrew St. Johnston. I was surprised that he had even heard of me. "You know what went wrong?" he shouted--he always shouted--"You let your programmers do things which you yourself do not understand."

The core of the solution is the same: milestones. (Which, taking a module to be a unit of work, is just information hiding yet again as the foundation of software engineering.)

$\endgroup$

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.