14
$\begingroup$

Imagine students in a course on algorithms & data structures, which are given homework assignments to implement things like trees, hash sets, sorting algorithms, etc.

Due to the high number of students per semester and limited resources, grading is heavily assisted by automatic unit tests. The TAs will still scan through the code, but if all or most of the test cases fail and there is no simple fix in sight, students will get 0 points on an assignment.

Unfortunately, this can happen very quickly. Consider implementing a balanced tree as an example. If students fail to correctly reference the nodes, insertion, removal and searching, - despite being implemented correctly if seen individually - will not work and can hardly be tested.

As expected, some students do complain about their poor grade and argue that "although they acknowledge that their data structure does not work entirely, they at least implemented various sub-tasks correctly". This may even be correct, but the TAs very often do not have the time to dig deep in the code and grade the internal functions of the students implementation. Especially for more complex data structures, it would also take too much time to fully comprehend and even debug the (erroneous) implementations.


My question is, if it is okay to grade programming assignments based on this external view (does the implementation pass the tests?) or should grading rather be based on an internal view (are the things implemented by the student correct if seen individually?).

One important remark: In this specific scenario, students were not given the unit tests, since they are expected to test their implementation themselves.

$\endgroup$
7
  • 2
    $\begingroup$ Students tend to complain a lot less when expectations and grading standards are shared explicitly in advance of the assignment due date. What are you telling your students about how their work will be graded? $\endgroup$
    – ff524
    Dec 11, 2018 at 21:45
  • 1
    $\begingroup$ Are they provided with an environment where they can run and test their code? $\endgroup$ Dec 11, 2018 at 21:46
  • $\begingroup$ @ff524 In this situation, students are aware of the test cases. The actual grading follows a rough catalogue but is mostly up to the TAs. One more thing for clarification: I am a student representative in this case and generally thinking about an opinion on that matter. $\endgroup$
    – blu3r4y
    Dec 11, 2018 at 22:00
  • $\begingroup$ @Acccumulation Yes, the students know about testing from previous courses and could easily write some along with the actual implementation. $\endgroup$
    – blu3r4y
    Dec 11, 2018 at 22:02
  • 1
    $\begingroup$ Assignments should be graded based on the educational goals of the course. I can see value in courses that teach writing code to fit a given specification, and different value in courses that teach implementing specific algorithms and data structures. The same code submission should be graded differently in those two classes. $\endgroup$
    – JeffE
    Dec 12, 2018 at 9:52

4 Answers 4

7
$\begingroup$

Actually, life isn't so simple as to choose one or the other here. Code needs to be well structured and it needs to be correct. Given that students are learning, you can't really expect that they will be perfect on either of these and so you need to take both in to account. A program could, be well designed and built but have some, as yet undiscovered, flaw that prevents it from passing its tests. If the students hack it up to get tests to pass they are going backwards, not forwards.

CS students have a lot to learn. Programming is important, but not the most important thing. But to advance on many of the other goals of the CS curriculum, students need to build good code. Even better, would be the ability to build maintainable code, since many of them will start their careers in programming-heavy jobs. Even if they go to grad school, much of what they do will depend on being able to write good code.

So, in grading, points should be gained or lost depending on both code and correctness. If the code is not correct, then someone needs to make an analysis about why the code is failing. It could be something simple or profound.

In my view if a program can't be understood easily and/or seems to be hacked together, it should receive few points. If a program, on the other hand, passes few tests then it is a much more subtle problem that needs deeper analysis. Don't neglect it.

In the question heading you talk about the "correct implementation". I worry about that. Students should not be graded on similarity to any reference implementation, certainly. There are more than a few ways to write a correct program.

$\endgroup$
3
$\begingroup$

I agree with Buffy, the question would be perfect for cseducators SE. But still, I try to give an answer which might be generalized to other disciplines as well:

Of course, this is absolutely ok - but it affects your teaching style and goal.

If you want the students to be 100% perfect, this is the way to go. It's a hard lesson, but in real world scenarios they are facing more or less the same situation. My Physcis teacher at school took the same attempt and argued, that it does not make a difference if you want to drive to school and crash into the tree next to school or next to your house - you don't make it!

I fully understand the burden of reading students code - I do the same and it can be a great pain. But if you do it, you can give individual feedback and you'll understand, which are the most common mistatkes and adress them in your course.

One compromise could be to offer different test sets, let's say some tests for each function needed for the complete task. That would allow to give points partially, but still you can not give points for a correct recursion, loop or whatever.

An additional thought: You might consider giving a few test cases so the students can at least test if their code understands the input and delivers the output in a format your test understands. I don't know if this is a real world problem with your students, but it would be an issue with mine ;-).

$\endgroup$
2
$\begingroup$

The 'external view' is important as it ensures the submission is correct. These tests should be similar to Acceptance Testing since you are checking whether the submission meets the 'business' requirements i.e. the requirements laid out in your question. I highly recommend automating this process with unit tests or console based tests (check output matches the expected output).

The 'internal view' is what motivates your students to do better since only a person can judge this properly. Students are not motivated to write good code when they are only being judged by automated tests. I recommend looking over each student's submission and ensuring that they have followed the correct approach have met proper code quality standards.

Of course, you can't give a lot of time to looking over each submission UNLESS the 'external view' tests are automated and you don't have to download and run the code for the 'internal view' reviews.

I suggest automating your process using AutoGradr (disclaimer: I am the author of this free tool)

$\endgroup$
-2
$\begingroup$

Unless the objective of the exercise is to copy an implementation (say from Java to C++), the "similarity" to any other implementation should have no weight in the grading. At least not in a positive sense. The similarity to another implementation could, however, be a warning sign for plagiarism. Everything else in the question is up for debate, so to speak.

The "fairness" of the grading only hangs on two principles: Consistent application of standards and advance notice of those standards. If the same standard, or evaluation, is applied to all the work received, and the assigned grade would be the same regardless of which student turned it in, or which TA assigned the grade, then the standards are applied consistently. Prior to the examination, or assignment, the students need to be informed as to how their work will be evaluated. In the course syllabus, or in the beginning of the course is the best time to inform them, of course. If they have been given such a notice, and then their work is evaluated according to that guide, there should be nothing to object to - from a "fairness" point of view. Knowing that there will be unit tests, even without knowing what those might be, is still acceptable, so long as the same unit tests are applied in a uniform manner to all the work evaluated.

How "fair" the evaluations are, however, likely has very small significance to how well the evaluations move the students toward the course objectives. Likewise, how such projects would be evaluated in a real-world workplace environment likely does not move them in the direction of meeting the course objectives. Of course, if the class is the final course in a series, the objective of which is to produce work-ready employees, then a purely external view of works vs. fails might be appropriate. In all other cases, however, such an extreme approach is not a very pedagogical approach. (As a side note, a binary go/no-go grade does not fit well into a scaled grading system either.)

In an idealized world, the student projects are a chance for the instructor to determine how well the student has apprehended the course materials thus far, discover where the student has stumbled, and take corrective measures to return the student to where they should be relative to the course materials. Teachers, of every stripe, are an over-worked, under-paid, unappreciated lot. (The TA's get to share in that "glory", and will become just as over-worked as the instructors.) As such many of the opportunities to learn about the difficulties and misunderstandings the students have will be missed when grading assignments. Having a rubric, published by preference, which the TA can apply to the evaluation often helps.

The course objectives, naturally, have greater weight in the rubric than other criteria. Yet non-course objectives should be included. While seldom included in the course specifications, clean code, naming conventions, documentation or self-documenting code, and adherence to coding style can all be covered by the rubric, and encouraged, thereby, in the students' work. Requiring adherence to such guidelines benefits the instructor (and TA's) in applying fine-grained grading evaluations. It can also aid the students in completing the projects, obtaining better results from the code, ease the burden of debugging the code prior to submission, and developing good habits for when they do enter the workforce, where go/no-go evaluations really matter. (I prefer to give more points to a clean almost-working program, than to a impenetrable kludge that passes every known test. The former can be maintained, including fixing the current version. The latter can probably only be maintained after being re-coded in some understandable manner.)

Unfortunately, my view is only one way of seeing the job of "instructor." As to it is okay to grade programming assignments based on this external view, the answer is; "While it is less than ideal, yes, it is okay to do so."

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.