How to write a grading rubric for coding assignments?

Question

I'm trying to improve the grading process and I need to write some guidelines for the graders. Students submit Java code for the assignments. Any ideas/suggestion about how to structure the grading?

Answer 1

This is what I have for my Java assignments. Notice that this is not a programming course, but an algorithms course.

1. Compiling and running (8 points)

   1.a 0 points if the program doesn't compile. No points for the rest. Grading complete.

1.b If the code compiles and runs:

a   Full points if it succeeds in all test cases.
b   0.5 deducted for each failing test case.

2. Readability and submission (2 points)

2.1 Documentation (Oracle official)
    a. 0.5 points for complete commenting
    b. 0.4/0.2/0 (%) if we detect 0-25/25-50/75-100 (%) missing comments where expected.
    You should at least use clear commenting for the following:

        a. Classes and interfaces
        b. Fields
        c. Methods, including all input parameters and return values

2.2 Indentation (Oracle official)

    a. 0.5 for appropriate indentation
    b. 0.4/0.2/0 (%) for 3/6/more violations

2.3 Submission:

(0) 1 points for (not) correctly following the requirements for submission.

I would appreciate your feedback on this.

Answer 2

Since the nature of your question implies sharing personal experience rather than providing some guidelines grounded on hard evidence, I'll take the liberty to share mine.

In the past I tried to develop a fine-grained scheme for my courses, but now I think it was a mistake. Such schemes cause more troubles than they do good: they are difficult to follow for the graders, students ask why someone got 8 points instead of 9 points, and there is too much room for subjective interpretation.

After years of teaching, I realised a simple thing: we must separate educational goals from the grading process. As an educator I strive to provide the best explanations as I can and I try to give them examples of good code and show the difference between good and bad practices. However, as a grader, I only have to rank students according to their performance, and my practice shows that regardless of the scheme the same students tend to go up or down. Therefore, I try to simplify my scheme every year.

Thus, currently I tend to focus on rather short assignments, each taking at most 2-3 hours to accomplish for a skillful student. Then I grade each of them on the scale 0-3 and give the same weight for each assignment (this is of course simplification as they aren't equal, but it works reasonably well).

The subsequent logic is as follows:

• 0 - empty submission / plagiarism / doesn't compile
• 1 - passes some tests
• 2 - passess most tests, but fails sometimes
• 3 - perfect submission

Sometimes I extend it to 0-5 if I want to ask some additional questions that had to be answered within their submission and/or want to give an additional point for coding style.

Answer 3

When students are learning to program they should be focused on more than getting the "right" answer. If a pro is writing quick-n-dirty code that will be discarded, the answer is important. But in all other cases, a program will have a lifetime and this needs to be considered. Therefore it also needs to be taught - and learned. Focusing too much on details works against this goal.

The rubric should reward correctness, of course, but it should also reward structure. It should not reward speculative programming, however: programming that might be useful in the future but isn't now.

But, in my view, the comments that are put on the students work should be more important than the points assigned. This is how you teach individual students and get them to advance. I realize that in the modern world, if you are teaching 800 students and have a staff of 40 to help you do it, someone else has to provide those specific comments. But if you ignore that altogether you are leaving the world of teaching and entering the world of pure performance - like a play - where there are not real requirements on the "audience".

It is also difficult to justify a too specific grading rubric. If one student has earned 88 points and another has 89 or 90, can you say that there is some essential difference in their learning? Broad grading categories and the opportunity to revise work are much more valuable to the student than a strict point scale. It also lessens the need, in their minds, to argue about small points - especially when they need guidance and practice on bigger ones.

If the scale is very large, it is useful to have a meeting with the graders before grading begins, but after some of the papers have been looked at (briefly). Only then can you get an idea about what is needed. Making it perfectly objective isn't the goal. The goal is to elicit change in the brains of the students. It will help, in such situations if the TAs have a stable set of students to work with, so that they can help with the individual guidance. To do this and to assure some uniformity can be difficult, of course. It might require that grades and comments are reviewed, perhaps by another TA than the one that normally works with a given student.

Note, that I may be defining "improve the grading process" differently than you originally did. To me, things only improve if the students skills improve. Everything else should work to achieve that.

Answer 4

What I'm doing is: I give points for each relevant idea / concept in the program. E.g. if you have to loop through a data structure and you have to do something with the content, I would give point for the loop construction and points for the content.

The graders can now check whether the loop is correct and if the inner content is correct and give points for this. If students use a different technique (e.g. while instead of for), they can still decide whether the loop is doing as intended and give the points.

If you are insisting in specific soliutions (e.g. you have to use a "for"-loop), you should note this. Still, it would be possible to get some points for the content.

For a simple test about python lists this could look like this:

feld = [5, 55, 555, 5555, 55555] # 1Pkt.

print("Output:")
print(feld[0]) # 0,5 Pkt.
print(feld[4]) # 0,5 Pkt.
print(feld[2]) # 0,5 Pkt.

feld[1] = 42 # 1Pkt.

for z in feld: # 1,5 Pkt
    print(z, end=", ") #0,5 Pkt. ("end" is not relevant)
print()

print(feld[:2]) # 0,5 Pkt.
print(feld[2:])     # 0,5 Pkt.
print(feld[::-1]) # 0,5 Pkt.
print(feld[::2]) # 0,5 Pkt.

summe = 0 #0,5 Pkt.
for z in feld: # 1 Pkt.
    summe += z # 1 Pkt.
print("Summe =", summe) #0,5 Pkt.

The problem becomes more difficult the larger the assignments are and the more variations you are having in the solutions. You could still try to decompose the overall problem and give points for parts of the solution.

In general, I suggest that you try to avoid to have different grades for the same (sub-)task - this ensures at least consistency of the grades.

Answer 5

Grading coding assignments should consist of the following parts

• automated tests
• plagiarism checks
• code quality (lint checks)
• a short manual review of each submission

If the tests are written properly, I would imagine that anyone who passes the automated tests, plagiarism checks, and code quality checks should qualify for at least a B (or the equivalent in your course). On top of that, the manual review of the code determines whether the student used the correct approach and whether the submission matches expectation. This is where the student can reach A or A+.

There are many tools that can help you streamline this process. I recommend checking AutoGradr.com (disclaimer: I'm one of the authors)