3
$\begingroup$

I'm trying to improve the grading process and I need to write some guidelines for the graders. Students submit Java code for the assignments. Any ideas/suggestion about how to structure the grading?

$\endgroup$
  • $\begingroup$ A course may have mixed objectives. For example, when given an assignment to sort an array, the student may struggle with the development environment, the algorithm, the syntax, the analysis of the problem, writing test data, naming variables, programming structure, proper indenting of code and of course the programming paradigm. The course must have a specific set of objectives and using these objectives the marking can be set accordingly by giving the appropriate weight to each of the above factors. $\endgroup$ – NoChance Jan 20 at 2:17
4
$\begingroup$

Since the nature of your question implies sharing personal experience rather than providing some guidelines grounded on hard evidence, I'll take the liberty to share mine.

In the past I tried to develop a fine-grained scheme for my courses, but now I think it was a mistake. Such schemes cause more troubles than they do good: they are difficult to follow for the graders, students ask why someone got 8 points instead of 9 points, and there is too much room for subjective interpretation.

After years of teaching, I realised a simple thing: we must separate educational goals from the grading process. As an educator I strive to provide the best explanations as I can and I try to give them examples of good code and show the difference between good and bad practices. However, as a grader, I only have to rank students according to their performance, and my practice shows that regardless of the scheme the same students tend to go up or down. Therefore, I try to simplify my scheme every year.

Thus, currently I tend to focus on rather short assignments, each taking at most 2-3 hours to accomplish for a skillful student. Then I grade each of them on the scale 0-3 and give the same weight for each assignment (this is of course simplification as they aren't equal, but it works reasonably well).

The subsequent logic is as follows:

  • 0 - empty submission / plagiarism / doesn't compile
  • 1 - passes some tests
  • 2 - passess most tests, but fails sometimes
  • 3 - perfect submission

Sometimes I extend it to 0-5 if I want to ask some additional questions that had to be answered within their submission and/or want to give an additional point for coding style.

$\endgroup$
  • $\begingroup$ And tests should be automated, and published so students know what they have to do, and can test their own code. $\endgroup$ – ctrl-alt-delor Jan 7 at 9:52
  • $\begingroup$ Personally, I think that if you are using an automated test suite, those test should not be published. Using a website like gradescope, you can allow students to run the tests as many times as they wish, but the risk is that they just hard-code in the solutions to the individual test cases. $\endgroup$ – Green Griffin Jan 11 at 22:27
4
$\begingroup$

This is what I have for my Java assignments. Notice that this is not a programming course, but an algorithms course.

  1. Compiling and running (8 points)

    1.a 0 points if the program doesn't compile. No points for the rest. Grading complete.

1.b If the code compiles and runs:

a   Full points if it succeeds in all test cases.
b   0.5 deducted for each failing test case.

2. Readability and submission (2 points)

2.1 Documentation (Oracle official)
    a. 0.5 points for complete commenting
    b. 0.4/0.2/0 (%) if we detect 0-25/25-50/75-100 (%) missing comments where expected.
    You should at least use clear commenting for the following:

        a. Classes and interfaces
        b. Fields
        c. Methods, including all input parameters and return values

2.2 Indentation (Oracle official)

    a. 0.5 for appropriate indentation
    b. 0.4/0.2/0 (%) for 3/6/more violations

2.3 Submission:

(0) 1 points for (not) correctly following the requirements for submission.

I would appreciate your feedback on this.

$\endgroup$
  • $\begingroup$ Welcome to Computer Science Educators and thank you for a nice anwer. Once you have enough Reputation (20pts.), why don't you stop by the sites chatroom The Classroom to discuss this, but my sense here is that this would make a good question on its own. The concept of feedback doesn't work well in comments, which are considered ephemeral here, but could make good answers to a question. Not that it is not also a good answer to this question, of course. $\endgroup$ – Gypsy Spellweaver Jan 10 at 22:12
3
$\begingroup$

When students are learning to program they should be focused on more than getting the "right" answer. If a pro is writing quick-n-dirty code that will be discarded, the answer is important. But in all other cases, a program will have a lifetime and this needs to be considered. Therefore it also needs to be taught - and learned. Focusing too much on details works against this goal.

The rubric should reward correctness, of course, but it should also reward structure. It should not reward speculative programming, however: programming that might be useful in the future but isn't now.

But, in my view, the comments that are put on the students work should be more important than the points assigned. This is how you teach individual students and get them to advance. I realize that in the modern world, if you are teaching 800 students and have a staff of 40 to help you do it, someone else has to provide those specific comments. But if you ignore that altogether you are leaving the world of teaching and entering the world of pure performance - like a play - where there are not real requirements on the "audience".

It is also difficult to justify a too specific grading rubric. If one student has earned 88 points and another has 89 or 90, can you say that there is some essential difference in their learning? Broad grading categories and the opportunity to revise work are much more valuable to the student than a strict point scale. It also lessens the need, in their minds, to argue about small points - especially when they need guidance and practice on bigger ones.

If the scale is very large, it is useful to have a meeting with the graders before grading begins, but after some of the papers have been looked at (briefly). Only then can you get an idea about what is needed. Making it perfectly objective isn't the goal. The goal is to elicit change in the brains of the students. It will help, in such situations if the TAs have a stable set of students to work with, so that they can help with the individual guidance. To do this and to assure some uniformity can be difficult, of course. It might require that grades and comments are reviewed, perhaps by another TA than the one that normally works with a given student.

Note, that I may be defining "improve the grading process" differently than you originally did. To me, things only improve if the students skills improve. Everything else should work to achieve that.

$\endgroup$
  • $\begingroup$ I can't remember the name of the study. But if you give sumative feedback (grades), then you negate formative feedback, and learning. You can measure or you can teach, you have to choose. It may be that you can separate these things. $\endgroup$ – ctrl-alt-delor Jan 7 at 9:58
1
$\begingroup$

What I'm doing is: I give points for each relevant idea / concept in the program. E.g. if you have to loop through a data structure and you have to do something with the content, I would give point for the loop construction and points for the content.

The graders can now check whether the loop is correct and if the inner content is correct and give points for this. If students use a different technique (e.g. while instead of for), they can still decide whether the loop is doing as intended and give the points.

If you are insisting in specific soliutions (e.g. you have to use a "for"-loop), you should note this. Still, it would be possible to get some points for the content.

For a simple test about python lists this could look like this:

feld = [5, 55, 555, 5555, 55555] # 1Pkt.

print("Output:")
print(feld[0]) # 0,5 Pkt.
print(feld[4]) # 0,5 Pkt.
print(feld[2]) # 0,5 Pkt.

feld[1] = 42 # 1Pkt.

for z in feld: # 1,5 Pkt
    print(z, end=", ") #0,5 Pkt. ("end" is not relevant)
print()

print(feld[:2]) # 0,5 Pkt.
print(feld[2:])     # 0,5 Pkt.
print(feld[::-1]) # 0,5 Pkt.
print(feld[::2]) # 0,5 Pkt.

summe = 0 #0,5 Pkt.
for z in feld: # 1 Pkt.
    summe += z # 1 Pkt.
print("Summe =", summe) #0,5 Pkt.

The problem becomes more difficult the larger the assignments are and the more variations you are having in the solutions. You could still try to decompose the overall problem and give points for parts of the solution.

In general, I suggest that you try to avoid to have different grades for the same (sub-)task - this ensures at least consistency of the grades.

$\endgroup$

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.