As a TA, the usual workflow for grading programming assignments for an introductory course would be:
- Some instructors used email as a submission mechanism (yes, really.)
- So, search for submissions in email and download every submission
- Organize it to make grading easier later on
- Go into each directory, compile it. Realize it doesn't work.
- If I email the student, I get a response 'Works on my computer'
- Debug it and make it compile on my computer
- Due to lack of test cases, every student has demonstrated their work with their own version of output on the console
- Decipher their output and hopefully they get a good grade
- Partial grading is hard because I have to dig through the code and see what parts are done and how far away they are from a working solution
All in all, it takes about 7-10 minutes per submission if I do things carefully. 100 students, 3 programming problems = 300 submissions. 10 mins per submission = 3000 minutes = 2 days of continuous grading. Impossible.
I have heard instructors make scripts, use some tools. Sometimes, they get super strict on the output of the program so they can grade it easily. Worse, they make problems easier so it's easier to grade.
How can I automate this efficiently?
EDIT: From the answers below, it seems like instructors have dabbled with online auto graders but haven't founder their final solution that solves the problem of manually grading these assignments.
@tucuxi has a really good grading setup but most of it is manual or script based. I highly suggest looking into a good auto grading tool that does all of the work in a scalable way that also benefits the students.
Having gone through this process, we created AutoGradr.com and want to share it with fellow instructors and TAs.