Reduce Grading & Improve Assignments

Question

I'm looking at ways to improve assignments and reduce the amount of grading. Previously, there were two in-person lab classes with 15-20 students each week. Since Covid, there is now one online lab class with 40+ students each week. This course builds off a previous course where students learn basic Java programming (if statements, while loops, I/O, methods, and objects) and lead up to a more advanced course on data structures and sorting algorithms. It's meant to teach students object-oriented programming in Java by going into concepts such as:

• Object-Oriented Design Principles
• JavaDoc
• Inheritance & Polymorphism
• Exception Handling
• GUI (Swing and JavaFx)

There are currently 6 Java programming assignments (each worth ~2.5% for a total of 15% of their final grade). In each assignment, the student is expected to build a fully functional program that basically creates objects from and saves the objects back to a database (text file, random access file, and MySQL). The last 3 assignments build off of each other (I'm not sure if this should be kept or not).

Another question was asked on how to automate grading (How can I automate the grading of programming assignments?) and the top answer suggests JUnit Testing, however, my students aren't introduced to JUnit testing until one of the last modules. I'm not sure if the Auto Grader is an option either because of our very limited budget, the size of the assignments, and pushback/criticism from other educators.

Summary

• Larger class sizes, less class time.
• Completely online instead of face to face.
• 6 large assignments with half of them building off of each other.
• Students aren't introduced to JUnit testing until the end of the course.
• Auto graders seem out of reach.

Questions

1. How can I reduce the amount of grading for the assignments?
2. How can the assignments be improved?
3. Any other feedback to improve the assessments?

Answer 1

One way that you could improve the output for the students (and thereby decrease the grading load on yourself) is to provide a sort of slapdash test-driven development.

With each assignment, provide a guiding class with a runner. In that runner, provide the tests you want the students to be able to pass, as well as intermediate tests, in order. Then comment out all of the tests.

You don't need anything as complicated as JUnit. Simple comments describing the output would be sufficient.

Instruct students to uncomment each section one at a time, and then create enough code to pass the test. If they ultimately cannot get everything working, they should submit the assignment with only as many sections uncommented as they can pass.

Explain that, other than uncommenting, they may not make any other changes to the file you provide.

This gives you complete compliance to an API and package system and a very good sense of what you're looking at as soon as you open their lab. Your grading job has now become changing a few numbers or strings around to make sure that they didn't "cheat" the output that they claim to have achieved, and then reading through the code quickly for whatever else you see.

I wouldn't personally make it a practice to hide any of the important tests, though that is your own choice.

Here is a tiny, trivial example:

    package planeLab;

    public class PlaneRunner {
        public static void main(String[] args) {
            /*
            // Should output []
            PointPlane p = new PointPlane();
            System.out.println(p);
            // */

            /*
            // Should output [(0,0)]
            p.addNode(new Point(0,0));
            System.out.println(p);
            // */

            /*
            // Should output 17.071068
            p.addNode(new Point(5,5));
            p.addNode(new Point(0,5));
            System.out.println(p.allVertices());
            // */

            /*
            // Should output 33.053326
            p.addNode(new Point(6,6));
            System.out.println(p.allVertices());
            // */
        }
    }

(Note that, along with the slapdash test-drive development approach here, there is also a slapdash way to easily uncomment sections. The // */ allows you to legally comment out the preceding /* line to pull a whole section back into live code.)

As to the labs that build on one another, you can do it, but I would strongly suggest providing answers/starting points to students whose prior labs didn't come out well. However, given the expense of providing an answer key to students, you might be better off just making the labs independent.

That said, you can always provide starter code to accomplish similar goals for the last two labs in the set.

Answer 2

Autograding doesn't have to be all-or-nothing

How can I reduce the amount of grading for the assignments?

The term "autograding" implies end-to-end work taking a zip file of submissions all the way to posting the completed grades onto your LMS via the API. I don't see the situation as all-or-nothing, and it seems like you'll only get so far without some automation.

You can create small tools that automate the repetitive stuff, like downloading and organizing submissions and compiling them so they're all ready to go for hand-grading, if you don't already.

For simple auto-grading, the only prerequisite is ensuring students adhere to input and output format guidelines, which is probably already done to help students understand the concept of delivering software that implements a specification and mitigating "works on my computer" or "guidelines weren't clear" excuses.

If the project is command-line based, it shouldn't be hard to write a simple test suite, either one that uses the classes directly or works on I/O and runs subprocesses. It doesn't have to enforce everything and can assist spot checks to ensure usage of OOP concepts.

MySQL increases the complexity a bit and is likely prohibitive to automate fully for you. However, depending on the nature of the assignment, something indirect like tearing down and respawning the solution process and wiping files between runs could validate that the student used the database without as much hassle as validating it directly. If you can establish that the correct I/O was emitted as a black box, you can grep or spot check the code by hand for the approproiate database calls.

Assignments should be designed to avoid imposing artificial constraints such as prohibiting specific common language features. For example, in a database assignment, choose a task that's difficult to do with files and you won't need to waste time validating that students used a database. Tasks well-suited to a tool motivate the concept better.

GUI programs are likely to stay hand-graded based on your circumstance, but a script that unzips and compiles all of the submissions seems like it'd give a nice boost and weed out early failures. You can also separate presentation from logic (as should be done anyway) and unit test the logic, then do a quick spot check on the GUI. I'm not sure how dominant GUI is in the projects.

Many problems with student code can be found with smoke tests. My experience is most submissions are nearly totally successful, with a minor deduction or two, or totally crash and burn, with a handful of submissions in the middle. You can use your knowledge of the nature of your assignments to guide the amount and flavor of automation to pick the low-hanging fruit.

Provide harnesses

Offering simple harnesses as suggested in other answers is a great idea (essential, I'd say), but keep in mind that students don't need to understand all provided tools, they only need to be able to operate them as users. I've taken multiple classes where I never took the time to read or understand much of the provided test suite, some of which were prohibitively complex or came in a binary.

Consider pushing JUnit forward

my students aren't introduced to JUnit testing until one of the last modules

If your curriculum already covers JUnit, you might try to work that in as early as possible, then use it as a foundation for grading and validation through the rest of the course. If you provide the harnesses and boilerplate, these suites need not be that much more complex than Ben's plain file, and may even be simpler if you can arrange the boilerplate to be compilable out of the box and avoid potential syntax errors associated with commenting and uncommenting or otherwise messing with the test suite code.

Unit testing is about as practical a skill as it gets, and it's unfortunate it wasn't a part of my undergraduate education. I only wrote unit tests after I started professional programming, and (as a client) from a coding challenge site Codewars. If you check out the 7- and 8-kyu (easiest) kata (challenges) on Codewars, I think you'll find them on level for your students, yet they are driven by (mostly) understandable, simple test suites (the ones that aren't are generally sloppily written by the kata creator).

Promote collaboration

A graduate-level programming course I was a TA for permitted students to share test harnesses and test cases on the class discussion board. This increased collaboration and happiness in the class and made it easier for students to validate their work.

Students that sought feedback on why their submissions failed were typically met with "did you run your code against the student-curated test suite?" Almost inevitably, they hadn't, and doing so led them to see their failing test cases and the power of collective effort.

This doesn't seem as feasible for CS101B-level students who aren't accustomed to whipping up tests on a dime, though, but there could be a useful variant specific to your case that provides a win-win for students while cutting back on your work.

Other suggestions/thoughts

How can the assignments be improved?

Any other feedback to improve the assessments?

It's hard to say based on a high-level description alone, so these ideas might be wildly inapplicable.

You mention "There are currently 6 Java programming assignments (each worth ~2.5% for a total of 15% of their final grade". Assuming the traditional cutoffs of 70/80/90 C/B/C, a student can get a B in the class without doing any of the programming assignments? I had this arrangement in two graduate-level classes on algorithms and data analytics which were weighted heavily on exams, but for a CS101B-level class, this is surprising. It's been a few years since my last experience with such a class so maybe I'm out of the loop. I typically expect to see projects at about 50-60% and exams 30-50% with some occasional filler for participation and quizzes at 0-10%. It could be that written homework assignments are a decent chunk of the projects portion of the pie, but that should still leave projects 30% or so.

It's interesting that you're asking for guidance with something that's a small fraction of the class yet seems to consume a disproportionate amount of your time, and presumably your students' time as well. If your students are investing as much time on these projects as you seem to be, the weight should reflect that.

You mention you have 3 assignments that build in sequence: text file to random access file to MySQL. I'd probably skip the random access file and go straight to the database. It's much more practical, and I think students might tire of iterating on the same thing too much.

Although files and databases are related in that they persist data, they're also unrelated in that they solve fundamentally different problems. I'd caution against trying too hard to create an artificial narrative as series projects sometimes do.

As much as I like databases and files, they're a bit dry, more means to and end than anything, so I try to integrate them into fun projects like games or social apps when possible, sneaking vegetables into the pizza.

In general, serial or long-term projects seem potentially divisive. I love a semester-long or serial project if it's something I'm invested in (usually because I had some control in picking it), but if I'm not, it tends to have the opposite effect. I associate long-term projects with upper-level courses where students have more agency and cohorts are smaller.

Regarding autograders, "pushback/criticism from other educators" is an interesting remark. I'd be curious to hear more. A CS101B-level Python course I took at a community college in 2016 had a homebrew service that the instructor wrote which he used to grade assignments. Students submitted their assignment code after SSHing into the college server. I suspect the resistance and frustration expressed by some students was due to low technical sophistication, but also feeling like it was an alienating, robotic and inhumane way to run a class (oh, the irony) -- the professor didn't even look at my code before flunking it! One student called the instructor something to the effect of a pretentious geek before dropping the class.

In hindsight, the professor hadn't done anything particularly shocking to an experienced programmer, and making students use SSH and the command line should have been seen by them as part of the programming knowledge skillset they were there to learn, so maybe the problem was creating the correct narrative behind the tool. I can see fellow educators being resistant to tooling as well for various reasons, but I also suspect we're all more comfortable with automation in education than we were in 2016, for better or worse.

Answer 3

Here is a simple "runner" type of program that has the students practice recursion. https://www.cs.colostate.edu/~fsieker/TestSemester/assignments/RECURSION/RECURSION.html. The assignment consists of four Java files, only one of which is modified by the student. The four are:

1. Shell.java - this is a shell-like program that takes a command from the keyboard, calls one method, and prints the result. The Shell program provides utilities to make it easy to enter arrays (String/int/double via CSVs) and easily produce empty and null values. The idea is to provide unit testing without JUnit. The actual tests can be entered via the keyboard or read from a text file. Processing commands from a file is the key to auto testing.
2. Recursion.java is an abstract class that defines the signatures of the methods the students must implement. It also contains the processing for the commands of this assignment. It implements a help command to print a list of the commands and how to use each one.
3. MyRecursion.java which is the starter file for the students to work on. They may use an IDE to generate stubs for each method. The student then has a fully functional program, albeit one that does not produce correct results. Thus, students can write and test one method at a time without having to comment out or uncomment code.
4. SampleFile.java was a sample code to demonstrate how to use the Java File class. It is not actually part of the assignment but provided as an example.

The pattern of using Shell.java, XXX.java, and MyXXX.java was reused over many assignments, including those that build on previous assignments. However, it is not suitable for anything involving a GUI. A sample test file with answers was provided to the students to help with debugging the assignment. Testing required a separate text file with different test values.

If you find this useful, let me know and I can provide additional information.