# If a program does not compile, should it get a mark of zero?

Following a discussion on the Academia SE, I would like to seek views here among Computer Science Educators on whether a mark of zero should be awarded if a student submits a program that does not compile for an undergraduate programming assignment. I will add an answer giving my view on this, but only later on so as not to bias the discussion. I'd be very grateful for your views.

• +1 I was going to ask after the same thing. Although I would have phrased the question: Is it reasonable to award zero for non-compilation? May 12 at 19:52
• Might help to edit the link to the Academia question in here. Most of the comment-discussion has now been moved to chat. General consensus in comments & answers is that should be a base expectation: academia.stackexchange.com/questions/185044/… May 12 at 20:11
• I had this exact constraint imposed in college back in the early 90s (I'm old). Prof was very clear on grading criteria for the whole class. Last project of the year "final" project grade - if I got 100%, or 0% I'd still get a "B" in the class (quirk of scoring and my previous tests / projects). I submitted a file entirely of comments, explaining what I would do, in pseudo code, if I had time, but had to study for other things. Made sure the entry/exit points were correct so it cleanly compiled, but did nothing when run. Got 15 / 20 points for demonstrating understanding. May 12 at 21:45
• A program that doesn't compile on the teacher's machine might well have compiled and ran fine on the student's machine. Sometimes a slightly different version of the exact same framework, or just what libraries are installed (and what versions they have), or what OS they are running, might make a difference. The only reason I didn't turn this into an answer because I don't know how well this possibility has been taken into account, as the question is quite short on details.
– vsz
May 13 at 17:20
• @vsz I give them a server to use that compiles and tests their program and formats it all as a .pdf file containing the code (formatted and the files in the right order), compiler messages, console output, output files etc. That means they have exactly the same target environment that the marker does. It saves a lot of marking time, and eliminates a lot of these sorts of difficulties. Lack of detail was intentional - the resulting discussion has been very informative - many thanks to all contributors - much appreciated! May 13 at 22:06

This answer assumes that the course's goal is to teach programming (as opposed to teaching advanced algorithms or numerics etc. to which the programming would only be an end). In that case I'd hold: It is a hard requirement that the program builds.

It's not so much that the submission would be graded with zero points; it's rather that it will be rejected because it is not gradable.

This should, of course, be announced beforehand, together with a defined development environment and, if possible, a "grading machine" that can be used to check the program before submission. Ideally, there would be an automated submit/build mechanism as suggested in other answers.

Rationale:

• Programming is the topic here. A program that doesn't build doesn't meet the minimal requirements for submission. It's like asking to grade an architect's house which collapses. "But it has a beautiful stairwell, if you'd only care to dig through the rubble!" No thanks.
• The teacher's time is limited. Grading by reading would take too long.
• The "must build" criteria is self-understood in the real world. It is good to get used to it. The customer won't pay you for a system that doesn't run, and the professor won't grade you.
• The programming environments for teaching programming are typically plain vanilla installations of Linux/gcc or the like which are easy to reproduce and are fairly compatible on the source code level.

Note because of ongoing discussion: No rule is absolute, nor should it be. This answer concerns programs that are submitted in a flawed state, not weird, unusual occurrences.

• For me the points here that mattered were: 1) students are clearly told that programs must compile and 2) it's not some "in the real world..." thing -- it's that fixing compile errors then rerunning the grading script is a huge pain. Of course if programs are hand-graded this wouldn't apply (but in practice I've never seen a non-compiling program that was worth more than a few points, once made to run). May 12 at 22:56
• This is reasonable, but only, ONLY, if the student has access to a standardized "grading machine". As Ben I. points out, builds fail for all sorts of reasons, and the student need to be given feed backback on that before the deadline runs out. May 13 at 7:21
• What do you mean by "it is not gradable". When I get a submission regardless of how it compiles, there are a number of things I should evaluate. e.g. "did they document it?" "is the structure reasonable?" "is the formatting and variable naming good?". It's not like the only thing determining a mark is the output and the resource use. May 13 at 14:28
• @Peter-ReinstateMonica not really, I often meet programs that I need to do my work that don't compile. Then how well documented they are, how nicely formatted and how logical the structure is matter rather a lot. Those things change it from a program that might take days to salvage, to something that I can recover in a few hours. So these things are actually even more important in a non-compiling program than in a program that does compile. May 13 at 14:49
• “It's like asking to grade an architect's house which collapses“ – which architecture class requires the students to build actual houses? And why should a house with a perfect first floor but a collapsed roof not give at least some points? “Grading by reading would take too long” – this is a poor excuse, no other field would even think of grading submissions without reading them. May 14 at 7:33

Of course not, and I have trouble believing that anyone would truly entertain the other position in any serious way. The closest analog in another field would be if you were to present an essay in a history class, and the professor reserved a technical score of zero if they found any grammatical errors.

Let's clear something up: a compiler error is an indicator of a problem in an assignment, and might well mean that the assignment is terrible. But it's only a clue, not a given. A compiler error in and of itself is only a clue, and there are enough reasons why something might not compile that aren't terribly concerning that the answer to "doesn't compile = automatic zero" should simply not be true.

Let's break that down a little. There are so many reasons something might not compile that are not under the students control, but could simply be environment-specific. I can't tell you the number of times a package statement at the top of a submitted Java file does not reflect my own directory structure. It compiled fine on the student's computer, and they were unaware of the name of the directory I'd be using to check their work. The appropriate deduction is no deduction.

We have students do large, group projects at the end of the year, and these get presented to the entire department in a CS Fair. I've had students submit android applications, iphone apps, web apps, unity projects, and do, so many other things. I quickly realized that I had to see the stuff working on their computer, not on my own. I would never even attempt to set up all of the environments I'd need to try it.

Then there are small accidents. A student recently submitted a Java file to me where everything worked, except for an added v at the beginning of the file, rendering the first command as vimport. Her code was otherwise excellent.

When I asked the student, she explained to me that she had opened the file up to make sure that she was submitting the correct version, and must have accidentally pressed a key prior to closing it. Her IDE auto-saves, so there was no Save prompt to warn her of the change Any of these is sufficient to make the answer to the question (as posed) an emphatic no.

Furthermore, students regularly submit work that works, but is terribly made. Once again, the compiler does not tell us anything relevant to grading.

The compiler can be a good first check of the work, and you might decide to deduct 5 points (instead of the zero I believe is proper) simply because it does not compile, but consider my poor v student, and you can immediately see why more than that would be unfair.

Philosophically, we are there to check student understanding and work, and as the prior to examples illustrate, compiling is a poor proxy for these. It is a clue, but nothing more.

If it doesn't compile, that suggests that there is possibly a serious problem, but that's all it can tell us. For the rest, we actually have to take the time to read through the submitted work, just like every other professor in every other field.

• Comments are not for extended discussion; this conversation has been moved to chat.
– Ben I.
May 17 at 14:15

Following further with @Ben I.'s answer, I think the real problem is often not giving feedback about whether the program builds to the student in an inspectable and rapid manner, while also allowing multiple submission attempts. (in a worst case, for example, an email attachment that comes back with a grade after the due date)

Instead, have a workflow where they upload a specific format to a build server, or better, submit via version-control.

Then their program must build there and the system must give back meaningful feedback (such as compiler/interpreter warnings and full or reduced test case results), and then they can choose a successful build as their submission.

Simple Workflow

• student creates work submission
• (after due date) final grade results come in (no opportunity to try teacher's environment unless they crashed office hours)

Better Workflow

• student creates work submission
• remotely build program -> feedback to student
• remotely test program -> feedback to student
• student has opportunity to submit assignment for grade
• (after due date) final grade results come in (perhaps run against a much larger test suite with those results)

This doesn't necessarily mean that you should or should not assign zero points to a submission that fails to build, but gives the option of letting the first few or a subset be acceptable! .. Rather, collecting the submissions will help make your/the department's build system more robust and also give a good understanding to yourself (and the class!) about what could be done to help them structure and format programs to build!

• Generally a good answer, except that version control capable code repositories (GIT, etc) are notoriously difficult for beginners to understand. May 12 at 19:27
• Or have an online grading tool - SPOJ was initially written by the TAs at the university I attended, and was used in grading assignments for basic coding and algorithms class. Sadly, this only works for programs which can be tested automatically. May 12 at 20:39
• Approaching this as a professional developer (rather than as an educator,) I'd give this another +1 just because of the inherent value of teaching students about version control and build servers from the beginning. Sure, some of its concepts can be a bit difficult to understand at first, but it will make their life much, much easier both during and after school once they've learned those concepts and learned to put everything in version control. And having a build server system where submitted revisions are automatically built with compiler feedback given is even better! May 12 at 22:14
• @Buffy But kids these days are often learning in Python and other magical import antigravity languages, so they should have plenty of brain power left over now for learning about version control instead. If people could learn FORTRAN and COBOL when they had to submit a program to run overnight on the mainframe, at least Git you will find out much faster that you screwed it up. May 13 at 0:35
• Honestly, I think this finds a lot of "computer science" woes and the subject of many Questions - where does one draw lines between CS being applied math theory, software design, real-world/legacy spelunking, and the history of assistive tools?Should new students be immediately ready to labour in the infinite Java mines or produce new works? In my experience, non-CS and largely math-focused CS students tend to understand and produce better programs. Did CS lose others? What should undergrads be taught first to improve later studies? Just lambda calculus? "normal" calculus? Java?(ever?) C? git?
– ti7
May 13 at 14:58

I think giving a zero for a non-compiling program can be reasonable and expected behavior in many circumstances. It's pretty close to what I do now (more below).

# The Original Academia SE Question

Note that the consensus in the motivating Academia SE question seems to mostly be that this would be the base expectation (mostly now moved from comments to chat: https://academia.stackexchange.com/questions/185044/grading-on-a-curve-in-required-doctoral-course-failing-as-a-result).

A few selections from different users in that chat:

Drake P: ... for the future, you should know that it's very standard to give 0s for code that doesn't compile, even at the undergrad level, as it makes grading ungodly for teachers otherwise...

ilkkachu: what's so tough here? If your code doesn't compile, it doesn't work. Nothing gets done. Expecting to get a good grade for code that doesn't work sounds like expecting to get a good grade for presenting e.g. mathematical proofs or other logical conclusions that are totally false...

RBarryYou: Technically, a "program that doesn't compile", isn't actually a program. And given how easy it is to check that it does actually compile, this seems like an obvious requirement for all but the most introductory CS and programming classes.

forest: It's not like you don't know if the code compiles until after you submit it. If the code doesn't compile, then fix it first.

# My Experience and Practice

I teach the first two semesters of programming with C++ in a two-year CS degree at an open-admissions community college in the northeast U.S., with generally very weak students. The assignment submissions I get are pretty frequently massively scrambled up, nonsensical, often show no understanding of the assignment, and no evidence of testing. There is no lower bound to the quality. So taking a random file and laboring on my end to make sense of it or fix it can take an unbounded amount of time and effort (which I don't have). Therefore it seems reasonable to use compilation as a first-pass check to see if the student has done the basics of making something that satisfies the definition of a program under our language rules. No, I'm not going to try and make sense or fix student work otherwise. Or, more accurately: I've tried that in the past and it's impossible in most circumstances where the code doesn't at least compile.

Now, does a non-compiling assignment get an immediate zero from me? Actually, no. It does get a zero for the correctness & user interface components, which account for 70% of the grade rubric. I still almost always award points for readability & documentation components, which are worth 30%. However -- the only reason I'm willing to do that is that I've written a custom style checker which automates that part of the process for the things I care about (and we've specified in class). If I didn't have that tool available, then it's pretty likely I'd give a zero for the whole thing, and not spend time and headache trying to make sense of a non-program.

# Justifying the Grade to Stakeholders

In addition to the preceding, there's the issue of justifying the grade being given. If I just mark a zero or low grade for something, a student (rightfully) will ask, "Why did I get a low grade?", and I need to have a clear response. I even need to think about the student challenging the grade I give at the department or college administration level. By being able to point to the clear and objective evidence of the compiler's failure on the student's submission, I have the strongest possible justification for the lowest-level grade being given.

We've made the point that non-compiling programs are most often garbage nonsense. In other answers and comments we also see some sentiments like this:

jamesdlin: Then your policy should be to give 0% for submitting nonsense, not for submitting code with a compilation error.

But "this is nonsense" won't be a comprehensible or satisfying response to the student, nor defensible at the administrative level if challenged. Pointing to the objective non-compiling issue is greatly clarifying for all involved. (In fact, if I didn't keenly highlight the compiling issue as a top priority, some of my students would likely not grok its importance at all, and never prioritize it as something they need to try and make happen.) Otherwise, finding ways to document and defend an evaluation of "this is nonsense" will take an egregious amount of time. This is analogous to the issue of "grading for completion" on elementary mathematics homework, for example, which I've also found to be unworkable in my context and experience.

I think some of the answers in the "do not give zero" camp interpret the question too narrowly to make a point. My interpretation is that we're looking for a best-practice that's profitable in a general case. I wouldn't argue for entirely mindlessly giving zeroes. For instance, I would assume that most of Ben's examples have been double-checked and taken off the table as possibilities. For my courses, I specify the standard environment and IDE on the syllabus. It's the same thing used in lectures, labs, and on my test machine for grading. If a submission doesn't compile, I double-check, and confirm it's due to something in violation of our protocols.

I think it's interesting that several of the "do not give zero" advocates, in the comments, have indicated that they are forced to deal with some kind of strange, out-of-control build environments. For example, the OP writes (in a comment under gnasher729's answer):

an issue that may prevent compilation is that my institution requires an LTS (Long Term Support) version of Java and we were using LTS11 (which is a bit long in the tooth) as LTS17 arrived just too late to be installed before the start of the academic year. Some students installed a more recent release (my instructions showed how to install LTS11 for compatibility) and their IDE prompted them to use more recent language features (switch) and their code doesn't compile despite being legal. Similar story for C++17 and C++20.

... a student installs the most recent toolchain rather than the one I tell them to (back in October) and has forgotten months later when the coursework is due (it is a full year course)... It is perfectly reasonable for a good student to have a more up to date toolchain for work on other projects (best if students enjoy hobby programming - it is something I encourage).

I, for instance, am required to take submissions through a system called "Schoology", which requires students to compress java files prior to submission (dont ask, it's really a problem).

Okay, so if you're teaching with an out-of-control build environment and routinely get "legal but doesn't compile" submissions, then I guess you have to deal with that. I'll say that in my situation the idea of students maintaining multiple build environments and toolsets is completely unimaginable and not something I've ever heard of, or that my students could conceivably be expected to manage. I feel highly responsible (and take quite a bit of time) in trying to ensure that there's one consistent environment used in the labs, lectures, my grading suite, etc., and oversee practice with it every week, so that students can develop confidence in their on-ramp. In 20 years I've simply never gotten an "oops I wrote this correctly for a different environment" submission. It's good for all of us to bear in mind the "Academia varies more than you think it does" principle. But my advice would be that if you are in that situation, then get your build environment under control.

As an added data point that such a practice is fairly common, consider the fairly widespread practice of using software autograder tools for programming assignments. E.g., one suggested at my school, used widely elsewhere at CUNY, is Gradescope (there are many others going back more than a decade now). The essence of a tool like this is that an automated suite compiles, throws certain inputs, and matches against expected outputs. There is, e.g., no facility for interface, readability, or documentation point awards. In short, there is no way for a tool like this to award any positive points for a program that fails to compile.

Is it reasonable to argue against using a tool like that? Sure, and I did so, and we declined to use it in our department. But that's because I wanted to give more feedback and points for styling issues with first-time programmers. From what I'm told of the widespread usage of this tool elsewhere in our system, that must be fairly rare. But is it completely unimaginable to use an autograder, in light of their giving zero to anything that doesn't compile? It seems hard to make sense of that, given their widespread usage.

# Conclusion

So: I do give zeroes for non-compilation, at least for a majority component of an assignment's grade. That's after double-checking for unforeseen causes that I failed to define properly for students -- which in practice hasn't happened in some years. Other credit is available for styling & documentation. But I've found it generally impossible to "fix" submissions from my students, and I use compiling as a first-pass to see whether something which counts as a program has been submitted.

On the other hand, evidence is that it's probably pretty widespread for other CS instructors to not go to the trouble that I do, and likely have a process that gives a zero by default for non-programs (e.g., with a standard autograder tool). If I attended someone else's class myself, that would be my default expectation.

• Comments are not for extended discussion; this conversation has been moved to chat.
– Ben I.
May 17 at 2:19
• @DanielR.Collins Your commentary on my comments in your answer is a gross and insulting misrepresentation of what was actually said. (i) we do not have to deal with " some kind of strange, out-of-control build environments" it is a long term support (LTS) release of the Java Development Kit. It is a very sensible policy not to use a bleeding edge release that is not guaranteed to be supported throughout the academic year. May 17 at 13:05
• (ii) "routinely get "legal but doesn't compile" submissions, " we don't "routinely" get submissions that don't compile. We get far fewer than we did before we implemented the preparation system, indeed that is exactly WHY we introduced it. (iii) " I'll say that in my situation the idea of students maintaining multiple build environments and toolsets is completely unimaginable " it is a shame, the students that do this tend to be very good and very keen students. IDEs are designed to allow that sort of thing. May 17 at 13:05
• The students are not required to have multiple toolchains, they choose to do so. (iv) " I feel highly responsible ..." implying that I don't? "... in trying to ensure that there's one consistent environment used in the labs, lectures, my grading suite, " oddly enough, that is exactly what the submission preparation tool does. It ensures that the build environment used is exactly the same as the one in the lab, the same one that I use in the lectures and the same one that is used by the markers, and more importantly, the student can see any compiler messages... May 17 at 13:06
• that the marker would see. It is also the same environment they use on their computers if they follow the instructions on installing it that I give them (video). It couldn't be more consistent. " (and take quite a bit of time)" oddly enough, a lot of time, thought and effort went into our system and it was well worth the reduction in submissions problems that it produced. May 17 at 13:06

### I enforced this exact policy ("doesn't compile = failure")

For several years I was the assessor on a postgraduate course on C++ (Financial computing with C++) given at the mathematics department, and I enforced this exact policy. If a student submitted work to me that didn't compile, it would get an immediate "D" grade (the lowest score on the scale, corresponding to 0%). Typically about 1-2 students a week would get stung by this for the first few weeks of the course. No exceptions, zero sympathy, without remorse, etc. Of course when students couldn't achieve code that compiled, it would be extremely frustrating for them to be in this scenario, but there was good reason for enforcing this, which I'll explain. Furthermore, I was also a student on this same course, so can understand the exact same pressure to get code past the compilation milestone, by any means necessary, even if the code becomes hacky.

### Why enforce such a strict policy

If the goal of the course is to teach numerical algorithms, computational linear algebra, or advanced scientific computing, or anything similar where the aim of the course is to teach the core science, then it's very rare to see anything use a compiled language, and I would be more inclined to disagree with such a policy. However, when courses are aiming to teach high performance computing languages and software engineering as its primary skill, compilation is essential. There are a few reasons for this:

• Code that doesn't compile would be rejected without question in any commercial or professional setting, without a moment's hesitation, as other answers have highlighted.
• A program that doesn't compile isn't really a program.
• Getting code to compile is a key skill, and crucially it is something the students are able to iterate on themselves before it needs to be submitted. That the code doesn't compile shouldn't surprise them.

### Is it just compilation?

Having code which compiles is the absolute baseline. However, it is not the last hurdle. If the program always encounters a runtime error straight after launch or gets caught in an infinite loop, these are viewed with equal frustration from the assessor, and if the code always seg faulted immediately, it would also fail for similar reasons. However, the compiler is usually your friend giving tips and suggestions about whats it's complaining about.

### Can the students be safeguarded?

It will vary from course to course, but naturally it is in everyone's interest if the code compiles. For our particular setting, we went to great lengths to set all the students up with a system agnostic build environment, used the same compiler, and shipped all the external libraries they would need with the projects. We also did a 2 day setup to ensure everyone could build and run a hello world, the homework assignments, and the exam assignments.

At the start of the course, I would tell all the students directly that failure to compile would result in a fail grade (for all the reasons stated above), and set out similar grading criteria.

### The myth that this is easier for the assessor

There seems to be the idea/myth that such a blanket policy is an easy option for the assessor. As someone who marked 30 or more code submissions each week, this is absolutely wrong. When a code came along that did not compile, while it got a failure immediately, the marking was not done there. I then had to go into the source code, set it up, compile, and then debug the compilation problems, so I could explain to the students what they needed to change and explain the compiler warnings/errors. Code that compiled but was wrong is easier to debug, but code that doesn't compile is a nightmare. Certainly it takes 10x longer than other mistakes that did compile, and it certainly was a lose-lose situation for the student and the assessor.

# Corner cases

While I agree there is scope for arguably valid corner cases, I have never seen one in the wild after having been marking these examples for several years, so think this is a null argument.

• " However, when courses are aiming to teach high performance computing languages " for an advanced level course, it makes more sense. I was mostly thinking of introductory courses. I wouldn't expect to see non-compiling code on an advanced level course. May 13 at 8:31
• I’ve rarely seen an intro level course for an academic subject done in anything other than: python, R, matlab, etc. all interpreted. It’s either advanced programming or advanced theory, rarely both at the same time. (Ironically my intro to programming for physics as an undergrad was in C - so there are some exceptions, but it was mostly for the programming). May 13 at 8:45
• Java is very commonly used. R is a special purpose language, and to a lesser extent MATLAB and only suitable for beginners in stats or numerical science. Python is a very useful language, but it is not a good language for teaching students to code well (worse than C which used ta attract that criticism). May 13 at 8:49
• I've heard of Java being more popular in Computer Science settings. In my experience for Physics it is mostly Python, C, and Matlab, and for Maths/Stats it is Matlab, Maple, R, Mathematica, Python, C++. For almost all courses I've seen it is "write a program" rather than "write a good/clean program". Students are rarely taught to code well, and in honesty I think academia isn't a environment that fosters nor encourages good programming. The standard for programming in academia in my experience is extremely poor unfortunately, because it's not the focus for most research areas. May 13 at 8:56
• I expect computer science would put good programming practices and standards in a high regard than e.g. maths and physics, but I wouldn't imagine it being too dissimilar. Personally, I wish coding quality were generally regarded as importantly (or even more so) as code correctness, but I've not seen this in my experience. While I've not been in a computer science department, I can comment on what I saw from the maths and physics side of things. For the mathematicians, I all to often see (sadly) that code correctness is binary, and code quality is a second class citizen. May 13 at 9:08

As a blanket policy, I would say that automatically assigning a grade of zero for any submission that doesn't compile is completely unreasonable. Aside from the reasons that Ben I. has already listed, depending on how you are accepting programming assignment submissions, the student's program may even have been corrupted by your university's e-mail servers (I personally remember this happening multiple times when I was in undergrad, including at least one case where empty .zip files were delivered to the professor and others where the e-mail just didn't show up at all.)

Unless you have a system like what ti7 suggested where students are submitting to a build server via a version control system directly with a strictly consistent build environment, corruption in transmission (e.g. by spam filters or virus scans) as well as configuration management problems are almost always a possibility. Giving a student a zero because the university's IT interfered with the submission or because you had a different version of the build tools, your environment was set up slightly differently, paths were different, etc. is completely unreasonable. Even deducting points for that is unreasonable unless you very clearly and precisely defined the environment and the student obviously strayed in a non-trivial way from what was defined. To give another example from personal experience, I recall a situation where I spent days trying to debug a program only to find that the professor was building and running it on a completely different operating system than the one my submission had been written for (and the one I wrote it for and documented it was written for was explicitly allowed for the assignment.)

Now with all of that being said, on the flip side, if a student submits complete gibberish that makes no sense either to a compiler or upon manual inspection of the source and/or demonstrates no intent to even try to complete the assignment, then a grade of zero would be reasonable. But that would be true regardless of whether the program compiles or not. For example, it would be completely reasonable to assign a grade of zero if a student submits even a working "Hello World!" program for an assignment where they were supposed to implement some complex data structure. Or if the assignment was in Java, but the student submitted a program in Whitespace.

Cases in between those extremes should receive grades between those extremes. A large, complex program submission that would have worked perfectly, had it not lacked a single semi-colon is a completely different animal from a case where the student didn't even try. It may deserve a few points off, but not a failing grade, let alone a zero. A program submission that demonstrates the student was trying, but just not getting it may deserve a failing grade, but probably not zero.

• Corrupted in transit but in a way that prevents that from being obvious? Single bit fl8p somewhere? Possible but unlikely, and regardless could always be countered by “But it works on my machine”… May 14 at 13:17
• @jmoreno Corruption in transit due to the network is unlikely (given that TCP is a reliable protocol.) Corruption in transit due to anti-virus software on the e-mail servers (for e-mail submissions) is much more likely. This can be anything from complete non-delivery to seemingly-random parts missing due to matching some overly-aggressive detection heuristic. May 14 at 19:22
• That is exactly my point. A policy of zero tolerance, doesn’t actually have to mean zero tolerance. If the file gets turned into garbage in transit, that’s different from a missing semicolon or using Integer when it should be int. If you have a policy of no make up test, and you accidentally run a student over in the parking lot, and they spend a couple of days in the hospital (missing the test) you better make an exception to your policy. May 14 at 20:58
• And if corruption happens on “your” machine, you need to deal with it not the student. May 14 at 21:01
• @jmoreno If there are files missing and/or parts of files missing, it's not always easy to know that it was the fault of your IT department rather than the student submitting it that way, though, which was my point. At any rate, that was only one reason. That still ignores the configuration management problems, for example. The over-arching point was that there are lots of potential reasons for a program to not compile that aren't the student's fault at all. Those were just some examples that I personally encountered; they were not intended to be an exhaustive list. May 14 at 22:57

I imagine this debate arises from two basic truisms: 1) the fact that nearly all programming languages have simple frame works that allow empty programs to be built in just a few lines; and 2) that even first semester students should be able to iron out basic syntax errors (or have had time to ask for help) by the time they are submitting their first project. Creating a functional program is trivial.

Borrowing the imagery from another poster... this is like an English teacher who is confronted with a paper illegibly scratched out in pencil, filled with grammar errors and spelling mistakes. Submitting a project that doesn't compile shows a level of inattentiveness and lack of effort so grand that a zero may be the only way to get through to the student.

With that in mind I can think of several scenarios where the debate of "Zero; to be or not to be?" is actually a question.

1. The student hit the "j" key as they were in the process of submitting the project and the compiler kicks a syntax error. It is obvious what the problem is and the teacher fixes the problem in less time then it would take to give a zero. This is also true of other issues like misaligned paths or unwanted package commands, where the code compiles and runs on the student computer but not on the teachers computer. Fix the problem, and grade the project as if the problem wasn't there.

The following points address situations when, without taking the time to ask for help, the student willfully submits a project that doesn't compile and run.

1. The student is taking a high level course, attention to detail is part of the grade and the student has been submitting projects for multiple semesters. The student should know enough to check their work. In this case I would consider a zero, or at least dropping the score by a full letter grade.

2. Projects shouldn't be collected in the first place. Programming is a collaborative art. Students should be encouraged to assist each other and ask for help from the teacher. Anything short of copying the code from another student is allowed. In this environment projects are evaluated as soon as the student completes the task, at the students computer, while the student is sitting there. This allows the teacher to judge success, offer assistance when help is needed and give suggestions for improvements. Formative assessment at its best. This works with even large groups of students because projects are scored based on milestones. When you complete a milestone, your grade jumps up to the next level. Evaluations take just a few seconds because the teacher knows what to expect at each milestone. And the first milestone is always "Did you get the project to run". Zeros do happen on rare occasions, but it takes a lot of will to fail in this environment.

3. The instructor is using an auto grader to accept project submissions. This type of system provides the student with instant feed back. First time students will be confronted with the problems as they occur, learning how to fix them and when to ask for help. Veterans will know that a submission doesn't qualify as a program if the system won't run it and will earn the zero it receives.

• Comments are not for extended discussion; this conversation has been moved to chat.
– Ben I.
May 15 at 14:52

It is an important but quite simple-to-answer question once you adopt a certain mindset. First, we need to ask ourselves: if the only thing that supposedly matters is whether the code compiles or not, why aren't students just told to send compiled binaries instead? The answer to that question is quite obvious: we are supposed to grade the source code, not just the executable it creates. If the source does not compile by the virtue of some factor, why should educators suddenly not care about the human-readable content of the file? Equating that situation to a student sending a corrupted or password-encrypted file is quite disrespectful and unfair to the student's time and effort, and honestly shows what could be interpreted as educator's laziness. And especially more so if the compiler error is caused by something that could be fixed within less than a minute, like a missing semicolon or a stray character.

If you really do want to reject programs based on technical aspects like compiler errors, instruct students not to send source code at all and tell them to send just the compiled binaries instead. Otherwise, it is intellectually dishonest.

• I believe the point being made is that a program that does not compile, especially one submitted through an auto grader, does equate to submitting a an encrypted assignment. More to the point, one written in French. Expecting translation is disrespectful of the instructors time. May 13 at 13:20
• @codingCat That is a valid point of view about that argument that I just don't happen to completely agree with. Auto graders are convenient tools used to facilitate easier job for instructors, but not to completely absolve them from any responsibility; I get your point, but one usually does not accidentally translate their essay from English to French by the virtue of a swift missclick, as in some cases of compiler errors. Regardless, I believe a fair compromise would be not to reject, but not to grade either: instead, get the grading on hold and allow student to fix the error {continues} May 13 at 13:29
• @codingCat themselves while the educator is watching. BTW I really appreciate your mentality of "little [being] more entertaining that a good argument", and no disrespect from me, just partial disagreement, cheers! May 13 at 13:29
• a zero is not necessarily a rejection. I will regularly enter zeros for assignments the moment that a deadline pass, but happily grade the late assignment (often without penalty) once it is handed in. The zero is to get their attention and to let them know that inattentiveness will not be tolerated. The same goes for a program that does not compile. May 13 at 13:49
• @DanielR.Collins I think you have merit there and that is valid criticism of my answer. And no feelings of being disrespected on my part, don't worry about this, I welcome the critical comments. I will think about updating my answer, maybe fixing the wording a bit to avoid the strawman thing. Cheers. May 13 at 19:25

My policy when correcting assignments for a similar course was close to this but also addresses some of the criticisms:

• Students are aware that they shall submit compiling code. They are sufficiently trained to at least locate sources of the vast majority of errors they can possibly encounter.
• If a student cannot get a part of the code to compile, they shall also submit the most complete code that does compile or similar and comment on this, e.g., like: “When I try to add this functionality, I get this compiler error. I fail to make sense of this, because …”
• If the student did not make use of the previous point and their code fails to compile, I spend less than a minute trying to fix the code myself.
• Any other non-compiling solution gets zero points, and I don’t spend further time on it.
• In this case, the student is granted the opportunity to fix their solution (without substantial changes) or demonstrate that it compiles on their machine. In that case, I will grade the assignment as usual. Since I can easily compare both solutions, this cannot be abused to gain time.

So, in other words, if your code does not compile, your submission should appropriately reflect that or it is upon you to convince me that you had reason to believe you submitted compiling code (and you get a delayed feedback).

The rationale is that unlike most errors you can make in assignments, failure to compile is a problem that is straightforward to notice before submission. Thus students should at the very least notice and comment on it. Last-minute typos or environment differences pose an exception to this, but those are straightforward to clarify as per the last of the above points.

I think a reasonable analogue to non-compiling code is: In a physics lab course, a student misunderstood how to read a measurement device and thinks any value yielded by the device is zero. The student doesn’t remark that something is wrong during the experiment or in their report, but instead blindly uses the zeros and obtains a totally nonsensical result for a natural constant.

• I like this passage "most complete code that does compile". I will be adding that to the project rubrics. :-) May 13 at 14:25

As a former IT Lecturer, In My point of view, as the first step in practical examination students can be given a "Written-test" where they can write the Algorithm & pseudo code/logic of the program so that we can also evaluate the level of knowledge of that student before compiling the program code. Also, they can be awarded several credits if the logic & algorithm, even if the student failed to compile the program on running.

• That is an interesting and valid take: Make both goals (algorithm/knowledge and technical/programming) explicit and grade them separately. May 13 at 9:18

I never understood why to otherwise highly intelligent people this is even a question. Like: Is an empty file to you literally the same as a 99.9% correct code with that missing semicolon or whatever ?

There is literally nothing more to say here except a clear and resounding NO you should obviously not award 0% of the points to a submission that may be 99% correct.

• Let me tell you the sad truth: A program that does not compile is not a working program. Almost counts in horseshoes and hand grenades - but not for programs. May 13 at 11:43
• @CharonX in maths a derivation that contains an error is not a derivation - should it be awarded zero, no matter how much of the derivation is correct? May 13 at 12:32
• @CharonX Almost does not count for hand grenades. A hand grenade that does not explode is not a hand grenade. Nor for horseshoes. A horseshoe that cannot go on a horse is not a horseshoe. Do you think computer programs are somehow special? May 13 at 14:13
• Respectfully: "99.9% correct code with that missing semicolon" is a straw-man argument; I've never seen that in 20 years of grading code submissions. IME, the usual case of a non-compiling program submission is sheer nonsense that would take an hour or more to fix up. May 13 at 17:30
• @DanielR.Collins Then your policy should be to give 0% for submitting nonsense, not for submitting code with a compilation error. May 14 at 18:02

Former student perspective here. A lot of people mentioned that in a commercial or professional enviornment, code that doesn't compile would be worthless. I would pose the following counter argument to that. While I was attending university, I rarely saw or have ever seen students treat being a student like a full time job doing 40 hours a week of both visting lectures and studying at home. Some may have come close to it while others simply couldn't due to working second or third jobs on the side.

A student who has to split his attention to multiple courses and probably has to work a second job shouldn't have to uphold the same standard of "doesn't work = failure" as someone working a paid full time job as a software developer.

Students will make mistakes, sometimes quite silly, sometimes more serious but it feels incredibly unfair and bitter if the turned in work isn't even properly looked at.

• Also in professional environments code that doesn't compile isn't really an issue. Professional software developers produce code that doesn't compile from time to time, but it is never shipped to customers. Because if that ever happens you have a problem with your continuous integration pipeline or your testing strategies, and not with your software developers. May 13 at 13:18
• @codingCat "a program that doesn't compile is equivalent to submitting math homework in an English class. " I don't think that is a reasonable analogy. It is more like a maths student handing in a proof with an error - it doesn't meet the requirements (an incorrect proof is not a proof), but it would not deserve a mark of zero if it demonstrated some competence and understanding. May 13 at 13:54
• @codingCat Accidentally adding "v" at the start of your file when submitting does not transmute English homework into math homework. May 13 at 14:15
• In the case of the accidental "v" this is more like submitting an otherwise valid proof with an arithmetic error in the tenth decimal place! The mistake is incredibly trivial. May 13 at 14:15
• @codingCat I was specifying the error in the mathematical proof, not the program. Omitting an operator means that the proof is syntactically incorrect and hence cannot be parsed, which seems to me to be directly analogous to a compiler error. I can't see how that would be different to e.g. omitting a semi-colon. May 13 at 14:35

I think zeroes for anything other than completely missing work are a sign of teacher laziness. The professional way to do this is to have a rubric. One element of the rubric can be "code compiles."

If it doesn't, there are several things you can do:

• If it's possible (that is, wasn't turned in on the last day of the semester), give it back to the student and tell them to fix the errors and make sure it compiles. Then grade it.
• If that's not an option, first see just how far is it from compiling? If the compiler finds a single error like a variable with the wrong case or a missing semicolon, I'd just make the change, deduct some points for not making sure it compiles, then run it and grade it.
• If it's not even close to compiling, and would take a lot of work to get it there, I'd look through the code to see if it's implementing the expected algorithms more or less correctly and in the right order. Any decent programming teacher should be able to see if a program is on the right track or not even close without having to run it. I'd give them points on the rubric for places I can see they did the right thing and deduct all the points for not compiling. They'd get a bad grade but not a zero.
• +1 this is pretty much what I do (20% of the marks are for getting the right answer, and they would lose that if it doesn't compile), except I don't have the option to give it back to them as resubmissions are handled centrally by the administrators and only permitted if there are "extenuating circumstances". May 13 at 21:56

No.

Some of the answers suggest that one would never get credit for a non-compiling piece of code when working in industry. I don't think this is true. People get "credit" for: power point presentations describing what they plan to do, power point presentations describing what they have done so far and pseudo-code that doesn't compile anywhere or on anything. Yes, they eventually have to produce code (if that is their job rather than specifying requirements, etc.), but aren't the goals of university education for the student to learn and show that they've learned something, rather than everything?

If you are going to do this, you should be very clear about it and know that you are encouraging some students to learn less than they would otherwise and to turn in less-complete work or even garbage that compiles rather than, as someone said above, 99.9% correct work.

Imagine you are an English literature instructor. You assign your class an essay on how Shakespeare uses foreshadowing in Macbeth. You receive the following submissions:

Shakespeare Macbeth -> Banquo. The foreshadowing Macduff, eeeeeeeee. ❤︎ Billy. The per./ TRUMP. x <= 53.3. Quadrilaterals cast. :), asdfpom, :-( salad regurgitate Hamlet, fly bar rat indoors necro: cup hole pole blanket! foreshadow

This isn't an essay. It is incomprehensible garbage, the equivalent to gibberish on Stack Exchange and patent nonsense on Wikipedia. This can't be graded because it can't be understood in any meaningful way as an attempt at an essay. The output of the process of reading this (the comprehension of the student's thesis in your mind) is undefined. Since it can't be graded, it can't be assigned any points. Assigning points would require you to understand why it is worthy of those points, which is impossible because you can't even understand it.

Yes, foreshadowing is used in Shakespeare's Macbeth. By turning Macbeth into a ghost, Banquo foreshadows the death of all of the kings. We see this on the seventh page when Malcolm is all like, "WTF is this shit, noob? Now you die by sword!!". Everyone dies and foreshadows the rise of Episode Two, which was supposed to come out in 1620 but never did. From this, we can see that Macbeth is the best book ever written and should be recommended to all children.

This is a terrible essay, but it is an essay. It attempts, however incompetently, to address the subject matter. You can read this essay and comprehend it enough to determine what the student expressed and what sort of grade might be appropriate. You can tell that the student recognized that the story involves kings, a sword, a ghost, and character death. You might give the student a few points for recognizing these things.

So, the same analogy works with code. Code that doesn't compile, like RBarryYou said, isn't a program at all. It can't be run, and its output is therefore undefined in the same way that the output of an incomprehensible "essay" is undefined. Code that runs, but doesn't accomplish the requirements specified in the assignment, is like a bad essay that you can at least read. Such an essay doesn't adequately address all of the requirements of your assessment rubric, but you can at least identify what it does address and how it addresses it.

• I don't agree with this at all. I very rarely see a program that doesn't compile, and I don't think I have seen one that is complete garbage where I can't understand any of it. Compilers are very fussy, human brains aren't (which is one of the reasons programming can be tricky to learn), so a compiler will reject code that does demonstrate understanding of programming, but which isn't quite righty. May 14 at 5:26

Non-compilation should be a zero if and only if learning to debug compilation errors is outside the scope of the course.

The purpose of any course is to cultivate understanding of the material, and in order to achieve that, the professor and TAs must be able to generate useful feedback on the subject and students must be able to receive it.

If the subject is "C++ for beginners," a question about the message "foobar is declared but not defined" is germane for office hours, and a submission failing with such an error message should be treated as a problem understanding the course material. The grader should be able to provide useful feedback, and should search the rest of the submission for other successes/failures of understanding (i.e. give partial credit.)

If the subject is "building widgets with C++" and requires the former as a prerequisite, compilation failures no longer represent a problem understanding the course material, but rather a problem in the student's understanding of their prerequisite courses. Providing useful feedback would be harder for the grader for two reasons - first, the material is more complex, and harder to assess for partial credit without the assistance of automated tools that only work on a compiled submission. Second, if the grader is a graduate or undergraduate TA, they may have no experience teaching the content of the beginner level course. They could fix the problem themselves, but may struggle to explain the fix in a pedagogically useful way.

In such a higher-level course, boneheaded mistakes (like accidentally editing a file before submission, or messing with environment settings and forgetting to set them back to course-default before testing a submission) can be covered by an allowance for resubmission. If the student understands their prerequisites, they'll have no issue resubmitting working code once alerted to the issue by the zero (or, ideally, by the build server when attempting a submission.)

But if the student really doesn't understand their prerequisites, that needs to be treated as a problem that will prevent them from succeeding in the course. Even if the grader puts in the extra effort to give them detailed feedback (partial credit) the student will be unprepared to receive and integrate that feedback, wasting both people's time and endangering the student's chances of success.

• "The purpose of any course is to cultivate understanding of the material," that seems inconsistent with not recognising the understanding shown in the program simply because it didn't compile. "In such a higher-level course, boneheaded mistakes ... can be covered by an allowance for resubmission." Not necessarily, at my institution we don't have that discretion and resubmission is only permissible if you have an "extenuating circumstance" (e.g. illness or SPLD). May 13 at 14:53
• @DikranMarsupial ah, bureaucracy. I think that it's possible to create policies with a similar end effect in many situations though. Wrzlprmft mentioned the grader spending up to 1 minute fixing trivial compilation errors. And a build server preventing noncompiling submissions solves the problem cleanly.
– John
May 13 at 15:03
• As for understanding, it would certainly be ideal to recognize the student's partial understanding, but what really matters is feedback about their misunderstandings so that they can learn. The grader is now looking at those through fog and the student is distracted by basic language problems, creating a real risk of falling behind under a mountain of multi-level understanding gaps. Forcing them to make their submissions compile means they can actually focus on the course-related feedback after grading. And if they truly can't figure out compilation, it can be triaged with their advisor.
– John
May 13 at 15:04
• bureaucracy indeed! We provide them with a build server that also tests and packages their results as a pdf file for submission, which clearly indicates when it doesn't compile. They only have one chance at submitting the work though and we have to mark what they submit. I only give ~20% for getting the right answer, and that is what they lose for non-compiling code, which seems reasonable. May 13 at 15:08
• "but what really matters is feedback about their misunderstandings so that they can learn." I don't agree there, they grade/mark should be representative of their level of understanding. " it can be triaged with their advisor. " I meet with every student that fails to put a work plan to get back on track (whether they follow it is up to them) May 13 at 15:11

A program will very often compile or not compile depending on its environment. Larger programs distributed as source code will often have configuration lines containing hundreds and hundreds of lines to configure the program to compile and run on a particular version.

For example, where I work a program may be required to run on MacOS, with different processors, with different SDKs, and on an iPhone, iPad, Apple Watch, Apple TV, simulators for the last four running on different processors. And we haven't even started looking at different Windows or Linux versions, Android, and then at more obscure machines. Getting a program to work on all versions is a lot of work.

If you just use a Mac, a Linux box, and a Windows PC, chances that a non-trivial program compiles on all three if it was only tested on one are low. If it doesn't compile on your machine, there's a good chance that you couldn't make it work on both your and my machine.

• I think you're comparing apples to oranges here. Most CS instructors here are focused on the majority case: introductory programming courses in the first few semesters. E.g., all of the assignments I give out of the textbook in the first year are submitted as a single C++ code file of maybe 40-100 lines, in a well-defined standard environment. In 20 years I've never seen environment factors ever come into play (aside from some students not knowing where their CPP file was on a MacOS to submit it in the first place). May 13 at 17:38
• @DanielR.Collins line endings frequently trip up Java programs (if the students don't use readline as I demonstrate in the lecture/seminar sessions) May 13 at 21:28
• There are plenty of instructors that provide a "well-defined standard environment" that isn't standard... May 13 at 22:29
• @DanielR.Collins an issue that may prevent compilation is that my institution requires an LTS (Long Term Support) version of Java and we were using LTS11 (which is a bit long in the tooth) as LTS17 arrived just too late to be installed before the start of the academic year. Some students installed a more recent release (my instructions showed how to install LTS11 for compatibility) and their IDE prompted them to use more recent language features (switch) and their code doesn't compile despite being legal. Similar story for C++17 and C++20. It does happen. May 14 at 5:21
• @DanielR.Collins I don't think they are irreconcilably different. I think the difference is the level at which we are teaching. "I get are pretty frequently massively scrambled up, nonsensical, often show no understanding of the assignment, " is something that I very rarely see, and it is not at all unusual to have students in my class that are already highly proficient (and happily maintain several toolchains on their IDEs for their own projects). The problems with my servers are almost always provoked by the very good students wanting to go beyond the learning objectives. May 15 at 16:07

TBH it depends on the level of abstraction, and hence, the likelihood that build fails result from compilation errors. If the program is written in a low-level language like C with strict regards to the runtime environment then you can rest assured the project should be a fail since the student was most probably aware of their build failing...

But in a higher-level language like Java, you can experience build failures when running the code in new environments with different configs, different user permissions, different platform features, the list goes on.

Having said that, if you can ensure that build failures are a direct result of failing the exercise in the question, i.e. getting to code to work, then it would be understandable to fail any programs that don't build. However this criteria is full of asterixis, mainly the question of "how can the student prove their build succeeded?". For less experienced programmers, it would be fair to allow them to demonstrate a build on their local machine. However, more advanced students should be aware of the circumstances when deploying to external environments.

The real question to ask is how trivial is the build error? Is it due to semantics? In which case it's a definite fail... is it due to mismatching configurations? discrepancies in runtime environments, execution platforms, external tools and dependencies? Then that's a less fundamental failure in my opinion.

As the teacher, you should really clarify the conditions for a build failure, i.e. the machine that their code will be built on. If possible, provide them with a virtual machine to deploy to (or some other pre-configured device). If possible, ask them to submit pre-compiled executables/binaries should their build fail or simply ask for a live demonstration orchestrated entirely by the student (especially convenient for remote-learners).

# Yes

Your job as an educator is not only to teach the knowledge of the subject at hand but also to prepare the student for the application of said knowledge in the real world.

No one gets credit for software that fails to compile when they start working in the real world. It is better to set that expectation now than when their job and livelihood depend on it.

• Does that mean that a maths student should get zero for a proof or derivation that contains an error invalidating the proof? May 15 at 6:20

Let me poke a hole in the “real world” argument. I work in the real world. And what we value more than anything is readable code.

Why? Because all code is broken. But I can fix readable code.

Give me code that works 99 times out of 100 that no one understands and you’re likely to kill someone.

But yeah, teachers are busy. Grade as best you can. Just remember these kids will code things that might some day be used to keep you alive.

I’d fail the ones that don’t build that you can’t see how to fix. That’s not readable code.

In my opinion, no, but I wouldn’t criticize any other instructor from grading otherwise. It really depends on the goals of the course and the goals (and rubric) for the specific assignment. I can see the instructor writing the assignment in such a way that if it doesn’t compile, the grade must be zero.