If a program does not compile, should it get a mark of zero?

Question

Following a discussion on the Academia SE, I would like to seek views here among Computer Science Educators on whether a mark of zero should be awarded if a student submits a program that does not compile for an undergraduate programming assignment. I will add an answer giving my view on this, but only later on so as not to bias the discussion. I'd be very grateful for your views.

Answer 1

This answer assumes that the course's goal is to teach programming (as opposed to teaching advanced algorithms or numerics etc. to which the programming would only be an end). In that case I'd hold: It is a hard requirement that the program builds.

It's not so much that the submission would be graded with zero points; it's rather that it will be rejected because it is not gradable.

This should, of course, be announced beforehand, together with a defined development environment and, if possible, a "grading machine" that can be used to check the program before submission. Ideally, there would be an automated submit/build mechanism as suggested in other answers.

Rationale:

• Programming is the topic here. A program that doesn't build doesn't meet the minimal requirements for submission. It's like asking to grade an architect's house which collapses. "But it has a beautiful stairwell, if you'd only care to dig through the rubble!" No thanks.
• The teacher's time is limited. Grading by reading would take too long.
• The "must build" criteria is self-understood in the real world. It is good to get used to it. The customer won't pay you for a system that doesn't run, and the professor won't grade you.
• The programming environments for teaching programming are typically plain vanilla installations of Linux/gcc or the like which are easy to reproduce and are fairly compatible on the source code level.

Note because of ongoing discussion: No rule is absolute, nor should it be. This answer concerns programs that are submitted in a flawed state, not weird, unusual occurrences.

Answer 2

Of course not, and I have trouble believing that anyone would truly entertain the other position in any serious way. The closest analog in another field would be if you were to present an essay in a history class, and the professor reserved a technical score of zero if they found any grammatical errors.

Let's clear something up: a compiler error is an indicator of a problem in an assignment, and might well mean that the assignment is terrible. But it's only a clue, not a given. A compiler error in and of itself is only a clue, and there are enough reasons why something might not compile that aren't terribly concerning that the answer to "doesn't compile = automatic zero" should simply not be true.

Let's break that down a little. There are so many reasons something might not compile that are not under the students control, but could simply be environment-specific. I can't tell you the number of times a package statement at the top of a submitted Java file does not reflect my own directory structure. It compiled fine on the student's computer, and they were unaware of the name of the directory I'd be using to check their work. The appropriate deduction is no deduction.

We have students do large, group projects at the end of the year, and these get presented to the entire department in a CS Fair. I've had students submit android applications, iphone apps, web apps, unity projects, and do, so many other things. I quickly realized that I had to see the stuff working on their computer, not on my own. I would never even attempt to set up all of the environments I'd need to try it.

Then there are small accidents. A student recently submitted a Java file to me where everything worked, except for an added v at the beginning of the file, rendering the first command as vimport. Her code was otherwise excellent.

When I asked the student, she explained to me that she had opened the file up to make sure that she was submitting the correct version, and must have accidentally pressed a key prior to closing it. Her IDE auto-saves, so there was no Save prompt to warn her of the change Any of these is sufficient to make the answer to the question (as posed) an emphatic no.

Furthermore, students regularly submit work that works, but is terribly made. Once again, the compiler does not tell us anything relevant to grading.

The compiler can be a good first check of the work, and you might decide to deduct 5 points (instead of the zero I believe is proper) simply because it does not compile, but consider my poor v student, and you can immediately see why more than that would be unfair.

Philosophically, we are there to check student understanding and work, and as the prior to examples illustrate, compiling is a poor proxy for these. It is a clue, but nothing more.

If it doesn't compile, that suggests that there is possibly a serious problem, but that's all it can tell us. For the rest, we actually have to take the time to read through the submitted work, just like every other professor in every other field.

Answer 3

Following further with @Ben I.'s answer, I think the real problem is often not giving feedback about whether the program builds to the student in an inspectable and rapid manner, while also allowing multiple submission attempts. (in a worst case, for example, an email attachment that comes back with a grade after the due date)

Instead, have a workflow where they upload a specific format to a build server, or better, submit via version-control.

Then their program must build there and the system must give back meaningful feedback (such as compiler/interpreter warnings and full or reduced test case results), and then they can choose a successful build as their submission.

---

Simple Workflow

• student creates work submission
• (after due date) final grade results come in (no opportunity to try teacher's environment unless they crashed office hours)

Better Workflow

• student creates work submission
• remotely build program -> feedback to student
• remotely test program -> feedback to student
• student has opportunity to submit assignment for grade
• (after due date) final grade results come in (perhaps run against a much larger test suite with those results)

This doesn't necessarily mean that you should or should not assign zero points to a submission that fails to build, but gives the option of letting the first few or a subset be acceptable! .. Rather, collecting the submissions will help make your/the department's build system more robust and also give a good understanding to yourself (and the class!) about what could be done to help them structure and format programs to build!

Answer 4

I think giving a zero for a non-compiling program can be reasonable and expected behavior in many circumstances. It's pretty close to what I do now (more below).

The Original Academia SE Question

Note that the consensus in the motivating Academia SE question seems to mostly be that this would be the base expectation (mostly now moved from comments to chat: https://academia.stackexchange.com/questions/185044/grading-on-a-curve-in-required-doctoral-course-failing-as-a-result).

A few selections from different users in that chat:

Drake P: ... for the future, you should know that it's very standard to give 0s for code that doesn't compile, even at the undergrad level, as it makes grading ungodly for teachers otherwise...

ilkkachu: what's so tough here? If your code doesn't compile, it doesn't work. Nothing gets done. Expecting to get a good grade for code that doesn't work sounds like expecting to get a good grade for presenting e.g. mathematical proofs or other logical conclusions that are totally false...

RBarryYou: Technically, a "program that doesn't compile", isn't actually a program. And given how easy it is to check that it does actually compile, this seems like an obvious requirement for all but the most introductory CS and programming classes.

forest: It's not like you don't know if the code compiles until after you submit it. If the code doesn't compile, then fix it first.

My Experience and Practice

I teach the first two semesters of programming with C++ in a two-year CS degree at an open-admissions community college in the northeast U.S., with generally very weak students. The assignment submissions I get are pretty frequently massively scrambled up, nonsensical, often show no understanding of the assignment, and no evidence of testing. There is no lower bound to the quality. So taking a random file and laboring on my end to make sense of it or fix it can take an unbounded amount of time and effort. Therefore it seems reasonable to use compilation as a first-pass check to see if the student has done the basics of making something that satisfies the definition of a program under our language rules. No, I'm not going to try and make sense or fix student work otherwise. Or, more accurately: I've tried that in the past and it's impossible in most circumstances where the code doesn't at least compile.

Now, does a non-compiling assignment get an immediate zero from me? Actually, no. It does get a zero for the correctness & user interface components, which account for 70% of the grade rubric. I still almost always award points for readability & documentation components, which are worth 30%. However -- the only reason I'm willing to do that is that I've written a custom style checker which automates that part of the process for the things I care about (and we've specified in class). If I didn't have that tool available, then it's pretty likely I'd give a zero for the whole thing, and not spend time and headache trying to make sense of a non-program.

Justifying the Grade to Stakeholders

In addition to the preceding, there's the issue of justifying the grade being given. If I just mark a zero or low grade for something, a student (rightfully) will ask, "Why did I get a low grade?", and I need to have a clear response. I even need to think about the student challenging the grade I give at the department or college administration level. By being able to point to the clear and objective evidence of the compiler's failure on the student's submission, I have the strongest possible justification for the lowest-level grade being given.

We've made the point that non-compiling programs are most often garbage nonsense. In other answers and comments we also see some sentiments like this:

jamesdlin: Then your policy should be to give 0% for submitting nonsense, not for submitting code with a compilation error.

But "this is nonsense" won't be a comprehensible or satisfying response to the student, nor defensible at the administrative level if challenged. Pointing to the objective non-compiling issue is greatly clarifying for all involved. (In fact, if I didn't keenly highlight the compiling issue as a top priority, some of my students would likely not grok its importance at all, and never prioritize it as something they need to try and make happen.) Otherwise, finding ways to document and defend an evaluation of "this is nonsense" will take an egregious amount of time. This is analogous to the issue of "grading for completion" on elementary mathematics homework, for example, which I've also found to be unworkable in my context and experience.

Response to Other Answers

I think some of the answers in the "do not give zero" camp interpret the question too narrowly to make a point. My interpretation is that we're looking for a best-practice that's profitable in a general case. I wouldn't argue for entirely mindlessly giving zeroes. For instance, I would assume that most of Ben's examples have been double-checked and taken off the table as possibilities. For my courses, I specify the standard environment and IDE on the syllabus. It's the same thing used in lectures, labs, and on my test machine for grading. If a submission doesn't compile, I double-check, and confirm it's due to something in violation of our protocols.

I think it's interesting that several of the "do not give zero" advocates, in the comments, have indicated that they are forced to deal with some kind of strange, out-of-control build environments. For example, the OP writes (in a comment under gnasher729's answer):

an issue that may prevent compilation is that my institution requires an LTS (Long Term Support) version of Java and we were using LTS11 (which is a bit long in the tooth) as LTS17 arrived just too late to be installed before the start of the academic year. Some students installed a more recent release (my instructions showed how to install LTS11 for compatibility) and their IDE prompted them to use more recent language features (switch) and their code doesn't compile despite being legal. Similar story for C++17 and C++20.

... a student installs the most recent toolchain rather than the one I tell them to (back in October) and has forgotten months later when the coursework is due (it is a full year course)... It is perfectly reasonable for a good student to have a more up to date toolchain for work on other projects (best if students enjoy hobby programming - it is something I encourage).

And Ben I. in comments under his answer says:

I, for instance, am required to take submissions through a system called "Schoology", which requires students to compress java files prior to submission (dont ask, it's really a problem).

Okay, so if you're teaching with an out-of-control build environment and routinely get "legal but doesn't compile" submissions, then I guess you have to deal with that. I'll say that in my situation the idea of students maintaining multiple build environments and toolsets is completely unimaginable and not something I've ever heard of, or that my students could conceivably be expected to manage. I feel highly responsible (and take quite a bit of time) in trying to ensure that there's one consistent environment used in the labs, lectures, my grading suite, etc., and oversee practice with it every week, so that students can develop confidence in their on-ramp. In 20 years I've simply never gotten an "oops I wrote this correctly for a different environment" submission. It's good for all of us to bear in mind the "Academia varies more than you think it does" principle. But my advice would be that if you are in that situation, then get your build environment under control.

The Case of Autograders

As an added data point that such a practice is fairly common, consider the fairly widespread practice of using software autograder tools for programming assignments. E.g., one suggested at my school, used widely elsewhere at CUNY, is Gradescope (there are many others going back more than a decade now). The essence of a tool like this is that an automated suite compiles, throws certain inputs, and matches against expected outputs. There is, e.g., no facility for interface, readability, or documentation point awards. In short, there is no way for a tool like this to award any positive points for a program that fails to compile.

Is it reasonable to argue against using a tool like that? Sure, and I did so, and we declined to use it in our department. But that's because I wanted to give more feedback and points for styling issues with first-time programmers. From what I'm told of the widespread usage of this tool elsewhere in our system, that must be fairly rare. But is it completely unimaginable to use an autograder, in light of their giving zero to anything that doesn't compile? It seems hard to make sense of that, given their widespread usage.

More Evidence of Student Practices

In addition to all that, an interesting added observation has appeared in the last year -- partly related to pandemic restrictions, I've switched to giving programming tests on a computer with the standard IDE available for students to use on the test. And the lectern computer in the classroom is set up by default to display what's happening on the student computer monitors, so I've accidentally found myself watching students code test questions in real-time just recently.

What I've learned is that -- despite my constant verbal and written recommendations -- the majority of our first-year CS major students will write code for the entirety of a project (e.g., on the final exam about 65 lines code across 6 different functions) without ever trying to compile any of it. At that point they'll hit the "compile" button for the first time and get a huge cascade of errors. Then they'll deal with those errors one at a time trying to get the compile to work -- frequently changing things even further from a correct solution, and adding more code that makes even less sense.

So having seen that widespread behavior for the first time, I'm even less happy about awarding partial credit for code that "kind-of-sort-of" echoes a correct solution, when I now know that it wasn't the product of students actually understanding what they wrote, but rather a semi-random process of shoveling fuzz at the compiler. Furthermore, I feel additionally motivated to try and communicate that a process like this is highly unlikely to actually produce a workable large project. They should be writing, compiling, and testing small components -- and apparently the current incentives are negative for practicing that behavior. If we made additional emphasis om the requirement to compile before earning any points, then perhaps students like mine would attend to that correct process more closely.

Conclusion

So: I do give zeroes for non-compilation, at least for a majority component of an assignment's grade. That's after double-checking for unforeseen causes that I failed to define properly for students -- which in practice hasn't happened in some years. Other credit is available for styling & documentation. But I've found it generally impossible to "fix" submissions from my students, and I use compiling as a first-pass to see whether something which counts as a program has been submitted.

On the other hand, evidence is that it's probably pretty widespread for other CS instructors to not go to the trouble that I do, and likely have a process that gives a zero by default for non-programs (e.g., with a standard autograder tool). If I attended someone else's class myself, that would be my default expectation.

Answer 5

I enforced this exact policy ("doesn't compile = failure")

For several years I was the assessor on a postgraduate course on C++ (Financial computing with C++) given at the mathematics department, and I enforced this exact policy. If a student submitted work to me that didn't compile, it would get an immediate "D" grade (the lowest score on the scale, corresponding to 0%). Typically about 1-2 students a week would get stung by this for the first few weeks of the course. No exceptions, zero sympathy, without remorse, etc. Of course when students couldn't achieve code that compiled, it would be extremely frustrating for them to be in this scenario, but there was good reason for enforcing this, which I'll explain. Furthermore, I was also a student on this same course, so can understand the exact same pressure to get code past the compilation milestone, by any means necessary, even if the code becomes hacky.

Why enforce such a strict policy

If the goal of the course is to teach numerical algorithms, computational linear algebra, or advanced scientific computing, or anything similar where the aim of the course is to teach the core science, then it's very rare to see anything use a compiled language, and I would be more inclined to disagree with such a policy. However, when courses are aiming to teach high performance computing languages and software engineering as its primary skill, compilation is essential. There are a few reasons for this:

• Code that doesn't compile would be rejected without question in any commercial or professional setting, without a moment's hesitation, as other answers have highlighted.
• A program that doesn't compile isn't really a program.
• Getting code to compile is a key skill, and crucially it is something the students are able to iterate on themselves before it needs to be submitted. That the code doesn't compile shouldn't surprise them.

Is it just compilation?

Having code which compiles is the absolute baseline. However, it is not the last hurdle. If the program always encounters a runtime error straight after launch or gets caught in an infinite loop, these are viewed with equal frustration from the assessor, and if the code always seg faulted immediately, it would also fail for similar reasons. However, the compiler is usually your friend giving tips and suggestions about whats it's complaining about.

Can the students be safeguarded?

It will vary from course to course, but naturally it is in everyone's interest if the code compiles. For our particular setting, we went to great lengths to set all the students up with a system agnostic build environment, used the same compiler, and shipped all the external libraries they would need with the projects. We also did a 2 day setup to ensure everyone could build and run a hello world, the homework assignments, and the exam assignments.

At the start of the course, I would tell all the students directly that failure to compile would result in a fail grade (for all the reasons stated above), and set out similar grading criteria.

The myth that this is easier for the assessor

There seems to be the idea/myth that such a blanket policy is an easy option for the assessor. As someone who marked 30 or more code submissions each week, this is absolutely wrong. When a code came along that did not compile, while it got a failure immediately, the marking was not done there. I then had to go into the source code, set it up, compile, and then debug the compilation problems, so I could explain to the students what they needed to change and explain the compiler warnings/errors. Code that compiled but was wrong is easier to debug, but code that doesn't compile is a nightmare. Certainly it takes 10x longer than other mistakes that did compile, and it certainly was a lose-lose situation for the student and the assessor.

Corner cases

While I agree there is scope for arguably valid corner cases, I have never seen one in the wild after having been marking these examples for several years, so think this is a null argument.

Answer 6

As a blanket policy, I would say that automatically assigning a grade of zero for any submission that doesn't compile is completely unreasonable. Aside from the reasons that Ben I. has already listed, depending on how you are accepting programming assignment submissions, the student's program may even have been corrupted by your university's e-mail servers (I personally remember this happening multiple times when I was in undergrad, including at least one case where empty .zip files were delivered to the professor and others where the e-mail just didn't show up at all.)

Unless you have a system like what ti7 suggested where students are submitting to a build server via a version control system directly with a strictly consistent build environment, corruption in transmission (e.g. by spam filters or virus scans) as well as configuration management problems are almost always a possibility. Giving a student a zero because the university's IT interfered with the submission or because you had a different version of the build tools, your environment was set up slightly differently, paths were different, etc. is completely unreasonable. Even deducting points for that is unreasonable unless you very clearly and precisely defined the environment and the student obviously strayed in a non-trivial way from what was defined. To give another example from personal experience, I recall a situation where I spent days trying to debug a program only to find that the professor was building and running it on a completely different operating system than the one my submission had been written for (and the one I wrote it for and documented it was written for was explicitly allowed for the assignment.)

Now with all of that being said, on the flip side, if a student submits complete gibberish that makes no sense either to a compiler or upon manual inspection of the source and/or demonstrates no intent to even try to complete the assignment, then a grade of zero would be reasonable. But that would be true regardless of whether the program compiles or not. For example, it would be completely reasonable to assign a grade of zero if a student submits even a working "Hello World!" program for an assignment where they were supposed to implement some complex data structure. Or if the assignment was in Java, but the student submitted a program in Whitespace.

Cases in between those extremes should receive grades between those extremes. A large, complex program submission that would have worked perfectly, had it not lacked a single semi-colon is a completely different animal from a case where the student didn't even try. It may deserve a few points off, but not a failing grade, let alone a zero. A program submission that demonstrates the student was trying, but just not getting it may deserve a failing grade, but probably not zero.

Answer 7

I imagine this debate arises from two basic truisms: 1) the fact that nearly all programming languages have simple frame works that allow empty programs to be built in just a few lines; and 2) that even first semester students should be able to iron out basic syntax errors (or have had time to ask for help) by the time they are submitting their first project. Creating a functional program is trivial.

Borrowing the imagery from another poster... this is like an English teacher who is confronted with a paper illegibly scratched out in pencil, filled with grammar errors and spelling mistakes. Submitting a project that doesn't compile shows a level of inattentiveness and lack of effort so grand that a zero may be the only way to get through to the student.

With that in mind I can think of several scenarios where the debate of "Zero; to be or not to be?" is actually a question.

1. The student hit the "j" key as they were in the process of submitting the project and the compiler kicks a syntax error. It is obvious what the problem is and the teacher fixes the problem in less time then it would take to give a zero. This is also true of other issues like misaligned paths or unwanted package commands, where the code compiles and runs on the student computer but not on the teachers computer. Fix the problem, and grade the project as if the problem wasn't there.

The following points address situations when, without taking the time to ask for help, the student willfully submits a project that doesn't compile and run.

2. The student is taking a high level course, attention to detail is part of the grade and the student has been submitting projects for multiple semesters. The student should know enough to check their work. In this case I would consider a zero, or at least dropping the score by a full letter grade.

3. Projects shouldn't be collected in the first place. Programming is a collaborative art. Students should be encouraged to assist each other and ask for help from the teacher. Anything short of copying the code from another student is allowed. In this environment projects are evaluated as soon as the student completes the task, at the students computer, while the student is sitting there. This allows the teacher to judge success, offer assistance when help is needed and give suggestions for improvements. Formative assessment at its best. This works with even large groups of students because projects are scored based on milestones. When you complete a milestone, your grade jumps up to the next level. Evaluations take just a few seconds because the teacher knows what to expect at each milestone. And the first milestone is always "Did you get the project to run". Zeros do happen on rare occasions, but it takes a lot of will to fail in this environment.

4. The instructor is using an auto grader to accept project submissions. This type of system provides the student with instant feed back. First time students will be confronted with the problems as they occur, learning how to fix them and when to ask for help. Veterans will know that a submission doesn't qualify as a program if the system won't run it and will earn the zero it receives.

Answer 8

It is an important but quite simple-to-answer question once you adopt a certain mindset. First, we need to ask ourselves: if the only thing that supposedly matters is whether the code compiles or not, why aren't students just told to send compiled binaries instead? The answer to that question is quite obvious: we are supposed to grade the source code, not just the executable it creates. If the source does not compile by the virtue of some factor, why should educators suddenly not care about the human-readable content of the file? Equating that situation to a student sending a corrupted or password-encrypted file is quite disrespectful and unfair to the student's time and effort, and honestly shows what could be interpreted as educator's laziness. And especially more so if the compiler error is caused by something that could be fixed within less than a minute, like a missing semicolon or a stray character.

If you really do want to reject programs based on technical aspects like compiler errors, instruct students not to send source code at all and tell them to send just the compiled binaries instead. Otherwise, it is intellectually dishonest.

Answer 9

My policy when correcting assignments for a similar course was close to this but also addresses some of the criticisms:

• Students are aware that they shall submit compiling code. They are sufficiently trained to at least locate sources of the vast majority of errors they can possibly encounter.
• If a student cannot get a part of the code to compile, they shall also submit the most complete code that does compile or similar and comment on this, e.g., like: “When I try to add this functionality, I get this compiler error. I fail to make sense of this, because …”
• If the student did not make use of the previous point and their code fails to compile, I spend less than a minute trying to fix the code myself.
• Any other non-compiling solution gets zero points, and I don’t spend further time on it.
• In this case, the student is granted the opportunity to fix their solution (without substantial changes) or demonstrate that it compiles on their machine. In that case, I will grade the assignment as usual. Since I can easily compare both solutions, this cannot be abused to gain time.

So, in other words, if your code does not compile, your submission should appropriately reflect that or it is upon you to convince me that you had reason to believe you submitted compiling code (and you get a delayed feedback).

The rationale is that unlike most errors you can make in assignments, failure to compile is a problem that is straightforward to notice before submission. Thus students should at the very least notice and comment on it. Last-minute typos or environment differences pose an exception to this, but those are straightforward to clarify as per the last of the above points.

I think a reasonable analogue to non-compiling code is: In a physics lab course, a student misunderstood how to read a measurement device and thinks any value yielded by the device is zero. The student doesn’t remark that something is wrong during the experiment or in their report, but instead blindly uses the zeros and obtains a totally nonsensical result for a natural constant.

Answer 10

As a former IT Lecturer, In My point of view, as the first step in practical examination students can be given a "Written-test" where they can write the Algorithm & pseudo code/logic of the program so that we can also evaluate the level of knowledge of that student before compiling the program code. Also, they can be awarded several credits if the logic & algorithm, even if the student failed to compile the program on running.

Answer 11

I never understood why to otherwise highly intelligent people this is even a question. Like: Is an empty file to you literally the same as a 99.9% correct code with that missing semicolon or whatever ?

There is literally nothing more to say here except a clear and resounding NO you should obviously not award 0% of the points to a submission that may be 99% correct.

Answer 12

Former student perspective here. A lot of people mentioned that in a commercial or professional enviornment, code that doesn't compile would be worthless. I would pose the following counter argument to that. While I was attending university, I rarely saw or have ever seen students treat being a student like a full time job doing 40 hours a week of both visting lectures and studying at home. Some may have come close to it while others simply couldn't due to working second or third jobs on the side.

A student who has to split his attention to multiple courses and probably has to work a second job shouldn't have to uphold the same standard of "doesn't work = failure" as someone working a paid full time job as a software developer.

Students will make mistakes, sometimes quite silly, sometimes more serious but it feels incredibly unfair and bitter if the turned in work isn't even properly looked at.

Answer 13

I think zeroes for anything other than completely missing work are a sign of teacher laziness. The professional way to do this is to have a rubric. One element of the rubric can be "code compiles."

If it doesn't, there are several things you can do:

• If it's possible (that is, wasn't turned in on the last day of the semester), give it back to the student and tell them to fix the errors and make sure it compiles. Then grade it.
• If that's not an option, first see just how far is it from compiling? If the compiler finds a single error like a variable with the wrong case or a missing semicolon, I'd just make the change, deduct some points for not making sure it compiles, then run it and grade it.
• If it's not even close to compiling, and would take a lot of work to get it there, I'd look through the code to see if it's implementing the expected algorithms more or less correctly and in the right order. Any decent programming teacher should be able to see if a program is on the right track or not even close without having to run it. I'd give them points on the rubric for places I can see they did the right thing and deduct all the points for not compiling. They'd get a bad grade but not a zero.

Answer 14

No.

Some of the answers suggest that one would never get credit for a non-compiling piece of code when working in industry. I don't think this is true. People get "credit" for: power point presentations describing what they plan to do, power point presentations describing what they have done so far and pseudo-code that doesn't compile anywhere or on anything. Yes, they eventually have to produce code (if that is their job rather than specifying requirements, etc.), but aren't the goals of university education for the student to learn and show that they've learned something, rather than everything?

If you are going to do this, you should be very clear about it and know that you are encouraging some students to learn less than they would otherwise and to turn in less-complete work or even garbage that compiles rather than, as someone said above, 99.9% correct work.

Answer 15

Imagine you are an English literature instructor. You assign your class an essay on how Shakespeare uses foreshadowing in Macbeth. You receive the following submissions:

Shakespeare Macbeth -> Banquo. The foreshadowing Macduff, eeeeeeeee. ❤︎ Billy. The per./ TRUMP. x <= 53.3. Quadrilaterals cast. :), asdfpom, :-( salad regurgitate Hamlet, fly bar rat indoors necro: cup hole pole blanket! foreshadow

This isn't an essay. It is incomprehensible garbage, the equivalent to gibberish on Stack Exchange and patent nonsense on Wikipedia. This can't be graded because it can't be understood in any meaningful way as an attempt at an essay. The output of the process of reading this (the comprehension of the student's thesis in your mind) is undefined. Since it can't be graded, it can't be assigned any points. Assigning points would require you to understand why it is worthy of those points, which is impossible because you can't even understand it.

Yes, foreshadowing is used in Shakespeare's Macbeth. By turning Macbeth into a ghost, Banquo foreshadows the death of all of the kings. We see this on the seventh page when Malcolm is all like, "WTF is this shit, noob? Now you die by sword!!". Everyone dies and foreshadows the rise of Episode Two, which was supposed to come out in 1620 but never did. From this, we can see that Macbeth is the best book ever written and should be recommended to all children.

This is a terrible essay, but it is an essay. It attempts, however incompetently, to address the subject matter. You can read this essay and comprehend it enough to determine what the student expressed and what sort of grade might be appropriate. You can tell that the student recognized that the story involves kings, a sword, a ghost, and character death. You might give the student a few points for recognizing these things.

So, the same analogy works with code. Code that doesn't compile, like RBarryYou said, isn't a program at all. It can't be run, and its output is therefore undefined in the same way that the output of an incomprehensible "essay" is undefined. Code that runs, but doesn't accomplish the requirements specified in the assignment, is like a bad essay that you can at least read. Such an essay doesn't adequately address all of the requirements of your assessment rubric, but you can at least identify what it does address and how it addresses it.

Answer 16

Non-compilation should be a zero if and only if learning to debug compilation errors is outside the scope of the course.

The purpose of any course is to cultivate understanding of the material, and in order to achieve that, the professor and TAs must be able to generate useful feedback on the subject and students must be able to receive it.

If the subject is "C++ for beginners," a question about the message "foobar is declared but not defined" is germane for office hours, and a submission failing with such an error message should be treated as a problem understanding the course material. The grader should be able to provide useful feedback, and should search the rest of the submission for other successes/failures of understanding (i.e. give partial credit.)

If the subject is "building widgets with C++" and requires the former as a prerequisite, compilation failures no longer represent a problem understanding the course material, but rather a problem in the student's understanding of their prerequisite courses. Providing useful feedback would be harder for the grader for two reasons - first, the material is more complex, and harder to assess for partial credit without the assistance of automated tools that only work on a compiled submission. Second, if the grader is a graduate or undergraduate TA, they may have no experience teaching the content of the beginner level course. They could fix the problem themselves, but may struggle to explain the fix in a pedagogically useful way.

In such a higher-level course, boneheaded mistakes (like accidentally editing a file before submission, or messing with environment settings and forgetting to set them back to course-default before testing a submission) can be covered by an allowance for resubmission. If the student understands their prerequisites, they'll have no issue resubmitting working code once alerted to the issue by the zero (or, ideally, by the build server when attempting a submission.)

But if the student really doesn't understand their prerequisites, that needs to be treated as a problem that will prevent them from succeeding in the course. Even if the grader puts in the extra effort to give them detailed feedback (partial credit) the student will be unprepared to receive and integrate that feedback, wasting both people's time and endangering the student's chances of success.

Answer 17

A program will very often compile or not compile depending on its environment. Larger programs distributed as source code will often have configuration lines containing hundreds and hundreds of lines to configure the program to compile and run on a particular version.

For example, where I work a program may be required to run on MacOS, with different processors, with different SDKs, and on an iPhone, iPad, Apple Watch, Apple TV, simulators for the last four running on different processors. And we haven't even started looking at different Windows or Linux versions, Android, and then at more obscure machines. Getting a program to work on all versions is a lot of work.

If you just use a Mac, a Linux box, and a Windows PC, chances that a non-trivial program compiles on all three if it was only tested on one are low. If it doesn't compile on your machine, there's a good chance that you couldn't make it work on both your and my machine.

Answer 18

TBH it depends on the level of abstraction, and hence, the likelihood that build fails result from compilation errors. If the program is written in a low-level language like C with strict regards to the runtime environment then you can rest assured the project should be a fail since the student was most probably aware of their build failing...

But in a higher-level language like Java, you can experience build failures when running the code in new environments with different configs, different user permissions, different platform features, the list goes on.

Having said that, if you can ensure that build failures are a direct result of failing the exercise in the question, i.e. getting to code to work, then it would be understandable to fail any programs that don't build. However this criteria is full of asterixis, mainly the question of "how can the student prove their build succeeded?". For less experienced programmers, it would be fair to allow them to demonstrate a build on their local machine. However, more advanced students should be aware of the circumstances when deploying to external environments.

The real question to ask is how trivial is the build error? Is it due to semantics? In which case it's a definite fail... is it due to mismatching configurations? discrepancies in runtime environments, execution platforms, external tools and dependencies? Then that's a less fundamental failure in my opinion.

As the teacher, you should really clarify the conditions for a build failure, i.e. the machine that their code will be built on. If possible, provide them with a virtual machine to deploy to (or some other pre-configured device). If possible, ask them to submit pre-compiled executables/binaries should their build fail or simply ask for a live demonstration orchestrated entirely by the student (especially convenient for remote-learners).

Answer 19

Yes

Your job as an educator is not only to teach the knowledge of the subject at hand but also to prepare the student for the application of said knowledge in the real world.

No one gets credit for software that fails to compile when they start working in the real world. It is better to set that expectation now than when their job and livelihood depend on it.

Answer 20

I don't, because when I grade a programming assignment, then I have a checklist of things I expect the student to understand. I grade the assignment by giving points about whether or not the student has demonstrated understanding of that concept. Here is one from my last test:

• Demonstrated understanding of program input (2 points)
• Demonstrated understanding of variables (2 points)
• Demonstrated understanding of loops (2 points)
• Demonstrated understanding of branches (2 points)
• Able to implement a simple algorithm according to specification (4 points)

One student didn't remember exactly how to write a loop in that programming language, so they couldn't get the program to compile. Which means no points for understanding loops. And I wasn't sure that what they thought how loops work would have resulted in the correct algorithm. So I could not award them those points either.

But what they had demonstrated was that they understood how to receive user input, how variables work and how branches work. So I awarded them 6/12 points.

Answer 21

Let me poke a hole in the “real world” argument. I work in the real world. And what we value more than anything is readable code.

Why? Because all code is broken. But I can fix readable code.

Give me code that works 99 times out of 100 that no one understands and you’re likely to kill someone.

But yeah, teachers are busy. Grade as best you can. Just remember these kids will code things that might some day be used to keep you alive.

I’d fail the ones that don’t build that you can’t see how to fix. That’s not readable code.

Answer 22

In my opinion, no, but I wouldn’t criticize any other instructor from grading otherwise. It really depends on the goals of the course and the goals (and rubric) for the specific assignment. I can see the instructor writing the assignment in such a way that if it doesn’t compile, the grade must be zero.