How to overcome the TDD/BDD time fallacy

Question

I've observed an unusual thing happen when teachers try to introduce Test-Driven Development (TDD) or Behavior-Driven Development (BDD) to their classes. Even though the teachers invariably encourage it and present it as a useful strategy, students come out of the class with mixed feelings or actively opposed to using TDD/BDD in the future.

The reason they give for this is time. And in a way, it makes sense. TDD and BDD have high upfront setup costs in selecting framework (e.g. JUnit or Cucumber), adding test hooks/libraries into the code (your import org.junit.Tests), thinking up test cases for all the various scenarios, and then of course actually writing all the tests themselves.

In industry, this isn't so bad, because tests are essentially infinitely reusable, so the cost of that work can be amortized over months or even years. For a class project though, the entire thing from start to finish is only going to be a handful of weeks long at best. So the students see all of the cost and only a small slice of the benefit.

In short, students sort of pick up on a perverse incentive to not use TDD/BDD because the benefits don't justify the cost. So, how can I convince students that those strategies really are good choices, when simply forcing them to use them and saying "it works better in the real world, I promise" isn't cutting it?

I've seen How to stress the importance of testing code?, but the answer there is just "use TDD." It's not wrong, but it's also not helpful for my particular question.

Answer 1

I believe that the reason why students tend to come out opposed to using TDD/BDD (testing) is because they've been assigned to use it when it isn't necessary. Testing a "Hello World" program is 100% a waste of time. Tests only become useful when you have a reasonably complex program. The way that you can avoid students disliking testing is by not assigning it on projects when it isn't needed.

A specific way in which you could do this would be by combining all the project you would assign over the year into one larger cumulative project. Eventually, as they're writing code, you can be fairly certain that they'll induce bugs or begin to find manual testing to be a pain. Once that starts happening, you can introduce automated testing to relieve that pain. That way, they'll see testing as a way to save work rather than create more of it.

In addition, I think the issue may be with this part:

TDD and BDD have high upfront setup costs in ... thinking up test cases for all the various scenarios.

If you are spending time thinking of test scenarios, you're doing TDD wrong. You should never just take time to think of test cases. You should think about how you want each new piece of code to behave, and then testing should be a way of writing down how you want it to behave. That's just good planning, and you should be thinking about that anyways.

Answer 2

The whole point of TDD is to try and reduce the number of defects in software. If your students are confident enough to think that they won't benefit from automated testing, fine — but have some tests prepared to run on their code to ensure that it's actually correct. If they fail your tests (which you need not specify in advance), then their solution is not a complete, working one, and they need to go back and figure out what went wrong.

I imagine you could gamify this a little bit if you wanted — I've talked about Codewars before, which uses 'hidden' unit tests to see if your solution works or not. You could do the same with your students' submissions, but you might not even need to use a framework to do that. You mentioned web services — why not just black-box test it a little to see what happens when you put in unusual results — does it fail when it should? You could even try fuzzing the code to see if completely random nonsense produces the right response.

Essentially: make the costs in the class the same as in industry — defective code is not a valid solution to your assignments. In a job, a serious bug might cost your employer a large sum of money. You could emphasise that in your classes, and show them cases where testing has not been done, leading to huge amounts of cost.

If you follow TDD, you shouldn't really be spending much more time testing, unless your students replace the writing tests before writing code with just writing code without testing. Otherwise, surely the costs are very similar in terms of time if the tests get written either way? That is, of course, ignoring the productivity benefits of having tests that you can rely on while writing your code, so TDD might even be quicker!

Answer 3

Give an assignment to write a function or class, and give stub code that includes tests. Initially all the tests fail.

Let the students know that these and other similar tests will determine score of their assignment. Encourage students to write further tests if they want to know their code is correct.

Answer 4

The point of TDD, like many of the other answers have been saying, isn't really to test the code and find bugs, but to codify your requirements.

Having requirements that are codified is useful when you have a lot of requirements that need to be tested that all intersect each other (like in many industry projects), and when those requirements are going to change very often (which is one of the key assumptions of Agile).

It would be tricky to recreate the first condition in a school setting. Instead, lets focus on the second one: Throughout the project, introduce new requirements that overwrite and contradict earlier ones.

They might start with:

we need to do A, B and C.

Once they finish that, mix things up:

we need to do A, C, E and F. Instead of B we need to do D instead. And D shouldn't change how A and C work.

If they have good unit tests, it should be relatively easy to refactor and change their implementation. If they don't, they'll have a bad time trying to wade through their legacy code.

Answer 5

The first mistake teachers usually make is not changing the point of view:

Unittests should be recognizes as "verifies what my code does" and not as "verifies my code is right".

Unittests are executable specification. With your set of unittests you can prove which parts of the requirements you really implemented.

One way to teach that to students could be like this:

• Split you students in teams of two.
• give each team a separate task to implement. This task should be too big to be completed.
• force half of the teams to do TDD, let the others go without tests.
• at the last day let the teams change their results so that TDD teams get the task of another TDD team and the "no tests" teams also change their results.
• let the teams work on the inherited project for half a day.
• after haft of the day do the same change as in the morning but this time give the TDD teams a "no tests" project and vice versa.
• Let them start with the same project state as in the morning and complete this newly inherited project.

The lessons leaned will be:

• without the unittests you first have to learn what the code already does by manually testing it. This is some effort even for small projects.

• with unittest in contrast you can simply read what the code already does and start implementing new behavior right away.

Answer 6

I always have mixed feelings about TDD. If TDD is to emphasize the important to writing test code, yes by all means. But if TDD is to make you to write test code before implementation, NO I am totally against it.

There is a lot of exploratory & discovery in the development process and only when you actually implement something (through experiment and iteration) you will know what good tests should look like. To exaggerate my point a little bit, if my Function X should take a name and output a greeting with that name, then writing the test code beforehand for my Function X sounds pretty sweet. Anything more complicated than that will make writing test code beforehand quite pointless. You may also refer to Test-Driven Development is Fundamentally Wrong. There are only some rare occasions that I find writing test first may make sense, one of them is to implement a homegrown bitarray (one of my assignments to the student) because even I have not figured out how to implement it I know what bitarray should behave so I can write test code first.

But this is a site for teaching and my answer is not to argue with others. Part of reason I answered it is I saw your words "Even though the teachers invariably encourage it and present it as a useful strategy".

As a teacher I will first emphasize the importance of writing test, then introduce TDD. But I emphasize that I am personally against writing test before implementation, but there are also many people embrace that idea, you probably need to try it yourself to figure out whether that is for you or not.

One of teacher's job to make students's eye open. So I may even go further (not for high school students) to mention that even we all agree writing unit test is a must but there is a crazy company called Tictok that they don't really write unit test and they still makes a huge success, check here for the detailed "5 crazy things about working for Tiktok"(youtube video) or here, the text version . The point here is that there are many ways to do things and your job as a student to experience as much as possible.

--- update ---

After reading "Chrome, 10 years later" I got another surprise, just like when I watched that Tiktok video. It makes me think again is UT really that important?

Working with Apple.

In retrospect I enjoyed this a lot, for two reasons. One is that getting code review from experienced engineers is one of the best ways to grow as a programmer, and the Apple engineers came from a totally foreign engineering culture — no unit tests! no comments! — that also was clearly producing a high-quality product.

Answer 7

I'd like to add two notes. First is that if you write both the unit tests and the code, then all they test is that the code does what you think it is supposed to do, not what it is actually supposed to do. That is why Acceptance Tests are also used in agile development to test that the code does what the client/customer thinks it is supposed to do. With an imperfect understanding of requirements (or an incomplete test suite) you can pass all the tests and still not have a valid program. Also, if you write the tests after the code, then all they test is that the code does what the code does. That is a waste of time, of course, except, perhaps, for regression.

Second, in the context of a course, if you are lucky enough to be able to project a computer screen for all to see then you can spend part of a class period developing a small application (100 or so lines) in front of the class, perhaps with a student "navigator" showing how TDD actually works. Bit of test, bit of code, repeat.

You can also expect that the number of lines of code in tests exceeds the number of lines in the application itself. That is normal. But if you use an adequate programming environment it isn't a burden. Especially since something like Eclipse will fill in the application method skeletons from what it finds in the tests (assuming JUnit, at least). Think of the test framework as your low level design space, where you design the public interface of your classes.

Answer 8

I think students might fail to appreciate the benefits of TDD if they are working primarily alone. They probably just view it as an impediment to doing the "fun stuff", up-front grunt work they have to do before they can start coding.

Even in the real world many developers fail to appreciate the benefits of thinking about testing up front until they've been through a few projects and see the chaos that can ensue when insufficiently tested code gets deployed to upper environments and/or production.

In the full software lifecycle, time will be spent identifying bugs and fixing them. If you employ TDD and write test-driven code, you spend the time up front. If instead you just start hacking away, don't test your code, and throw it over to the testing group, time will still be spent finding and fixing bugs. It might not be spent by you but someone, somewhere will hopefully discover your bugs through testing, filing bugs, and fixing bugs. But it's usually more time effective to think about it up front. So while spending time doing TDD may seem annoying, it usually leads to overall time savings from the perspective of the entire project.

TDD in my opinion addresses two challenges that are present on almost every software project:

1) It's hard to collect requirements, document them, and turn them into test cases with a sufficient amount of detail that the software can be tested against actual requirements. While it may sound simple, many requirements are identified through conversations, meetings, industry research, ideas, etc. In a team environment, not everyone is aware of every requirement. And usually business people don't make good developers, developers don't make good testers, and testers don't make good business people, but all 3 groups have to work together. And nobody wants to write a super long and comprehensive requirements document and try to keep it up to date, and documentation tends to become obsolete immediately. Test driven development tries to address this by making every developer and team member part of the solution.

2) This last point is probably the most important point, many developers prefer only to do the "fun stuff" which is the coding, and have a tendency to assume that things like testing, accuracy, performance, integration, etc. are "someone else's problem". They can develop a mentality of writing code, doing limited testing, and then checking it in, assuming that someone else will somehow complete it. Some people say that this is the difference between being "done" and "done done". Software needs to be "done done" before it can work correctly but many developers only get it "done". TDD serves to address this problem by introducing a different mindset to the team, to make every developer aware that their code must be tested, and requires them to understand the context within which their code was developed and participate in all aspects of the software lifecycle and not just the coding.

Answer 9

When I was in industry, my boss said to me: this test driven development thing, [that we introduced recently] its real value is not that it finds bugs, or that we can re-factor without fear. The real value is that it saves us time, because it makes us think before writing the code.

How to encourage its use

For the project, give them code that is already weeks into development, complete with unit-tests. Then, have them add features to the code. Using TDD.

Answer 10

Any technique of reasonable complexity requires fluency and experience. People who use TDD/BDD are using it as one of the many tools. Starting a new project with these tools enabled should take a competent professional very little time. Minutes for small projects like utility libraries. So the idea that it takes "more time" is like arguing that riding a bike takes too long vs walking - if you don't know how to ride a bike efficiently and are just learning and are wobbly and falling a lot and don't feel safe. Same with TDD/BDD. For a novice it's just as big of a deal as learning to ride a bike. A year later they won't even remember why it was supposedly a big deal. It's just something you do.

Many programming environments make testing almost zero-effort. Say python doctests. Most Python IDEs let you run those tests from a keyboard shortcut, or a menu/toolbar control. Setup is minimal.

Tests also highlight how hard it may be to write bug-free software. Even "simple" library code may have serious bugs lurking within. If you don't test well, you'll only find them when things don't work or work flakily.