I Pity The Fool Who Doesn't Write Unit Tests

J. Timothy King has a nice piece on the twelve benefits of writing unit tests first. Unfortunately, he seriously undermines his message by ending with this:

This is a companion discussion topic for the original blog entry at: http://www.codinghorror.com/blog/2006/07/i-pity-the-fool-who-doesnt-write-unit-tests.html

I too pity fools who don’t write unit tests. But writing the tests first? Not for me, too limiting!

During development, I like to keep the code as loose and maleable as possible. So often I find that I really need to play with code and experiment with strucuture and idioms, to make the code clear and concise. Sometimes its not clear how to do that until you are well into the development cycle. I often massively rip apart and rearrange lots of code because things I didn’t think of when first designing it became apparent much later.

I hate too much upfront design in the code, I think its unrealistic to rely on getting the design right with nothing concrete to validate assumptions. TDD creates all this inertia around the initial interfaces in the code, which start to affect the design. Or at least I think it does, I’ve never actually tried it. But that’s my fear.

And dammit, I just don’t want to…because I’m special and different. And you’re not the boss of me.

I do see tremendous value in unit tests, but I think TDD is going overboard. I could be wrong, who has time to try out every amazing new methodology?.

Hi, Jeff. I’m glad you wrote this. You are correct that I didn’t do much to sell test-first to developers who don’t do unit tests at all. Actually, I was writing primarily to developers who already do test-first. For the other group, you’re right that any step in the right direction is better than nothing. I myself did a lot of test-after-and-only-when-I-feel-like-it before I did test-first. And before that, I avoided automated tests completely.

So, yes, I’ve been there. I used to feel the way Damien does. And I tried a little testing, and a little more, and a little more, and now I’m a religious zealot. :slight_smile:

My one problem with test religion is that it can totally blind someone to design mistakes, or entirely unconsidered effects. For instance, if you don’t properly study the API of a library you’re using, or worse if it’s badly (or un-)documented, functions can have side effects that the caller has never even considered, and thus code that passes all the tests still fails on other input. (Fuzzers can help where unit tests aren’t sufficient.) Or the tester will simply never consider the possibility that a closed file handle (as a simple example) could be passed to the function. The permutations of failure cases can easily overrun the ability of tests to find them; what they can do is to keep a coder from forgetting a case they found six months ago, and document it for others who wouldn’t see it.

(As for design mistakes, I must not be the only one to catch myself thinking “it passed all the tests, so it must be a correct solution…” at some point.)

I’m with Damien. My coding process is a lot more unstructured and involves a lot of changes, code refactoring, deleting, reorganizing …

When I’ve tried test-first I find myself frustrated that I need to go back and “port” the tests to the new design. You could argue that I should be thinking about the design more before I start coding, but I find writing code, trying out little snippets and getting some real output helps me think through the problem in a way that I just can’t on paper or a whiteboard.

The times when test-first works is when I’m implementing something that’s already designed (porting a class from another language, implementing an algorithm, etc.) But that tends to be the exception.

The value is in having the tests. How you get them written is less important (IMO). Test-first/during/after? Its all good.

My focus in software development is always on economic value. Do you get more out of something than you put into it? For me (and each individual is, well, individual), it’s not paying off. I wrote the unit tests for a 400 line class using NUnit. I spent about equal time writing the class and writing/maintaining the unit tests. I could have done as good a job with less effort without NUnit.

I believe this is true for me because of my coding style. I’m very careful when I code, making sure each line of code is exactly what I want. Never guess, never assume, always look things up if not sure, write lots of little test programs. This makes me a slow coder compared to others, but I think it pays back in the end.

Hopefully, the unit testing technology will improve over time so that I can find economic value in unit testing.

Ironically, the first half of Damien’s post sounds like a TDDers description of why they TDD. Part of the beauty of unit tests is that you can refactor your code and still have confidence that it works. Otherwise, how do you know? Manual retesting? Stepping through the debugger? Also, using the tests to drive the design reduces the need for upfront design, not increases it.

One of the advantages of writing the tests first is that it guarantees that your code is testable. It’s far, far too easy to write a bunch of code, go back to write unit tests, and realized that everything is so tightly coupled that it’s extremely difficult to unit test. I’ve seen it time and time again.

My current project, we did a good amount of upfront requirements development and design – and have a good idea of what we’re doing. Usually we’re taking a day or so to code up what we wrote down, and then the unit tests, and that’s working pretty well.

"Part of the beauty of unit tests is that you can refactor your code and still have confidence that it works."
I think he was referring to low-level functions that may be radicaly changed or eliminated entirely and new ones created. Obviously a unit test is less than useless on each individual function in that case, but at a higher level you’re entirely correct, assuming you haven’t completely tossed out your API for a new one. If you go through a few iterations of that, TDD is just extra overhead guaranteed to waste your company’s time. (It’s like copy editing a first draft - you might not think of it that way because most professionals do their first few drafts in their head or on paper/meetings/email.)

In this case you would test with as much confidence as you have that code will last. Once you feel you’ve started to settle on a stable design, work testing through it.

It all comes down to how testing can fit into your - and your team’s - code-writing style (assuming your style is useful and produces timely, working code), not making your process more inefficient by forcing tests into every step. Happily marrying the two is always the best solution.

I’m a big fan of TDD, and I don’t feel you need to be an experienced unit test writer before you can start writing test-first. Certainly there are advantages to having unit tests, regardless of when they are written, but I feel the extra advantages of test-first are significantly greater. Case in point: just yesterday I was writing, well, a bunch of things - but one of them was a hashset of a new, small class. I wrote a test to put an item into the set twice, then wrote the obvious code to make it pass - and it failed. I had forgotten (this is Java, fwiw) to write a hashCode() function for my new class. A moment or two and it was working right. Here’s the thing. Writing test-after, I wouldn’t have bothered. I’ve got a set, I can put things into it, what could possibly go wrong? And when the bug surfaced, I’d have been blindsided - had a very hard time finding this stupid mistake. With TDD, I make (roughly) the same number of mistakes, but I catch them a lot sooner and they’re much easier to fix. It’s not a panacea, and not every programmer likes it, but it’s mighty good for some of us.

I’ve just recently started trying TDD on a new subproject in an existing large project. I like the idea and I’m seeing some benefit from it. I even found a bug in pre-existing code almost right away, though it’s something that would have turned up very quickly even without the unit tests, it was just code that hadn’t been used very much. But it was still a nice plus that I found it without ever having run the actual program, just the tests, and thus felt like a compile-time error.

The bigger problem I’m facing is that the program has a great deal of database and network access going on, and that stuff is hard to test standalone. To really do a lot of it effectively, there would have to be a whole test-only version of the database layer so that everything that’s calling into or out of it could be tested without an actual database connection. Writing that would be a large undertaking, and I am very skeptical that it would be worthwhile.

Nevertheless, I’m writing tests where I can, and often before code, but not always. Just about any metholodogy can be harmful when taken to extremes, and TDD seems no different. But there’s still a lot of good that can come from it, and a lot of good times to use it.

Actually I thought it was an atrocious article, full of sloppy thinking and riddled with condescension. These twelve supposed benefits of TDD range the gamut from ridiculously untrue (“unit tests prove that your code actually works”) to personal preferences (“it’s more fun to code with tests than without”), to unscientific generalizations from limited personal anecdotes or perhaps selected case studies (“it’s faster than writing code without tests”), to solutions to problems few people encounter (“it virtually eliminates coder’s block”). He lost me LONG before the last sentence.

No, I’m with James Bach. There are No Best Practices; a “Best Practice” is a rule devoid of context[1]. As Andy Hunt points out in his awesome presentation, “Herding Racehorses, Herding Sheep”[2], context-free rules are only the first stage towards holistic comprehension. They’re great for novices, but experienced practicioners violate these rules all time. In chess, for example, a beginner might learn “never lose a piece without winning a piece of equal or greater value” - and that piece of advice, if taken to heart, will improve their game a great deal. But it’s not a silver bullet. In fact, their chess ability will reach an early plateau if they don’t start learning the situations in which they NEED to sacrifice material in exchange for position.

When I hear someone say, “we should use TDD all the time”, it’s very clear to me that they’re a software engineering novice. They haven’t quite understood in what circumstances TDD works best, which circumstances it has no effect either way, and yes, in which circumstances it’s just a plumb waste of time.

Where does TDD shine? TDD works best in new “from scratch” development, in code which lends itself to well-defined, testable outputs (eg, text or API calls, not-eg UI or AI algorithms), in code which is algorithmically simple enough to run in sub-millisecond times, and in code which is largely self-contained (rather than dependent on a network, database, or large object model environment or outside framework).

Let’s face it. These are very limiting criteria, and there’s a lot of real-world code which can’t be shoehorned to fit this model. Don’t think that because TDD worked so well for you in your college project or cool little app that you wrote, that it’s the One True Way that All Development Must Be Done.

I have been written to write a developer test from time to time. Who hasn’t? Sometimes it’s a unit test, sometimes it’s an system integration test (testing the interfaction of several units). But I don’t write tests for everything. Like I said, some code I simply can’t test. Other code is so simple, I’d be wasting my time trying to test it. But beyond that, I’m in the practice of looking at the bugs that I myself, or my coworkers have written, and asking myself, “how could this have been avoided”? And the answer is almost never “a unit test would have caught that”. I just don’t see where the ROI is in banging out all these tests that don’t catch anything.

And quite frankly, the few merits TDD has (“unit tests make better designs”) is a level of hand-holding I outgrew long ago. I’ve learned (mostly from painful experience maintaining other people’s code) what code smells I should avoid. The question “how should this function be used?” comes far more naturally to me than “how should this function be implemented?”. I don’t need TDD to force me to think that way. I know to keep my objects small, to write referentially transparent code, to not write Rube Goldberg machines, to minimize my usage of if statements, to tell rather than ask, to avoid casting and pointer arithmetic, to avoid code which transfers ownership of objects, to use std::string and std::vector rather than fixed-sized C-style buffers, to avoid premature optimization and premature abstraction, to prefer composition and aggregation over inheritence, to be careful to distinguish between a count of bytes or a count of characters, and the zillion other things which practicioners of TDD eventually stumble upon.

For me, TDD zealots are just one of a long line of religious loonies knocking on my door, trying to prove that my way of doing things is irrepairably broken and the only path to salvation is Jesus, Mohammed, Krishna, Dianetics, or Unit Testing. And maybe there’s a good idea or two to be gleaned from all the mountains of bullshit. But you know what? Keep your crazy religion, 'cause I don’t need it.

[1] - http://www.rtpspin.org/Information/030327prs.ppt
[2] - http://www.satisfice.com/blog/archives/27

Yes, there’s a good reason if I ever heard one. When the customer comes into the meeting to ask for some silly feature he dreamt up, the proper answer is:

a. "That’s the dumbest thing I ever heard."
b. "While we could look into it, it would mean a lot of lost time and extra costs, and my gut tells me the benefit is not worth the cost."
c. “We have tests! Tests! One must not desecrate the holy tests!”

And if it really is a great idea you wish you’d thought of, you’ll still want to work it in or shuffle things around so it can be worked in cleanly in the future, anyway.

I’m with Alyosha. 100%. Unit tests are for beginners. Beyond that they’re a monumental waste of time and effort.

PS This is just an opinion. It’s mine and there’s no reason at all for it to be yours.

Alyosha said:
“And those people can do less with the wet-behind-the-ears TDD zealots with their condescension, their pity, their starry-eyed idealism and their “if you’ve never tasted the kool-aid you’ll never know what it’s like”.”

But it is still true, if you never try kool-aid, you will never know what it is like. Every now and then you do have to taste the kool-aid to experience new things.

I am a Coke drinker. Back in the day when they had those taste tests between Coke and Pepsi, I always knew which one was which. So, I would always go to the Pepsi ones, and choose Coke. Why? Because that was what I liked. I didn’t want to give those Pepsi drinkers the satisfaction of saying I liked Pepsi more.

To this day I only drink Coke. Does it limit my options? Yes, half the restraunts in the US serve Pepsi (might even be more than half). So, I am disappointed half the time. Maybe if I wasn’t so set in my ways, I might be able to appreciate Pepsi every now and then.

I think you are picking a very hard line stance like I do with my drink choice. I feel that if you opened up a little, you just might be able to appreciate the ‘other side’ more.

You are definately allowed your own opinion of TDD, but from some of the stuff you said, you may not have found the correct techniques that make it successful for a great many developers. The reason I feel this way is because of these comments:

"When I hear someone say, “we should use TDD all the time”, it’s very clear to me that they’re a software engineering novice.“
"Let’s face it. These are very limiting criteria, and there’s a lot of real-world code which can’t be shoehorned to fit this model. Don’t think that because TDD worked so well for you in your college project or cool little app that you wrote, that it’s the One True Way that All Development Must Be Done.”

So, it doesn’t matter if you do it or not. You may write some great code that doesn’t need testing. I can appreciate that. I don’t test everything, but I am moving to more testing in my development. This testing is in real-world line of business applications (inventory, accounting, etc). So to say it only works in college projects is completely false!

Hi Jeff

I read a lot of your articles in CodeProject like:

Reading your topic about Unit Testing, I ask: Do you unit test asp.net applications? Do you use NUnitAsp? Or what?

Fabio Alves

Sadly, it’s the “senior” developers that are often most resistent to trying new techniques like TDD - years of doing things a certain has them set in their ways.

The fact is, there’s very little code that can’t be test-driven in some way. I’ve test driven complex web UIs, database code, OLAP code, Web Services. They may be trickier to get a handle on at first, but once you learn some tricks of the trade, it’s totally doable. Maybe your coverage won’t be quite as good, but as Jeff says, some tests are better than no tests. If not for you, then for the next guy who has to take over your work.

It’s amazing how often your here “TDD doesn’t work”, followed shortly thereafter by “oh, but I’ve never tried it”. Unfortunately, TDD is one of those things that you often can’t convince someone the value of until they do try it - people are very attached to the way they are used to doing things, and the advantages of TDD can be subtle and difficult to appreciate without trying it out.

Foxy - while I generally don’t think of unit tests as a “customer” specification (that’s what customer acceptance tests are for), unit tests are great specifications about what the expected behavior of the code is.

Have you ever inherited a bit of code, gone to change it, and realized you don’t actually know what the repercussions of your change would be? Having a battery of unit tests to validate your change is immensely valuable.


Not to answer for Jeff, but if you use the Model View Presenter pattern for testing, your UI should have almost no code in it that needs testing. Just a little info for you if you don’t know about it.

Jeremy Miller is a pretty good resource for TDD. Here is a post on the MVP pattern for ASP.Net:

Sorry, but Jeremy Millder is totally, totally out to lunch. He either works on extremely simple web apps, or he doesn’t know what the hell he’s talking about. Either way, it is in fact possible to have a view that is a little bit more than trivial and needs to be tested. The totally contrived example he provides is near useless.