Why I Practice TDD

TL; DR: I practice because I make mistakes; each mistake I recognize is an opportunity to learn, and thereby not make that mistake where it would be more expensive to do so.

Last night, I picked up the Fibonacci kata again.

I believe it was the Fibonacci kata where I finally got my first understanding of what Kent Beck meant by "duplication", many years ago.  So it is something of an old friend.  On the other hand, there's not a lot of meat to it - you go through the motions of TDD, but it is difficult to mine for insights.

Nonetheless, I made three significant errors, absolute howlers that have had me laughing at myself all day.

Error Messages

Ken Thompson has an automobile which he helped design. Unlike most vehicles, it has neither a speedometer, nor gas gauge, nor any of the other numerous idiot lights which plague the modern driver. Rather, if the driver makes a mistake, a giant "?" lights up in the center of the dashboard. "The experienced driver," says Thompson, "will usually know what's wrong."

In the early going of the exercise, I stopped to review my error messages, writing up notes about the motivations for doing them well.

Mid exercise, I had a long stare at the messages.  I was in the middle of a test calibration, and I happened to notice a happy accident in the way that I had made the test(s) fail.

But it wasn't until endgame that I finally discovered that I had transposed the expected and actual arguments in my calls to the Assertions library.

The contributing factors -- I've abandoned TestNG in favor of JUnit5, but my old JUnit habits haven't kicked back in yet.  While all the pieces are still fresh in my mind, I don't really see the data.  I just see pass and fail, and a surprise there means revert to the previous checkpoint.

Fence Posts

The one really interesting point in Fibonacci is the relationship between a recursive implementation and an iterative one.  A recursive implementation falls out pretty naturally when removing the duplication (as I learned from Beck long ago), but as Steve McConnell points out in Code Complete: recursion is a really powerful technique and Fibonacci is just an awful application of it.

What recursion does give you is a lovely oracle that you can use to check your production implementation.  It's an intermediate step along the way.

In "refactoring" away from the recursive implementation, I managed to introduce an off-by-one error in my implementation.  As a result, my implementation was returning a Fibonacci number, just not the one it was supposed to.

Well that happens, right? you run the tests, they go red, and you discover your mistake.

Not. So Much.

I managed to hit a perfect storm of errors.  At the time, I had failed to recall the idea of using an oracle, so I didn't have that security blanket.  I went after the recursion as soon as it appeared, so I was working in the neighborhood of fibonacci(2), which is of course equal to fibonacci(1).

My other tests didn't catch the problem, because I had tests that measured the internal consistency
of the implementation, rather than independently verifying the result.  One of the hazards that comes about from testing an implementation "in the calculus of itself."  Using property tests was "fine", but I needed one more unambiguous success before relying on them.

Independent Verification

One problem I don't think I've ever seen addressed in a Fibonacci kata is integer overflow.  The Fibonacci series grows roughly geometrically.  Java integers have a limited number of bits, so ending up with a number that is too big is inevitable.

But my tests kept passing - well past the point where my estimates told me I should be seeing symptoms.

The answer?  In Java, integer operators do not indicate overflow or underflow in any way. JLS 4.2.2

And that's just as true in the test code as it is in the production code.  In precisely the same way that I was only checking the internal consistency of my own algorithm, I also fell into the trap of checking the internal consistency of the plus operator.

There are countermeasures one can take -- asserting a property that the returned value must be non-negative, for instance.

Conclusions

One thing to keep in mind is that production code makes a really lousy test of the test code, especially in an environment where the tests are "driving" the implementation.  The standard for test code should be that it obviously has no deficiencies, which doesn't leave much room for creativity.