TDD: When the API changes

If I discover my API is bad after writing 60 tests, I have to change a lot! -- shared by James Grenning

My experience with the TDD literature is that it tends to concentrate of situations that are fixed.  Here's a problem, implement a solution in bite sized steps, done.

And to be fair, in a lot of cases -- where the implications of "bite sized steps" are alien, that's enough.  There's a lot going on, and eliminating distractions via simplification is, I believe, a good focusing technique.  Red-Green-Refactor is the "new" idea, and we want to keep that centered until the student can exercise the loop without concentration.

But -- if we intend that TDD be executed successfully outside of lessons -- we're going to need to re-introduce complicating factors so that students can experience what they are going to face in the wild.

The implications of a design change after a large number of tests have been introduced in one of those things.

Let's consider an idealized case (simplification again!)  We've got 60 tests written against one interface, and we have now learned that the interface used by those tests is inadequate.  Now what?

We can start over, of course -- but that loses a bunch of the work that we have already done.  We can change the test - but that's a lot of busywork, a distraction from delivering value to the customer.  Is there another option?

What I've found to be the easiest answer is to introduce an extra layer of indirection.  In the easiest case, the old API simply forwards the work to the new API.  In effect, we introduce the API using the same techniques that we would use to ensure that the new work is backwards compatible with the old (test) clients.

This is really easy when the interface between the test and the subject is simple; when we have a simple function, it is straight forward to redirect the work to a new function, perhaps with additional default arguments that we had not considered when writing the first 60 tests.

This is more complicated when there is a lot of back and forth interaction between the test and the subject - when we are dealing with a protocol - a rich contract provided by the test subject, which the test touches multiple times.

One interpretation of Test Driven Design: that we make easy testing (affordances) first class requirements in our designs.

So, it if it complicated to test things with one design, and easier with the other, then we should be biasing our design decisions toward the latter.

In my work, what this typically looks like is a test design (Michael Bolton and James Back would say check design.  They are correct, of course, but I'm not ready to lead a check-driven-design crusade today) where the measurement (what behavior do I see when I...) and the assertion are separate ideas.  I pass values to some function to get a measurement (the "actual" value) and then compare that measurement against a specification (the "expected" value).

The measurement interface always looks like a function, so it affords an easy change to the API, even when the act of taking the measurement is complicated - the complexity is abstracted away from the description of the desired behavior, so we can change from the old API to the new API in bulk.

In other words, when I'm 60 tests deep into a design and discover a flaw in my API, I still have only a manageable number of measurements to change.

An important implication: my tests are not the demonstration of how to use my API.  The tests describe the behaviors, but the exampleware is the measurement.  I find this separation actually makes things clearer (we are co-mingling two different documentation concerns), but it can catch people by surprise if they are accustomed to one big block of test.

Also, it is still true that incompatible changes to a published interface are a headache (as they should be).  Our test code doesn't typically discriminate between published and unpublished test subjects, so we need to be a little bit careful about "just" porting our old tests to the new API (via a new measurement) when in fact we still have obligations to maintain the old API.

If you refactor your tests, aggressively identifying duplication and taking steps to manage it, you may find that (a) measurements fall out naturally and (b) introducing changes to an unpublished interface are low impact, because your test suite is a many examples of behavior using a small number of measurements.

My experiences so far suggest that "property based" testing has the same basic shape -- many examples get condensed into a small number of properties, but we still see properties that share measurements.