Kevin Swiber, of SIREN fame, just taught me a nifty trick for describing commands to a resource
Background: commands are semantically unsafe operations, as described in the HTTP specification. They are messages that expect to induce a change of state on the server, they invalidate caches, and so on.
In considering a design that conforms to the REST architectural style, it's useful to ask "how did we do it on the web?" We had URI, and HTTP, and HTML; those elements alone were enough to support catastrophic success.
Now, in HTML, the only readily available link for unsafe operations was the form, which supported only GET and POST. When there's only one road, you take it: servers would send HTML representations that would include Forms, that would describe to the client how to create the command message.
Notice that this is very much a hypermedia approach - if the server wishes to communicate that the command should not be sent, it simply removes the form from the representation. Elements that are no longer necessary can be removed from the form. New optional elements can be added by simply including a new input control and providing a reasonable default value.
What this teaches us is that a POST message with an application/x-www-form-urlencoded payload is a perfectly satisfactory way of modeling a command sent to a resource.
But I've got to admit, that doesn't feel much like "REST".
So here's another way of thinking about it.
HTTP Patch affords the modification of resources. We tend to think about the payload as a list of changes to make; but another way of thinking about it is that the payload describes a list of commands. For example, application/json-patch+json is a document that describes operations; add, remove, and so on.
First trick: any resource that supports PATCH application/json-patch+json can just as easily provide the same functionality via POST application/json-patch+json. The semantics of PATCH are more specific than those of POST, but they don't fundamentally conflict with each other.
Second trick: application/json-patch+json is just a representation of a sequence of operations; with those operations taken specifically from the vocabulary of manipulating a json document. We don't particularly want to be borrowing those semantics unless they happen to be a particularly good fit for our domain; we likely have our own names for operations, particular parameters, and so on. So we instead might choose our own patch document representation; and write it up so that clients can be coded against it.
Third trick: these media types don't have any magical properties; they are just rules for taking semantic meaning and encoding that meaning as bytes, and then reversing the process at the other end. We can use any byte representation we like, provided that the client and the server agree on the rules being used. We could use text/plain, or application/json. Alas, both of those representations suffer from the problem that we need additional out of band communication to ensure that both ends of the conversation understand the meaning the same way.
Fourth trick: if our command message can be represented as a collection of key/value pairs (likely the case, if we account for hierarchical keys), then application/x-www-form-urlencoding is another possible representation that we might use. It shares the same problem we just saw: the client and server need to agree on semantics. But with one important difference: we already possess a standard format with facilities for describing application/x-www-form-urlencoded representations in band: HTML.
So, yeah -- it's REST.
That still doesn't mean "easy"; the client still has to know what it wants to do, how to find the right form to do it, how to find the fields it wants to set before submitting the form. In much the same way that JSON Patch puts semantics on top of JSON, you need something that describes your domain semantics on top of HTML (or whatever other hypermedia representation you prefer).
An advantage in this approach is that it allows us to take advantage of the cache invalidation semantics of POST; which is to say, because we are sending the message to be handled by "the" resource, as opposed to some other resource that understands the semantics of a specific command, the caches are able to invalidate the representations that are actually being changed, rather than invalidating an irrelevant side channel.
Monday, January 15, 2018
Thursday, January 4, 2018
Property based testing: Mars Rover
My go to TDD exercise has been the mars rover kata. Trying a property test first approach to it has been a struggle. Compared with an example driven approach, there seems to be a whole lot of front loaded pain without an obvious payoff.
I reached out to Paul Holser to see if I was missing a bet, and of course the discussion turned to what sorts of properties I had discovered in the design.
First property: for all inputs, the output is an ordered collection of representations of rover positions. If we are checking within the domain boundary, this is pretty trivial -- we can use the type system to enforce this constraint. If we are checking from outside of the boundary, then we're checking that the lines of text/bytes that have been returned to us can be parsed; three entries per represe, the first two entries are coordinates/integers, the last entry is a member of the set of cardinal directions.
Second property: for all inputs, the number of representations in the output equals the number of rovers described by the input. Rovers are conserved.
Third property: for any inputs, you get the same output each time you run it.
Fourth property: for any pair of inputs, reversing the order of execution does not change the outputs.
In other words, its a pure function. I don't actually know the battery of tests that you should run to establish this fact.
A rover with an empty instruction set remains at the same coordinates in the same orientation. This gives us our fifth property: for all inputs, if we replace a single rover's instructions with an empty instruction set (thereby creating a different input), the corresponding output will have the same position. Note that this indirectly demonstrates that the order of the outputs matches the order of the inputs.
This property would not apply if the rovers moved other than by consuming their own instructions. For example, if one rover were to push another rover out of the way, then the property would not hold.
The above identity leads to two additional properties, both variations of different paths, same destination.
The sixth property takes advantage of the identity to allow us to establish that the behavior of the rovers is consistent with their programs being executed in the right order. So for all inputs, we can select any rover, and create a new program in the following way: the positions an instructions of the rovers before the selection in the list are copied directly, the positions of the remaining rovers are copied with empty instructions. We get the corresponding output, and then build a second program -- the output positions are taken as inputs, with empty instruction sets assigned to the rovers before the selected rover, and the original instruction sets assigned to the remainder. The property to be checked is that the output of this second program matches the original that we started with.
The seventh property is similar; instead of breaking up the original program along neat rover boundaries, we can also split the selected rover's instructions into two pieces, running part of the instruction set with the rovers ahead of it, then running the remaining instructions with the subsequent rovers.
Both of these are basically establishing that we're dealing with a stateless process; the next collection of positions depends only on the current collection of positions and the next instruction.
Up until this point, we haven't really been looking at the semantics of the output at all; we've got a list of coordinates and orientations, but all we've done is check for equality.
Putting this another way, the simplest thing that could possibly work is still to ignore the instruction sets entirely, and simply return the original positions and orientations in each case.
The plateau is rectangular, and presumably the boundary effects are equivalent on all sides. If you flip the rectangle upside down -- exchanging north for south and east for west -- you get the same configuration of rovers in a different coordinate system. Right and Left have the same effect on orientation that they did before the flip. Any interactions with the boundary will still appear at the same points in the evaluations of the rovers.
This yields our eighth property: for any input, if we flip the input position, compute the output, and then flip the output, we should get the same answer that we would get from running the original output as is.
There is a similar result for quarter rotations, if you adjust the input so that the plateau itself is square.
Rotations are mostly immune to boundary effects and collisions, so it's probably reasonable to start there.
One assumption that we probably need to make is that all valid inputs have rovers that start within the region of the specified plateau. The problem statement doesn't address that point. I don't think property based testing helps much here - running the checks can only tell you if the test and implementation made compatible assumptions, not correct assumptions.
Property nine: for all inputs, we can select any rover and replace it's original instruction set with one that has all of the move instructions removed. When we run this program:
This establishes that we've got four-symmetry, that left and right cancel each other out, and that rotations preserve position.
Property ten: for all inputs, we can select any rover and replace it's original instruction set with a one that elides all of the left and right instructions. When we run this program, the orientation of the selected rover will be unchanged.
EDIT: come to think of it, any single instruction leaves at least two of the three properties unchanged; the remaining property changes by at most one unit. The complication of the move instruction is that you need the orientation to know which property is changing in which direction.
We can introduce into this domain the concept of a displacement - we're in a taxicab geometry. The number of moves in an instruction set establishes an upper bound on the displacement that can be achieved; we know that the rover will be found somewhere in the circle.
Property eleven: for all inputs, we can measure the displacement of the output coordinates from the input coordinates. For each rover, the displacement will be less than or equal to the count of moves in its instruction set.
Furthermore, if the circle doesn't reach the boundary of the plateau, then we don't need to worry about boundary effects for that rover. Taking the initial positions of the rovers, and the size of the bounding circles allows us to compute a bounding box; if the box doesn't reach the plateau boundaries, then the simulation is completely free of boundary effects.
If the plateau is large enough to enclose the bounding box at two different locations, then we can establish twelfth property - that of translation symmetry in the following way. Given any valid input, we can compute the extents of a bounding box that encloses the possible programs of all rovers. Choose a displacement in positive x and y. Compute the dimensions of a plateau by adding the displacement to the extent of the bounding box. Create two inputs using this plateau: for the first, compute the initial positions of the rovers such that the lower left corner of the bounding box is at the origin; for the second, compute initial positions of the rovers such that the upper right corner of the bounding box is at the upper right corner of the plateau. Note that the relative displacment of the input coordinates of the corresponding rovers should be the same. Compute outputs for both programs; the displacements of the corresponding outputs should all match.
The thirteenth property also plays with bounding boxes to establish this property - that if the bounding circles of two rovers don't overlap, then they don't have any interference effects. The check goes something like this: first you run a single program with all of the rovers participating. Then you run a program that isolates each rover on the same plateau, and confirm that the final positions of the isolated rovers matches that predicted by the single program containing all of the rovers.
It's not clear how you transform input with overlapping bounding circles into one that doesn't have them. Do you move the rovers? shrink the bounding circles? skip examples that don't match the criteria?
But in the main, it seems to be a very powerful pattern to use an input as a seed from which you can compute an input that has the properties you want to verify -- using an identity transformation wherever possible.
It's not so clear to me that these constraints are driving the design in any useful way. For example, we don't have any properties that establish that left and right are correctly oriented. We don't have any tests that actually demonstrate that the rover has moved.
I reached out to Paul Holser to see if I was missing a bet, and of course the discussion turned to what sorts of properties I had discovered in the design.
First property: for all inputs, the output is an ordered collection of representations of rover positions. If we are checking within the domain boundary, this is pretty trivial -- we can use the type system to enforce this constraint. If we are checking from outside of the boundary, then we're checking that the lines of text/bytes that have been returned to us can be parsed; three entries per represe, the first two entries are coordinates/integers, the last entry is a member of the set of cardinal directions.
Second property: for all inputs, the number of representations in the output equals the number of rovers described by the input. Rovers are conserved.
Third property: for any inputs, you get the same output each time you run it.
Fourth property: for any pair of inputs, reversing the order of execution does not change the outputs.
In other words, its a pure function. I don't actually know the battery of tests that you should run to establish this fact.
A rover with an empty instruction set remains at the same coordinates in the same orientation. This gives us our fifth property: for all inputs, if we replace a single rover's instructions with an empty instruction set (thereby creating a different input), the corresponding output will have the same position. Note that this indirectly demonstrates that the order of the outputs matches the order of the inputs.
This property would not apply if the rovers moved other than by consuming their own instructions. For example, if one rover were to push another rover out of the way, then the property would not hold.
The above identity leads to two additional properties, both variations of different paths, same destination.
The sixth property takes advantage of the identity to allow us to establish that the behavior of the rovers is consistent with their programs being executed in the right order. So for all inputs, we can select any rover, and create a new program in the following way: the positions an instructions of the rovers before the selection in the list are copied directly, the positions of the remaining rovers are copied with empty instructions. We get the corresponding output, and then build a second program -- the output positions are taken as inputs, with empty instruction sets assigned to the rovers before the selected rover, and the original instruction sets assigned to the remainder. The property to be checked is that the output of this second program matches the original that we started with.
The seventh property is similar; instead of breaking up the original program along neat rover boundaries, we can also split the selected rover's instructions into two pieces, running part of the instruction set with the rovers ahead of it, then running the remaining instructions with the subsequent rovers.
Both of these are basically establishing that we're dealing with a stateless process; the next collection of positions depends only on the current collection of positions and the next instruction.
Up until this point, we haven't really been looking at the semantics of the output at all; we've got a list of coordinates and orientations, but all we've done is check for equality.
Putting this another way, the simplest thing that could possibly work is still to ignore the instruction sets entirely, and simply return the original positions and orientations in each case.
The plateau is rectangular, and presumably the boundary effects are equivalent on all sides. If you flip the rectangle upside down -- exchanging north for south and east for west -- you get the same configuration of rovers in a different coordinate system. Right and Left have the same effect on orientation that they did before the flip. Any interactions with the boundary will still appear at the same points in the evaluations of the rovers.
This yields our eighth property: for any input, if we flip the input position, compute the output, and then flip the output, we should get the same answer that we would get from running the original output as is.
There is a similar result for quarter rotations, if you adjust the input so that the plateau itself is square.
Rotations are mostly immune to boundary effects and collisions, so it's probably reasonable to start there.
One assumption that we probably need to make is that all valid inputs have rovers that start within the region of the specified plateau. The problem statement doesn't address that point. I don't think property based testing helps much here - running the checks can only tell you if the test and implementation made compatible assumptions, not correct assumptions.
Property nine: for all inputs, we can select any rover and replace it's original instruction set with one that has all of the move instructions removed. When we run this program:
- The coordinates of the output should match those of the input
- The orientation of the output will match that of the input if, and only if, the difference in the count of left instructions and the count of right instructions is a multiple of four.
This establishes that we've got four-symmetry, that left and right cancel each other out, and that rotations preserve position.
Property ten: for all inputs, we can select any rover and replace it's original instruction set with a one that elides all of the left and right instructions. When we run this program, the orientation of the selected rover will be unchanged.
EDIT: come to think of it, any single instruction leaves at least two of the three properties unchanged; the remaining property changes by at most one unit. The complication of the move instruction is that you need the orientation to know which property is changing in which direction.
We can introduce into this domain the concept of a displacement - we're in a taxicab geometry. The number of moves in an instruction set establishes an upper bound on the displacement that can be achieved; we know that the rover will be found somewhere in the circle.
Property eleven: for all inputs, we can measure the displacement of the output coordinates from the input coordinates. For each rover, the displacement will be less than or equal to the count of moves in its instruction set.
Furthermore, if the circle doesn't reach the boundary of the plateau, then we don't need to worry about boundary effects for that rover. Taking the initial positions of the rovers, and the size of the bounding circles allows us to compute a bounding box; if the box doesn't reach the plateau boundaries, then the simulation is completely free of boundary effects.
If the plateau is large enough to enclose the bounding box at two different locations, then we can establish twelfth property - that of translation symmetry in the following way. Given any valid input, we can compute the extents of a bounding box that encloses the possible programs of all rovers. Choose a displacement in positive x and y. Compute the dimensions of a plateau by adding the displacement to the extent of the bounding box. Create two inputs using this plateau: for the first, compute the initial positions of the rovers such that the lower left corner of the bounding box is at the origin; for the second, compute initial positions of the rovers such that the upper right corner of the bounding box is at the upper right corner of the plateau. Note that the relative displacment of the input coordinates of the corresponding rovers should be the same. Compute outputs for both programs; the displacements of the corresponding outputs should all match.
The thirteenth property also plays with bounding boxes to establish this property - that if the bounding circles of two rovers don't overlap, then they don't have any interference effects. The check goes something like this: first you run a single program with all of the rovers participating. Then you run a program that isolates each rover on the same plateau, and confirm that the final positions of the isolated rovers matches that predicted by the single program containing all of the rovers.
It's not clear how you transform input with overlapping bounding circles into one that doesn't have them. Do you move the rovers? shrink the bounding circles? skip examples that don't match the criteria?
But in the main, it seems to be a very powerful pattern to use an input as a seed from which you can compute an input that has the properties you want to verify -- using an identity transformation wherever possible.
It's not so clear to me that these constraints are driving the design in any useful way. For example, we don't have any properties that establish that left and right are correctly oriented. We don't have any tests that actually demonstrate that the rover has moved.
Tuesday, January 2, 2018
Property based testing: thoughts of a novice.
when i write tests, i do it because writing tests lets me make more features faster. that's the only reason.— Michael D. Hill (@GeePawHill) April 11, 2017
Writing tests first forces you to think about the problem you're solving. Writing property-based tests forces you to think way harder.— Jessica Kerr (@jessitron) April 25, 2013
The tension between these two ideas drives me nuts. Thinking way harder means that I'm not delivering features faster.
Example based testing is straight forward; choose an input, hard code the output, remove the duplication, repeat until you have no more examples that produce the wrong output. You may even be able to estimate the minimum required number of tests just by thinking about the cyclomatic complexity of the problem.
But this in turn means that you can't easily judge "complete" just from looking at the demonstrated examples. As Scott Wlaschin points out, a finite suite of example tests can be beaten by a pathological implementation that is fitted to the suite.
Property based tests handle this concern better -- they explore the domain, rather than just sampling it. That's a lie, of course; what property based tests are really doing is sampling a lot of the domain -- enough that the pathological fake is more expensive than just solving the problem.
My most startling test result, ever, came about from a property test which revealed that I had completely failed to recognize that the properties that I thought would hold were not consistent with the implementation I had chosen.
But it didn't come from randomly exploring the space, but rather choosing an example from the domain that I recognized was "weird". Fortunately, there were lots of weird values in the domain, and they all demonstrated that my implementation didn't support the property in question. So I got
"lucky" without having to write four billion tests.
I'm not at all sold on the idea of using a random input generator to find a needle in the haystack.
Friday, December 22, 2017
Injection: a refactoring exercise.
Yesterday, I needed to extend a program that had a broken design.
The existing program works fine; but it has hard wired into it an identifier for a specific database host. That host name is used during execution to create the URL string that is passed to JDBC, by merging the hard coded host with a configured database name.
An additional constraint is that I need to have the old and new versions of the program running in parallel, in the same environment. We're going to run the two reports in parallel for a time, and then decommission the legacy.
So, how to dig my way out? Here's the outline of how I did it....
As is often the case, I rely on Parnas -- the legacy implementation is the result of a decision that I want to change. Therefore, I begin by creating a boundary around that decision.
In this case, the code in question was computing a new string from the hard coded literal and a member variable. So I applied an "extract method" refactoring to give me a member function that promised to provide the string that I needed. The method simply returned the calculated string, as before.
I then created a new type; the configuration model already included DatabaseName as an explicit concept, but it didn't have an understanding for the URL. So I introduced a DatabaseUrl type, that was simply an alias for a string.
I then inserted that type into the middle of the method -- so we compute the string, use the string to create a new DatabaseUrl, then use the DatabaseUrl to yield the string to return.
Next, I apply extract method again, creating a function that would convert the DatabaseName to the DatabaseUrl.
Next, I take DatabaseUrl, which had been a local variable, and make it a private member of the object; it now has the same lifetime and scope as DatabaseName, the only different is that it is being initialized at a different point in the program.
Next, I copy the logic to initialize the DatabaseURL to the method where the DatabaseName was initialized. I now have a redundant copy, which I can remove -- the query that produces the string representation of the url when I need it is just a toString() on the member value
At this point, the only place where the DatabaseName is used in in the initializer, so I can demote that to a local variable in the method, and inline it away.
I don't think we should be using the old method on the API any more, so I mark it as deprecated.
Reviewing at this point -- through a sequence of refactorings, I've managed to extend the existing API, adding the new method I need to it, while at the same time ensure that my existing tests are exercising that code (because the implementation of the old API feeds into the new one).
The legacy implementation includes a factory, which when passed the configuration (captured within a Properties object) returns the interpreted configuration. So the new piece I need is a factory with the same signature, that understands the new configuration.
But before I can think about creating a new factory implementation, I have to address a second problem -- I was sloppy ensuring the integrity of the composition root. The factory I need to replace is allocated behind two different default constructors.
So I need to perform a second round of refactoring. The member variable for the factory that I need to replace is too specific, as it names the implementation directly. So the spelling of that type needs to be changed to the interface. Fortunately, the interface I need is already available, so I can just replace it.
Then we delete the assignment at the member variable declaration. At this point, everything goes red, because we aren't assigning a final field. Next, we fix the compilation problem by allocating a new instance of the factory in the constructor body.
Next, we work our way back up the stack to the composition root. There are a couple possibilities; you can either develop a completely new path from the root to the assignment so that the factory can be passed along, and then with everything wired up remove the duplication, or you can create new signatures, but play criss cross -- the existing implementation gets a new signature, and a new method appears which creates a default factory and then routes to the real implementation.
I prefer the latter course; you only really need to preserve the signatures that are part of the public API. Anything that is private to the module you can simply change, watching for local compile errors.
At the end of the game, I have the starting position I'm waiting for: me legacy implementation, with the creation of the default factory at the composition root.
Now I'm ready to make the easy change - implement the new factory, clone the composition root, and replace the legacy factory with the new implementation.
The existing program works fine; but it has hard wired into it an identifier for a specific database host. That host name is used during execution to create the URL string that is passed to JDBC, by merging the hard coded host with a configured database name.
An additional constraint is that I need to have the old and new versions of the program running in parallel, in the same environment. We're going to run the two reports in parallel for a time, and then decommission the legacy.
So, how to dig my way out? Here's the outline of how I did it....
for each desired change, make the change easy (warning: this may be hard), then make the easy change— Kent Beck (@KentBeck) September 25, 2012
As is often the case, I rely on Parnas -- the legacy implementation is the result of a decision that I want to change. Therefore, I begin by creating a boundary around that decision.
In this case, the code in question was computing a new string from the hard coded literal and a member variable. So I applied an "extract method" refactoring to give me a member function that promised to provide the string that I needed. The method simply returned the calculated string, as before.
I then created a new type; the configuration model already included DatabaseName as an explicit concept, but it didn't have an understanding for the URL. So I introduced a DatabaseUrl type, that was simply an alias for a string.
I then inserted that type into the middle of the method -- so we compute the string, use the string to create a new DatabaseUrl, then use the DatabaseUrl to yield the string to return.
Next, I apply extract method again, creating a function that would convert the DatabaseName to the DatabaseUrl.
Next, I take DatabaseUrl, which had been a local variable, and make it a private member of the object; it now has the same lifetime and scope as DatabaseName, the only different is that it is being initialized at a different point in the program.
Next, I copy the logic to initialize the DatabaseURL to the method where the DatabaseName was initialized. I now have a redundant copy, which I can remove -- the query that produces the string representation of the url when I need it is just a toString() on the member value
At this point, the only place where the DatabaseName is used in in the initializer, so I can demote that to a local variable in the method, and inline it away.
I don't think we should be using the old method on the API any more, so I mark it as deprecated.
Reviewing at this point -- through a sequence of refactorings, I've managed to extend the existing API, adding the new method I need to it, while at the same time ensure that my existing tests are exercising that code (because the implementation of the old API feeds into the new one).
The legacy implementation includes a factory, which when passed the configuration (captured within a Properties object) returns the interpreted configuration. So the new piece I need is a factory with the same signature, that understands the new configuration.
But before I can think about creating a new factory implementation, I have to address a second problem -- I was sloppy ensuring the integrity of the composition root. The factory I need to replace is allocated behind two different default constructors.
So I need to perform a second round of refactoring. The member variable for the factory that I need to replace is too specific, as it names the implementation directly. So the spelling of that type needs to be changed to the interface. Fortunately, the interface I need is already available, so I can just replace it.
Then we delete the assignment at the member variable declaration. At this point, everything goes red, because we aren't assigning a final field. Next, we fix the compilation problem by allocating a new instance of the factory in the constructor body.
Next, we work our way back up the stack to the composition root. There are a couple possibilities; you can either develop a completely new path from the root to the assignment so that the factory can be passed along, and then with everything wired up remove the duplication, or you can create new signatures, but play criss cross -- the existing implementation gets a new signature, and a new method appears which creates a default factory and then routes to the real implementation.
I prefer the latter course; you only really need to preserve the signatures that are part of the public API. Anything that is private to the module you can simply change, watching for local compile errors.
At the end of the game, I have the starting position I'm waiting for: me legacy implementation, with the creation of the default factory at the composition root.
Now I'm ready to make the easy change - implement the new factory, clone the composition root, and replace the legacy factory with the new implementation.
Sunday, November 5, 2017
TDD: Mars Rover
I took some time this weekend to practice the Mars Rover kata again.
My focus in the exercise was duplication - can we proceed from hard coded outputs, and arrive at a viable implementation simply by removing the duplication that must be present? Of course, the answer is yes -- duplication is duplication; you can remove it any time you like.
I shan't walk through the process here, it should be reasonably straight forward to follow along from the commit history.
With just the one example, I needed almost no investment in the test harness at all; all of my time was spent working in my production code. Setting aside the one error that forced me to revert, practically all of the work was in the green bar.
My guess is that the process went as smoothly as it did because my previous experiments in the kata act as spike solutions -- I'm not discovering how to achieve the result at the same time that I'm discovering how to remove the duplication. Removing the duplication is comfortable at each step, because I can see the code evolving towards a design that is already familiar.
This code is tightly coupled to the data representations that we were given for this problem. Do we need to invest now in the abstractions that would make it easier to change data representations later? YAGNI says, no, not really; and that's certainly true for a kata in a limited time box. Make the next change easy is much easier after you know what the next change is going to be.
Kent Beck talked a bit about this in his discussion of triangulation
Horses, of courses; but I find that answer unsatisfying.
I think it's closer to true to recognize that the tests aren't driving anything; they are instruments that measure whether or not our current implementation is satisfactory over some range of examples. In particular, they can give us awareness of whether a change we've made has certain classes of unintended side effects. They inform us, a little bit, about a possible interface.
With the instrumentation in place, we apply changes to the code in increments; not for the design, but because the short feedback cycle makes it easier to localize the errors we introduce along the way.
There are quite a few kata that progress toward a single implementation in increments, what I've come to think of as extending a particular set of requirements. I think we need more examples of a different sort, where we have to make something new of code originally written for another purpose.
My focus in the exercise was duplication - can we proceed from hard coded outputs, and arrive at a viable implementation simply by removing the duplication that must be present? Of course, the answer is yes -- duplication is duplication; you can remove it any time you like.
I shan't walk through the process here, it should be reasonably straight forward to follow along from the commit history.
With just the one example, I needed almost no investment in the test harness at all; all of my time was spent working in my production code. Setting aside the one error that forced me to revert, practically all of the work was in the green bar.
My guess is that the process went as smoothly as it did because my previous experiments in the kata act as spike solutions -- I'm not discovering how to achieve the result at the same time that I'm discovering how to remove the duplication. Removing the duplication is comfortable at each step, because I can see the code evolving towards a design that is already familiar.
This code is tightly coupled to the data representations that we were given for this problem. Do we need to invest now in the abstractions that would make it easier to change data representations later? YAGNI says, no, not really; and that's certainly true for a kata in a limited time box. Make the next change easy is much easier after you know what the next change is going to be.
Kent Beck talked a bit about this in his discussion of triangulation
Smaller bites makes sense, I think, when the way forward isn't clear. But the Feynman algorithm also works. So what does that teach us?Triangulation feels funny to me. I use it only when I am completely unsure of how to refactor. If I can see how to eliminate duplication between code and tests and create the general solution, then I just do it. Why would I need to write another test to give me permission to write what I probably could have written in the first place.
Horses, of courses; but I find that answer unsatisfying.
I think it's closer to true to recognize that the tests aren't driving anything; they are instruments that measure whether or not our current implementation is satisfactory over some range of examples. In particular, they can give us awareness of whether a change we've made has certain classes of unintended side effects. They inform us, a little bit, about a possible interface.
With the instrumentation in place, we apply changes to the code in increments; not for the design, but because the short feedback cycle makes it easier to localize the errors we introduce along the way.
There are quite a few kata that progress toward a single implementation in increments, what I've come to think of as extending a particular set of requirements. I think we need more examples of a different sort, where we have to make something new of code originally written for another purpose.
Sunday, October 29, 2017
Aggregate Semantics
First, we need an abstraction for encapsulating references within the model.This past week, I had a brief, on-line discussion with Yves Reynhout about repositories. He pointed out that there is a difference between persistence oriented repositories and collection oriented repositories.
A REPOSITORY represents all objects of a certain type as a conceptual set. It acts like a collection....
(Repositories) present clients with a simple model for obtaining persistent objects and managing their life cycle.The invocation of the repository might be as simple as
The key insight is that this is just a semantic; behind the interface, the domain model is free to choose its implementation.
We're not really -- at least, not necessarily -- "invoking a method on an entity in the model". What we are actually doing in this design is capturing two key pieces of information
- Which document in our data model are we trying to modify
- Which behavior in our domain model are we modifying that document with
This pattern bothered me for a long time -- we shouldn't "need" getters and setters on the domain objects, yet somehow we need to get the current state into and out of them. Furthermore, we've got this weird two phase commit thing going on, where we are mutating an entity in memory and the book of record.
A different analogy to consider when updating the model is the idea that we are taking a command -- sent to the domain model -- and transforming it into a command sent to the data. In other words, we pass the behavior that we want to the data model, saying "update your state according to these business rules".
Tell, don't ask.
Sunday, October 22, 2017
Test Driven Diamonds
I've been reviewing, and ruminating upon, a bunch of experiences folks have reported with the diamond kata. A few ideas finally kicked free.
George's answer is "back up and work in smaller steps", but I don't find that answer particularly satisfactory. I believe that there's something deeper going on.
And Kent Beck in turn replied
Big changes can be decomposed into many small changes. So if we are working in small increments, we are going to be running the tests a lot. Complex behaviors can be expressed as many simple behaviors; so we expect to have a lot of tests that we run a lot.
This is going to suck unless our test suite has properties that make it tolerable; we want the tests to be reliable, we want the interruption of running the tests to be as small as possible -- we're running the automated checks to support our real work.
One way to ensure that the interruption is brief is to run the tests in parallel. Achieving reliability with concurrently executing tests introduces some constraints - the behaviors of the tests need to be isolated from one another. Any assets that are shared by concurrently executing tests have to be immutable (otherwise you are vulnerable to races).
This constraint effectively means that the automated checks are pure functions; given-when-then describes the inputs: a deterministic representation of the initial configuration of the system under test, a message, a representation of the specification to be satisfied, and the output is a simple boolean.
This test function is in turn composed of some test boiler plate and a call to some function provided by the system under test.
Beck's ability to discover duplication has been remarked on elsewhere, and I suspect that this principle is a big part of it. Much of the duplication is implicit -- you need to dig to make the implicit explicit.
We've got a passing test, so before we proceed: where is the duplication? One bit of duplication is that the "A" that appears in the output is derived from the `A` in the input. Also, the number of lines in the output is derived from the input, as is the relative position of the A in that output.
There's nothing magic about the third test, or even the second, that removes the shackles - you can attack the duplication as soon as you have a green bar. All of the essential complexity of the space is included in the first problem, so you can immediately engage the design dynamo and solve the entire problem.
So when do we need more than one test? I think of the bowling game, where the nature of the problem space allows us to introduce the rules in a gradual way. The sequencing of the tests in the bowling game allows us to introduce the complexity of the domain in a controlled fashion - marks, and the complexities of extra throws in the final frame are deferred.
But it doesn't have to be that way - the mars rover kata gives you a huge slice of navigation complexity in the first example; once you have eliminated the duplication between the input and the output, you are done with navigation, and can introduce some other complexity (wrapping, or collisions).
Of course, if the chunk of complexity is large, you can easily get stuck. Uncle Bob touched on this in his write up of the word wrap kata; passing the next test can be really difficult if you introduce too many new ideas at once. Using a test order that introduces new ideas one at a time is an effective way to build up a solution. Being willing to suspend complex work to find a more manageable slice is an important skill.
And, in truth, I did the same thing in the rover; essentially pushing as far as I could with the acceptance test alone, then suspending that work in favor of a new test isolating the next behavior I needed, refactoring my solution into my acceptance work, and continuing on until I next got stuck.
In an exercise dedicated to demonstrating how TDD works, that's an important property for a problem to have: a rule set with multiple elements that can be introduced into the solution separately.
Of course, that's not the only desirable property; you also need an exercise that can be explained in a single page, one in which you can make satisfactory progress with useful lessons in a single session. That introduces a pressure to keep things simple.
But there's a trap: if you strip down the problem to a single rule (like the diamond, or Fibonacci), then you lose the motivation for the increments - the number of test cases you introduce before actually solving the problem becomes arbitrary.
On the other hand, if simply returning a hard coded value is good enough to pass all of our acceptance tests, then why are we investing design capital in refactoring the method? The details clearly aren't all that important to the success of the project yet.
My mental model right now is something along the ideas of a budget; each requirement for a new variation in outputs gives us more budget to actually solve the problem, rather than just half assing it. For a problem like Fibonacci, teasing out the duplication is easy, so we don't need many iterations of capital before we just solve the damn thing. Teasing the duplication out of diamond is harder, so we need a commensurately larger budget to justify the investment.
I found the approaches that used property based testing to be absolutely fascinating. For quite a while I was convinced that they were wrong; but my current classification of them is that they are alien.
In the incremental approach, we target a specific example which is complete, but not general. Because the example is complete, it necessarily satisfies all invariants that are common to all possible outputs. Any acceptance test that only needs the invariant satisfied will be able to get buy with this single solution indefinitely.
The property based approach seems to explore the invariant generally, rather than targeting a specific example. For diamond
For the mars rover kata, such an approach might look like paying attention first to the shape of the output; that each row of output describes a position and an orientation, perhaps a rule that there are no collisions in the grid, that all of the positions are located within the input grid, that there is an output row for each input row, and so on.
I don't think property based tests really help in this kata. But I think they become very interesting if the requirements change. To offer a naive example, suppose we complete the implementation, and the acceptance tests are passing, and then during the playback the customer recognizes that the requirements were created incorrectly. The correct representation of the diamonds should be rotated 90 degrees
With a property driven test suite, you get to keep a lot of the tests; you retire the ones that conflict with the corrected requirements, and resume iterating until the new requirements are satisfied? With the example based? approach, the entire test suite gets invalidated in one go -- except for the one trivial example where the changed requirements didn't change the outcome.
Actually, that's kind of interesting on its own. The trivial example allows you to fully express, by means of aggressively eliminating the duplication, a complete solution to the diamond problem in one go. But what it doesn't give you is an artifact that documents for your successor that the original orientation of the diamond, rather than its successor, is the "correct" one.
The more complicated example makes it easier to distinguish which orientation is the correct one, but is more vulnerable to changing requirements.
I don't have an answer, just the realization that I have a question.
Overview
- Seb Rose, 2014-11-23
- Alistair Cockburn, 2014-11-29
- George Dinwiddle, 2014-11-30
- Mark Seeman, 2015-01-10
- Nat Pryce, 2015-01-10
The usual approach is to start with a test for the simple case where the diamond consists of just a single ‘A’Alistair expressed the same conclusion more precisely
The next test is usually for a proper diamond consisting of ‘A’ and ‘B’; It’s easy enough to get this to pass by hardcoding the result.
We move on to the letter ‘C’: The code is now screaming for us to refactor it, but to keep all the tests passing most people try to solve the entire problem at once.
all the work that they did so far is of no use to them whatsoever when they hit ‘C’, because that’s where all the complexity is. In essence, they have to solve the entire problem, and throw away masses of code, when they hit ‘C’
Background
Long ago, I was trying to demonstrate simplest thing that could possibly work on the testdrivendevelopment discussion group. I had offered this demonstration:And Kent Beck in turn replied
Do you have some refactoring to do first?Sadly, I wasn't ready for that particular insight that day. But I come back to it from time to time, and am starting to feel like I'm beginning to appreciate the shape of it.
On Unit Tests
Unit tests are a check that the our latest change didn't break anything -- the behavior specifications that were satisfied before the change are still satisfied after the change.Big changes can be decomposed into many small changes. So if we are working in small increments, we are going to be running the tests a lot. Complex behaviors can be expressed as many simple behaviors; so we expect to have a lot of tests that we run a lot.
This is going to suck unless our test suite has properties that make it tolerable; we want the tests to be reliable, we want the interruption of running the tests to be as small as possible -- we're running the automated checks to support our real work.
One way to ensure that the interruption is brief is to run the tests in parallel. Achieving reliability with concurrently executing tests introduces some constraints - the behaviors of the tests need to be isolated from one another. Any assets that are shared by concurrently executing tests have to be immutable (otherwise you are vulnerable to races).
This constraint effectively means that the automated checks are pure functions; given-when-then describes the inputs: a deterministic representation of the initial configuration of the system under test, a message, a representation of the specification to be satisfied, and the output is a simple boolean.
This test function is in turn composed of some test boiler plate and a call to some function provided by the system under test.
Implications
... since every piece of matter in the Universe is in some way affected by every other piece of matter in the Universe, it is in theory possible to extrapolate the whole of creation — every sun, every planet, their orbits, their composition and their economic and social history from, say, one small piece of fairy cake.If we are, in fact, testing a function, then all of the variation in outputs must be encoded into the inputs. That's the duplication Beck was describing.
Beck's ability to discover duplication has been remarked on elsewhere, and I suspect that this principle is a big part of it. Much of the duplication is implicit -- you need to dig to make the implicit explicit.
Example
To show what I mean, let's take a look at the production code for the Moose team after the first passing test.We've got a passing test, so before we proceed: where is the duplication? One bit of duplication is that the "A" that appears in the output is derived from the `A` in the input. Also, the number of lines in the output is derived from the input, as is the relative position of the A in that output.
There's nothing magic about the third test, or even the second, that removes the shackles - you can attack the duplication as soon as you have a green bar. All of the essential complexity of the space is included in the first problem, so you can immediately engage the design dynamo and solve the entire problem.
So when do we need more than one test? I think of the bowling game, where the nature of the problem space allows us to introduce the rules in a gradual way. The sequencing of the tests in the bowling game allows us to introduce the complexity of the domain in a controlled fashion - marks, and the complexities of extra throws in the final frame are deferred.
But it doesn't have to be that way - the mars rover kata gives you a huge slice of navigation complexity in the first example; once you have eliminated the duplication between the input and the output, you are done with navigation, and can introduce some other complexity (wrapping, or collisions).
Of course, if the chunk of complexity is large, you can easily get stuck. Uncle Bob touched on this in his write up of the word wrap kata; passing the next test can be really difficult if you introduce too many new ideas at once. Using a test order that introduces new ideas one at a time is an effective way to build up a solution. Being willing to suspend complex work to find a more manageable slice is an important skill.
And, in truth, I did the same thing in the rover; essentially pushing as far as I could with the acceptance test alone, then suspending that work in favor of a new test isolating the next behavior I needed, refactoring my solution into my acceptance work, and continuing on until I next got stuck.
In an exercise dedicated to demonstrating how TDD works, that's an important property for a problem to have: a rule set with multiple elements that can be introduced into the solution separately.
Of course, that's not the only desirable property; you also need an exercise that can be explained in a single page, one in which you can make satisfactory progress with useful lessons in a single session. That introduces a pressure to keep things simple.
But there's a trap: if you strip down the problem to a single rule (like the diamond, or Fibonacci), then you lose the motivation for the increments - the number of test cases you introduce before actually solving the problem becomes arbitrary.
On the other hand, if simply returning a hard coded value is good enough to pass all of our acceptance tests, then why are we investing design capital in refactoring the method? The details clearly aren't all that important to the success of the project yet.
My mental model right now is something along the ideas of a budget; each requirement for a new variation in outputs gives us more budget to actually solve the problem, rather than just half assing it. For a problem like Fibonacci, teasing out the duplication is easy, so we don't need many iterations of capital before we just solve the damn thing. Teasing the duplication out of diamond is harder, so we need a commensurately larger budget to justify the investment.
Increments vs Iterations
I found the approaches that used property based testing to be absolutely fascinating. For quite a while I was convinced that they were wrong; but my current classification of them is that they are alien.
In the incremental approach, we target a specific example which is complete, but not general. Because the example is complete, it necessarily satisfies all invariants that are common to all possible outputs. Any acceptance test that only needs the invariant satisfied will be able to get buy with this single solution indefinitely.
The property based approach seems to explore the invariant generally, rather than targeting a specific example. For diamond
- The output is a square
- With sides of odd length
- With the correct length
- With top-bottom symmetry
- With left-right symmetry
- With the correct positions occupied by symbols
- With the correct positions occupied by the correct symbols
For the mars rover kata, such an approach might look like paying attention first to the shape of the output; that each row of output describes a position and an orientation, perhaps a rule that there are no collisions in the grid, that all of the positions are located within the input grid, that there is an output row for each input row, and so on.
I don't think property based tests really help in this kata. But I think they become very interesting if the requirements change. To offer a naive example, suppose we complete the implementation, and the acceptance tests are passing, and then during the playback the customer recognizes that the requirements were created incorrectly. The correct representation of the diamonds should be rotated 90 degrees
With a property driven test suite, you get to keep a lot of the tests; you retire the ones that conflict with the corrected requirements, and resume iterating until the new requirements are satisfied? With the example based? approach, the entire test suite gets invalidated in one go -- except for the one trivial example where the changed requirements didn't change the outcome.
Actually, that's kind of interesting on its own. The trivial example allows you to fully express, by means of aggressively eliminating the duplication, a complete solution to the diamond problem in one go. But what it doesn't give you is an artifact that documents for your successor that the original orientation of the diamond, rather than its successor, is the "correct" one.
The more complicated example makes it easier to distinguish which orientation is the correct one, but is more vulnerable to changing requirements.
I don't have an answer, just the realization that I have a question.
Subscribe to:
Posts (Atom)