I haven't seen their API, but by all accounts it is terrible.
However, I have seen the bike shed they painted, and _it_ is terrific.
Tuesday, February 28, 2017
Shedding
Sunday, February 19, 2017
TDD Probability Theory
I recently recommended that a friend review the probability kata; which got me back into it again.
The probability kata was published by Greg Young in June, 2011. The core of the problem statement was
Well, that certainly seems straight forward enough....
WARNING: Spoilers ahead
The probability kata was published by Greg Young in June, 2011. The core of the problem statement was
Write a probability value object. It should contain the following methods:
- Probability CombinedWith(Probability)
- Probability Either(Probability)
- Probability InverseOf()
Well, that certainly seems straight forward enough....
WARNING: Spoilers ahead
Sunday, February 5, 2017
Aggregates and RFC 2119.
The language used to describe the relationship between aggregates and commands is a confusing one.
The usual language is that the aggregate protects the business invariant -- one can reasonably read such at thing, and come away with the idea that aggregates are going to be throwing some flavor of DomainException when a command would break the rules.
Fortunately, we have experts trying to offer guidance.
Commands should not fail in collaborative domainI struggled with understanding this for quite a while, because it sounded to me as though he was talking about checking that the command is valid before dispatching it to the domain.
The language that turned my brain around, was to consider that the aggregate is responsible for restoring the business invariant.
Udi Dahan, observing that Race Conditions Don't Exist, observed:
A microsecond difference in timing shouldn’t make a difference to core business behaviors.Reordering two commands should not change the behavior that you observe. In particular, you shouldn't be rejecting a command that you would accept if the ordering were different.
This turned up recently in a calendar domain, where the aggregate is responsible for ensuring that events don't overlap. So let us imagine an attempt to schedule two conflicting events, A and B. In a naive implementation, the order used to process the commands will determine how the conflict is resolved.
This is an interpretation of the invariant that is consistent with MUST or MUST NOT from RFC 2119. And that's fine, if that's what the business really needs. But it's not the only way to interpret a business invariant.
This model of the domain is analogous to SHOULD or SHOULD NOT. We don't want conflicts to happen, but if we do the aggregate has the responsibility of tracking the conflict and whether or not it has been resolved. In a sense, the responsibility of the aggregate is to detect and track conflicts -- maintaining the schedule is a side effect.
To reduce the rate of conflicts, the caller is expected to make a good faith effort to ensure that there are no conflicts before dispatching the command. After all, if the aggregate is tracking events and conflicts, then the caller can check for both before dispatching the command. When the data that the caller is working from is stale, that judgment may be off, but the aggregate has a fail safe available to cover that contingency.
A similar example might occur in banking, where instead of rejecting a transaction, the bank instead invokes an overdraft contingency.
If your requirements include a MUST NOT, then push back and check: what is the cost to the business of allowing the behavior to occur? Especially in business that run on human input, the processes that run the business are designed to mitigate these kinds of problems. We can do the same thing in code.
Because SHOULD NOT is going to have much nicer scaling properties.
Thursday, January 26, 2017
A RESTful Kitchen Oven
Some time back, I chatted with Asbjørn Ulsberg about an example of a REST api using an oven. Shortly there after, he presented his conclusions at the Nordic APIs 2016 Platform Summit. [slides]
What follows is my own work, but clearly I was influenced by his point of view.
In my kitchen, there is a free standing gas range. On the inside, it's got various marvels that we no longer think about much: an ignition system, and a thermostat, safety valves, etc.
But as a home cook, I really don't need to worry about those details at all. I work with the touch pad at the top of the unit. Bake, plus-plus-plus-plus, Start, and I get a beep and a display message letting me know that the oven is preheating. Some time after that, another beep let's me know that the oven has reached the target temperature and on good days the actual temperature stays near that target until the oven is shut off.
Let's explore, for a time, what it would look like to control the oven from a web browser.
For our first cut, we can look at an imperative approach.
HTTP/1.0 gave us everything we need. GET allows us to retrieve the information identified by the request uri, and POST allows us to provide a block of data to a data handling process.
In a browser, it might look like this: we load the bookmark URI. That would get some information resource -- perhaps a menu of the services available, perhaps a representation of the current state of the stove, maybe both. One of the links would include a relation (as well as human readable cues) that communicates that this is the link to follow to set the oven temperature. Following that link would GET another resource that is a web form; in this case it would just be an annotated field for temperature (set with the default value of 350F), semantic cues, and a submit button. Submitting the form would post the contents to the server, which would in turn interpret the submitted document as instructions to present to the oven. In other words, the web server reads the submitted data, and pushes the control buttons for us.
Having done that, the server would return a 200 status to us, with a representation of the action it just took, and links from there to perhaps the status page of the oven, so that we can read updates to know if the oven has reached temperature. In a simple interface like the one on my stove, the updates will only announce that the oven is preheating. A richer interface might include the current temperature, an estimate of the expected wait time, and so on.
Great, that gets us a website that allows a human being to control the stove from a web browser, but how do we turn that into a real API? Answer: that is a real API. Anybody can grab their favorite http client and their favorite html parser, write an agent that understands the link relations and form annotations (published as part of this API). The agent uses the client to load the bookmark url, loads the result into the parser, searches the resulting DOM for the elements it needs, submits the form, and so on. Furthermore, the agent can talk to any stove at all.
And -- bonus trick: if you want to test the agent, but don't have a stove handy, you can just point it at any web server with test cases represented as graphs of html documents. After all, the only URI that the agent actually knows anything about is the start point. From that point on, it's just following links.
That's a nice demonstration of hypermedia, and the flexibility that comes from using the uniform interface. But it misses two key lessons, so let's try again.
This time, we'll go with a declarative approach. We need another verb, PUT, defined in the HTTP/1.1 spec (although it had appeared in the quarantine of the earlier appendix D). Puts early definition got right to the heart of it.
To the oven, it means nothing -- but we aren't implementing an oven, we're implementing a web api for an oven. To be a web-api for something interesting means to express the access to that bit of interesting as though it were a document store.
In other words, the real goal here is that we should be able to control the stove from any HTTP aware document editor.
Here's how that might work. We start from the bookmark page as we did before. This includes a hypermedia link to a representation of the current temperature settings of the oven. For instance, that representation might be a json document which includes a temperature field somewhere in the schema. So the document can GET the current state represented as a json document (important note: this representation does NOT include hyperlinks -- we're not going to allow the document editor to modify those. Instead, transitions away from the editable representation of the document are described in the Link header field.)
We load the document of the current state into our editor, which then follows the "self" relation with an OPTIONS call to discover if PUT is supported. Discovering that this is so, the editor enables the editing controls. The human operator can now replace the representation of the current temperature of the oven with a representation of the desired temperature, and click save. The document editor does a simple PUT of the saved representation to the web server.
PUT is analogous to replace, in this use case. In the case of a document store, we would simply overwrite the existing copy of the document. We need to give some thought to what it means for an oven to mimic that behavior.
One possibility is that the resource in question represents, not the current *state* of the oven, but the current *program*. So we initially connect to the oven the state of the program would be do nothing, and we would replace that with a representation of the program "set the temperature to 300 degrees", and the API would "store" that program by actually commanding the oven to heat to the target temperature. PUT of a new program actually matches very well with the semantics of my oven, which treats entering the desired state as an idempotent operation.
A separate, read-only, resource would be used to monitor the current state of the oven.
In this idiom, "turning off the oven" has a natural analog for our document editor -- deleting the document. The editor can just as easily determine that DELETE is supported as PUT, and enable the property controls for the user.
If we don't like the "update the program" approach, we can work with the current state document directly. We enable PUT on the resource for the editable copy of the current oven state, enabling the edit controls in the document editor. The agent can describe the document that they want, and submit the result.
The tricky bit, that is key to the declarative approach: the API needs to compare the current state with the target state, and determine for itself what commands need to be sent to the oven to bring about that change. Just as the keypad insulates us from the details of the burners and valves, so to does the API insulate the consumers from the actual details of interacting with the oven.
Reality ensues: the latency for bringing the oven up to temperature isn't really suitable for an HTTP response SLA. So we need to apply a bit of lateral thinking; the API reports to the document store that the proposed edit has been Accepted (202). It's not committal, but it is standard. The response would likely include a link to the status monitor, which the client could load to see that the oven was preheating.
Once again, automating this is easy -- you teach your agent how to speak oven (the standard link relations, the media-types that describe the states and the programs). You use any HTTP away document editor to publish the agents changes to the server. You test by directing the agent to a document store full of fake oven responses.
Do we need two versions of the API now? one for an imperative approach, another for the declarative approach? Not at all -- the key is the bookmark URL. We require that the agents be forgiving about links they do not recognize -- those can simply be ignored. So on the bookmark page, we have a link relation that says "this way to the imperative setOvenTemperature interface", and another that says "this way to the declarative setOvenTemperature interface", and those two links can sit next to each other on the page. Clients can follow whichever link they recognize.
Document editing -- especially for small documents -- is reasonably straight forward in html as well; the basic idea is that we have a resource which renders the current representation of our document inside a text area in a form, which POSTs the modified version of the document to a resource that interprets it as a replacement state or a replacement program as before.
You can reasonably take two different approaches to enabling this protocol from the bookmark page. One approach would be to add a third link (inventing a new link relation). Another would be to use content negotiation -- the endpoint of the setOvenTemperature interface checks to see if the Accept-Type is html (in which case the client is redirected to the entry point of that protocol, otherwise directed back to the previously implement PUT based flow).
Using the HTML declarative flow also raises another interesting point about media types. Text areas aren't very smart, they support free form text, so you end up relying on the operator not making any data entry errors. With a standardized media-type, and a document editor that is schema aware, the editor can help the operator do the right thing. For instance the document editor may be able to assist the agent with navigation hints and a document object model, allowing the agent to seek directly to the document elements relevant to its goal.
Edit: go figure -- having written up my thoughts, I went back to look at Asbjørn Ulsberg's slides and realized that we had originally been talking about toasters, not ovens.
What follows is my own work, but clearly I was influenced by his point of view.
In my kitchen, there is a free standing gas range. On the inside, it's got various marvels that we no longer think about much: an ignition system, and a thermostat, safety valves, etc.
But as a home cook, I really don't need to worry about those details at all. I work with the touch pad at the top of the unit. Bake, plus-plus-plus-plus, Start, and I get a beep and a display message letting me know that the oven is preheating. Some time after that, another beep let's me know that the oven has reached the target temperature and on good days the actual temperature stays near that target until the oven is shut off.
Let's explore, for a time, what it would look like to control the oven from a web browser.
For our first cut, we can look at an imperative approach.
HTTP/1.0 gave us everything we need. GET allows us to retrieve the information identified by the request uri, and POST allows us to provide a block of data to a data handling process.
In a browser, it might look like this: we load the bookmark URI. That would get some information resource -- perhaps a menu of the services available, perhaps a representation of the current state of the stove, maybe both. One of the links would include a relation (as well as human readable cues) that communicates that this is the link to follow to set the oven temperature. Following that link would GET another resource that is a web form; in this case it would just be an annotated field for temperature (set with the default value of 350F), semantic cues, and a submit button. Submitting the form would post the contents to the server, which would in turn interpret the submitted document as instructions to present to the oven. In other words, the web server reads the submitted data, and pushes the control buttons for us.
Having done that, the server would return a 200 status to us, with a representation of the action it just took, and links from there to perhaps the status page of the oven, so that we can read updates to know if the oven has reached temperature. In a simple interface like the one on my stove, the updates will only announce that the oven is preheating. A richer interface might include the current temperature, an estimate of the expected wait time, and so on.
Great, that gets us a website that allows a human being to control the stove from a web browser, but how do we turn that into a real API? Answer: that is a real API. Anybody can grab their favorite http client and their favorite html parser, write an agent that understands the link relations and form annotations (published as part of this API). The agent uses the client to load the bookmark url, loads the result into the parser, searches the resulting DOM for the elements it needs, submits the form, and so on. Furthermore, the agent can talk to any stove at all.
And -- bonus trick: if you want to test the agent, but don't have a stove handy, you can just point it at any web server with test cases represented as graphs of html documents. After all, the only URI that the agent actually knows anything about is the start point. From that point on, it's just following links.
That's a nice demonstration of hypermedia, and the flexibility that comes from using the uniform interface. But it misses two key lessons, so let's try again.
This time, we'll go with a declarative approach. We need another verb, PUT, defined in the HTTP/1.1 spec (although it had appeared in the quarantine of the earlier appendix D). Puts early definition got right to the heart of it.
What the heck does that mean for an oven?The PUT method requests that the enclosed entity be stored under the supplied Request-URI.
To the oven, it means nothing -- but we aren't implementing an oven, we're implementing a web api for an oven. To be a web-api for something interesting means to express the access to that bit of interesting as though it were a document store.
In other words, the real goal here is that we should be able to control the stove from any HTTP aware document editor.
Here's how that might work. We start from the bookmark page as we did before. This includes a hypermedia link to a representation of the current temperature settings of the oven. For instance, that representation might be a json document which includes a temperature field somewhere in the schema. So the document can GET the current state represented as a json document (important note: this representation does NOT include hyperlinks -- we're not going to allow the document editor to modify those. Instead, transitions away from the editable representation of the document are described in the Link header field.)
We load the document of the current state into our editor, which then follows the "self" relation with an OPTIONS call to discover if PUT is supported. Discovering that this is so, the editor enables the editing controls. The human operator can now replace the representation of the current temperature of the oven with a representation of the desired temperature, and click save. The document editor does a simple PUT of the saved representation to the web server.
PUT is analogous to replace, in this use case. In the case of a document store, we would simply overwrite the existing copy of the document. We need to give some thought to what it means for an oven to mimic that behavior.
One possibility is that the resource in question represents, not the current *state* of the oven, but the current *program*. So we initially connect to the oven the state of the program would be do nothing, and we would replace that with a representation of the program "set the temperature to 300 degrees", and the API would "store" that program by actually commanding the oven to heat to the target temperature. PUT of a new program actually matches very well with the semantics of my oven, which treats entering the desired state as an idempotent operation.
A separate, read-only, resource would be used to monitor the current state of the oven.
In this idiom, "turning off the oven" has a natural analog for our document editor -- deleting the document. The editor can just as easily determine that DELETE is supported as PUT, and enable the property controls for the user.
If we don't like the "update the program" approach, we can work with the current state document directly. We enable PUT on the resource for the editable copy of the current oven state, enabling the edit controls in the document editor. The agent can describe the document that they want, and submit the result.
The tricky bit, that is key to the declarative approach: the API needs to compare the current state with the target state, and determine for itself what commands need to be sent to the oven to bring about that change. Just as the keypad insulates us from the details of the burners and valves, so to does the API insulate the consumers from the actual details of interacting with the oven.
Reality ensues: the latency for bringing the oven up to temperature isn't really suitable for an HTTP response SLA. So we need to apply a bit of lateral thinking; the API reports to the document store that the proposed edit has been Accepted (202). It's not committal, but it is standard. The response would likely include a link to the status monitor, which the client could load to see that the oven was preheating.
Once again, automating this is easy -- you teach your agent how to speak oven (the standard link relations, the media-types that describe the states and the programs). You use any HTTP away document editor to publish the agents changes to the server. You test by directing the agent to a document store full of fake oven responses.
Do we need two versions of the API now? one for an imperative approach, another for the declarative approach? Not at all -- the key is the bookmark URL. We require that the agents be forgiving about links they do not recognize -- those can simply be ignored. So on the bookmark page, we have a link relation that says "this way to the imperative setOvenTemperature interface", and another that says "this way to the declarative setOvenTemperature interface", and those two links can sit next to each other on the page. Clients can follow whichever link they recognize.
Document editing -- especially for small documents -- is reasonably straight forward in html as well; the basic idea is that we have a resource which renders the current representation of our document inside a text area in a form, which POSTs the modified version of the document to a resource that interprets it as a replacement state or a replacement program as before.
You can reasonably take two different approaches to enabling this protocol from the bookmark page. One approach would be to add a third link (inventing a new link relation). Another would be to use content negotiation -- the endpoint of the setOvenTemperature interface checks to see if the Accept-Type is html (in which case the client is redirected to the entry point of that protocol, otherwise directed back to the previously implement PUT based flow).
Using the HTML declarative flow also raises another interesting point about media types. Text areas aren't very smart, they support free form text, so you end up relying on the operator not making any data entry errors. With a standardized media-type, and a document editor that is schema aware, the editor can help the operator do the right thing. For instance the document editor may be able to assist the agent with navigation hints and a document object model, allowing the agent to seek directly to the document elements relevant to its goal.
Edit: go figure -- having written up my thoughts, I went back to look at Asbjørn Ulsberg's slides and realized that we had originally been talking about toasters, not ovens.
On Anemic Domain Models
The data model has the responsibility to store data, whereas the domain model has the responsibility to handle business logic. An application can be considered CRUD only if there is no business logic around the data model. Even in this (rare) case, your data model is not your domain model. It just means that, as no business logics is involved, we don’t need any abstraction to manage it, and thus we have no domain model.Ouarzy, It Is Not Your Domain.
I really like this observation, in that it gives me some language to distinguish CRUD implementations, where we don't have any invariant to enforce, and anemic domains, where there is an invariant, but we don't keep the invariant and the state changing behavior in the same logical unit.
We probably need more variations here
- There is no invariant (CRUD)
- There is an invariant, but it is maintained by human beings -- the invariant is not present in the code
- There is an invariant in the code, but the code is arranged such that modification of state can happen without maintaining the invariant (anemic)
- There is an invariant in the code, and the code is arranged so that any change of state is checked against the invariant.
But if the software is any good, bypassing the domain model should be unusual enough that additional steps to disable the safeties is acceptable.
Thursday, January 19, 2017
A RESTful supply closet
At stackflow, another question was submitted about REST API design for non CRUD actions. In thinking about it, I found an analogy that may help explain the point.
Imagine a supply closet; an actual physical closet in the real world. The stock includes boxes of pencils. What does a web API for the supply closet look like?
As Jim Webber pointed out years ago, HTTP is a document management application. So the first thing to realize is that we are trying to create an interface that supports the illusion that the supply closet is a document store. If we want to know about the current state of the closet, we read the latest report out of the store. To change the state of the closet, we propose changes to the document store.
How do we convert the current state of the closet to a document? In the real world (think 1950s office), we would ask the quartermaster for the latest inventory document. If a recent one is available, the quartermaster gives us a copy of that document. Otherwise, he can look in the closet, count the boxes, and produce send us a copy of the fresh report.
That, fundamentally, is GET. We ask the API for a copy of the inventory report. Maybe the API just copies the report that's posted on the closet door, maybe the API goes inside the closet to count everything, maybe the closet isn't accessible, so we just get the last report the API saw. Doesn't matter, we got a document.
Now, key in the next stage is to realize that the document is not the closet; when we edit the document, boxes of pencils don't magically appear in the closet. What we need to implement is the illusion that the closet really is a document store.
In our real world model, we read the inventory report, and there aren't enough pencils. So we create a new document -- a memo to the quartermaster that says "stock more pencils". When we deliver the memo to the quartermaster, he decides how to get more pencils for the closet -- maybe he gets boxes out of storage, or buys some from the store next door. Maybe he updates his todo list (another document) and tells you he'll get back to you.
This is the basic idiom of HTML forms. We create (POST) a new document to the API, and the API interprets that document as changes to be made to the closet. The requisition document and the inventory document are different resources. For that matter, the collection of requisition documents and the inventory document are different resources. So you need a different namespace of identifiers to work with.
HTTP (but not HTML) also supports another approach. Instead of interacting with the closet by submitting new documents, we could interact with the closet by proposing edits to the existing documents.
This, to my mind, feels a bit more declarative -- you describe in the edited document the state that you want the closet to be in, and its up to the API to figure out the details of making that happen. In our analogy, we've sent to the quartermaster a copy of the inventory with a bunch of corrections made to it, and he changes the state of the closet to match the document.
This is PUT -- specifically a PUT to the inventory resource. Notice that it doesn't change what work the quartermaster needs to do to fill the closet; it doesn't change his schedule for doing that work, it doesn't change the artifacts that he generates while doing the work. It just changes which document manipulation illusion we are choosing to support.
Now, HTTP is specific about the behavior of the imaginary document store we are mimicking, which is that PUT is an upsert. If we want fine grained control of the contents of the closet ("more pencils, leave everything else alone"), then we need to upsert to a resource with a matching grain.
PATCH is another alternative to introducing finer grained resources; we send the patch to the server, it compares the patched version of the inventory document to the original version, and from there decides what changes need to be made to the closet.
These are all variations of the same fundamental idea - the HTTP request describes the desired end state, and the implementation sitting behind the API figures out how to realize that end.
Imagine a supply closet; an actual physical closet in the real world. The stock includes boxes of pencils. What does a web API for the supply closet look like?
As Jim Webber pointed out years ago, HTTP is a document management application. So the first thing to realize is that we are trying to create an interface that supports the illusion that the supply closet is a document store. If we want to know about the current state of the closet, we read the latest report out of the store. To change the state of the closet, we propose changes to the document store.
How do we convert the current state of the closet to a document? In the real world (think 1950s office), we would ask the quartermaster for the latest inventory document. If a recent one is available, the quartermaster gives us a copy of that document. Otherwise, he can look in the closet, count the boxes, and produce send us a copy of the fresh report.
That, fundamentally, is GET. We ask the API for a copy of the inventory report. Maybe the API just copies the report that's posted on the closet door, maybe the API goes inside the closet to count everything, maybe the closet isn't accessible, so we just get the last report the API saw. Doesn't matter, we got a document.
Now, key in the next stage is to realize that the document is not the closet; when we edit the document, boxes of pencils don't magically appear in the closet. What we need to implement is the illusion that the closet really is a document store.
In our real world model, we read the inventory report, and there aren't enough pencils. So we create a new document -- a memo to the quartermaster that says "stock more pencils". When we deliver the memo to the quartermaster, he decides how to get more pencils for the closet -- maybe he gets boxes out of storage, or buys some from the store next door. Maybe he updates his todo list (another document) and tells you he'll get back to you.
This is the basic idiom of HTML forms. We create (POST) a new document to the API, and the API interprets that document as changes to be made to the closet. The requisition document and the inventory document are different resources. For that matter, the collection of requisition documents and the inventory document are different resources. So you need a different namespace of identifiers to work with.
HTTP (but not HTML) also supports another approach. Instead of interacting with the closet by submitting new documents, we could interact with the closet by proposing edits to the existing documents.
This, to my mind, feels a bit more declarative -- you describe in the edited document the state that you want the closet to be in, and its up to the API to figure out the details of making that happen. In our analogy, we've sent to the quartermaster a copy of the inventory with a bunch of corrections made to it, and he changes the state of the closet to match the document.
This is PUT -- specifically a PUT to the inventory resource. Notice that it doesn't change what work the quartermaster needs to do to fill the closet; it doesn't change his schedule for doing that work, it doesn't change the artifacts that he generates while doing the work. It just changes which document manipulation illusion we are choosing to support.
Now, HTTP is specific about the behavior of the imaginary document store we are mimicking, which is that PUT is an upsert. If we want fine grained control of the contents of the closet ("more pencils, leave everything else alone"), then we need to upsert to a resource with a matching grain.
PATCH is another alternative to introducing finer grained resources; we send the patch to the server, it compares the patched version of the inventory document to the original version, and from there decides what changes need to be made to the closet.
These are all variations of the same fundamental idea - the HTTP request describes the desired end state, and the implementation sitting behind the API figures out how to realize that end.
Subscribe to:
Posts (Atom)