Saturday, March 18, 2017

RPC vs REST

Inspired by Why should I prefer HTTP REST over HTTP RPC JSON-Messaging style in conjunction with CQRS?

As with most REST questions, the easy answers can be found by looking at the web through a browser.

In RPC, the client creates an application/x-www-url-encoded document and sends it to the server.
In REST, the client completes a form and submits it.

In both cases, the client-stateless-server architectural constraint still applies; the server has no way of knowing anything about the nature of the client.  The message looks the same regardless of whether the client walks through the guided path or skips to the end.

In 2008, Roy T. Fielding made the following observation of a solution he was outlining.
I should also note that the above is not yet fully RESTful, at least how I use the term. All I have done is described the service interfaces, which is no more than any RPC. In order to make it RESTful, I would need to add hypertext to introduce and define the service, describe how to perform the mapping using forms and/or link templates, and provide code to combine the visualizations in useful ways.
My translation: having identified the endpoint of the protocol, you work toward REST by relaxing the understanding of the endpoint required by the client.

The RPC approach to a form submission requires that the client know in advance
  1. The location of the resource that accepts the request
  2. The appropriate HTTP method to use
  3. The correct spellings of the keys in the key/value pairs
  4. The semantic meaning of the keys/value pairs
In the REST approach, this information is delivered on demand, in the hypermedia.  So the location, the method, the keys are all covered.  The client just loads the form, uses semantic cues to recognize which fields to change, and the submits the form.

Of course, this also means that the client needs to understand forms and how to interpret them.  That's not free, but it shifts the problem from knowing in advance about the specific service to knowing in advance about a generic media type, and the conventions for semantic cues.

The riddle of finding the form gets pushed up to the bookmarked representation.  The client still needs a starting point, which is the bookmark.  It loads a representation of the bookmark provided by the server, and uses the available semantic cues to find a link to the required form.

The server can direct the client to a new representation of the form by changing the representation of the bookmark.  The new form representation can include additional fields with new semantic cues; the client, knowing only about the original semantics, simply ignores these fields when filling out the form; which in turn means that the values submitted will be those provided by the server in the form representation itself.

This isn't free, by any stretch -- we're buying the decoupling of the client and server by doing more upfront api design.
REST is intended for long-lived network-based applications that span multiple organizations. If you don’t see a need for the constraints, then don’t use them.
Stable APIs require a design investment.  The stability constraint, however, is optional; you should reject that constraint when the benefits are insufficient to offset the costs.

Thursday, March 2, 2017

DDD Repository Interfaces

Composed in response to Vladimir Khorikov.

One issue is that the above interface doesn’t constitute an actual abstraction. It just duplicates the concrete class’s functionality. The Principle of Reused Abstractions tells us that, in order for an interface to become one, it needs to have more than one implementation.

If we reboot the Wayback Machine, and take a look at the description provided by Eric Evans, we see at once the existence of other implementations.

Another transition that exposes technical complexity that can swamp the domain design is the transition to and from storage. This transition is the responsibility of another domain design construct, the REPOSITORY

The idea is to hide all the inner workings from the client, so that client code will be the same whether the data is stored in an object database, stored in a relational database, or simply held in memory.
I would certainly expect to see an in memory implementation, used by tests that protect me from errors in refactoring -- I'm going to burn the world as soon as the test passes anyway, so neither writing nor running integration boilerplate adds value.

But Vladimir raises an interesting point
Note that neither integration tests, nor unit tests would require seams that “abstract” the database out from the rest of the code. Unit tests just don’t involve anything other than isolated domain logic. Integration tests verify the database directly as part of the bounded context.
I love that -- it really shows that he has dug deeper into the question, to really think about the principles involved and whether or not they fit.
An application database (a database fully devoted to a single bounded context) is one of such systems. It belongs to your application only and not shared with anyone else.


An application with multiple writers is sharing. Your isolated domain logic doesn't share anything, so you can't check the behavior of conflicting writes that way. Trying to introduce conflicts, in all the paths that you need, during integration testing threatens many nightmares because of the combinatoric explosion of possibilities. If you are going to be refactoring your contingency pathways, you need a seam that discounts the overhead of checking the error to the point that you will actually pay the price. That requires a seam somewhere between the command handler and the process boundary, and the price drops each as you get closer to the handler.

In addition, that seam is a natural place to introduce an in memory cache; why reload an object from the book of record when the copy that you saved is still available? Why treat that optimization as an all or nothing affair when each composition root could be making its own choice on a case by case basis?

Vladimir is absolutely right that the repository (as written here) doesn't really align properly with boundaries. That thought is worth exploring in more detail.

Tuesday, February 28, 2017

Shedding

I haven't seen their API, but by all accounts it is terrible.
However, I have seen the bike shed they painted, and _it_ is terrific.

Sunday, February 19, 2017

TDD Probability Theory

I recently recommended that a friend review the probability kata; which got me back into it again.

 The probability kata was published by Greg Young in June, 2011.  The core of the problem statement was

Write a probability value object. It should contain the following methods:
  • Probability CombinedWith(Probability)
  • Probability Either(Probability)
  • Probability InverseOf()


Well, that certainly seems straight forward enough....

WARNING: Spoilers ahead

Sunday, February 5, 2017

Aggregates and RFC 2119.


The language used to describe the relationship between aggregates and commands is a confusing one.

The usual language is that the aggregate protects the business invariant -- one can reasonably read such at thing, and come away with the idea that aggregates are going to be throwing some flavor of DomainException when a command would break the rules.

Fortunately, we have experts trying to offer guidance.
Commands should not fail in collaborative domain
I struggled with understanding this for quite a while, because it sounded to me as though he was talking about checking that the command is valid before dispatching it to the domain.

The language that turned my brain around, was to consider that the aggregate is responsible for restoring the business invariant.

Udi Dahan, observing that Race Conditions Don't Exist, observed:
A microsecond difference in timing shouldn’t make a difference to core business behaviors. 
Reordering two commands should not change the behavior that you observe.  In particular, you shouldn't be rejecting a command that you would accept if the ordering were different.

This turned up recently in a calendar domain, where the aggregate is responsible for ensuring that events don't overlap.  So let us imagine an attempt to schedule two conflicting events, A and B.  In a naive implementation, the order used to process the commands will determine how the conflict is resolved.



This is an interpretation of the invariant that is consistent with MUST or MUST NOT from RFC 2119.  And that's fine, if that's what the business really needs.  But it's not the only way to interpret a business invariant.


This model of the domain is analogous to SHOULD or SHOULD NOT. We don't want conflicts to happen, but if we do the aggregate has the responsibility of tracking the conflict and whether or not it has been resolved. In a sense, the responsibility of the aggregate is to detect and track conflicts -- maintaining the schedule is a side effect.

To reduce the rate of conflicts, the caller is expected to make a good faith effort to ensure that there are no conflicts before dispatching the command.  After all, if the aggregate is tracking events and conflicts, then the caller can check for both before dispatching the command.  When the data that the caller is working from is stale, that judgment may be off, but the aggregate has a fail safe available to cover that contingency.

A similar example might occur in banking, where instead of rejecting a transaction, the bank instead invokes an overdraft contingency.

If your requirements include a MUST NOT, then push back and check: what is the cost to the business of allowing the behavior to occur?  Especially in business that run on human input, the processes that run the business are designed to mitigate these kinds of problems.  We can do the same thing in code.

Because SHOULD NOT is going to have much nicer scaling properties.

Thursday, January 26, 2017

A RESTful Kitchen Oven

Some time back, I chatted with Asbjørn Ulsberg about an example of a REST api using an oven.  Shortly there after, he presented his conclusions at the Nordic APIs 2016 Platform Summit. [slides]
What follows is my own work, but clearly I was influenced by his point of view.

 In my kitchen, there is a free standing gas range.  On the inside, it's got various marvels that we no longer think about much: an ignition system, and a thermostat, safety valves, etc.

But as a home cook, I really don't need to worry about those details at all.  I work with the touch pad at the top of the unit.  Bake, plus-plus-plus-plus, Start, and I get a beep and a display message letting me know that the oven is preheating.  Some time after that, another beep let's me know that the oven has reached the target temperature and on good days the actual temperature stays near that target until the oven is shut off.

Let's explore, for a time, what it would look like to control the oven from a web browser.


For our first cut, we can look at an imperative approach.

HTTP/1.0 gave us everything we need.  GET allows us to retrieve the information identified by the request uri, and POST allows us to provide a block of data to a data handling process.

In a browser, it might look like this: we load the bookmark URI.  That would get some information resource -- perhaps a menu of the services available, perhaps a representation of the current state of the stove, maybe both.  One of the links would include a relation (as well as human readable cues) that communicates that this is the link to follow to set the oven temperature.  Following that link would GET another resource that is a web form; in this case it would just be an annotated field for temperature (set with the default value of 350F), semantic cues, and a submit button.  Submitting the form would post the contents to the server, which would in turn interpret the submitted document as instructions to present to the oven.  In other words, the web server reads the submitted data, and pushes the control buttons for us.

Having done that, the server would return a 200 status to us, with a representation of the action it just took, and links from there to perhaps the status page of the oven, so that we can read updates to know if the oven has reached temperature.  In a simple interface like the one on my stove, the updates will only announce that the oven is preheating.  A richer interface might include the current temperature, an estimate of the expected wait time, and so on.

Great, that gets us a website that allows a human being to control the stove from a web browser, but how do we turn that into a real API?  Answer: that is a real API.  Anybody can grab their favorite http client and their favorite html parser, write an agent that understands the link relations and form annotations (published as part of this API).  The agent uses the client to load the bookmark url, loads the result into the parser, searches the resulting DOM for the elements it needs, submits the form, and so on.  Furthermore, the agent can talk to any stove at all.

And -- bonus trick: if you want to test the agent, but don't have a stove handy, you can just point it at any web server with test cases represented as graphs of html documents.  After all, the only URI that the agent actually knows anything about is the start point.  From that point on, it's just following links.

That's a nice demonstration of hypermedia, and the flexibility that comes from using the uniform interface.  But it misses two key lessons, so let's try again.

This time, we'll go with a declarative approach.  We need another verb, PUT, defined in the HTTP/1.1 spec (although it had appeared in the quarantine of the earlier appendix D).  Puts early definition got right to the heart of it.
The PUT method requests that the enclosed entity be stored under the

supplied Request-URI.
What the heck does that mean for an oven?


To the oven, it means nothing -- but we aren't implementing an oven, we're implementing a web api for an oven.  To be a web-api for something interesting means to express the access to that bit of interesting as though it were a document store.

In other words, the real goal here is that we should be able to control the stove from any HTTP aware document editor.

Here's how that might work.  We start from the bookmark page as we did before.  This includes a hypermedia link to a representation of the current temperature settings of the oven.  For instance, that representation might be a json document which includes a temperature field somewhere in the schema.  So the document can GET the current state represented as a json document (important note: this representation does NOT include hyperlinks -- we're not going to allow the document editor to modify those.  Instead, transitions away from the editable representation of the document are described in the Link header field.)

We load the document of the current state into our editor, which then follows the "self" relation with an OPTIONS call to discover if PUT is supported.  Discovering that this is so, the editor enables the editing controls.  The human operator can now replace the representation of the current temperature of the oven with a representation of the desired temperature, and click save.  The document editor does a simple PUT of the saved representation to the web server.

PUT is analogous to replace, in this use case.  In the case of a document store, we would simply overwrite the existing copy of the document.  We need to give some thought to what it means for an oven to mimic that behavior.

One possibility is that the resource in question represents, not the current *state* of the oven, but the current *program*.  So we initially connect to the oven the state of the program would be do nothing, and we would replace that with a representation of the program "set the temperature to 300 degrees", and the API would "store" that program by actually commanding the oven to heat to the target temperature.  PUT of a new program actually matches very well with the semantics of my oven, which treats entering the desired state as an idempotent operation.

A separate, read-only, resource would be used to monitor the current state of the oven.

In this idiom, "turning off the oven" has a natural analog for our document editor -- deleting the document.  The editor can just as easily determine that DELETE is supported as PUT, and enable the property controls for the user.

If we don't like the "update the program" approach, we can work with the current state document directly.  We enable PUT on the resource for the editable copy of the current oven state, enabling the edit controls in the document editor.  The agent can describe the document that they want, and submit the result.

The tricky bit, that is key to the declarative approach: the API needs to compare the current state with the target state, and determine for itself what commands need to be sent to the oven to bring about that change.  Just as the keypad insulates us from the details of the burners and valves, so to does the API insulate the consumers from the actual details of interacting with the oven.

Reality ensues: the latency for bringing the oven up to temperature isn't really suitable for an HTTP response SLA.  So we need to apply a bit of lateral thinking; the API reports to the document store that the proposed edit has been Accepted (202).  It's not committal, but it is standard.  The response would likely include a link to the status monitor, which the client could load to see that the oven was preheating.

Once again, automating this is easy -- you teach your agent how to speak oven (the standard link relations, the media-types that describe the states and the programs).  You use any HTTP away document editor to publish the agents changes to the server.  You test by directing the agent to a document store full of fake oven responses.
 
Do we need two versions of the API now? one for an imperative approach, another for the declarative approach?  Not at all -- the key is the bookmark URL.  We require that the agents be forgiving about links they do not recognize -- those can simply be ignored.  So on the bookmark page, we have a link relation that says "this way to the imperative setOvenTemperature interface", and another that says "this way to the declarative setOvenTemperature interface", and those two links can sit next to each other on the page.  Clients can follow whichever link they recognize.

Document editing -- especially for small documents -- is reasonably straight forward in html as well; the basic idea is that we have a resource which renders the current representation of our document inside a  text area in a form, which POSTs the modified version of the document to a resource that interprets it as a replacement state or a replacement program as before.

You can reasonably take two different approaches to enabling this protocol from the bookmark page.  One approach would be to add a third link (inventing a new link relation).  Another would be to use content negotiation -- the endpoint of the setOvenTemperature interface checks to see if the Accept-Type is html (in which case the client is redirected to the entry point of that protocol, otherwise directed back to the previously implement PUT based flow).

Using the HTML declarative flow also raises another interesting point about media types.  Text areas aren't very smart, they support free form text, so you end up relying on the operator not making any data entry errors.  With a standardized media-type, and a document editor that is schema aware, the editor can help the operator do the right thing.  For instance the document editor may be able to assist the agent with navigation hints and a document object model, allowing the agent to seek directly to the document elements relevant to its goal.

Edit: go figure -- having written up my thoughts, I went back to look at Asbjørn Ulsberg's slides and realized that we had originally been talking about toasters, not ovens.





On Anemic Domain Models


The data model has the responsibility to store data, whereas the domain model has the responsibility to handle business logic. An application can be considered CRUD only if there is no business logic around the data model. Even in this (rare) case, your data model is not your domain model. It just means that, as no business logics is involved, we don’t need any abstraction to manage it, and thus we have no domain model.
 Ouarzy, It Is Not Your Domain.

I really like this observation, in that it gives me some language to distinguish CRUD implementations, where we don't have any invariant to enforce, and anemic domains, where there is an invariant, but we don't keep the invariant and the state changing behavior in the same logical unit.

We probably need more variations here
  • There is no invariant (CRUD)
  • There is an invariant, but it is maintained by human beings -- the invariant is not present in the code
  • There is an invariant in the code, but the code is arranged such that modification of state can happen without maintaining the invariant (anemic)
  • There is an invariant in the code, and the code is arranged so that any change of state is checked against the invariant.
Having some anemic code paths isn't wrong -- it's a desirable property for the software to be able to get out of the way so that the operator in the present can veto the judgment of the implementer in the past. 

But if the software is any good, bypassing the domain model should be unusual enough that additional steps to disable the safeties is acceptable.