If I'm shopping for books on Amazon, I can look at my shopping list, click on a book, and have it added to my shopping cart. For some items, Amazon will decline to add the item to my cart, and inform me, perhaps, that the item is no longer available.
At the grocery store, no matter how many times I click on the shopping list, the Fruit Loops don't appear in my cart. I have to place the box in my cart by hand, next to the salad dressing that my phone says I can't put in the cart because it has been discontinued, and the milk that I can't put in my cart because it has expired.
If creating a user isn't a lot more fun than sending a command message to an aggregate, you are doing it wrong.
We often want representations of entities that we don't control, because the approximation we get by querying our representations is close enough to the answer we would get by going off to inspect the entities in question, while being much more convenient.
But if the entities aren't under our control, we have no business sending commands to the representations. Our representations don't have veto power over the book of record.
Aggregates only make sense when your domain model is the book of record.
Which means that you have no ability to enforce an invariant outside of the book of record. You can only query the available history, detect inconsistencies, and perhaps initiate a remediation process.
Wednesday, April 20, 2016
On Read Models
I learned something new about read models today.
Most discussions I have found so far emphasize the separation of the read model(s) from the write model.
For example, in an event sourced solution, the write model will update the event history in the book of record. Asynchronously, new events are read out of the book of record, and published. Event handlers process these new events, and produce updated projections. The read model answers queries using the most recently published projection. Because we are freed from the constraints of the write model, we can store the data we need in whatever format gives us the best performance; reads are "fast".
But by the time the read models can access the data; the data is old -- there's always going to be some latency between "then" and "now". In this example, we've had to wait for the events to be published (separately from the write), and then for the event handlers to consume them, construct the new projections, and store them.
What if we need the data to be younger?
A possible answer; have the read models pull the events from the book of record directly, then consume the events directly. It's not free to do this -- the read model has to do its own processing, which adds to its own latency. There record book is doing more work (answering more queries), which may make your writes slower, and so on.
But it's a choice you can make; selecting which parts of the system get the freshest data, and which parts of the system can trade off freshness for other benefits.
In an event sourced solution, one option is to return, from the command, the version number of the aggregate that was just updated. This version number becomes part of the query used to refresh the view.
In handling the query, you compare the version number in the query with that used to generate the projection. If the projection was assembled from a history that includes that version, you're done -- just return the existing projection.
But when the query is describing a version that is still in the future (from the perspective of the projection), one option is to suspend the query until history "catches up", and a new projection is assembled. An alternative approach is to query the book of record directly, to retrieve the missing history and update a copy of the projection "by hand". More work, more contention, less latency.
If the client is sophisticated enough to be carrying a local copy of the model, it can apply its own changes to that model, provisionally; reconciling the model when the next updates arrive from the server. That supports the illusion of low latency without creating additional work for the book of record (but might involve later reconciliation).
Most discussions I have found so far emphasize the separation of the read model(s) from the write model.
For example, in an event sourced solution, the write model will update the event history in the book of record. Asynchronously, new events are read out of the book of record, and published. Event handlers process these new events, and produce updated projections. The read model answers queries using the most recently published projection. Because we are freed from the constraints of the write model, we can store the data we need in whatever format gives us the best performance; reads are "fast".
But by the time the read models can access the data; the data is old -- there's always going to be some latency between "then" and "now". In this example, we've had to wait for the events to be published (separately from the write), and then for the event handlers to consume them, construct the new projections, and store them.
What if we need the data to be younger?
A possible answer; have the read models pull the events from the book of record directly, then consume the events directly. It's not free to do this -- the read model has to do its own processing, which adds to its own latency. There record book is doing more work (answering more queries), which may make your writes slower, and so on.
But it's a choice you can make; selecting which parts of the system get the freshest data, and which parts of the system can trade off freshness for other benefits.
Example
In some use cases, after handling a command, you will want to refresh the client with a projection that includes the effect of the command that just completed; think Post/Redirect/Get.In an event sourced solution, one option is to return, from the command, the version number of the aggregate that was just updated. This version number becomes part of the query used to refresh the view.
In handling the query, you compare the version number in the query with that used to generate the projection. If the projection was assembled from a history that includes that version, you're done -- just return the existing projection.
But when the query is describing a version that is still in the future (from the perspective of the projection), one option is to suspend the query until history "catches up", and a new projection is assembled. An alternative approach is to query the book of record directly, to retrieve the missing history and update a copy of the projection "by hand". More work, more contention, less latency.
If the client is sophisticated enough to be carrying a local copy of the model, it can apply its own changes to that model, provisionally; reconciling the model when the next updates arrive from the server. That supports the illusion of low latency without creating additional work for the book of record (but might involve later reconciliation).
Tuesday, February 9, 2016
Event Sourcing: on Event Handlers
One of the things I've been doing in my toy "study" problem, has been to implement an in memory event store. That means no persistence, per se, but all of the block and tackle of getting data to move from the "write model" to the "read model".
In particular, I've been taking pains to ensure that the asynchronous points in the data transfer are modeled that way -- I'm using a DirectExecutorService to run the asynchronous tasks, but I want to make sure that I'm getting them "right".
So, for this toy event store, I use the streamIds as keys to a hash; the object that comes out is a description of the stream, including a complete list of the events in that stream. Each write is implemented as a task submitted to the executor service, which uses a lock to ensure that only one thread writes to the event store at a time. The commit method replaces a volatile reference to the hash with a reference to an updated copy, producing an atomic commit. As the toy problem has very forgiving SLAs, writes are not merely appends to the stream, but actually check for conflicts, duplication, and so on.
Riddle: how to now update the read model. The transaction is the write to the volatile memory location, and if that part succeeds the client should be informed. So we really can't do any sort of synchronous notification of the read model. Instead, another task is scheduled to perform the update.
What should that task do? Pub/sub! which is right, but deceptively so. The basic idea is fine - we're going to asynchronously dispatch a message to an event queue, and all the subscribers will pick up that update and react.
What's the message though. I had been thinking that we could just enumerate the events, or possible the collection of events, but that makes a mess on the downstream side. The two basic issues being (a) the broadcast is asynchronous, so you really need the message handling to be idempotent, and (b) being asynchronous, the messages can arrive out of order.
Which means that simply publishing each of the domain events onto an asynchronous bus is going to make a mess of the event handlers, which all need a bunch of sequencing logic to repair the inevitable ordering edge cases.
Too much work.
The key clue is that the event sourced projections, process managers, and so on aren't really interested in a stream of events, so much as they are interested in a sequence of events. That sequence already exists in the write model, so the key idea is to not screw it up; we should be pushing/polling for updates to the sequence, rather than trying to track things at the level of the individual domain events.
The answer is to think in terms of publishing the cursor position for each stream.
In the write model, we push the events to the store as before. But we keep track of the positions in the stream that we have just written. After the transaction has been committed, we schedule an asynchronous task to push an event describing the new cursor position to the pub/sub system. Each event handler subscribes to that queue, and on each message compares the cursor position to its own high water mark; if there is further progress to be made, the handler fetches an ordered sub sequence of the events from the stream.
A potentially interesting byproduct of this idea: the write can return the cursor position to the caller, which can then use that position to rebuild it's next view. A reader that knows the specific position that it is waiting for can block until the read model has been updated to that point.
Because each of the event handlers is tracking its own high water mark, the cursor update messages are trivial to handle idempotently; the incorrectly ordered update messages are trivial to recognize and drop.
In particular, I've been taking pains to ensure that the asynchronous points in the data transfer are modeled that way -- I'm using a DirectExecutorService to run the asynchronous tasks, but I want to make sure that I'm getting them "right".
So, for this toy event store, I use the streamIds as keys to a hash; the object that comes out is a description of the stream, including a complete list of the events in that stream. Each write is implemented as a task submitted to the executor service, which uses a lock to ensure that only one thread writes to the event store at a time. The commit method replaces a volatile reference to the hash with a reference to an updated copy, producing an atomic commit. As the toy problem has very forgiving SLAs, writes are not merely appends to the stream, but actually check for conflicts, duplication, and so on.
Riddle: how to now update the read model. The transaction is the write to the volatile memory location, and if that part succeeds the client should be informed. So we really can't do any sort of synchronous notification of the read model. Instead, another task is scheduled to perform the update.
What should that task do? Pub/sub! which is right, but deceptively so. The basic idea is fine - we're going to asynchronously dispatch a message to an event queue, and all the subscribers will pick up that update and react.
What's the message though. I had been thinking that we could just enumerate the events, or possible the collection of events, but that makes a mess on the downstream side. The two basic issues being (a) the broadcast is asynchronous, so you really need the message handling to be idempotent, and (b) being asynchronous, the messages can arrive out of order.
Which means that simply publishing each of the domain events onto an asynchronous bus is going to make a mess of the event handlers, which all need a bunch of sequencing logic to repair the inevitable ordering edge cases.
Too much work.
The key clue is that the event sourced projections, process managers, and so on aren't really interested in a stream of events, so much as they are interested in a sequence of events. That sequence already exists in the write model, so the key idea is to not screw it up; we should be pushing/polling for updates to the sequence, rather than trying to track things at the level of the individual domain events.
The answer is to think in terms of publishing the cursor position for each stream.
In the write model, we push the events to the store as before. But we keep track of the positions in the stream that we have just written. After the transaction has been committed, we schedule an asynchronous task to push an event describing the new cursor position to the pub/sub system. Each event handler subscribes to that queue, and on each message compares the cursor position to its own high water mark; if there is further progress to be made, the handler fetches an ordered sub sequence of the events from the stream.
A potentially interesting byproduct of this idea: the write can return the cursor position to the caller, which can then use that position to rebuild it's next view. A reader that knows the specific position that it is waiting for can block until the read model has been updated to that point.
Because each of the event handlers is tracking its own high water mark, the cursor update messages are trivial to handle idempotently; the incorrectly ordered update messages are trivial to recognize and drop.
Monday, February 8, 2016
More thoughts on REST
Continuing to explore REST...
Lesson: REST is not about building nice little web applications like blogs, or Amazon, or Google. REST is about building web scale applications like...
the Web.
Lesson: the Web is already web scale. If you are building your application on the back of HTTP, all you need to do is not screw it up...
for example, by replacing text/html with some other media type that doesn't include controls. Oops.
Lesson: a friend of mine teaches his students that, when they are asked to implement a protocol, their first step should be to obtain the appropriate state machine compiler. In reverse, if an expert writes a thesis on Representational STATE Transfer, you should be thinking about how that maps to your application protocols...
and maybe not so much mapping it to your data model. Or your persistence layer.
Lesson: HTML is a perfectly cromulent media-type. GET and POST will serve as hypermedia controls. application/www-form-urlencoded is a little bit clumsy for hierarchical data, but can be made to serve.
So it should be straight forward to use a browser to navigate your application protocol?
Lesson: the modern web works. You point your browser at a url, and the browser downloads a bunch of java script that renders a single page application, and starts pinging some json api and using the responses to update the DOM.... That's REST. Event if those json endpoints aren't actually providing hypermedia controls - neither do images in html documents.
Lesson: REST is not about building nice little web applications like blogs, or Amazon, or Google. REST is about building web scale applications like...
the Web.
Lesson: the Web is already web scale. If you are building your application on the back of HTTP, all you need to do is not screw it up...
for example, by replacing text/html with some other media type that doesn't include controls. Oops.
Lesson: a friend of mine teaches his students that, when they are asked to implement a protocol, their first step should be to obtain the appropriate state machine compiler. In reverse, if an expert writes a thesis on Representational STATE Transfer, you should be thinking about how that maps to your application protocols...
and maybe not so much mapping it to your data model. Or your persistence layer.
Lesson: HTML is a perfectly cromulent media-type. GET and POST will serve as hypermedia controls. application/www-form-urlencoded is a little bit clumsy for hierarchical data, but can be made to serve.
So it should be straight forward to use a browser to navigate your application protocol?
Lesson: the modern web works. You point your browser at a url, and the browser downloads a bunch of java script that renders a single page application, and starts pinging some json api and using the responses to update the DOM.... That's REST. Event if those json endpoints aren't actually providing hypermedia controls - neither do images in html documents.
Friday, January 22, 2016
REST: on Resources
I've been trying to make more progress with REST by reviewing the questions that other programmers are asking about it
Sidestepping for the moment the whole riddle of hypermedia representations, the most common issue seems to come about when the programmer assumes that an entity in the domain must have a single URI that does everything with nothing but Four Verbs and the Truth.
The internal mapping seems to be that entities are nouns, and resources are nouns, and therefore entities are resources -- since the URI is "the" identifier for the resource, it must also be "the" identifier for the entity, and suddenly everything looks like a nail.
Clearly, identifiers should be 1-to-1 with resources -- you really need to pervert the definition of "uniform resource identifier" to reach any other conclusion. Although, it turns out that is something that we had to learn over time.
For instance, in 1999, the definition of the PUT method described resources this way
The latter interpretation allows you to sidestep one of the complicating issues with interpreting a PUT method -- the body is supposed to be a replacement for the resource, but nothing in "the rules" prevents you from inventing a resource with the specific purpose of being replaced.
Jim Webber hinted at the same thing:
Another idea that helped clear things for me is that a view of an immutable resource can be cached. Trying to cache a projection of a mutable entity brings you face to face with one of the two hard problems, but caching the immutable history of a mutable entity is fine. In other words, we have different resources that describe the state of an entity at different times. Couple that with the idea that there are many ways of expressing a moment of time, that each in turn maps to a different resource, and you get an explosion of possible resources that you can exploit.
Sidestepping for the moment the whole riddle of hypermedia representations, the most common issue seems to come about when the programmer assumes that an entity in the domain must have a single URI that does everything with nothing but Four Verbs and the Truth.
The internal mapping seems to be that entities are nouns, and resources are nouns, and therefore entities are resources -- since the URI is "the" identifier for the resource, it must also be "the" identifier for the entity, and suddenly everything looks like a nail.
Clearly, identifiers should be 1-to-1 with resources -- you really need to pervert the definition of "uniform resource identifier" to reach any other conclusion. Although, it turns out that is something that we had to learn over time.
For instance, in 1999, the definition of the PUT method described resources this way
A single resource MAY be identified by many different URIs. For example, an article might have a URI for identifying "the current version" which is separate from the URI identifying each particular version. In this case, a PUT request on a general URI might result in several other URIs being defined by the origin server.In 2014, the definition of the PUT method changed.
A PUT request applied to the target resource can have side effects on other resources. For example, an article might have a URI for identifying "the current version" (a resource) that is separate from the URIs identifying each particular version (different resources that at one point shared the same state as the current version resource). A successful PUT request on "the current version" URI might therefore create a new version resource in addition to changing the state of the target resource, and might also cause links to be added between the related resources.
The latter interpretation allows you to sidestep one of the complicating issues with interpreting a PUT method -- the body is supposed to be a replacement for the resource, but nothing in "the rules" prevents you from inventing a resource with the specific purpose of being replaced.
Jim Webber hinted at the same thing:
You should expect to have many many more resources in your integration domain than you do business objects in your business domain.I think the CQRS language helps here -- from our event history, we build lots of different projections that are each especially suitable to a particular use case.
Another idea that helped clear things for me is that a view of an immutable resource can be cached. Trying to cache a projection of a mutable entity brings you face to face with one of the two hard problems, but caching the immutable history of a mutable entity is fine. In other words, we have different resources that describe the state of an entity at different times. Couple that with the idea that there are many ways of expressing a moment of time, that each in turn maps to a different resource, and you get an explosion of possible resources that you can exploit.
Friday, January 8, 2016
Coordinated Entities
Question: how do you decide if two different entities belong in the same aggregate?
I've been puzzling over this for a while now, looking for the right sorts of heuristics to apply.
The book answer is straight forward, without being useful. The aggregate boundary encompasses all of the state required to maintain the business invariant. So if you know what the business invariant, then the problem is easy. You start with an aggregate of a single entity, then you fold in all of the business rules that reference the state of the entity, then you fold in all of the entities touched by those rules, and then fold in more rules... it's turtles until you reach a steady state. Then that aggregate, at least, is complete. You set it aside, pick a new entity, and repeat the process until all the entities in the domain have been assigned to an aggregate.
In any interesting problem space, the invariant is not so clearly defined. Most of the discussions describing the evolution of a model talk about the discovery that the model is missing some element of the Ubiquitous Language, and that inspires someone to recognize why some use case has been broken, or incredibly difficult to implement. Or that the Ubiquitous Language has actually been missing some important concept, that -- one expressed -- brings new clarity to the actual requirements of the business. Most of the refactoring exercises I have seen have described cases where entities were too tightly coupled; contention between unrelated entities was making the system harder to use.
Lesson I learned today:
Thinking about the happy path doesn't inform anything. Any composition of the objects will do when the command history never violates any business rules. The interesting cases are partial failures.
Contention, as noted previously, is a primary pressure to separate entities. Commands are being applied to different entities, where there should be no interplay between the affected states. Yet if both commands are being run through the same aggregate root, then one otherwise satisfactory command will fail because it happened to be trying to commit after a different command has already advanced the history of the aggregate. This is a failure of interference between uncoordinated commands. The inverse problem are two coordinated commands are broadcast to separate entities, where one command succeeds and the other fails.
Thought experiment: suppose that we were to model these two entities in separate aggregates, so that they are participating in different transactions. What would this coordination failure look like in the event stream? Well, you would be watching the events go by, and you would see the history of the successful command, and then you would wait, and wait, and you would never see the history from the other aggregate.
Let's put a scope on it - we have a coordination contingency if some specified amount of time passes without seeing the missing history. That we are watching the event history, and thinking about the passage of time, announces at once that we are considering a process manager; which is an entity that implements a state machine. Within their own transactions, a process manager will emit events describing the changes to the state machine, asynchronously schedule calls to itself (a time trigger), and perhaps dispatch asynchronous commands to the domain model.
There's some block and tackle to be done at this point -- the processManager is an entity in its own right, and we need to be sure that the observed events are dispatched to "the right one". We're going to need some meta data in the events to ensure that they are all going to the right destination.
Back to our experiment; the history of the first command arrives. We load a process manager and pass the event to it. The process manager uses its SLA to schedule a message to itself at some time in the future. Time passes; the scheduled message is delivered. The process manager fires the timeout trigger into its state machine, arrives at the Contingency state, and writes that event into the log.
How does that help?
It gives us something to look for in the Ubiquitous Language. If the coordinated entities really do belong in separate aggregates, then this contingency is a thing that really happens in the business, and so somebody should know about it, know the requirements for mitigating the contingency, what events should appear in the log to track the mitigation progress, and so on.
On the other hand, if the domain experts begins saying "that can't happen", "that MUST NOT happen", "that is too expensive when it happens, which is why we have you writing software to prevent it", and so forth; then that is strong evidence that the two entities in question need to be modeled as part of the same aggregate.
I've been puzzling over this for a while now, looking for the right sorts of heuristics to apply.
The book answer is straight forward, without being useful. The aggregate boundary encompasses all of the state required to maintain the business invariant. So if you know what the business invariant, then the problem is easy. You start with an aggregate of a single entity, then you fold in all of the business rules that reference the state of the entity, then you fold in all of the entities touched by those rules, and then fold in more rules... it's turtles until you reach a steady state. Then that aggregate, at least, is complete. You set it aside, pick a new entity, and repeat the process until all the entities in the domain have been assigned to an aggregate.
In any interesting problem space, the invariant is not so clearly defined. Most of the discussions describing the evolution of a model talk about the discovery that the model is missing some element of the Ubiquitous Language, and that inspires someone to recognize why some use case has been broken, or incredibly difficult to implement. Or that the Ubiquitous Language has actually been missing some important concept, that -- one expressed -- brings new clarity to the actual requirements of the business. Most of the refactoring exercises I have seen have described cases where entities were too tightly coupled; contention between unrelated entities was making the system harder to use.
Lesson I learned today:
Thinking about the happy path doesn't inform anything. Any composition of the objects will do when the command history never violates any business rules. The interesting cases are partial failures.
Contention, as noted previously, is a primary pressure to separate entities. Commands are being applied to different entities, where there should be no interplay between the affected states. Yet if both commands are being run through the same aggregate root, then one otherwise satisfactory command will fail because it happened to be trying to commit after a different command has already advanced the history of the aggregate. This is a failure of interference between uncoordinated commands. The inverse problem are two coordinated commands are broadcast to separate entities, where one command succeeds and the other fails.
Thought experiment: suppose that we were to model these two entities in separate aggregates, so that they are participating in different transactions. What would this coordination failure look like in the event stream? Well, you would be watching the events go by, and you would see the history of the successful command, and then you would wait, and wait, and you would never see the history from the other aggregate.
Let's put a scope on it - we have a coordination contingency if some specified amount of time passes without seeing the missing history. That we are watching the event history, and thinking about the passage of time, announces at once that we are considering a process manager; which is an entity that implements a state machine. Within their own transactions, a process manager will emit events describing the changes to the state machine, asynchronously schedule calls to itself (a time trigger), and perhaps dispatch asynchronous commands to the domain model.
There's some block and tackle to be done at this point -- the processManager is an entity in its own right, and we need to be sure that the observed events are dispatched to "the right one". We're going to need some meta data in the events to ensure that they are all going to the right destination.
Back to our experiment; the history of the first command arrives. We load a process manager and pass the event to it. The process manager uses its SLA to schedule a message to itself at some time in the future. Time passes; the scheduled message is delivered. The process manager fires the timeout trigger into its state machine, arrives at the Contingency state, and writes that event into the log.
How does that help?
It gives us something to look for in the Ubiquitous Language. If the coordinated entities really do belong in separate aggregates, then this contingency is a thing that really happens in the business, and so somebody should know about it, know the requirements for mitigating the contingency, what events should appear in the log to track the mitigation progress, and so on.
On the other hand, if the domain experts begins saying "that can't happen", "that MUST NOT happen", "that is too expensive when it happens, which is why we have you writing software to prevent it", and so forth; then that is strong evidence that the two entities in question need to be modeled as part of the same aggregate.
Monday, November 9, 2015
Domain Driven Design vs REST
For a couple of weeks now, I've been banging my head against Domain Driven Design (DDD), Command Query Responsibility Segregation (CQRS), and Representational State Transfer (REST).
I had been making a big, and probably common mistake: I started looking for nouns. The ubiquitous language gives me lovely nouns, and they seemed a natural fit for resources.
But I couldn't get the same natural feeling from the verbs. In the ubiquitous language, I've got all of these lovely expressive verbs to motivate change in my business model. In my link relations, I've got GET, PUT, POST, DELETE.
I finally tracked down Jim Webber's DDD in the Large presentation. Yeah, that helped.
Application State.
Why am I thinking about trying to represent my aggregate roots as resources? Those are two completely different layers! The application layer talks to the domain layer, there's an interface between the two, but there's no particular reason to expect a one to one correspondence.
All of the RESTful bits are going to be over there, with the anti corruption logic.
What Jim's talk clarified for me: that's not cheating, it's the whole damn point.
The hand wavy argument is that we are looking for nouns, and the commands, as messages, are the noun that we want. No kidding, our resources are representations of little pieces of paper, a ToDo on a post it note, that the client is passing to the server.
It still feels like a cheat to me.
But Jim in his talks points out, rightly, that if you are communicating over HTTP, then you are using a document management system to communicate your applications state. So if you aren't passing documents, you're clearly Doing It Wrong.
That's getting closer, but I needed to make one more connection to sell myself.
Stepping back from the problem; ignore the REST constraint completely. The client, over there, needs to communicate with the server here. We crossing a boundary; if we watch on the wire, we're not going to see objects in the transfer, but data. We see the same thing when the client queries a projection - Data Transfer Objects are being exchanged.
Data Transfer Object is a synonym for document. Oh.
I find it a little bit easier to sneak up on the idea by looking at the interaction with the read model. The client sends us a query, and we send back some projection of the model. It doesn't make sense to think about caching a model (it changes in time), and caching the projection (which also changes in time) is similarly dubious, but caching a snapshot of the projection at some point in time -- that does make sense. "Get me the report as of Time.now()" sure sounds like a document to me.
Something similar happens with domain events, and the communication between the read model and the write model. "Stream of Events" might not sound like a document, but journal, ledger, log -- those certainly are.
Commands as documents? I mentioned the ToDo analog earlier, but another good fit would be orders.
The read model, that's easily managed -- the queries arrive, the appropriate event history is loaded into the projection, and report is generated and delivered. All of these steps are idempotent and safe - we might change some state in memory, like caching the projection data for a time in case we are about to need it, but the model and the event history are not changed at all.
The write model is more difficult - changing the model is a side effect of the arrival of the command, and the command may arrive more than once.
For instance, the client puts a command, the command is received and executed, but the acknowledgement of the command is lost in transit. As PUT is supposed to be idempotent, the client may send the command document a second time.
The model, it should be running the commands given to it. So you need either that the commands in the model handle commands idempotently, or you need the anti-corruption layer to do the right thing when the duplicated command arrives.
An example scenario: Alice puts command.id:1 to the server. The server executes the command, updating the history of the model, and publishes a reply. That reply is lost in transit. Bob puts command.id:2 to the server, updating the model further. Alice times out waiting for the acknowledgment that her command arrived, and resends it. Charlie puts command.id:3 to the server, but is data is stale because he was working from a state prior to Alice's first command.
What should the responses look like?
Bob, clearly, successfully delivered a command that was executed, he should see a status 201 Created message. Charlie's command should be rejected, because the preconditions under which he submitted the command were not met, which probably means a 409 or 411 response. Alice's first command should like Bob, get a 201. When she resubmits the same command a second time, it is still supposed to be an idempotent operation, so she should still be getting the 201 response (and not the error code seen by Charlie).
That probably means a message store: to cache the response in the application layer for a time so that it can be replayed without interacting with the model.
My feeling is that the command should be considered immutable by the client; a second put that doesn't agree with the first should be rejected. The delete by the client might be a way to incorporate an acknowledgement, and expire the data in the message store.
I had been making a big, and probably common mistake: I started looking for nouns. The ubiquitous language gives me lovely nouns, and they seemed a natural fit for resources.
But I couldn't get the same natural feeling from the verbs. In the ubiquitous language, I've got all of these lovely expressive verbs to motivate change in my business model. In my link relations, I've got GET, PUT, POST, DELETE.
I finally tracked down Jim Webber's DDD in the Large presentation. Yeah, that helped.
Application vs Domain
Taking it very slowly: the key idea underlying REST is "Hypermedia as the Engine of Application State".Application State.
Why am I thinking about trying to represent my aggregate roots as resources? Those are two completely different layers! The application layer talks to the domain layer, there's an interface between the two, but there's no particular reason to expect a one to one correspondence.
All of the RESTful bits are going to be over there, with the anti corruption logic.
Commands as Resources
Resources were the second bit that I had flat out gotten wrong. It had occurred to me that I could just cheat, turn the problem around, and use my commands as resources.What Jim's talk clarified for me: that's not cheating, it's the whole damn point.
The hand wavy argument is that we are looking for nouns, and the commands, as messages, are the noun that we want. No kidding, our resources are representations of little pieces of paper, a ToDo on a post it note, that the client is passing to the server.
It still feels like a cheat to me.
But Jim in his talks points out, rightly, that if you are communicating over HTTP, then you are using a document management system to communicate your applications state. So if you aren't passing documents, you're clearly Doing It Wrong.
That's getting closer, but I needed to make one more connection to sell myself.
Stepping back from the problem; ignore the REST constraint completely. The client, over there, needs to communicate with the server here. We crossing a boundary; if we watch on the wire, we're not going to see objects in the transfer, but data. We see the same thing when the client queries a projection - Data Transfer Objects are being exchanged.
Data Transfer Object is a synonym for document. Oh.
I find it a little bit easier to sneak up on the idea by looking at the interaction with the read model. The client sends us a query, and we send back some projection of the model. It doesn't make sense to think about caching a model (it changes in time), and caching the projection (which also changes in time) is similarly dubious, but caching a snapshot of the projection at some point in time -- that does make sense. "Get me the report as of Time.now()" sure sounds like a document to me.
Something similar happens with domain events, and the communication between the read model and the write model. "Stream of Events" might not sound like a document, but journal, ledger, log -- those certainly are.
Commands as documents? I mentioned the ToDo analog earlier, but another good fit would be orders.
Model Change
Of course, the whole point to this mess is to have an application that can interact with the model, so something needs to connect the two.The read model, that's easily managed -- the queries arrive, the appropriate event history is loaded into the projection, and report is generated and delivered. All of these steps are idempotent and safe - we might change some state in memory, like caching the projection data for a time in case we are about to need it, but the model and the event history are not changed at all.
The write model is more difficult - changing the model is a side effect of the arrival of the command, and the command may arrive more than once.
For instance, the client puts a command, the command is received and executed, but the acknowledgement of the command is lost in transit. As PUT is supposed to be idempotent, the client may send the command document a second time.
The model, it should be running the commands given to it. So you need either that the commands in the model handle commands idempotently, or you need the anti-corruption layer to do the right thing when the duplicated command arrives.
An example scenario: Alice puts command.id:1 to the server. The server executes the command, updating the history of the model, and publishes a reply. That reply is lost in transit. Bob puts command.id:2 to the server, updating the model further. Alice times out waiting for the acknowledgment that her command arrived, and resends it. Charlie puts command.id:3 to the server, but is data is stale because he was working from a state prior to Alice's first command.
What should the responses look like?
Bob, clearly, successfully delivered a command that was executed, he should see a status 201 Created message. Charlie's command should be rejected, because the preconditions under which he submitted the command were not met, which probably means a 409 or 411 response. Alice's first command should like Bob, get a 201. When she resubmits the same command a second time, it is still supposed to be an idempotent operation, so she should still be getting the 201 response (and not the error code seen by Charlie).
That probably means a message store: to cache the response in the application layer for a time so that it can be replayed without interacting with the model.
My feeling is that the command should be considered immutable by the client; a second put that doesn't agree with the first should be rejected. The delete by the client might be a way to incorporate an acknowledgement, and expire the data in the message store.
Subscribe to:
Posts (Atom)