Sunday, 27 October 2013

REST APIs and restaurants [Take II*]

[Level C3]

 - "... and sauteed with the finest aromatic saffron - we import directly from our suppliers in Tibet - and garnished with the finest quality ocra from the heart of Morocco and ..."

- : ".. hmmm, bu..."

- : " and Sir, the King Shrimps day fresh from our suppliers in Belgium mildly cooked with a hint of lime zest and aromatic Thai basil snuggled with a swirl of Cornish Creme Fraiche and blossoms of Burmese jasmine and ..."

- : "... I mean, ..."

- : " and I should remind you that our head chef has used a special blend of exotic spices, marinated for 42 minutes exact, to create a sensual oriental effect of ... "

- : "I REALLY DON'T CARE! This stuff is awful!"

⚑  ⚑  ⚑

In more than one way a REST API is similar to a restaurant. They both hide a very complex process of accomplishing a service (serving food or data) behind a nice, warm and welcoming presentation. And we need to bear in mind that presentation is the key: no matter what goes on behind the scene in the kitchen, the food and its presentation is the key. And your API is the menu, the door to the services provided by the server. Whatever goes on in the kitchen (your server's internal domain) is not the concern of your customers as long as the output and presentation is good.

Commonly, the perspective in which we look at our services is wrong. They are server-centric. [I have talked before about the definition of server (and client) and their interaction] In brief, when we talk about the server, this is what comes in mind (as per CSDS):

This view is useful when building a server. It highlights inherent complexity of the server and how it should hide its complexity behind its API. But it is way too easy to get attached to the bird's eye view of the server where all internals of the server is visible.

And it is too easy to forget that this is how the client sees the server:

Client perspective of a server, where it sees a black box (here it's blue!) and
the gateway to the server is through its API.
When talking of the domain in an API, we are very likely to mean the server's private domain - its internals and complex process. Reality is server's internal domain does not surface beyond the API layer. API Layer is responsible to define resources which in turn define a coherent public domain composed of API and its messages - abstracted away from server's internal domain. If the server is an online retail API, I can create, edit or cancel orders. The fact that this is achieved through contact to many suppliers and their own custom processes is not a concern of the client.

In fact, in the same way that kitchen of a restaurant and its dining area are completely abstracted away, server's public domain is an abstraction on the top of server's internal domain. This is good for the chef and the customer. It is true the food that is served comes out of the kitchen but the whole process of preparation is irrelevant for the customer. 

Why talking of server's public domain is important? Because we commonly look down on the public domain. When we talk of the DTO messages passed to and fro, we regard them as View Models so they are not even proper models, they are just DTOs, no behaviour just some property bag. We use a mapper to strip down the sensitive data from our real models and spit out the view models to the client. And when we talk about DDD, we only mean server - we think view models do not deserve the love of care of DDD.

But this results in a brittle API that has to be changed by every change of the server. It becomes a cut down version of the server's internal API. We all know examples of good and bad API but too many changes in an API is a sign of bad design usually from Server Chauvinism. So always do a separate DDD for your server's public domain. It might sound strange since I am in favour of no-business-logic-in-api-layer. Well, it is true - we do not implement any business logic in the API Layer but the biggest responsibility of API Layer is talking HTTP. This translates to defining resources in REST.

So practice DDD in your API Layer. Define its aggregate roots, entities and value types - it is simplistic to think such concepts can be entirely hidden from the clients. Specify the commands and queries - with commands usually being the payload of the PUT, DELETE and POST and queries usually expressed in the URL itself to make them cache friendly.

What we see in the above picture define server's public domain: it is API (set of resources), input messages (commands and queries) and output messages (entities and value types). The case of queries - as we said - is a bit special since it is normally expressed in URL. This has the benefit of cacheability in contract with sending a payload (using other methods) which not suitable for caching. As an example, here is twitter's API for getting status timelines:

Get /1.1/statuses/user_timeline.json?screen_name=twitterapi&count=2

Defining parameters as query strings is useful as they can also imply optionality. One downside to this approach is the fact that ordering of parameters cannot be expressed in query string and while any combination can be valid, semantically equal combinations will be considered different resources and cached separately. For example, screen_name=twitterapi&count=2 and screen_name=twitterapi&count=2 are semantically equal while they are considered different resources. This reduces effectiveness of caching for GET requests. For this reason, URL segments separated by / are superior although in a big API can result in prohibitive complexity of routing.

Items returned in this API are Entities. it is evident by having an ID (id_str) which is returned as part of data.

Looking at what each timeline contains, we can see list of urls, where each url returned is a Value Type. It has three properties: url, indices and expanded_url.

In the case of commands, we have a similar situation. Looking at an example in twitter, tweeting (sending a tweet) is achieved using this command:

POST /1.1/statuses/update.json


Perhaps a better implementation would have been to omit the update.json totally - as this naming has an RPC smell. Using singular or plural is mainly down to taste - although I prefer singular. The command can be expressed by the simple class below:

Not all commands have to have a payload. Deleting a tweet can get send an ID in the URL with the DELETE command to /1.1/statuses/update.json although actual twitter API uses a POST instead.

Resources are basically all that can be accessed through the API although a more useful definition is the grouping around identities. For example Flickr API defines cameras, people, photos, photo comments, etc. It is true that Flickr is not considered a good example of a REST API when it comes to HTTP semantics, however, it does a good job in defining a coherent domain represented by resources while hiding its internals. A simple example is how it sends you previous and next photo instead of letting the client guess what these are based on the internal implementations.

In brief, it is responsibility of the API to create a coherent set of resources that can layout a cohesive picture in front of the client.

Common Anti-Patterns

As we said, the main danger is to have a server-centric perspective and neglect the public domain of your API.

While performing appropriate-sized DDD exercise is important, we need to be careful not expose too much of the server. One common anti-pattern is exposing server's internal to outside world. OData is a RESTful protocol which defines semantics for querying (mainly tabular) data store.

GET /api/products?$filter=name eq 'Gizmo 3'&$orderby=price desc&$top=1

While OData itself is not necessarily prescribing exposing server's internals, most of the implementations are actually built straight on the top of an RDBMS exposing its internal schema and and potentially allowing ad-hoc changes using SQL-like commands rather than domain-rich commands. Netflix who initially warmed up to the idea of exposing its films database to the public, reached the conclusion this is not such a RESTful idea and retired its OData service.

Another issue is the server-centric design: server assuming how its services will be used. I believe (Sorry Roy!) HATEOAS touches the subjects discussed here. When we talk of Hypermedia As The Engine OF Application State, which application do we mean? OK, let's consider Twitter's API. How can you think same hypermedia can equally drive the state of a twitter client, a game that posts scores on twitter, 4Square app that declares you a mayor on twitter or a twitter realtime map showing location of tweets using the firehose data pipe? The relationship from clients to servers are not one-to-one any more and effectiveness of HATEOAS is questionable. Application is defined by the client and not the server, as such server's hypermedia cannot be the engine of the application.

At the end of the day, none of us want to sound like the pompous chef who bores the customers with self-flattery. Server serves, so design it for servant-hood.

* this post was originally posted slightly differently under the name: "Server Chauvinism - API's public domain"


  1. I like the term Server Chauvinism :) Btw you mean screen_name=twitterapi&count=2 and count=2&screen_name=twitterapi being semantically equal.

    1. Yes :) That is what the post was originally called. And yes, They are semantically equal but in HTTP terms, completely different resources.