Machine Learning and APIs: introducing Mills in REST API Design

Level [C3]

REST (REpresentational State Transfer) was designed with the "state" at its heart, literally, standing for the S in the middle of the acronym.

TL;DR: Mill is a special type of resource where server's authority purely comes from exposing an algorithm, rather than "defining, exposing and maintaining integrity of a state". Unlike an RPC style endpoint, it has to adhere to a set of 5 constraints (see below). 

Historically, when there were only a few thousand servers around, the state was predominantly documents. People were creating, editing and sharing a lot of text documents, and some HTML. With HTTP 1.1, caching and concurrency was built into the protocol and enabled it to represent richer distributed computing concerns and we have been building . With the rising popularity of REST over the last 10 years, much of today's web has been built on top of RESTful thinking, whether what is visible or what is behind the presentation (most external layer) servers. Nowadays when we talk of state, we normally mean data or rather records persisted in a data store (relational or non-relational). A lot of today's data, directly or indirectly, is created, updated and deleted using REST APIs. And this is all cool, of course.

When we design APIs, we map the state into REST Resources. It is very intuitive to think of resources as collection and instance. It is unambiguous and useful to communicate these concepts when for example we refer to /books and /books/123 URLs, as the collection or instance resources, respectively. We interact with these resources using verbs, and although HTTP verbs are not meant to be used just for CRUD operations, interacting with the state that exists on the server is inherent in the design.

But that is not all the story. Mainstream adoption of Machine Learning in the industry means we need to expose Machine Learning applications using APIs. The problem is the resource oriented approach of REST (where the state is at the heart of the design) does not work very well.

By the way, I am NOT 51...

How-old.net is an example of a Machine Learning application where instead of being an application, it could have been an API. For example (just for illustration, you could use other media types too):

POST /age_gender_classifier HTTP/1.1
Content-Type: image/jpeg

And the response:

200 OK
Content-Type: application/json

{
    "gender":"M"
    "age":37
}

Server is generating a response to the request by carrying out complex face recognition and running a model, most likely a deep network model. Server is not returning a state stored on the server, in fact this whole process is completely stateless.

And why does this matter? Well I feel if REST is supposed to move forward with our needs and use cases, it should define, clarify, internalise and finally digest edge cases. While such edge cases were pretty uncommon, with the rise and popularity of Machine Learning, such endpoints will be pretty standard.

A few days ago, on the second day of APIdays Mediterranea 2015 conference, I presented a talk on Machine Learning and APIs. And in this talk I presented simple concept of Mills. Mills, where you take your wheat to be ground and you carry back the flour.

Basically, it all goes back to the origin of a server's authority. To bring an example, a "Customer Profile" service, exposed by a REST API, is the authority to go to when another service requires access to a customer's profile. The "Customer Profile" service has defined a state, which is profile of the customers, and is responsible for ensuring integrity of the state (enforcing business rules on the state). For example, marketing email preference can have values of None, WeeklyDigest or All, it should not allow the value to be set to MonthlyDigest. We are quite used to these type of services and building REST APIs on top: CustomerProfile becomes a resource that we can query or interact with.

On the other hand, a server's authority could be exposing an algorithm. For example, tokenisation of text is a non-trivial problem that requires not only breaking the text to its words, but also maintaining muti-words and named entities intact. A REST API that exposes this functionality will be a mill.

5 constraints of a Mill

1) It encapsulates an algorithm not a state

Which was discussed ad nauseum, however, the distinction is very important. For example let's say we have an algorithm that you provide the postcode and it returns to you houses within 1 mile radius of that postcode - this is not an example of a mill.

2) Raw data in, processed data out

For example you send your text and get back the translation.

3) Calls are both safe and idempotent

Calling the endpoint should not directly change any state within the server. For example, the endpoint should not be directly mapped to the ML training engine, e.g. sending a text 1000 times skew the trained model for that text. The training endpoint is usually a normal resource, not a mill - see my slides

4) It has a single specialty

And as such, it accepts a single HTTP verb apart from OPTIONS, normally POST (although a GET with entity payload would be more semantically correct but not my preferred option for practical reasons).

5) It is named not as a verb but as a tool

A mill that exposes tokenisation, is to be called tokeniser. In a similar way, classifier would be the appropriate name for a system that classifies on top of a neural network, for example. Or normalising text, would have a normaliser mill.

No this is not the same as an RPC endpoint. No RPC smell. Honest :) That is why those 5 constraints exists.

REST APIs and restaurants [Take II*]

[Level C3]

 - "... and sauteed with the finest aromatic saffron - we import directly from our suppliers in Tibet - and garnished with the finest quality ocra from the heart of Morocco and ..."

- : ".. hmmm, bu..."

- : " and Sir, the King Shrimps day fresh from our suppliers in Belgium mildly cooked with a hint of lime zest and aromatic Thai basil snuggled with a swirl of Cornish Creme Fraiche and blossoms of Burmese jasmine and ..."

- : "... I mean, ..."

- : " and I should remind you that our head chef has used a special blend of exotic spices, marinated for 42 minutes exact, to create a sensual oriental effect of ... "

- : "I REALLY DON'T CARE! This stuff is awful!"

⚑  ⚑  ⚑

In more than one way a REST API is similar to a restaurant. They both hide a very complex process of accomplishing a service (serving food or data) behind a nice, warm and welcoming presentation. And we need to bear in mind that presentation is the key: no matter what goes on behind the scene in the kitchen, the food and its presentation is the key. And your API is the menu, the door to the services provided by the server. Whatever goes on in the kitchen (your server's internal domain) is not the concern of your customers as long as the output and presentation is good.

Commonly, the perspective in which we look at our services is wrong. They are server-centric. [I have talked before about the definition of server (and client) and their interaction] In brief, when we talk about the server, this is what comes in mind (as per CSDS):

This view is useful when building a server. It highlights inherent complexity of the server and how it should hide its complexity behind its API. But it is way too easy to get attached to the bird's eye view of the server where all internals of the server is visible.

And it is too easy to forget that this is how the client sees the server:

Client perspective of a server, where it sees a black box (here it's blue!) and
the gateway to the server is through its API.

When talking of the domain in an API, we are very likely to mean the server's private domain - its internals and complex process. Reality is server's internal domain does not surface beyond the API layer. API Layer is responsible to define resources which in turn define a coherent public domain composed of API and its messages - abstracted away from server's internal domain. If the server is an online retail API, I can create, edit or cancel orders. The fact that this is achieved through contact to many suppliers and their own custom processes is not a concern of the client.

In fact, in the same way that kitchen of a restaurant and its dining area are completely abstracted away, server's public domain is an abstraction on the top of server's internal domain. This is good for the chef and the customer. It is true the food that is served comes out of the kitchen but the whole process of preparation is irrelevant for the customer. 

Why talking of server's public domain is important? Because we commonly look down on the public domain. When we talk of the DTO messages passed to and fro, we regard them as View Models so they are not even proper models, they are just DTOs, no behaviour just some property bag. We use a mapper to strip down the sensitive data from our real models and spit out the view models to the client. And when we talk about DDD, we only mean server - we think view models do not deserve the love of care of DDD.

But this results in a brittle API that has to be changed by every change of the server. It becomes a cut down version of the server's internal API. We all know examples of good and bad API but too many changes in an API is a sign of bad design usually from Server Chauvinism. So always do a separate DDD for your server's public domain. It might sound strange since I am in favour of no-business-logic-in-api-layer. Well, it is true - we do not implement any business logic in the API Layer but the biggest responsibility of API Layer is talking HTTP. This translates to defining resources in REST.

So practice DDD in your API Layer. Define its aggregate roots, entities and value types - it is simplistic to think such concepts can be entirely hidden from the clients. Specify the commands and queries - with commands usually being the payload of the PUT, DELETE and POST and queries usually expressed in the URL itself to make them cache friendly.

What we see in the above picture define server's public domain: it is API (set of resources), input messages (commands and queries) and output messages (entities and value types). The case of queries - as we said - is a bit special since it is normally expressed in URL. This has the benefit of cacheability in contract with sending a payload (using other methods) which not suitable for caching. As an example, here is twitter's API for getting status timelines:

Get /1.1/statuses/user_timeline.json?screen_name=twitterapi&count=2

Defining parameters as query strings is useful as they can also imply optionality. One downside to this approach is the fact that ordering of parameters cannot be expressed in query string and while any combination can be valid, semantically equal combinations will be considered different resources and cached separately. For example, screen_name=twitterapi&count=2 and screen_name=twitterapi&count=2 are semantically equal while they are considered different resources. This reduces effectiveness of caching for GET requests. For this reason, URL segments separated by / are superior although in a big API can result in prohibitive complexity of routing.

Items returned in this API are Entities. it is evident by having an ID (id_str) which is returned as part of data.

Looking at what each timeline contains, we can see list of urls, where each url returned is a Value Type. It has three properties: url, indices and expanded_url.

In the case of commands, we have a similar situation. Looking at an example in twitter, tweeting (sending a tweet) is achieved using this command:

POST /1.1/statuses/update.json

status=Maybe%20he%27ll%20finally%20find%20his%20keys.%20%23peterfalk

Perhaps a better implementation would have been to omit the update.json totally - as this naming has an RPC smell. Using singular or plural is mainly down to taste - although I prefer singular. The command can be expressed by the simple class below:

Not all commands have to have a payload. Deleting a tweet can get send an ID in the URL with the DELETE command to /1.1/statuses/update.json although actual twitter API uses a POST instead.

Resources are basically all that can be accessed through the API although a more useful definition is the grouping around identities. For example Flickr API defines cameras, people, photos, photo comments, etc. It is true that Flickr is not considered a good example of a REST API when it comes to HTTP semantics, however, it does a good job in defining a coherent domain represented by resources while hiding its internals. A simple example is how it sends you previous and next photo instead of letting the client guess what these are based on the internal implementations.

In brief, it is responsibility of the API to create a coherent set of resources that can layout a cohesive picture in front of the client.

Common Anti-Patterns

As we said, the main danger is to have a server-centric perspective and neglect the public domain of your API.

While performing appropriate-sized DDD exercise is important, we need to be careful not expose too much of the server. One common anti-pattern is exposing server's internal to outside world. OData is a RESTful protocol which defines semantics for querying (mainly tabular) data store.

GET /api/products?$filter=name eq 'Gizmo 3'&$orderby=price desc&$top=1

While OData itself is not necessarily prescribing exposing server's internals, most of the implementations are actually built straight on the top of an RDBMS exposing its internal schema and and potentially allowing ad-hoc changes using SQL-like commands rather than domain-rich commands. Netflix who initially warmed up to the idea of exposing its films database to the public, reached the conclusion this is not such a RESTful idea and retired its OData service.

Another issue is the server-centric design: server assuming how its services will be used. I believe (Sorry Roy!) HATEOAS touches the subjects discussed here. When we talk of Hypermedia As The Engine OF Application State, which application do we mean? OK, let's consider Twitter's API. How can you think same hypermedia can equally drive the state of a twitter client, a game that posts scores on twitter, 4Square app that declares you a mayor on twitter or a twitter realtime map showing location of tweets using the firehose data pipe? The relationship from clients to servers are not one-to-one any more and effectiveness of HATEOAS is questionable. Application is defined by the client and not the server, as such server's hypermedia cannot be the engine of the application.

At the end of the day, none of us want to sound like the pompous chef who bores the customers with self-flattery. Server serves, so design it for servant-hood.


* this post was originally posted slightly differently under the name: "Server Chauvinism - API's public domain"

5 levels of media type

[Level C4] HTTP spec defines use of media type in the value of serveral headers including content-type and Accept. Server and client can engage in the process of content negotiation to decide on the best suitable media type.

With the rising popularity of the REST systems and adoption of pure HTTP APIs, we are using media type not only for delivering content formatting information but also metadata for domain level application data. Advanced use of the media type and its controversies has been discussed before.

IANA is responsible for registering internet media types. RFC 4288 defines the process of media type registration through IANA.

As can be seen below, registration of media types increased by 2000 dot-com boom and then declined. This trend was reversed again by the REST awareness and resurgence of web 2.0 around 2006. Recently, we see a slight decline in the registrations - partly explained by the use of private media types.

Source: IANA - http://www.iana.org/assignments/contact-people.html

While there will always be a need for registering new formats, media type has been used to described not only the format but also the schema and domain level description of the application data. Use of media type for versioning resources is a controversial yet fairly popular trend.

The problem with creating new media types for describing anything other than format is that you you will be requiring the clients to understand the new media type - hence the client burden of the media type. Such clients could be very well capable of handling the format (and all they need might be to use understand the format) but unable to comprehend the media type. For example an XmlSerializer is capable of handling the XML format and that is all it cares about.

One such attempt to conserve the formatting information of the media types yet provide higher level constructs is the use of + in the second part of the media types such as application/rss+xml which combines formatting with schema (see below). But as I explained before many systems use a dictionary-based media type processing and cannot separate format and schema information and also this is rather a convention and not a canonical implementation.

I will review the logical levels of media type and then propose a solution for the current issues we are experiencing in the industry.

5 levels of media type

Media type deals with different levels of information. Any solution needs to take into account backward-compatibility, interoperability and extensibility. Media type can provide information at different levels. As the level goes up, the number of clients able to comprehend and interact with that level diminishes.

Lowest level of information is whether the content is human-readable. This was initially envisaged in the text/* media type but there again, it was mixing human-readability with the formatting. As we know text/xml and text/javascript later were converted to application/xml and application/javascript.

Next level is formatting, i.e. how a parser/processor can read and understand the media type. This is the most important aspect of a media type. Examples are application/xml, image/* and video/*.

Schema is a common superset of the formatting. Here we define different schema commonly within the same  format. Examples of this are application/rss+xml, application/atom+xml and application/collection+json.

Domain level is where we have a lot of new interest. Many companies are using private and public APIs for exposing their data and services. As such, a recent trend is to take the schema to the next level where it defines a domain object model. As we have described before here, domain model is part of the server's public domain and could be in the format of command or query messages.

Including version information for a domain object model is the highest level of a media type. This is useful by only a subset of the clients capable of version coherence and version content negotiation.

In fact clients each can use the media type at a particular level:

So forcing he higher levels of media type upon clients will reduce interoperability.

Solution

I propose using additional properties for preserving the interoperability and backward-compatibility yet allowing rich higher level information. Currently HTTP 1.1 allows for custom additional parameters to be defined.

For example, if I am using application/atom+xml for passing customer domain object model in a CRM API, I can keep the format in the value of the content-type header and include the rest of the information (Note: we have to replace / with _ since it is not allowed in the parameter value according to HTTP spec):

content-type: application%2fxml;schema=application%2fatom+xml;is-text=true;domain-model=MyDomain.Customer;version=1.0.2.0

Or alternatively use the schema level information as the main value:

content-type: application%2fatom+xml;format=application%2fxml;is-text=true;domain-model=MyDomain.Customer;version=1.0.2.0

Please note the values of parameters need to be UrlEncoded.

Compared to single-token approach, this will preserve the interoperability and backward-compatibility while allowing for extensibility.

Difference of single-token vs. 5LMT. Please note the the values need to be UrlEncoded so application/xml will appear as application%2fxml

Conclusion

Registering media types should be done mainly for new formats. The problem with using a single token is that by setting the token at a any level, lower levels need to be inferred - as such client needs to understand the exact media type.

5-level media type compared to the single token approach provides a more robust solution for extensibility and interoperability of clients and servers in private and public APIs.

Introducing Client-Server Domain Separation

[Level C4]

If you have followed my posts on REST and its client-server implications, you already know I have a thing for the client-server relationship.

I have been thinking about Client-Server Domain Separation CSDS for a while. And I think it is time to do a brain dump.

TLDR;

So here is the definition CSDS - if you don't want to read the whole post. CSDS defines a single constraint which is just an expansion on REST's client-server constraint:

Client and server must define and live in their own bounded context. 

This will lead to 1) cleaner separation of concerns among clients and servers and 2) adoption of API as the building blocks of complex systems in an SOA world. CSDS is also not compatible with HATEOAS - as we will see. If you need to find out how such a seemingly trivial constraint can have such an impact, read the rest.

Background

REST defines a set of constraints that will lead to better architecture and design - well that is the claim but I personally do believe in it. One of those constrains is client-server. As far as REST is concerned, client and server are decoupled entities - these ideas were successfully used in the design of HTTP. Yet considering limitations of the clients back in the day REST dissertation was being written, I think we need to re-visit this constraint.

I suppose it all started with smartphones. We now have more processing power in our pockets than the Apollo that landed on the moon for the first time. Native apps allow for developing pretty complex applications while HTML/JS app has become a reality (better browsers, adoption of HTML5, better javascript runtime and development tools). 

The dilemma we are faced now is to decide where to implement a functionality (in other words put the business logic): client or server. Back in the late 90s or early 2000s we did not have a choice - we had to implement most of the functionality on the servers and use client-side code mainly for limited validation. We lived in a time of server dominance. Now we have the liberty to implement a sizeable chunk of functionality in both places but getting the balance right is difficult and has lead to mainly two opposing camps: Single Page Application followers and server domination supporters. I am more inclined towards the first but shall explain below why CSDS will lean more on the SPA side rather than server domination - although not as a matter of taste but as a matter of principle - question is whose concern is a functionality.

Other changes in the industry have contributed to the need to define client and server. Nowadays, it is only incidental that the client code in an html/js application is served from the server - we can package up javascript files with the application as in PhoneGap or Windows 8 metro applications. In mash-up applications, there is no single server defining the flow as such HATEOAS is meaningless.

Introduction

In here I briefly re-iterate what I explained in two related posts.

First of all, in order to decide where a functionality belongs - server or client - we need to understand whose concern it is. I talked about server-concern, client-server and mixed-concern and I explained their anti-patterns, each with an example.

In this post I tried to define client and server - as they stand now. So I am going to go back to the same definitions - with client definition slightly changed.

Server is responsible for defining a domain (server domain) and maintaining its state and consistency/integrity. Server is usually very complex but a good server hides its complexity behind its API. Server should not expose its internals to the outside world.

Server in CSDS

Client is responsible for using server(s) services to provide value to the end user - either directly if it is a client device or indirectly if client itself is a server. Client can also maintain a state but it is not its primary function.

We also touched on Application. For me application => value => user => client. Application, use and usability is mainly a client concern. Having said that, server will define a secondary level of API which could use underlying basic APIs and present a more useful representation of its state. As such, server is not completely oblivious to the user/value. For example, a high-street bank could have an API returning 10 most recent transactions on your account defined as a resource at /account/{id}/transaction/mostrecent,since this is a very common query. This is instead/or in addition to providing an API which allows client to define date ranges and number of transactions returned. The application sitting on a client device does have the liberty to show only 6 of them if its usability mandates such restriction.

CSDS definition

So CSDS can be seen as a superset style on the top of REST with a single constraint on REST's client-server constraint. This is similar to, for example, HATEOAS which builds upon hypermedia constraint. So the constraint is:

Client and server must define and live within their own bounded context.

In other words, decision to where put a functionality is to ascertain whose concern it is. Is that it? Yes, that is it. Yet, this is going to have quite a big impact as well as important repercussions.

First of all since each defines its own boundary, client's domain is separate from server's domain. Their interaction is only through the API. "Domain objects" in the diagram above are usually regarded as view-models which are a translated version of the corresponding models in these two different domains - which in DDD terms are called context map.

By keeping client and server in their own bounded context, internals of each can be changed independently. By separating the domains, we achieve the client-server decoupling which is the goal of the client-server constraint - as Fielding puts it.

So here are some of the aspects and implications of CSDS:

Client has full coherence of server's public domain

This means that client is free to have full coherence of server's public domain including all its public API, domain objects and schemata. It is able to call, discover and make full use of the public API in any order or fashion it needs.

Server is responsible for versioning its public domain

In CSDS, client building dependency on the top of the public domain is not regarded harmful and actually is seen as essential. Server already knows that by changing the public domain, it will be breaking clients as such server is responsible for visioning its public domain.

Server has got no clue about the client

In CSDS, server has no reliance on its knowledge about the client calling it. Of course, in HTTP, it can use user-agent header for statistical purposes. Or in the case of OAuth, it can know the name of the application and perhaps even limit the scope of the public API according to that but this is an authorization concern - authentication and authorization of the calls are server concerns. In other words, it should not make any assumptions about the client, client device or its capabilities.

Server got no clue about the client - one of the clients could be a server itself (not shown, could not find the original visio to add the server :( )

CSDS, HATEOAS and hypermedia

CSDS is not compatible with HATEOAS. Why? Well, HATEOAS talks about hypermedia (a server concern but part of public domain) as the engine of the application. What application? Server got no clue about it. When I am listening to Spotify, I can tweet the song I am listening to. Publishing this tweet is no different to doing this from a twitter client, TuneIn radio client, etc. Server does not know what application is using it (although as we said it could know the name of the application in OAuth as a string) or where in the application this tweet happens. As such it cannot be the engine of the application. Also in a mash-up application, no single server could be the engine - there are multiple servers. 

CSDS regards hypermedia an important aspect of REST; it is a semantic web of interconnected resources. Client will have full coherence of the axes of such relationships and can effectively use to navigate the semantic web - since it is part of the public domain. But for it to become the engine of the application is server dominance.

Server has a lot to worry about

CSDS acknowledges utmost complexity of the server. Reliable storage, big data, high availability HA, sharding, resilience, redundancy, etc are all server concerns. Implementing the right server-side architecture is not easy as such server is best to focus on its own concerns rather than dominating the client by implementing client's concerns too.

CSDS leads to a cleaner SOA, especially when client itself is a server

Recent server-side challenges and trends in achieving a scalable and highly available architecture has added the focus for achieving the right balance in the client-server separation.

Listening to Adrian Cockcroft's talk in Cambridge on Netflix's architecture and having read Daniel Jacobson's book, I have a lot of appreciation for what these guys are doing and I think this will become a roadmap for a cleaner and more decoupled SOA. Adrian explained how in Netflix, they have used a web of micro-SOA services through REST APIs to create a resilient architecture whereby they even send chaos monkeys and gorillas to bring down servers or even server zones. I believe this is only possible with separating domains of each micro-SOA service. So a lot of kudos to them and it is a place to watch.

What does REST's Client-Server mean now?

Introduction

[Level C4] In "What does coupling mean ...", we reviewed three client-server patterns (with their anti-patterns) based on the concern. In this post, we will carefully examine the new meaning of client-server web applications. This will serve as a primer on my new work to define Client Server Domain Separation (CSDS).

Motivation

As I explained in the coupling post, I was challenged by recent emergence of attempts to define client-server relationship, one of which being ROCA. There seems to be a recent trend in the REST-aware community by disgruntled developers who believe too much control has been shifted to the client-side. As such they are trying to come up with practices/styles to move some of the control back to the server.

Background

REST has a strong emphasis on separation of client and server. Fielding in his PhD dissertation outlines the constraint:

Separation of concerns is the principle behind the client-server constraints. By separating the user interface concerns from the data storage concerns, we improve the portability of the user interface across multiple platforms and improve scalability by simplifying the server components

As we saw in the coupling post, we need to understand if a functionality is client-concern, server-concern or mixed concern. Implementing the functionality in the wrong side of wire can be initially tolerated but finally takes its toll. Below we will try defining building blocks of our discussion.

When we talk about the domain below, we refer to the concerns implemented/expose in the client or server.

Server

Server is responsible for defining a domain (server domain) and maintaining its state and consistency/integrity. Server usually has very complex components yet it hides its complexity behind its services.

Figure 1 - Server exposes a public domain hiding its complexity

Service is composed of API and domain objects. In a RESTful world, API is HTTP REST. Domain objects are representative of the server's domain model. While Server can have a complex domain, we refer to server domain as only the publicly available domain.

Server should not have a knowledge of the client type. Although such information can be shared with the server (for example through User-Agent header), it should not be used other than for auditing or statistics.

Domain objects sent to the client are usually looked down upon and treated as second class citizen. They are  sometimes called DTO (Data Transfer Object) or ViewModel. While this is OK in a server development scenario when the focus is to decide how much of the server's whole domain to be exposed, these models are not to be mistaken with Value Objects since they are entities (they have identity according to DDD).

Domain object can be a fully rendered HTML (markup) in its semantic form (i.e. no display semantics such as <b> or <i>). Documents are domain objects for example a blog domain has a post domain object.

Server itself can be a client of one or several servers. ifttt is a beautiful example of it. 

Client

Client is responsible for using server(s) services to provide value to the user. In the process, it defines a domain which is usually different from server domain, although there is always an overlap. Client has a life of its own, able to maintain some level of functionality with no server access.

Figure 2 - Client and server domains - now and before

Please note that we did not mention user on the definition of server. Server is being abstracted away from the user by the client. Client can still provide some of its values to the user even when server down. Funny enough, I lost connectivity for half an hour while I was typing this blog and I carried on typing. When re-connected, Blogger saved my worked.

So let's bring an example to highlight a few important points:

I love listening to online radio while working. I use TuneIn on my android to listen to music based on my mood. TuneIn is a great app that allows you to search online radios and podcasts. So one of the functionalities is the directory service of radio stations. This functionality is provided by TuneIn servers (that maintain the state and its consistency/integrity). It defines a server model that consists of name, URL, style, icon, etc. Client does not know how the directory is created or how often the information gets updated.
On the other hand, when I click on a station to listen, I connect to the radio server. Domain of the server has music streaming, current artist, etc. For the radio server, it is all the same if you listen to music in your browser or in the native client on your phone. Client does not know how music is stored, chosen, etc.
Now if I really like a song, I can share it in twitter. Twitter server, while for security reasons (OAuth) knows the application I am using, it does not really care. Publishing my tweet is all the same for it.

So as you can see, Twitter's server domain is fully concerned with users, their tweets, re-tweeting, etc while TuneIn client only cares about publishing a tweet, so their domains have a tiny (yet important) overlap.

Some the client functionality has nothing to do with server. Playing music on the device is fully a client concern so does not exist on the server domain. For example, online radio's streaming servers would not know if the data they are sending will be even heard by the user (e.g. speaker could be on mute or worse, the client uses the data for illegal dumping of the songs).

Let's have a look at Figure 2. I think our TuneIn example fully described the "Now" diagram so let's focus on the "classic" case. This is the classic client (a web application running on the browser - see below) where all the logic served by the server. In extreme cases, server even generates client scripts apart from hosting the static logic (Javascript). Client's domain is fully engulfed by the server meaning server is aware of all the client logic.

Does the classic client look to you as the REST Eutopia? Having read Fielding's dissertation snippet above, which one do you think represent REST better?

Web Application

What is a web application? This definition has been drastically changed over the last few years with the emergence of diverse client devices capable of running advanced Javascript or native code. Web applications can be found in the forms below (not particularly ordered and probably some missing):

• Single Page Application (SPA) running on various devices including top-end mobile devices
• Native rich clients on desktop/laptop
• Native client apps on mobile devices
• Bundled HTML/Javascript apps running on mobile devices (PhoneGap) or desktop (Windows 8)
• Traditional web applications run by browsers
• Browser as an application to display markup

Figure 3 - Server cannot really see behind the cloud and which client is using it

Web application is the usage abstraction of the client. While the server domain used to pretty much define the web application, client is becoming more and more important in defining it.

Now what does web do in the "web application"? Nowadays, almost every application is a web application: the client uses one or more cloud services to enrich the experience. Unless it is a simple drawing or editing tool, most applications are web applications.

Some of the forms deserve more attention. Bundled HTML/Javascript apps remind us that in some cases it is in fact incidental that the server hosts the files. Some Javascript files are hosted on CDN and downloaded and cached for a long time.

Also not all logic is Javascript. Microsoft RIA service (regardless of whether I like it or not - and I don't!) sends server's domain rules to the client very much like Javascript. Generating Javascript or binaries is all bad the same as it breaches client's independence.

Web Application and REST

Basing your web application on REST will help achieving better separation as well as making an efficient use of the web.

Having said that, today's world demands more and more from the computing industry. Server Push model (gaining popularity in node.js/websocket world) requires a stateful server which is a REST no-no but today's virtualisation and cloud elasticity has made scalability a much smaller problem.

For most web applications, however, following all REST constraints is the best practice as very few applications require Server Push model.

So what do I think of the new trend?

Well, I think the trend cannot resist the wind of change. Computing industry has been pushed to provide more value and has to do it cheaper, faster and richer. Separating client and server will help to achieve this more effectively:

With regard to ROCA I must say it is a worthwhile effort to understand the client and server interactions and contains useful common-sense practices. However I cannot subscribe to it since:

• must-server advocates "classic" model (engulfed client) and ignores the client server separation prescribed by REST. 
• It does not respect client domain (must-no-duplication) and it rules and logic.
• Single-Page-Application is not ROCA-compliant (see discussions).
• Not all clients are browsers, in fact less and less clients are browsers. ROCA is heavily targeted for browser applications with many of the constraints directly related to HTML, CSS or Javascript
• It does not fully appreciate the inherent complexity of the client domain (must-jslimits and mustnot-jsengine)
• A ROCA client will not be able to provide any useful offline feature
• Includes lower-end non-browser clients yet does not appreciate upper-end clients (must-non-browser)

Conclusion

Client and server have their own domain - as REST prescribes. Server defines a domain and maintains its state and integrity. It hides its complexity behind its services while other than authentication and authorization does not need to know anything about the client.

Client through the usage of server(s) services provides value to the user. It defines its own domain that has overlap with server(s) domain.

Web application is a plethora of different devices and technologies. As such, defining a style requires considering all such scenarios.

We will talk more about Client-Server Domain Separation (CSDS) in the upcoming posts.