Monday, 18 March 2013

CacheCow 0.4 released: new features and a breaking change

Version 0.4 is out and with it a couple of features such as attribute-based cache control and cache refresh (see below). For the first time I felt that I have got to write a few notes about this version, least of which because of one breaking change in this release - although it is not likely to break your code as I explain further. Changes have been in the server components.

I and other contributors have been working on CacheCow for the last 8 months. I thought with a couple of posts I have explained the usage of CacheCow. But I now feel that with concept counts increasing, I need to start a series on CacheCow. Before doing that I am going to explain new concepts and the breaking change.

Breaking change

The breaking change was a change in the signature of CacheControlHeaderProvider from Func<HttpRequestMessage, CacheControlHeaderValue> to Func<HttpRequestMessage, HttpConfiguration, CacheControlHeaderValue> to accept HttpConfiguration.

If you have provided your own CacheControlHeaderProvider, you need to provide HttpConfiguration as well - which should be very easy to fix whether web-host or self-host.

Cache Control Policy

So defining cache policy against resource have been by setting the value of CacheControlHeaderProvider which you would define whether a resource is cacheable and if it is, what is the expiry (and other related stuff):

public Func<HttpRequestMessage, HttpConfiguration, CacheControlHeaderValue> CacheControlHeaderProvider { get; set; }

So by default CacheCow sets Func to return a header value for private caching with immediate expiry for all resources:

CacheControlHeaderProvider = (request, cfg) => new CacheControlHeaderValue()
{
 Private = true,
 MustRevalidate = true,
 NoTransform = true,
 MaxAge = TimeSpan.Zero
};

Immediate expiry actually means that the client can use the expired resource as long as it validates the resource using a conditional GET - as explained before here.

But what if you want to individualise cache policy for each resource? We could use per-route handlers but that is not ideal and generally it depends on the resource organisation approach. I have explained in my previous post that resource organisation is one of the areas that needs to be looked at. But this is not within the scope of CacheCow. We are looking into solving this as part of another project while ASP.NET team are also looking into this. So I have decoupled the resource organisation project from CacheCow.

Having said that, in the meantime, I am going to provide some help with doing cache policy set up less painful. This means that CacheCow will come with a few pre-defined functions that help you with defining your cache control policy.

Good news! Cache policy definition using attributes

So now you can define your cache policy against your actions or controllers or both - although action attribute always takes precedence over controller. Using the popular ASP.NET Web API sample:

    [HttpCacheControlPolicy(true, 100)]
    public class ValuesController : ApiController
    {

        public IEnumerable<string> Get()
        {
            return new[] { "cache", "cow" };
        }

        [HttpCacheControlPolicy(true, 120)]
        public string Get(int id)
        {
            return "cache cow... mooowwwww";
        }

So GET call to the first action (/api/Values) will have a max-age of 100 while GET to the second action (e.g. /api/Values/1) will return a max-age of 120.

In order to set this up, all you have to do is to set the CacheControlHeaderProvider property of your CachingHandler to GetCacheControl method of an instance of AttributeBasedCacheControlPolicy:

cachingHandler.CacheControlHeaderProvider = new AttributeBasedCacheControlPolicy(
 new CacheControlHeaderValue()
  {
   NoStore = true
  }).GetCacheControl;

So in above we have passed a default caching policy of no-caching. This table defines which attribute value (or default provided in the constructor) is used:



Cache Refresh Policy

CacheCow works best when HTTP API is actually a REST API. In other words, it uses uniform interface (i.e. HTTP Verbs) to modify resources and this means that the caching handler will get the opportunity to invalidate and remove the cache when POST, PUT, DELETE or PATCH is used.

Problem is commonly HTTP API sits on the top of a legacy system where it has not control over modifications of resources and acts as a data provider. In such a case, the API will not be notified on resource changes and application will be responsible for removing cache metadata directly on the EntityTagStore used. And this is not always possible.

I am providing a solution for defining a time based cache refresh policy using attributes in a very similar fashion to Cache Control Policy - even the above table applies. Removal of items from cache store on the server happens upon the first request after the refresh interval has passed not immediately after interval. So we add the refresh policy provider:

cachingHandler.CacheRefreshPolicyProvider = new AttributeBasedCacheRefreshPolicy(TimeSpan.FromSeconds(5 * 60 * 60))
    .GetCacheRefreshPolicy;

We have defined 5 hour refresh policy as default. And we override using controller or action attributes.

Future posts

As promised, next few posts will be a CacheCow walk-through. 

Sunday, 10 March 2013

Pro tip: don't try abstracting the code world from the HTTP world

[Level T2] I have explained in my Part 5 of the Web API series how the world of HTTP (URI, headers, method, payload) meets the world of code (controller, actions, parameters, etc). This has partly been done in ASP.NET Web API in the media type formatter. And then I talked about the Tower Bridge of MediaTypeFormatter. The problem is these two worlds will eventually come to collide hard - if you ever try to abstract them away.

So what am I on about?

Problem is routing - as I have raised a few times before. And here I will try to explain why - and propose a solution.

The routes basically live in the application setup. You define them outside your controllers and actions - usually inside glolbal.asax or outside while being called on application startup. The idea is that no matter what your routing is, if set up correctly, the values will be populated in your actions - all parameters, and your data. And then your return some data which will be interpreted and serialised in your MediaTypeFormatters.

So what happens if I want to set the location header in the response to a POST method (when an item has been created)? Now I need to provide the parameters to the route to create a virtual link. Now here although I have been abstracted away from the route, I have to know the parameters to pass - this is where things start to get a bit hairy. I have defined a route somewhere else and yet I need to know about it here.

And that by itself is not a big problem. A bigger issue is that ASP.NET does the route matching linearly. So if I happen to define 1000 routes, and my 1000th route gets call often, I can be in trouble as performance can suffer: ASP.NET should always try to match to 999 routes until it hits 1000th.

But why should I have 1000 routes? What is wrong with /api/controller/id? I could end up with handful of routes? OK, here is the catch: routing was designed for a flat routing structure (and originally for ASP.NET MVC) while resource organisation in a Web API has to be hierarchical.

Another issue is hypermedia. Have you tried doing clean and robust hypermedia in Web API using the existing routing? Good luck with that one!

Another problem is the resource in the Web API's really elegant HTTP pipeline (which I love) is URL until it hits the controller dispatcher and then tries to find a matching controller and then action. As such, code part of the resource (controller, action, parameters) cannot programmatically (using attributes, etc) define a strategy to the delegating handlers since we do not yet know which controller or action will respond.

Resource organisation is the responsibility of the application

There were a few issues  raised with CacheCow in terms of cache invalidation of related resources. This is actually one of the fundamental scenarios that I designed CacheCow for. Problem is resource organisation is the responsibility of the application as such CacheCow cannot make any assumptions about the organisation of the resource. So the solution is for the application to describe resource organisation and the relationship of resources. This is definitely not easy - as again, world of routing is a disjoint one.

So here is what we need:

  • A hierarchical routing with a natural setup rather than arbitrary route definition 
  • Resources aware of related resources. As such a resource can define its hypermedia for the most part
  • Ability for resources to define their strategy when it comes to caching, etc


Once asked by Youssef Moussaoui - a developer in ASP.NET team - on what to improve in ASP.NET Web API, my immediate answer was routing. And not just routing, it is resource organisation.

So I know that ASP.NET will look into this but I probably will have a stab at this. We have defined the project Resourx which will look at this and try to achieve the 3 goals above. Watch the space!

Sunday, 10 February 2013

CacheCow project - Current status and future roadmap

For those of you who might not know, CacheCow is an open source project that I started around 8 months ago to address the server-side and client-side caching in the new Microsoft HTTP stack - also known as ASP.NET Web API. I have been fortunate to get regular help, feedback and contributions from geek friends  (Tugberk, Alex Zeitler, Sayed Hashimi) and I am pleased to announce that version 0.3 is out now. As part of this release, cache storage implementation for Memcached has been released.

There are currently 9 packages available - have a look here.

CacheCow.Server

This is the server side component of CacheCow which works as a DelegatingHandler and looks at the request/response and implements server-side caching according to HTTP spec. This includes inspecting headers, generating ETag and adding headers, responding to conditional GET and PUT, etc. The idea is that you should only have to declare your caching strategy against the handler and it should take care of the rest.

Storing ETag and other important headers such as LatsModified (which I collectively call Cache Metadata) against the resources requires a persistent or in-memory storage. CacheCow.Server by default comes with an In-Memory implementation of the storage. However, these other storages are currently available:
  1. MongoDb
  2. RavenDB
  3. SQL Server

CacheCow.Client

As part of ASP.NET Web API, a new shiny HttpClient has been developed which uses the same HTTP pipeline model as the server. While you might be consuming Web API using JavaScript on the browser, I see that more often than not, this will be consumed by HttpClient on the server as more and more APIs move to Web API from WCF and classic (ASMX) web services.

As such, I believe the client side story of the caching is an important - and somehow neglected - one. So you will find a client-side CachingHandler in CacheCow.Client. Now on the client instead of storing Cache Metadata, we need to store actual responses. There are even more storage implementations available on the client:
  1. Redis
  2. File-based storage
  3. SQL Server
  4. Memcached

Future roadmap

There is a lot of work there to be done. [If you would like to contribute, please let me know]

Basically the plan is to implement below in the next 4-6 months:
  1. Turning all storage API to asynchronous. So IEntityTageStore and ICacheStore will turn completely async.
  2. Implementing total and per-domain storage quota for the CacheCow.Client. Part of this work has already been done but need to be consolidated.
  3. Implement all storages for both client and server:
    • Server-side:
      • Redis
      • Memcached
    • Client-side
      • RavenDb
      • MongoDB
  4. Implement resource organisation so declaring cache plan for a resource is easy (currently is not)
So watch this space. If you would like to contribute, please message me in twitter.

Saturday, 2 February 2013

Performance series: GUIDs vs. IDENTITY INT in RDBMS

[Level C2] We have been discussing with a few colleagues about whether we should adopt GUIDs as our primary keys. At the same time, we had a meeting with our DBAs to discuss other scenarios and the topic came up.

When the systems become more complex and you have more layers between your user interface and database, not knowing the ID of the aggregate root before storing can pose unnecessary challenges. On the other hand, the cost of choosing GUID as the primary key of the database is usually unknown. That is why I set out to find out the exact cost of choosing GUID instead of IDENTITY INT as the primary key of the table.

Pros and cons

This is not a new topic. There are quite a few resources that discuss this topic and list the pros and cons. Jeff Atwood's blog post from 6 years ago is a short and sweet one explaining why StackOveflow decided to move the IDs (at least some of them) to GUID. It also contains many references to other related posts that discuss the topic - all of which are good reads especially this one.

Why GUID?

For me, the main reason to choose a GUID is to know the ID of the entity even before persisting. If you do not have this requirement, you might still want to use GUID and avoid IDENTITY INT especially if you are using sharding or master-master replication. But for me, the first one is vital. Why?

If you use CQRS and commands to store your entities, you would naturally implement commands as an asynchronous operation. Your UI would have to find out about the result of the operation by polling or subscribing to the event published after processing the command. If you do not know the ID of your entity before persistence, you would end up using a different ID as a reference.

Problem with GUIDs

Apart from GUIDs being unreadable, there are performance implications for using GUIDs. The main ones are:

  1. GUID is a 16-byte type while INT is a 4-byte type. With storage nowadays very cheap, this normally is not a problem from the storage point of view. The problem is that reads (and writes) will be multiplied by 4x.
  2. The most serious problem is when you use the GUID ID as the clustered index of the table. In this case, with every INSERT you would be changing the layout of the data potentially having to move many data pages. This is unlike IDENTITY INT clustered indexes where data is stored sequentially providing the best performance.

Benchmarks

This old article provides a benchmark of the cost of the writes. Considering very little has been changed with the basic functionality of the SQL Server, this benchmark is still relevant - if not accurate. According to the results, cost of writing GUID primary keys in a database containing 1,000,000 records is 10x the cost of writing INT primary keys. This cost goes up exponentially when the table contains more rows.

One of the solutions presented in the article is to have a semi-sequential GUID generated in the database. In fact since SQL Server 2005, there is an option to do that using NEWSEQUENTIALID(). This will change the overhead to a mere linear 10% on both reads and writes.

The problem with these solutions is that the GUID is still generated in database so does not solve the problem of knowing the ID before storing the entity.

My solution and benchmark

As discussed, the problem generally is to do with the non-sequential nature of the GUIDs. But who said we should use the primary key as the clustered index?!

Basically my solution is to:

  1. Add an IDENTITY INT column (I called it order Id - nothing to do with customer orders) and set it to be the clustered index. If you do not want to use an additional column, you could use a DATETIME column to store a timestamp which is usually very useful
  2. Add a UNIQUEIDENTIFIER column (GUID data type in SQL Server) as the main ID. Set that to be the primary key.
  3. Add a non-clustered index for the GUID ID

This will keep the data storage sequential while benefiting from system generated IDs instead of database generated IDs. There are two drawbacks: 1) storing an additional column 2) having an additional non-clustered index which takes some space and makes reads and writes slightly slower. Bear in mind, this is  recommended only if you keep the table only for transactional usage and to store and retrieve by key and not for reporting. In most cases, this is what you would normally do especially if you are implementing CQS or CQRS.

* Using an IDENTITY INT as the clustered index

As can be seen, using this technique, performance of the GUID primary key is close to the performance of the INT primary key (roughly 10% overhead).

Conclusion

Performance of a GUID primary key is acceptable and adds mere a 10% overhead, if we use another INT or DATETIME timestamp as the clustered index.


Sunday, 20 January 2013

Review of .NET Framework cryptography and symmetric algorithms benchmark

[Level T2] .NET Framework provides a set of cryptography services in System.Security.Cryptography namespace. This namespace has been there for years and I have been using some of its classes on and off. Yet, I felt I need to make some research into all the services it provides. And this post is the result.

Symmetric and asymmetric algorithms

Symmetric algorithms use a secret key/password (along with an Initialisation Vector IV) to encrypt and decrypt data. A number of these algorithms have been described and documented. Secret key needs to be stored safely as whoever gets hold of the key can encrypt/decrypt the data. These algorithms are efficient and fast and can encrypt/decrypt large amount of data - they can even encrypt streams of data.

Initialisation Vector is used for encrypting the first block of data. It is generally not considered to be a secret and can be passed as plain text. However, the same IV needs to be used for both encryption and decryption. More information can be found here.

Algorithms available in .NET Framework:

  • DES: One of the oldest. Uses 8-byte keys.
  • RC2: Can use keys of 5-16 bytes.
  • TripleDES or 3DES: An advanced and more secure DES. Uses keys of 16 or 24 bytes.
  • Rijndael: A purely managed implementation. Can use keys of 16, 24 or 32 bytes.
  • AES: A purely managed implementation. Uses 16-byte keys. [Please see @leastprivilege comment at the end as AES is an alternative Rijndael implementation]

Performance of these algorithms are compared here (code to re-produce this can be found on GitHub):


As can be seen, AES and Rinjdael provide the best performance, partly due to the fact that they are purely managed. RC2, DES and 3DES are older algorithms and in terms of security not the most reliable so best to be avoided. So when it comes to choosing between an algorithm, use AES or Rijndael.

HMACSHA1 is a symmetric algorithm but is not used for encryption. This is mainly used for creating hash-based message authentication code which is a kind of signing.

Asymmetric algorithms on the other hand use a public/private key pair. These algorithms can be used for encryption as well as signing. Public key is used for encryption and verifying signatures while private key is used for decryption and generating signatures. So the idea is that the public key is shared publicly while the private key has to be stored securely. X509 certificates are a common type of public/private key pairs and usually installed on the machine's keystore so the operating system (and resource ACL) is responsible for its security.

Asymmetric algorithms are used for encrypting small pieces of data. So unlike symmetric algos, they cannot be used for encrypting large data or streams. So they are usually used to encrypt symmetric algo secret keys. Signing is an important use of asymmetric algorithms. .NET Framework itself uses it for signing assemblies (strong naming assemblies).

The most widely used algorithm is RSA which is implemented in.NET framework.

An example of using symmetric and asymmetric algorithms

SSL/TLS is an example of using both symmetric and asymmetric algorithms in the same security context session. In this protocol, client and server negotiate a size for the encryption key (as well as other aspects including the algorithm to use) and then client generates a random secret key of the agreed size for symmetric encrypting the communication. Then it uses public key of the server to encrypt the secret key and sends it to the server. Server, being the only entity in possession of the private key, decrypts the secret key. This secret key is used for the lifetime of the secure transmission to symmetrically encrypt (and decrypt) the data.

Other cryptography services

Random generation

Most likely you have used the class Random in the .NET Framework. This class is capable of generating random bytes that can be used as secret keys:
var random = new Random();
var buffer = new byte[1024];
random.NextBytes(buffer);
While this code works, Random class is incapable of generating completely random values, i.e. the values it produces are pseudo-random as it chooses from a finite set of numbers.

For producing numbers with cryptography-level randomness use RNG:
var buffer = new byte[1024];
using (var rng = new RNGCryptoServiceProvider())
{
    rng.GetBytes(buffer);
}

Hashing

Hashing is the process by which a signature of small fixed size is created for a much larger piece of data. Hashing algorithms such as MD5 or SHA can be used to generate such hashes. Hashing algorithms can also use a secret key which can be used while generating the hash. Ability to produce the hash with a given secret key can be used as a sign of the ownership of the secret without having to send the secret. This technique is used in OAuth using HMACSHA1 algorithm. [OAuth2 is different as per @leastprivilege comment. Please see comments]

There are a number of algorithms implemented in .NET Framework. MD5 is best to be avoided and one of the alternatives of SHA algo should be used. In case of a need for key-based hashing, use HMACSHA1.

How to store the secret key of the encryption

We are commonly in need of encrypting pieces of sensitive data and storing them. These could include content of cookies, user sensitive information, etc. For using a symmetric encryption, we need to use a secret key. So where should we store the secret key?

I have seen many cases where the secret key is hard-coded within the code, usually as a string. First of all, such secret keys can be easily retrieved using reverse engineering tools such as Reflector. Also, these strings are commonly meaningful strings, that usually can be guessed easily.

One way to have a secure secret key is to:
  1. Create a random secret key (of required size) using algorithms such as RNG
  2. Install an X509 certificate on the servers (private and public keys)
  3. Use asymmetric encryption to encrypt the key and store it for example as a file
  4. At the time of using the key, use the private key to decrypt the secret key and then use the key for symmetric encryption/decryption


Conclusion

Avoid older encryption or hashing algorithms such as MD5, DES, RC2 or 3DES. Use newer and managed-only implementations such as AES, Rijndael (and SHA for hasing). AES provides the best performance out of all symmetric encryption algorithms. Use combination of symmetric and asymmetric algorithms for secure encryption. Use RNG if you need crypto-level randomness.

Special thanks to Dominick Baier @leastprivilege for reviewing this article and posting comments which you can read in the comments section.

Monday, 7 January 2013

Performance series: Is thread contention bad?

[Level T2] For better or worse, I have recently been working a lot on the area of performance optimisation and bottleneck hunting.

Performance monitoring is an art as much as it is a science. On one hand you have all various metrics (simple, complex or derived) that a system can produce (at each level, from the client to presentation layer to middleware to database) and on the other hand you require the art of comparison and the ability to decide to ignore or act upon a particular change. The work can be exciting at times while slow and frustrating at other times - finding performance bottlenecks could be difficult but when found they are extremely gratifying!

I am going to have a separate post on performance monitoring an ASP.NET web application but I will use this post to look at a particular scenario and that is the thread contention. This is especially important in the face of increasing popularity of asynchronous programming and use of .NET 4.0's TPL and .NET 4.5's async/await.

So let's imagine this is the problem statement:
From version 1 of the software to version 2 of the software, thread contention has increased substantially. 
And imagine you are responsible for your application's performance. What would you do?

What is Thread Contention?

In .NET, contention rate/second is the performance counter to measure the level of contention as far as the managed threads are concerned. This measures number of unsuccessful managed lock attempts per second. Where would you take a lock? At the time of synchronisation. So contention rate is more a measure of synchronisation rate rather than the multi-threading. 

To confirm the above statement, let's use a console application and monitor the contention rate / sec in the performance monitor for the console application process. Initially, let's try this code:

static void Main(string[] args)
{
    Parallel.For(1, 1000000, (i) => Math.Asin(Math.Log(i)));
    Console.Read();
}

In this case contention will stay at 0 since we have used multi-threading yet we do not have a shared resource requiring locking hence contention is zero. Second scenario is when we write the iterator/counter to the console:

static void Main(string[] args)
{
    Parallel.For(1, 1000000, (i) => Console.WriteLine(i));
    Console.Read();
}

Now performance monitor will show a very high number for contention rate (on my machine it is average of 200). But wait, we did not use any locking here?! Well, we did not but Console did - all calls to Console class is synchronised so it internally uses locking.

Can Thread Contention be healthy?

Absolutely! Contention is just a measurement and you should not base your judgement on Guide Metrics. Guide metrics such as "% Processor Time", "Contention Rate/sec", "Number of Threads" are to be looked at only in the presence of a deteriorated Goal Metrics such as throughput (related to scalability) and performance. For example increase in response time or a drop maximum number of concurrent requests/users is something to look at but increase in % Processor Time can be a sign of higher/better utilisation of the system in the case of removing a logical bottleneck (for example coarse locking).

So always look at guide metrics in the light of goal metrics.

An important example of this in case of contention is use of IO completion ports. Windows defines two types of threads: worker threads and IO completion ports. Worker threads are known to most developers and can be used for processing a piece of work in the background. .NET framework (and .NET runtime) has a ThreadPool which contains worker threads. IO completion ports are Windows's low overhead threads that can carry out an IO-bound operation and notify the commissioning thread when the job is done. These threads will be used by the .NET runtime as soon as you use .BeginXXX and .EndXXX on an IO related class such as FileStream.

ContentionTest is a simple project that illustrates use of IO completion ports. Just try it for yourself and see that setting async variable to true will improve the performance but will make the contention rate to go rocket high.

Conclusion

Always look at guide metrics objectively. Do not try to improve guide metrics, only focus on goal metrics. If a drop in goal metrics coincides with a change in a particular guide metrics, review the relationship.

Thread contention rate is a measurement of thread synchronisation. This rate can go up in various conditions such as increased throughput, use of IO completion ports or removal of a bottleneck in which case there is no cause for concern. If high contention coincides low throughput, consider refactoring the synchronisation.

Saturday, 8 December 2012

5 levels of media type

[Level C4] HTTP spec defines use of media type in the value of serveral headers including content-type and Accept. Server and client can engage in the process of content negotiation to decide on the best suitable media type.

With the rising popularity of the REST systems and adoption of pure HTTP APIs, we are using media type not only for delivering content formatting information but also metadata for domain level application data. Advanced use of the media type and its controversies has been discussed before.

IANA is responsible for registering internet media types. RFC 4288 defines the process of media type registration through IANA.

As can be seen below, registration of media types increased by 2000 dot-com boom and then declined. This trend was reversed again by the REST awareness and resurgence of web 2.0 around 2006. Recently, we see a slight decline in the registrations - partly explained by the use of private media types.

Source: IANA - http://www.iana.org/assignments/contact-people.html
While there will always be a need for registering new formats, media type has been used to described not only the format but also the schema and domain level description of the application data. Use of media type for versioning resources is a controversial yet fairly popular trend.

The problem with creating new media types for describing anything other than format is that you you will be requiring the clients to understand the new media type - hence the client burden of the media type. Such clients could be very well capable of handling the format (and all they need might be to use understand the format) but unable to comprehend the media type. For example an XmlSerializer is capable of handling the XML format and that is all it cares about.

One such attempt to conserve the formatting information of the media types yet provide higher level constructs is the use of + in the second part of the media types such as application/rss+xml which combines formatting with schema (see below). But as I explained before many systems use a dictionary-based media type processing and cannot separate format and schema information and also this is rather a convention and not a canonical implementation.

I will review the logical levels of media type and then propose a solution for the current issues we are experiencing in the industry.

5 levels of media type

Media type deals with different levels of information. Any solution needs to take into account backward-compatibility, interoperability and extensibility. Media type can provide information at different levels. As the level goes up, the number of clients able to comprehend and interact with that level diminishes.

Lowest level of information is whether the content is human-readable. This was initially envisaged in the text/* media type but there again, it was mixing human-readability with the formatting. As we know text/xml and text/javascript later were converted to application/xml and application/javascript.


Next level is formatting, i.e. how a parser/processor can read and understand the media type. This is the most important aspect of a media type. Examples are application/xml, image/* and video/*.

Schema is a common superset of the formatting. Here we define different schema commonly within the same  format. Examples of this are application/rss+xml, application/atom+xml and application/collection+json.

Domain level is where we have a lot of new interest. Many companies are using private and public APIs for exposing their data and services. As such, a recent trend is to take the schema to the next level where it defines a domain object model. As we have described before here, domain model is part of the server's public domain and could be in the format of command or query messages.

Including version information for a domain object model is the highest level of a media type. This is useful by only a subset of the clients capable of version coherence and version content negotiation.

In fact clients each can use the media type at a particular level:
So forcing he higher levels of media type upon clients will reduce interoperability.

Solution

I propose using additional properties for preserving the interoperability and backward-compatibility yet allowing rich higher level information. Currently HTTP 1.1 allows for custom additional parameters to be defined.




For example, if I am using application/atom+xml for passing customer domain object model in a CRM API, I can keep the format in the value of the content-type header and include the rest of the information (Note: we have to replace / with _ since it is not allowed in the parameter value according to HTTP spec):
content-type: application%2fxml;schema=application%2fatom+xml;is-text=true;domain-model=MyDomain.Customer;version=1.0.2.0
Or alternatively use the schema level information as the main value:
content-type: application%2fatom+xml;format=application%2fxml;is-text=true;domain-model=MyDomain.Customer;version=1.0.2.0
Please note the values of parameters need to be UrlEncoded.

Compared to single-token approach, this will preserve the interoperability and backward-compatibility while allowing for extensibility.
Difference of single-token vs. 5LMT. Please note the the values need to be UrlEncoded so application/xml will appear as application%2fxml

Conclusion

Registering media types should be done mainly for new formats. The problem with using a single token is that by setting the token at a any level, lower levels need to be inferred - as such client needs to understand the exact media type.

5-level media type compared to the single token approach provides a more robust solution for extensibility and interoperability of clients and servers in private and public APIs.