Sunday, 22 July 2012

Introducing CacheCow: An HTTP caching framework for server and client


CacheCow

[Level T2] This is a short post to introduce CacheCow, an Open Source framework for HTTP caching on the client and server in ASP.NET Web API.

As some of you probably know, I have been working on caching for a while. If you go back to my post on CachingHandler, you will see that I contributed server-side HTTP caching implementation to the WebApiContrib project and included samples and tests.

However, I realised that even more important part of the caching needs to be implemented on the client. Also implementations of the IEntityTagStore on various databases (in-memory and persisted) and client-side's cache storage all need their own project so this is bigger than just a feature on WebApiContrib. As such, I have decided to start a new project and port the server-side from WebApiContrib. [Please bear in mind, the code in the WebApiContrib will be maintained and supported so if you are using it and have problems or experience bugs, please ping me in twitter or GitHub.]

So CacheCow framework has been born. The name itself is a word game with Cash Cow, meaning it does the heavy lifting for caching with minimal set up hence promises good return on investment :). Project is open source and hosted on GitHub. And yes, I do accept pull requests; Tugberk has done a great job (also with some help from Sayed Ibrahim Hashimi which I am so grateful for) and automated the whole build and NuGet package generation and his PR was merged pretty much immediately. But please contact me before-hand on the work you would like to do - there is ton of interesting work to do.

How to use CacheCow.Server

At the moment, only server-side CacheCow is ready for use. All you need to do is to use NuGet to get the package:

PM> Install-Package CacheCow.Server

This will add the CacheCow.Server and CacheCow.Common DLLs and the rest is all the same as the CachingHandler post and samples. Just add CachingHandler to the config:

GlobalConfiguration.Configuration.MessageHandlers.Add(new CachingHandler());

This will add this handler with all the default settings and will store the cache state in the memory. There are many dials that you can turn and configure the handler according to your resource organisation - just see the sample in WebApiContrib. This sample connects to the CarManager sample and tests various scenarios for a fairly complex resource organisation.

How to use CacheCow.Server.EntityTagStore.SqlServer

As I pointed out above, by default cache state is stored in memory. This is OK for single server or test scenarios but in case of a server farm you would like the cache state to be maintained for whole farm and when a resource cache invalidated, it is done for all servers. In this case you need a central EntityTagStore (cache state store).

Building a cache state store is pretty easy and all you have to do is to implement IEntityTagStore interface which has 5 methods. Since cache might be invalidated not just for a CacheKey (previously called EntityTagKey) but also for a RoutePattern (see CachingHandler post), key-value stores are not rich enough to provide this but conventional databases can be used.

So I have implemented this for SQL Server. In order to get this EntityTagStore, just use NuGet:

PM> Install-Package CacheCow.Server.EntityTagStore.SqlServer

This will download the DLL and also a script file named script.sql located in <project root>\packages\CacheCow.Server.EntityTagStore.SqlServer.0.1.0\scripts

So create a database and run this script against it, and you will get one table and several stored procedures. Then in order to use SqlServerEntityTagStore, create an instance pass it to the CachingHandler constructor. Default constructor relies on a connection string named "EntityTagStore" to be there in your web.config and pointing to your database.

GlobalConfiguration.Configuration.MessageHandlers.Add(
 new CachingHandler(new SqlServerEntityTagStore()));


Alternatively, pass the connection string to the constructor. That is all you have to do use SQL Server EntityTagStore.

Roadmap

I am working on the client CachingHandler to be used with HttpClient. This will initially come with an InMemoryCacheStore but then various persistent cache store implementations can be done (for example file-based, SQL CE, etc)

Also CarManager sample needs to be ported for CacheCow which I am hoping to do very soon - although the old sample does work well.

Any question or comment please ping me on twitter (@aliostad) or GitHub.


Tuesday, 10 July 2012

What does REST's Client-Server mean now?

Introduction

[Level C4] In "What does coupling mean ...", we reviewed three client-server patterns (with their anti-patterns) based on the concern. In this post, we will carefully examine the new meaning of client-server web applications. This will serve as a primer on my new work to define Client Server Domain Separation (CSDS).

Motivation

As I explained in the coupling post, I was challenged by recent emergence of attempts to define client-server relationship, one of which being ROCA. There seems to be a recent trend in the REST-aware community by disgruntled developers who believe too much control has been shifted to the client-side. As such they are trying to come up with practices/styles to move some of the control back to the server.

Background

REST has a strong emphasis on separation of client and server. Fielding in his PhD dissertation outlines the constraint:
Separation of concerns is the principle behind the client-server constraints. By separating the user interface concerns from the data storage concerns, we improve the portability of the user interface across multiple platforms and improve scalability by simplifying the server components
As we saw in the coupling post, we need to understand if a functionality is client-concern, server-concern or mixed concern. Implementing the functionality in the wrong side of wire can be initially tolerated but finally takes its toll. Below we will try defining building blocks of our discussion.

When we talk about the domain below, we refer to the concerns implemented/expose in the client or server.

Server

Server is responsible for defining a domain (server domain) and maintaining its state and consistency/integrity. Server usually has very complex components yet it hides its complexity behind its services.

Server
Figure 1 - Server exposes a public domain hiding its complexity

Service is composed of API and domain objects. In a RESTful world, API is HTTP REST. Domain objects are representative of the server's domain model. While Server can have a complex domain, we refer to server domain as only the publicly available domain.

Server should not have a knowledge of the client type. Although such information can be shared with the server (for example through User-Agent header), it should not be used other than for auditing or statistics.

Domain objects sent to the client are usually looked down upon and treated as second class citizen. They are  sometimes called DTO (Data Transfer Object) or ViewModel. While this is OK in a server development scenario when the focus is to decide how much of the server's whole domain to be exposed, these models are not to be mistaken with Value Objects since they are entities (they have identity according to DDD).

Domain object can be a fully rendered HTML (markup) in its semantic form (i.e. no display semantics such as <b> or <i>). Documents are domain objects for example a blog domain has a post domain object.

Server itself can be a client of one or several servers. ifttt is a beautiful example of it. 

Client

Client is responsible for using server(s) services to provide value to the user. In the process, it defines a domain which is usually different from server domain, although there is always an overlap. Client has a life of its own, able to maintain some level of functionality with no server access.


Figure 2 - Client and server domains - now and before
Please note that we did not mention user on the definition of server. Server is being abstracted away from the user by the client. Client can still provide some of its values to the user even when server down. Funny enough, I lost connectivity for half an hour while I was typing this blog and I carried on typing. When re-connected, Blogger saved my worked.

So let's bring an example to highlight a few important points:

I love listening to online radio while working. I use TuneIn on my android to listen to music based on my mood. TuneIn is a great app that allows you to search online radios and podcasts. So one of the functionalities is the directory service of radio stations. This functionality is provided by TuneIn servers (that maintain the state and its consistency/integrity). It defines a server model that consists of name, URL, style, icon, etc. Client does not know how the directory is created or how often the information gets updated.
On the other hand, when I click on a station to listen, I connect to the radio server. Domain of the server has music streaming, current artist, etc. For the radio server, it is all the same if you listen to music in your browser or in the native client on your phone. Client does not know how music is stored, chosen, etc.
Now if I really like a song, I can share it in twitter. Twitter server, while for security reasons (OAuth) knows the application I am using, it does not really care. Publishing my tweet is all the same for it.

So as you can see, Twitter's server domain is fully concerned with users, their tweets, re-tweeting, etc while TuneIn client only cares about publishing a tweet, so their domains have a tiny (yet important) overlap.

Some the client functionality has nothing to do with server. Playing music on the device is fully a client concern so does not exist on the server domain. For example, online radio's streaming servers would not know if the data they are sending will be even heard by the user (e.g. speaker could be on mute or worse, the client uses the data for illegal dumping of the songs).

Let's have a look at Figure 2. I think our TuneIn example fully described the "Now" diagram so let's focus on the "classic" case. This is the classic client (a web application running on the browser - see below) where all the logic served by the server. In extreme cases, server even generates client scripts apart from hosting the static logic (Javascript). Client's domain is fully engulfed by the server meaning server is aware of all the client logic.

Does the classic client look to you as the REST Eutopia? Having read Fielding's dissertation snippet above, which one do you think represent REST better?

Web Application

What is a web application? This definition has been drastically changed over the last few years with the emergence of diverse client devices capable of running advanced Javascript or native code. Web applications can be found in the forms below (not particularly ordered and probably some missing):
  • Single Page Application (SPA) running on various devices including top-end mobile devices
  • Native rich clients on desktop/laptop
  • Native client apps on mobile devices
  • Bundled HTML/Javascript apps running on mobile devices (PhoneGap) or desktop (Windows 8)
  • Traditional web applications run by browsers
  • Browser as an application to display markup
Figure 3 - Server cannot really see behind the cloud and which client is using it


Web application is the usage abstraction of the client. While the server domain used to pretty much define the web application, client is becoming more and more important in defining it.

Now what does web do in the "web application"? Nowadays, almost every application is a web application: the client uses one or more cloud services to enrich the experience. Unless it is a simple drawing or editing tool, most applications are web applications.

Some of the forms deserve more attention. Bundled HTML/Javascript apps remind us that in some cases it is in fact incidental that the server hosts the files. Some Javascript files are hosted on CDN and downloaded and cached for a long time.

Also not all logic is Javascript. Microsoft RIA service (regardless of whether I like it or not - and I don't!) sends server's domain rules to the client very much like Javascript. Generating Javascript or binaries is all bad the same as it breaches client's independence.

Web Application and REST

Basing your web application on REST will help achieving better separation as well as making an efficient use of the web.

Having said that, today's world demands more and more from the computing industry. Server Push model (gaining popularity in node.js/websocket world) requires a stateful server which is a REST no-no but today's virtualisation and cloud elasticity has made scalability a much smaller problem.

For most web applications, however, following all REST constraints is the best practice as very few applications require Server Push model.

So what do I think of the new trend?

Well, I think the trend cannot resist the wind of change. Computing industry has been pushed to provide more value and has to do it cheaper, faster and richer. Separating client and server will help to achieve this more effectively:

With regard to ROCA I must say it is a worthwhile effort to understand the client and server interactions and contains useful common-sense practices. However I cannot subscribe to it since:

  • must-server advocates "classic" model (engulfed client) and ignores the client server separation prescribed by REST. 
  • It does not respect client domain (must-no-duplication) and it rules and logic.
  • Single-Page-Application is not ROCA-compliant (see discussions).
  • Not all clients are browsers, in fact less and less clients are browsers. ROCA is heavily targeted for browser applications with many of the constraints directly related to HTML, CSS or Javascript
  • It does not fully appreciate the inherent complexity of the client domain (must-jslimits and mustnot-jsengine)
  • A ROCA client will not be able to provide any useful offline feature
  • Includes lower-end non-browser clients yet does not appreciate upper-end clients (must-non-browser)

Conclusion

Client and server have their own domain - as REST prescribes. Server defines a domain and maintains its state and integrity. It hides its complexity behind its services while other than authentication and authorization does not need to know anything about the client.

Client through the usage of server(s) services provides value to the user. It defines its own domain that has overlap with server(s) domain.

Web application is a plethora of different devices and technologies. As such, defining a style requires considering all such scenarios.

We will talk more about Client-Server Domain Separation (CSDS) in the upcoming posts.

Sunday, 1 July 2012

The place of Extension Methods in Software Design


Introduction

[Level T3] Extensions methods - introduced back in .NET 3.0 - are useful tools in a .NET developer's toolset. Apart from their usefulness, extension method is not an inherently object oriented concept yet we use them more and more in our API designs.

Extension methods initially were used for those classes where we did not own the source code for. But nowadays we are using them increasingly for types where we do own the source.

This post aims to have an in-depth look at the place of extension methods in the API design.

Background

Definition of Extension Methods according to MSDN is:
Extension methods enable you to "add" methods to existing types without creating a new derived type, recompiling, or otherwise modifying the original type.
So as we all know, in order to create an extension method, we need to:
  1. Create a static non-generic class
  2. Create a static method
  3. Make the first parameter as the type we are trying to add the method, with the keyword this
For example (and one of my favourites), we can add this extension method to the object to replicate the T-Sql's IN operator:

public static bool IsIn(this object item, params object[] list)
{
 if (list == null || list.Length == 0)
  return false;
 return list.Any(x => x == item);
}

Now I can use this like an instance method:

string ali = "ali";
var isIn = ali.IsIn("john", "jack", "shabbi", "ali"); // isIn -> true

If I may digress a little bit here, this is not such a great implementation since:

var isInForInt = 1.IsIn(2, 3, 1); // isInForInt -> false!

As you have probably guessed, defining the extension method for object type will cause the boxed integers objects to be compared instead of integers themselves and they surely won't be equal. So a generic implementation will solve the problem:

public static bool IsIn<T>(this T item, params T[] list)
{
 if (list == null || list.Length == 0)
  return false;
 return list.Any(x => EqualityComparer<T>.Default.Equals(x, item));
}

Reality is, extension method only gives the illusion of method being on the type and what is being compiled is nothing but a plain old static method call. Having a look at the IL generated confirms this:

IL_0056:  call       bool ConsoleApplication1.ExtensionMethods::IsIn<int32>(!!0, !!0[])

ExtensionMethods above is the name of the static class I created for this method.

So extension methods are basically the same utility or helper static methods we have been writing only glamorised to look like instance methods. Yet they have the additional benefit of:

  1. It leads to much more readable and natural code.
  2. I do not have to know the name of the helper class whose static methods I am using - in fact the behaviour has nothing to do with the static class. That class is not really a class in a true sense since it does not exert state or behaviour. And that is why it has to be declared static: to make clear its design intentions.
  3. Fluent API can be easily designed for older types without touching them.
  4. Since it is not really an instance method call, it can be called on null instances. This is a desirable side effect since we can check for nulls in the extension method and cater for them (none of the "object reference not set to an instance..." nonsense!)
  5. Since it can be called on null instances, some type information for the null instance can be determined in the extension method (although it can be a base type or an interface) while this is not possible for a null object.

Extension methods when we do not own the type

This has been the typical scenario. We always wonder if for example string had a such and such method and there was no way to achieve this. Now using extension methods we can. This scenario can also apply to cases where a historic API has been released (and you own the API) but cannot be changed. In this case, your API can be enhanced using extension methods.

With such usage, there is no decision to be made hence the design has already been done. Extension methods serve mere as a nice utility and syntactical sugar.

One of the most useful use cases I have found is the function composition in functional programming in C# (see some examples in my other posts here and here). This is especially important since you can achieve readability by method chaining. For example:

  usingReflection
   .Repeat(TotalCount)
   .OutputPerformance(stopwatch, performanceOutput)();

In addition to the examples above, let's have a look at a simple example to swallow the exception and optionally log the error (note how the implementation reuses itself to swallow errors that could arise from logging):

public static class WrapSwallowExtension
{
 public static Action<T> WrapSwallow<T>(this Action<T> action, Action<Exception> logger = null)
 {
  return (T t) =>
       {
           try
           {
            action(t);
           }
           catch (Exception e)
           {
      if (logger != null)
       logger.WrapSwallow()(e);             
           }

       };
 }
}

So I can use:

string myString = null;
Action<string> action = (s) => { s.ToLower(); }; // reference null exception! 
action.WrapSwallow()(myString); // swallowed

Now here I created a new exception but when I am working in a functional scenario, I already have my actions and functions.

Extension methods when we own the type

I have heard some saying "Why would you wanna use an extension method when you own the code? Just add the method to the type."

There are cases where you own the type yet you would still use an extension method. Here we have a look at a few scenarios below.

Extension methods for interfaces

This is the most obvious use case. Most of the Linq library is implemented using extension methods (while Microsoft owns the types). An interface cannot have the implementation but you can use extension methods to add implemented enhancement to your interfaces.

Without getting into the debate whether implementing ForEach against IEnumerable<T> is semantically correct or not (don't! I am not going there) you might have noticed that the function only exists for the List<T> so you have to use ToList() to use the feature. Well, this can be easily done for IEnumerable<T> too:

public static class IEnumerableExtensions
{
 public static IEnumerable<T> ForEachOne<T>(this IEnumerable<T> enumerable, Action<T> action)
 {
  foreach (var t in enumerable)
  {
   action(t);
   yield return t;
  }
 }
}

In this particular example, I do not own the source for IEnumerable<T> but even if I had, I would only be able to associate implementation with the interface using extension methods.

Overloading

This is the next common case. If you are familiar with ASP.NET MVC, you probably have noticed that the most of the functionality of HtmlHelper class has been implemented using extension methods.

Html.TextBoxFor(x => x.Name)

In fact all different overloads of HtmlHelper for Textbox, RadioButton, Checkbox, TextArea, etc are implemented using extension methods. So the HtmlHelper class itself implements a core set of functionality which will be called by these extension methods.

Now lets look at this fictional interface:

public interface IDependencyResolver
{
   object Resolve(Type t);
   T Resolve<T>();
}

The interface has two methods for resolving the type, one using the generic type the other with the type instance. Whoever implements this will be most likely implementing the non-generic method and then make generic method call the non-generic one:

public interface IDependencyResolver
{
 object Resolve(Type t);
}

public static class IDependencyResolverExtension
{
 public static T Resolve<T>(this IDependencyResolver resolver)
 {
  return (T) resolver.Resolve(typeof (T));
 }
}


This will help to:
  • Trim down the interface and make it terser so it can express its design intentions more clearly
  • Save all implementers of the interface having to repeat the same bit of code
When I look at the interface IQueryProvider, I wonder if it was designed before extension methods were available:

public interface IQueryProvider
{
    IQueryable<TElement> CreateQuery<TElement>(Expression expression);
    IQueryable CreateQuery(Expression expression);
    TResult Execute<TResult>(Expression expression);
    object Execute(Expression expression);
}

So the 4 methods could have been reduced to 2. Considering the fact that Linq and extension methods both came in .NET 3.0, my suspicion seems very likely!

Dependency layering

Another case where you might decide to use an extension method rather than exposing a direct method on the type is when a type's sole dependency on another type is confined to a single method. This is very common in cases where the dependency is on a layer above the dependent type - while naturally must be the other way around.

For example, let's look at this case:

// THIS WILL NOT WORK!

// sitting at entity layer
public class Foo
{
 // ...

 public Bar ToBar()
 {
  // ...
 }
}

// sitting at business layer
public class Bar
{
 // ...  
}

Now in this example, I have laid out these two classes in different logical layers to better illustrate the case - but it does not have to be, this is all about managing dependencies, in the same layer or other layers. We have baked in Foo's the dependency to Bar for the sake of ToBar(). The solution is to create an extension method for the ToBar().

So we can write (and completely decouple to classes):

public class Foo
{
 
}

public class Bar
{

}

public static class FooExtensions
{
 public static Bar ToBar(this Foo foo)
 {
  return new Bar();
 }
}


Providing implementation for enumerations

This is one that probably many of us have done. Enumerations - unfortunately - cannot contain implementations so extension methods are a good place to put the implementation code for enums. This is usually to do to conversion, parsing and formatting.

Delay decision on API signatures

With regards to an API, anything that goes into the public interface of the type is difficult to change. As such attempting to provide all possible overloads and use cases of the type on its public interface is likely to fail.

Delaying such decisions with providing a base functionality on the type and then providing more and more extension methods with each release is a useful process. ASP.NET team have used this technique for ASP.NET MVC and recently with ASP.NET Web API.

Drawbacks

Extension methods are static methods. As such they cannot be mocked using standard mocking frameworks. An extension method should not have any dependency other than the ones passed to it.

Let's look at this case:

public class Foo
{
 public string FileName { get; set; }

 public void Save()
 {
  // ...
 }
}

public static class FooExtensions
{
 public static void SafeSave(this Foo foo)
 {
  var directoryName = Path.GetDirectoryName(foo.FileName);
  if (!Directory.Exists(directoryName))
   Directory.CreateDirectory(directoryName);
  foo.Save();
 }
}

In this case, unit testing any class that uses SafeSave becomes a nightmare. What we need here is to create an interface IFileSystem and pass along with the extension method to abstract it from using the real file system.

Conclusion

Availability of extension methods has changed the way we design software APIs in the .NET world. We have started to build the basic functionality in the actual types and use extension methods to provide overloading.

There are 5 reasons to use extension methods when you own the type:

  • To associate implementation with interfaces
  • Overloading of an API
  • Removing dependency especially in logical layering
  • Providing implementation for enumerations
  • Delaying decision on API signatures
An extension method should not have any dependencies other than the ones passed to it.

Thursday, 28 June 2012

Using PocoHttp to consume Classic OData services


Introduction

[Level T2] In the last post, I introduced PocoHttp which I have been working on. As I promised there, we can use it to access classic OData services in addition to as much as OData support we have in ASP.NET Web API. OData functionality currently provided in PocoHttp is limited at the moment but I will be building upon.

[UPDATE: PocoHttp package v0.1 is not available in NuGet. Just type in the package manager console PM>Install-Package PocoHttp or search for PocoHttp]

Why OData?

During the last  few years, Microsoft has invested a lot in OData. Some other technologies have also been built upon it such as RIA services which is heavily used in the Silverlight world. So Microsoft has been pushing OData and marketing it as an open protocol but reality is outside Microsoft, adoption has been nill. I believe this has to do with issues regarding query syntax and paylaod.

Firstly, as we talked about before, community believes that OData services are not RESTful as query syntax is unique to OData (Darrel explains here). It also exposes the bowels of the server to the client which does not respect REST's client-server model.  Personally I believe there is a place for some of the simpler query constructs of OData such as top and skip that could be implemented by non-OData services as well.

And payload is limited to specific data structures in XML and JSON. As for XML payload, I think AtomPub as data wrapping format is heavy and inefficient - not to say that it has not been realised of recent. Sending metadata to the client that consumes the data every second is not a very good idea - metadata was used once to build the client but the client is eternally doomed to receive that information until the day sun stops shining (there might be a way to turn it off but that is not the default design). So it is like a web service that sends its WSDL all the time.

JSON payload is definitely lighter - it contains some metadata but nothing to be concerned about. The JSON format has changed between versions 1 and 2 of OData and server could be sending V1 or V2 (etc) depending on the content. There is DataServiceVersion custom header that can be used to get the version number from the response but I think as a canonical HTTP implementation, it should have been implemented as part of content-type:
Content-Type: application/json;DataServiceVersion=2.0
Or
Content-Type: application/odata2.0+json
Anyhow, that is not a big problem and I will be using JSON format in this sample. You can ask for JSON using an Accept header or appending $format=json.

OK, with all this so why OData then? For two reasons.

First of all, ASP.NET team are working to support alternative formats that do not have all the extra metadata: plain JSON or XML. So OData is going to live much longer in an improved implementation.

Second reason is there is already a considerable adoption inside the Microsoft community. Many teams while moving to new shiny Web APIs might still have an existing OData services that they need to use along with the new ones. 

Why use PocoHttp to access OData?

While OData has the problems we talked about, it is not short of one thing: tools and SDKs. You can easily use DataService<T> to consume OData services and the API even allows for automatic generation of data models for Entity Framework. You are fully abstracted from OData itself while using its all benefits.

So why would you ever want to use anything other than the existing tools to comsume OData services? Well, a few reasons.

With adoption of Web API increasing (which can be seen from the enthusiasm in the community), you will most likely start using Web APIs alongside OData ones. You would be building custom handlers to take care of authentication, tracing, caching, auditing, exception handling, etc and start to rely on them. But you would not be able to use these them with existing client tools since you need a client that is built upon the new HTTP pipeline in System.Net.Http.dll. That is PocoHttp!

Also you might want to harness the extreme powers that the client has in terms of queries it can run (which we touched in the last post) and limit them to a few handful essential but safe constructs - protecting your database and services. Then again that is one of the goals of PocoHttp.

New sample on the block

So I have created this sample on the PocoHttp that connects to OData sample services exposing Northwinds database. 

This basically sets up PocoClient to retrieve data in JSON format. Since the version of the JSON could be V1 or V2, the code needs to be able to support both.

Show us some code

OK! Let's look at request setup. There are two ways to request for JSON: by setting the Accept header or by appending $format=json:

// 1
pocoConfig.RequestSetup = (request) => request.Headers.Add("Accept", "application/json");

// 2
pocoConfig.RequestSetup = (request) => request.RequestUri = new 
   Uri(request.RequestUri.ToString() + "&$format=json",UriKind.Relative);

PocoConfiguration is a set of configuration parameters which is passed to the client constructor. One the parameters is RequestSetup which is an Action<HttpRequestMessage> that allows for the request to be modified before sending it to the server.

On the way back we use ResponseReader function to read the content into the objects. It receives the response, formatters and element types and must return objects it has read (will be IEnumerable<T>)

public Func<HttpResponseMessage, IEnumerable<MediaTypeFormatter>, Type, Task<object>> 
   ResponseReader { get; set; }

So this is a property sitting on the PocoConfiguration. PocoHttp provides a default implementation but since the data we get back is wrapped inside OData top level envelopes, we have to provide our own implementation. We need to define Payload<T> types that represent the structure of the data we are receiving. I have defined one for V1 and one for V2.


The gist of what we need to do is:
  1. Create a generic type of the entity using MakeGenericType
  2. Call non-generic ReadAsAsync passing the type
  3. Use reflection on the result to read off the properties to unwarp and get to the entities we need
For example, the code for V1 looks like this:

private static object ReadFromV1(HttpResponseMessage response, 
 IEnumerable<MediaTypeFormatter> formatters, Type type)
{
 var outermostType = typeof(PocoHttp.Samples.ClassicOData.V1.Payload<>)
  .MakeGenericType(type);
 var result = response.Content.ReadAsAsync(outermostType, formatters).Result;
 return outermostType.GetProperty("d").GetValue(result, null);
}

In the example, I have used only Take and Skip but you can use direct Where clauses as well.

Conclusion

While ASP.NET Web API is the future of providing services and data, there is a considerable number of existing OData services that still need to be consumed. 

While existing OData client tools expose full features of OData abstracting the client from HTTP detailed, we outlined two reasons why you would want to use PocoHttp instead to consume these services: re-using HTTP pipeline handlers and limiting the client in terms of queries it can run so that the.

Monday, 25 June 2012

Introducing PocoHttp: Consuming HTTP data services


Introduction

[Level T2] PocoHttp is a non-opinionated Open Source .NET library for seamless consumption of HTTP data services using a familiar IQueryable<T> interface. Version 0.1 of the library is now available on GitHub - NuGet package to follow soon.

This post explains the background and motivation for this library as well as its usage and possible future directions.

Motivation and background

ASP.NET Web API exposes full richness of the HTTP spec. As such, many new avenues have been opened for creating HTTP client-server applications. Having said that, with HTTP spec aimed to be Touring-Complete, there is a client burden involved in implementing clients capable of deep protocol coherency.

ASP.NET Web API has made it easy to achieve protocol coherency as part of the HTTP pipeline - modelled as Russian Dolls (see Part 6 of the series).

With a lot of WCF services (that commonly were serving domain's value objects) soon to be moved to Web API, there is a requirement for abstracting HTTP-level aspects for seamless consumption of such services. PocoHttp is designed to provide an IQueryable<T> interface which is familiar to developers who have been working with ORMs such as Entity Framework, NHibernate, etc.

WCF Data Services are able to provide this seamless communication but they
  • Can only generate AtomPub payloads
  • As such no content negotiation or ability to generate plain XML or JSON (it does honour Accept header but returns OData specific format)
  • Use OData query syntax which is deemed by many in the community not RESTful since it assumes non-HTTP syntax coherence in the form of query string parameters on the client
  • Fully expose the data to the outside world
Hence there is a need for a .NET client library to take advantage of new HTTP features exposed in System.Net.Http to be able to consume HTTP data services whether developed in ASP.NET Web API or in any other platform. PocoHttp rises up to that challenge.

Minimal example

Before going much further into details, it might be useful to present a minimal usage of how to use it. Example below is from the PocoHttp samples (which hosts a minimal server too) available from GitHub.

var pocoClient = new PocoClient()
        {
            BaseAddress = new Uri("http://localhost:12889/api/")
        };
var list = pocoClient.Context<Car>() // calling "/api/Cars"
 .Take(1).ToList(); // getting the first item
Console.WriteLine(list[0]);

As can be seen above, pocoClient is initialised with a BaseAddress (Note the trailing slash which is mandatory for correct URI composition) and then a generic context of Car type is queried. PocoClient assumes (using the naming convention setup) that the Car should be exposed at BaseAddress + "Cars" so it makes an HTTP request and turns the result into Car objects.

PocoHttp's Grammar model

Basically PocoHttp translates IQueryable<T> queries into HTTP semantics (query string parameter in case of GET or a query criteria object in the body in case of POST) using a grammar and a set of naming convention settings - and then turns the result into .NET types using media type formatting provided by the ASP.NET Web API.



Currently, two built-in grammars are provided: OData and Pagination. OData implementation does not implement full features of OData (see below). Grammar can be implemented to translate the queries into other providers such as MongoDB or other custom syntaxes.

Normally, a grammar modifies the request by adding query string parameters. While it is not recommended, a grammar can modify the request to turn it into an HTTP POST and send a payload containing the criteria. The reason this is not recommended is that using POST method for GETting resources is not REST-friendly and responses to POST results cannot be cached while caching results of calls to data services can be very useful.

OData implementation

Current implementation of OData grammar supports below features:
  • Where: current implementation supports direct property expression. Complex property expressions (such as Car.Make.StartsWith("M")) not yet supported
  • AND, OR, greater than, lesser than, lesser than or equal, greater than or equal, not equal in WHERE expressions
  • Take
  • Skip
  • OrderBy
  • OrderByDescending 

Exposing an HTTP service in ASP.NET Web API (so that it can be consumed in PocoHttp) is simple: you just need to return IQueryable<T> from your action and decorate the action with [Queryable] attribute.

Having said that, while this feature is currently available in ASP.NET Web API RC, the future of OData support on Web API is a bit unclear as the Queryable attribute has been removed in ASP.NET Web API source code and it might not ship with RTM. As far as we can find out from ASP.NET team, OData support will be implemented using OData libraries and might ship out of band after the RTM. So the team is committed to full OData support but the timeline is unclear. As for now, you can happily use Queryable attribute in ASP.NET Web API RC.

As for PocoHttp, I am committed to implement full OData query syntax in the future releases.

Using PocoHttp against existing AtomPub OData services

Nothing prevents us to consume classic OData services that return AtomPub by just adding AtomPub media type formatter to the formatters. This will allow you to take advantage of all the niceties of HttpClient while using your existing OData services.

I will be having a post on this soon.

Disclaimer

As I initially said, this is a non-opinionated framework. You can expose your full domain model through an IQueryable<T> interface out to the client but this is not necessarily a good thing. Exposing bowels of your server and data to the outside world is an anti-pattern and could be a costly mistake. For example you can easily expose your database to the outside world allowing this query:

pocoClient.Context<Car>()
 .Where(car => car.Make.EndsWith("L"))

turn into this SQL statement that can bring down the server by having to look into each record:
SELECT * FROM CAR WHERE MAKE LIKE '%L'

Also free form Linq queries enable the client to select data using criteria which involves columns that do not have indices upon.

Pagination grammar

Pagination grammar is a simple non-OData syntax which is REST-friendlier. An example of the syntax is /api/Cars?skip=100&count=20.

Pagination vocabulary only supports 4 constructs:

  1. Skip: normally represented by skip
  2. Take: normally represented by count
  3. OrderBy: normally represented by order
  4. OrderByDescending: normally represented by orderDesc

All above representations can be changed by the client. A typical server implementation supporting these parameters will be

public IEnumerable<Car> Get(int skip, int count)
{
    return _repository.Skip(skip).Take(count);
}
As it can be seen, server implementation only supports skip and count and ignores the OrderBy and OrderByDescending.

Overriding naming convention and routing

There are times that you might want to override default route:
pocoClient.Context<Person>()
 .Where(p => p.Name == "Ali"); // calls /api/Persons?$filter=(Name eq 'Ali')
while you might want it to go to /api/Employees.

You have to ways to achieve this:

  1. To decorate your entity with EntityUriAttribute. In this case [EntityUri("Employees")]
  2. Pass the full URI as below when creating context:

pocoClient.Context<Car>("http://server/api/Employees")
 .Where(car => car.Make.EndsWith("L"))

Conclusion

In this post, we have introduced PocoHttp which is an emerging open source client library for consuming HTTP data services.

PocoHttp can be used to consume ASP.NET Web API services where IQueryable<T> is returned, existing OData services or any other HTTP data service built in other platforms - provided the grammar is implemented for the query syntax.

Using the CachingHandler in ASP.NET Web API


Introduction


[NOTE: Please also see updated post and the new framework called CacheCow. 
This class has been removed from WebApiContrib code and NO LONGER SUPPORTED]


[Level T3] Caching is an important concept in HTTP and comprises a sizeable chunk of the spec. ASP.NET Web API exposes full goodness of the HTTP spec and caching can be implemented as a message handler explained in Part 6 and 7. I have implemented a CachingHandler and contributed the code to the WebApiContrib in GitHub.

This post will have two sections: first a primer on HTTP caching and then how to use the handler. This code uses ASP.NET Web API RC and with .NET 4 (VS 2010 or 2012).

Background

NOTE: This topic is fairly advanced and complex. You do not necessarily need to know all this in order to use CachingHandler and are welcomed to skip it but more in-depth knowledge of HTTP could go a long way. 

Caching is a very important feature in the HTTP. RFC 2616 extensively covers this topic in section 13. Review of all the spec is beyond the scope of this post but we will briefly touch on the subject.

First of all let's get this straight: this is not about putting an object in memory as we do with HttpRuntime.Cache. In fact server does not store anything (more on this below), it only tells the client (or mid-stream cache or proxy servers) what can be cached, how long and validates the caching.

Basically, HTTP provides semantics and mechanism for the origin server, client and mid-stream proxy/cache servers to effectively reduce traffic by validating the version of the resource they have against the server and retrieve the resource only if it has changed. This process is usually referred to as cache validation.

In HTTP 1.0, server would return a LastModified header with the resource. A client/user agent could use this value and send it in the If-Modified-Since header in the subsequent GET requests. If the resource was not changed, server would respond with a 304 (Not Modified) otherwise the resource would be sent back with a new LastModified header. This would be also useful in PUT scenarios where a client sends a PUT request to update a resource only if it has not changed: a If-Unmodified-Since header is sent. If the resource has not changed, server fulfils the request and sends a 2xx response (usually 202 Accepted) otherwise a 412 (Precondition Failed) is sent.

For many reasons (including the fact that HTTP dates do not have milliseconds) it was felt that this mechanism was not adequate. In HTTP 1.1, ETag (Entity Tag) was introduced which is an opaque Id in the format of a quoted string that is returned with the resource. ETags can be strong (refer to RFC 2616 for more info) or weak in which case they start with w/ such as w/"12345". ETag works like a version ID for a resource - if two ETags (for the same resource) are equal, it means those two versions of the resource are the same.

ETag for the same resource could be different according to various headers. For example a client can send an Accept-Language header of de-DE or en-GB and server would send different ETags. Server can define this variability for each resource by using Vary header.

Cache validation for GET and PUT requests are similar but instead will use If-Match (for PUT) and If-None-Match (for GET).

Well... complex? Yeah, pretty much and yet I have not covered some other aspects and the edge cases. But don't worry! You will be abstracted from a lot of it if you use the CachingHandler.

Using CachingHandler right out of the box

OK, using the CachingHandler is straightforward especially if you go with the default settings. You can have a look at the CarManager sample in the WebApiContrib project. CarManager.Web sample demonstrate server side setup of the caching and CarManager.CachingClient for making calls to the server and testing various caching scenarios.

All you have to do is to create an ASP.NET Web API project and add the CachingHandler as a delegating handler as we learnt in Part 6:

GlobalConfiguration.Configuration.MessageHandlers.Add(cachingHandler);

And you are done! Now let's try the server with some scenarios. I would suggest that you use the CarManager sample, otherwise create a simple handler and implement GET, PUT and POST on it.

Now let's make some request and look at the response. I would suggest using Fiddler or Chrome's Postman to send requests and view the response.

So if we send a GET request:

GET http://localhost:8031/api/Cars HTTP/1.1
User-Agent: Fiddler
Host: localhost:8031

We get back this response (or similar; some headers removed and body truncated for clarity):

HTTP/1.1 200 OK
ETag: "54e9a75f2dbb4edca672f7a2c4a73dca"
Vary: Accept
Cache-Control: no-transform, must-revalidate, max-age=604800, private
Last-Modified: Thu, 21 Jun 2012 23:35:46 GMT
Content-Type: application/json; charset=utf-8

[{"Id":1,"Make":"Vauxhall","Model":"Astra","BuildYear":1997,"Price":175.0....

So we see the ETag header here along with important caching headers. Now if we make another GET call, we get back the same response but ETag and last modified stays the same.

Generating same ETag is fine (showing our CachingHandler is doing something) but we have not yet seen any caching. That is where client has to do some work to do. It has to use If-None-Match with the ETag to conditionally ask for the resource: if it matches server, it will get back 304 but if not, server will return the new resource:
GET http://localhost:8031/api/Cars HTTP/1.1
User-Agent: Fiddler
Host: localhost:8031
If-None-Match: "54e9a75f2dbb4edca672f7a2c4a73dca"
Here we get back 304 (Not modified) as expected:
HTTP/1.1 304 Not Modified
Cache-Control: no-cache
Pragma: no-cache
Expires: -1
ETag: "54e9a75f2dbb4edca672f7a2c4a73dca"
Server: Microsoft-IIS/8.0
Date: Sun, 24 Jun 2012 07:34:29 GMT

A typical server that can use CachingHandler

CachingHandler makes a few RESTful assumptions about the server for effective caching. Some of these assumptions can be overridden but generally not recommended. These assumptions are:

  • HTTP verbs (POST/GET/PUT/DELETE for CRUD operations) are used to modify resources - and not RPC-style resources (such as POST /api/Cars/Add). This is the most fundamental assumption.
  • All resources are to be modified through the same HTTP pipeline that implements caching. If a resource modified outside the pipeline, cache state needs to be updated by the same process.
  • Resource are organised in a natural cache-friendly manner: invalidation of related resources can be done with a minimal setup (more details upcoming).

Defining cache state

As we said, this has nothing to do with HttpRuntime.Cache! Unfortunately ASP.NET implementation makes it really confusing between the HTTP cache (where the resource gets cached on the client or midstream cache servers) and server caching (when the rendered output gets cached on the server).

Cache state is a collection of data that helps keep track of each resource along with its last modified date and ETag. It might initially seem that for a resource, there exists a single of such pieces of information. But as we touched upon above, a resource can have different representations each of which needs to be stored separately on the client while they will most likely invalidated together. 

For example, resource /api/disclaimer can exist in multiple languages as such client has to cache each representation separately but when disclaimer changes, all such representations need to be invalidated. That will require a storage of some sort to keep track of all this data. Current implementation comes with an in-memory storage but in a web farm scenario this needs to be a persistent store.

Cache state storage and cache invalidation

So we do not need to store the cache on the server, but we DO need to store ETag and various states on the server. If we only have a single server, this state can be stored in the memory. In a web farm (or even web garden) scenario a persisted store is needed. This store is called Entity Tag Store and is represented by a simple interface:

public interface IEntityTagStore
{
 bool TryGetValue(EntityTagKey key, out TimedEntityTagHeaderValue eTag);
 void AddOrUpdate(EntityTagKey key, TimedEntityTagHeaderValue eTag);
 bool TryRemove(EntityTagKey key);
 int RemoveAllByRoutePattern(string routePattern);
 void Clear();
}

We need to have an implementation of this interface so that the cache management can be abstracted away from controllers and instead done in the DelegatingHandlers.

Introducing some concepts (you may skip and come back to it later)

This store will keep track of the Etag (and related state stored in TimedEntityTagHeaderValue) based on an Entity Tag key which is calculated based on the resource URI and content of the important headers (which their list will be on the Vary header). In-Memory implementation for a single server is provided out of the box with CachingHandler - I would be creating a SQL-Server implementation of it very soon; watch this space.

It is important to note that change in the resource will most likely invalidate all forms of the resource so all permutations of important headers will be invalidated. So invalidation is usually performed at the resource level. Sometimes several related resources can be represented as a RoutePattern. By default, URI of a resource is its RoutePattern.

Also in some cases, change in a resource will invalidate linked resources. For example, a POST to /api/cars to add a car will invalidate /api/cars/fastest and /api/cars/mostExpensive. In this case, "/api/cars/*" can be defined as the linked RoutePattern (since /api/cars does not qualify for /api/cars/*).

Some assumptions in CachingHandler

  1. Re-emphasising that resource can only change through the HTTP API (using PUT, POST and DELETE verbs). If resources are to be changed outside the API, it is responsibility of the application to use IEntityTageStore to invalidate the cache for those resources.
  2. If no Vary header is defined by the application, CachingHandler creates weak ETags.
  3. Change in the resource will invalidates all forms of the resources (all permutations of important header values)

CarManager sample

I have considered a pretty complex and interrelated routing and caching requirement for the sample to display what is possible. Resources available are:

  1. /api/Car/{id}: GET, PUT and DELETE
  2. /api/Cars: GET and POST
  3. /api/Cars/MostExpensive: GET
  4. /api/Cars/Fastest: GET
So creating a car would invalidate cache for 2, 3 and 4. Updating car with id=1 invalidates 2,3 and 4 in addition to the car itself at /api/Car/1. Deleting a car will invalidate 2, 3 and 4.

The client sample (CarManager.CachingClinet) displays how to call the server with headers to validate the cache.

Configuring CachingHandler

Best place to start is the CarManager.Web sample to give an idea on how various setup can be used to configure complex caching requirements. Basically the points below can be used to configure the CachingHandler

Constructor

A list of request header names can be passed that will define the vary headers. If no vary header is passed, system only generates weak ETags. Also optionally an implementation of IEntityTagStore otherwise by default InMemoryEntityTagStore will be used.


EntityTagKeyGenerator

This is an opportunity to provide linked resource for a resource.

 Other customisation points

CachingHandler provides properties in the form of functions with default implementation that can be changed  to override the default behaviour which we will cover in the upcoming post on "Extending CachingHandler".


Conclusion

CachingHandler is a server-side DelegatingHandler which can be used to abstract away caching from individual ApiControllers so that controller logic can be coded without having to worry about caching. 

The  code is hosted on GitHub (currently sitting at my fork waiting to be merged merged!) and comes with both a client and server sample - CarManager.Web and CarManager.CachingClient.

Thursday, 14 June 2012

ASP NET Web API Series - Part 9: MediaTypeFormatter on the client


Introduction

[Level T3] In part 5 we talked about MediaTypeFormatter on the server. We have another side of the story - perhaps not as big - on the client. MediaTypeFormatters will helps us converting our ViewModels/DTOs to and from HTTP content. HttpClient exposes full goodness of server side formatting as we have seen in part5. This post is going to be short and simple - especially if you have read the server-side post.

Why should I care?

You should care if you want to do anything useful consuming an HTTP API from a .NET client - and not Javascript. 

As we discussed on the server-side post, MediaTypeFormatter marries the world of HTTP (headers and body) to the world of .NET (strongly-typed objects). It will be very likely that you will be sending some strongly typed objects across the wire and expecting the same back. MediaTypeFormatter will help you achieve that.

ObjectContent and ObjectContent<T>

ASP.NET Web API abstracts the HTTP message's body and its body-related headers as HttpContent class - this applies to both request and response. So what are body-related headers? Here is a list:
  • Allow
  • Content-Disposition
  • Content-Encoding
  • Content-Language
  • Content-Length
  • Content-Location
  • Content-MD5
  • Content-Range
  • Content-Type
  • Expires
  • Last-Modified
As we have highlighted, Content-Type is the key header here - as for the media type formatting. These are in fact the same headers that can appear in each part of a Multipart message. As we covered briefly before, in a multi-part message, each part gets its own share of the body and body-related headers.

While HttpContent generally abstracts HTTP body as a stream, ObjectContent which inherits from HttpContent abstracts it as a strongly typed object - especially the generic type ObjectContent<T>. 

Creating an ObjectContent<T>

ObjectContent<T> has two constructors:

public ObjectContent(T value, MediaTypeFormatter formatter) : this(value, formatter, null)
{
}

public ObjectContent(T value, MediaTypeFormatter formatter, string mediaType) : base(typeof(T), value, formatter, mediaType)
{
}

We are supposed to pass the value of generic type T and pass the actual MediaTypeFormatter we want to format the content. The content then can be used to be sent to the server, for example in a post:

var content = new ObjectContent<foo>(new Foo(), new JsonMediaTypeFormatter);
var httpClient = new HttpClient();
httpClient.PostAsync(url, content);

Since it is the client who initiates the HTTP call, it would know what format to send the request body in. So the content-negotiation does not apply here. Having said that, I would have liked to be able to POST a request without having to decide on a MediaTypeFormatter and let Web API to choose a default formatter for me. I have not been able to find this in the framework yet.

Choosing a default media type formatter is not that hard in fact. All we have to do is to create a new MediaTypeFormatterCollection and then choose the JsonFormatter or XmlFormatter property.

MediaTypeFormatterCollection

ASP.NET Web API comes with a non-generic collection for MediaTypeFormatter. Unlike most collections which are a bag/list of their items, MediaTypeFormatterCollection has some knowledge and understanding of the formatters.

For starter, constructor of this collection creates the collection with these three default formatters:

public MediaTypeFormatterCollection() : this(CreateDefaultFormatters())
{
}

private static IEnumerable<MediaTypeFormatter> CreateDefaultFormatters()
{
    return new MediaTypeFormatter[] { new JsonMediaTypeFormatter(), 
          new XmlMediaTypeFormatter(), 
          new FormUrlEncodedMediaTypeFormatter() };
}

Also it has three properties: JsonFormatter, XmlFormatter and FormUrlEncodedFormatter. With using the parameterless constructor, each of these properties will point to one of the formatters constructed. However, if we create our own IEnumerable<MediaTypeFormatter> and pass to constructor, collection will find these formatters and assign the properties. I call these common formatters since they are bread and butter of the web.

Formatting objects into HTTP content (Request)

As we saw earlier, we can directly create an ObjectContent<T> or do that as part of POST/PUT:

client.PostAsync<Foo>(uri, new Foo(),
 new MediaTypeFormatterCollection().JsonFormatter);

This is not really much more complex than that.

Reading object from HTTP content (response)

This is where ASP.NET Web API provides on the client its full media type formatter selection on the server. Normally you would use ReadAsAsync<T> on the HttpResponseMessage. For example:

Foo result = client.GetAsync(uri)
 .Result.Content.ReadAsAsync<Foo>().Result;

As you can see, no media type formatter had to be passed and we used the organic media type formatter selection within the framework. How? Well, parameterless ReadAsAsync<T> is in fact an extension method that creates a new MediaTypeFormatterCollection (as we saw above) and pass as a set of formatters:

public static Task<T> ReadAsAsync<T>(this HttpContent content)
{
    return content.ReadAsAsync<T>(new MediaTypeFormatterCollection());
}

This in turn will call other methods and in the end, FindReader method of the MediaTypeFormatterCollection will be used to select a suitable formatter based on type and Contet-Type header.

Example: using Rotten Tomatoes web API

Rotten Tomatoes is a popular website for movie reviews. So if you are a movies fan like me, and you are into Web API, you might be tempted to expose the search movie search functionality in your application/mashup.

So what you need is an API key. It is free to acquire and you just need a Mashery ID and register for an API key in Rotten Tomatoes here. After you got your api key, you need to append this key as a query string parameter to all your calls.

So let's imagine we want to search films having "matrix" word in them. I have done that and here is the JSON.

Next step is to create classes to represent your JSON. You are welcome to do that by hand but there is a useful online tool there that can do that for you: json2csharp.

So here is what it has generated:

public class ReleaseDates
{
    public string theater { get; set; }
    public string dvd { get; set; }
}

public class Ratings
{
    public string critics_rating { get; set; }
    public int critics_score { get; set; }
    public string audience_rating { get; set; }
    public int audience_score { get; set; }
}

public class Posters
{
    public string thumbnail { get; set; }
    public string profile { get; set; }
    public string detailed { get; set; }
    public string original { get; set; }
}

public class AbridgedCast
{
    public string name { get; set; }
    public string id { get; set; }
    public List<string> characters { get; set; }
}

public class AlternateIds
{
    public string imdb { get; set; }
}

public class Links
{
    public string self { get; set; }
    public string alternate { get; set; }
    public string cast { get; set; }
    public string clips { get; set; }
    public string reviews { get; set; }
    public string similar { get; set; }
}

public class Movy
{
    public string id { get; set; }
    public string title { get; set; }
    public int year { get; set; }
    public string mpaa_rating { get; set; }
    public int runtime { get; set; }
    public string critics_consensus { get; set; }
    public ReleaseDates release_dates { get; set; }
    public Ratings ratings { get; set; }
    public string synopsis { get; set; }
    public Posters posters { get; set; }
    public List<AbridgedCast> abridged_cast { get; set; }
    public AlternateIds alternate_ids { get; set; }
    public Links links { get; set; }
}

public class Links2
{
    public string self { get; set; }
    public string next { get; set; }
}

public class RootObject
{
    public int total { get; set; }
    public List<Movy> movies { get; set; }
    public Links2 links { get; set; }
    public string link_template { get; set; }
}

As you can see, it does a very good job in converting them but there are some naming conventions that you might have to go back and re-visit (In C# we use Pascal-Casing). Also Movy really needs to be Movie but for our purpose it is just fine.

So let's open Visual Studio 2010 and create a new console application project. Then add a code file and paste the codes above in there.

Then in the "Package Manager Console", type

PM> Install-Package System.Net.Http.Formatting
This should add all client libraries you need. Then we just need to write

var httpClient = new HttpClient();
   var rootObject = httpClient.GetAsync("http://api.rottentomatoes.com/api/public/v1.0/movies.json?q=matrix&page_limit=2&page=1&apikey=[YOUR_API_KEY]")
    .Result.Content.ReadAsAsync<RootObject>();

You should have the data correctly deserialised back to your objects. Well, not quite, there is a problem since the API uses legacy content-type "text/javascript" for the JSON instead of  "application/json". With the technique we learnt back in Part 7 and 8, we can create a message handler to change the legacy content type to the correct one and add it to the pipeline:

public class LegacyJsonMediaTypeConverterDelegatingHandler : DelegatingHandler
{
 protected override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
 {
    
  return base.SendAsync(request, cancellationToken)
     
   .ContinueWith(task =>
             {
                 var httpResponseMessage = task.Result;
        if (httpResponseMessage.Content.Headers.ContentType.MediaType == "text/javascript")
         httpResponseMessage.Content.Headers.ContentType.MediaType = "application/json";
                 return httpResponseMessage;
             });
 }
}

Now all we have to do is to add the handler to the pipeline and make the call:

static void Main(string[] args)
{

 var httpClient = new HttpClient(new LegacyJsonMediaTypeConverterDelegatingHandler() { InnerHandler = new HttpClientHandler()});
   
 var rootObject = httpClient.GetAsync("http://api.rottentomatoes.com/api/public/v1.0/movies.json?q=matrix&page_limit=2&page=1&apikey=YOUR_API_KEY")
  .Result.Content.ReadAsAsync<RootObject>();


 Console.Read();
}


Conclusion

ObjectContent<T> is an important class in ASP.NET Web API which abstracts the HTTP content (or a part within a multipart message) which includes its body and body-related headers.

MediaTypeFormatter on the client is as important as on the server. For sending requests, since client is the initiator of the request, it needs to explicitly set the formatter. For processing response data, however, a suitable formatter is selected based on type of the object to be read and content-type.