Saturday, 21 December 2013

Thank you Microsoft, and so long ...

[Level C1]

Ideas and views expressed here are purely my own and none of my employer or any related affiliation.

Yeah, I know. It has been more than 13 years. Still remember how it started. It was New Years Eve of the new millennium (yeah year 2000) and I was standing on the shore of the Persian Gulf in Dubai. With my stomach full of delicious Lebanese grilled food and a local newspaper in hand looking for jobs. I was a frustrated Doctor, hurt by the patient abuse and discouraged by the lack of appreciation and creativity in my job. And the very small section of medical jobs compared to pages, after pages, after pages of computer jobs. Well, it was the dot.com boom days, you know, it was the bubble - so we know now. And that was it. A life-changing decision to self-teach myself network administration and get MCSE certificate.

Funny thing is I never did. I got distracted, side-tracked by something much more interesting. Yeah programming, you know, when it gets its hooks in you, it doesn't leave you - never. Hence it started by Visual Basic, and then HTML, and obviously databases which started from Microsoft Access. Then you had to learn SQL Server, And not to forget web programming: ASP 3.0. I remember I printed some docs and it was 14 pages of A4. It had everything I needed to write ASP pages, connect to database, store, search, everything. With that 14 pages I built a website for some foreign Journalists working in Iran: IranReporter - It was up until a few years ago. I know it looks so out-dated and primitive but for a 14-page documentation not so bad.

Then it was .NET, and then move to C# and the rest of the story - I don't bore you with. But what I knew was that the days of 14-page documentation were over. You had to work really hard to keep up. ASP.NET Webforms which I never liked and never learnt.  With .NET 3.0 came LINQ, WCF, WPF and there was Silverlight. And there was ORMs, DI frameworks, Unit Testing and mocking frameworks. And then there was Javascript explosion.

And for me, ASP.NET Web API was the big thing. I have spent best part of the last 2 years, learning, blogging, helping out, writing OSS libraries for it and then co-authoring a book on this beautiful technology. And I love it. True I have not been recognised for it and am not an MVP - and honestly not sure what else could do to deserve to become one - but that is OK, I did what I did because I loved and enjoyed it.

So why Goodbye?

Well, I am not going anywhere! I love C# - I don't like Java. I love Visual Studio - and eclipse is a pain. I adore ASP.NET Web API. I love NuGet. I actually like Windows and dislike Mac. So I will carry on writing C# and use ASP.NET Web API and Visual Studio. But I have to shift focus. I have no other option and in the next few lines I try to explain what I mean.

Reality is our industry is on the cusp of a very big change. Yes, a whirlwind of the change happened already over the last few years but well, to put it bluntly I was blind.

So based on what I have seen so far I believe IN THE NEXT 5 YEARS:

1. All data problems become Big Data problems

If you don't like generalisations you won't like this one too - when I mean all I mean most. But the point is industries and companies will either shrink or grow based on their IT agility. The ones shrinking not worth talking about. The ones growing will have big data problems if they do not have them yet. This is related to the growth of internet users as well as realization of Internet of Things. And the other reason is the adoption of Event-Based Architecture which will massively increase the amount of data movement and data crunching.

Data will be made of what currently is not data. Your mouse move and click is the kind of data you would have to deal with. Or user eye movement. Or input from 1000s of sensors inside your house. And this stuff is what you will be working on day-in day-out.

Dear Microsoft, Big Data solutions did not come from you. And despite my wishes, you did not provide a solution of your own. I love to see an alternative attempt and a rival to Hadoop to choose from. But I am stuck with it, so better embrace it. There are some alternatives emerging (real-time, in-memory) but again they come from the Java communities.

2. Any server-side code that cannot be horizontally scaled is gonna die

Is that even a prediction? We have had web farms for years and had to think of robust session management all these years. But something has changed. First of all, it is not just server-stickiness. You have to recover from site-stickiness and site failures too. On the other hand, this is again more about data.

So the closer you are to the web and further from data, the more likely that your technology will survive. For example, API Layer technologies are here to stay. As such ASP.NET Web API has a future. Cannot say the same thing for middleware technologies, such as BizTalk or NServiceBus. Databases? Out of question.

3. Data locality will still be an issue so technologies closer to data will prevail

Data will still be difficult to move in the years to come. While our bandwidth is growing fast, our data is growing faster so this will probably be even a bigger problem. So where your data is will be a key in defining your technologies. If you use Hadoop, using Java technologies will make more and more sense. If you are Amazon S3, likelihood of using Kafka than Azure Service Bus. 

4. We need 10x or 100x more data scientists and AI specialists

Again not a prediction. We are already seeing this. So perhaps I can sharpen some of my old skills in computer vision.

So what does it mean for me?

As I said, I will carry on writing C#, ASP.NET Web API and read or write from SQL Server and do my best to write the best code I can. But I will start working on my Unix skills (by using it at home) and pick up one or two JVM languages (Clojure, Scala) and work on Hadoop. I will take Big Data more seriously. I mean real seriously... I need to stay close to where innovation is happening which sadly is not Microsoft now. From the Big Data to Google Glass and cars, it is all happening in the Java communities - and when I mean Java communities I do not mean the language but the ecosystem that is around it. And I will be ready to jump ships if I have to.

And still wish Microsoft wakes up to the sound of its shrinking, not only in the PC market but also in its bread and butter, enterprise.

PS: This is not just Microsoft's problem. It is also a problem with our .NET community. The fight between Silverlight/XAML vs. Javascript took so many years. In the community that I am, we waste so much time fighting on OWIN. I had enough, I know how to best use my time.

16 comments:

  1. I disagree with all of your predictions, but it sounds like you should learn googles protocol buffers. It's a big part of how Google does big data. I use them all the time from C# and love the forward and backwards compatibility and being able to rename fields without breaking anything! There's a great library called protobuf-net.

    ReplyDelete
    Replies
    1. Protobuf-net? I think you missed the point on the article.

      Delete
  2. You need to read up about the following things:

    F#
    Azure, particularly the new "Orleans" feature set that's coming.
    HDInsight (on Azure)

    ReplyDelete
    Replies
    1. Read up? I am using HDInsight already...

      Delete
  3. I too wish MS wasn't so behind. I truly love working with their technology it is the Mercedes Benz of the development world l, it can cost initially but then its well worth the proce.

    I think the stuff happening with Web Api 2.0, SignalR 2.0, the acknowledgement that JavaScript/HTML 5 is the king of UI and the front end are all great steps on the right direction.

    the only thing lacking is a strong big data alternative but hopefully once its here it will be as nice to work with as the other technologies.

    I think MS is making a turn and will deliver in the enterprise. There is a huge paradigm shift going on though, the enterprise is no longer driving technology, the consumer side is....

    ReplyDelete
  4. "big data" has been dramatically oversold, not suprisingly by the too-many companies trying to commercialize hadoop. who here really deals with petabytes of data?

    ReplyDelete
  5. I do not use Microsoft products if I can help it, but I think they will dig in and deliver an attractive alternative to other providers. I think they will be a shadow of their former self but Microsoft aren't going anywhere soon. I think in order to succeed they need to let their own company compete with their own company, for example google stealing the march with collaborative editing, and Microsoft until recently wanting you to buy Office in the traditional way.

    Interesting times ahead providing we can lock all the lawyers in a box.

    ReplyDelete
  6. Huh? I get the sense you are not as dialed in on technological trends as you think you are. Nothing beats Microsoft tools for enterprise web application development. Not PHP, not Ruby, not Linux. No other stack is more scalable/robust and since Visual Studio 2012 no other stack is faster. Visual Studio 2013, Sql Management Studio 2012 and Azure are so amazingly well integrated together I kept having "There is no way that is going to work moments" on a small business app I built. Microsoft is going anywhere and coders that don't like Microsoft or PC's just have never used them much plain and simple.

    ReplyDelete
    Replies
    1. This comment has been removed by the author.

      Delete
  7. This comment has been removed by the author.

    ReplyDelete
  8. This comment has been removed by the author.

    ReplyDelete
  9. I like your sharp vision into the future ... yes you are right. My question is what is the effort needed by the community to create a .net version of Hadoop or big data services ... ?
    I mean we have copied alot of things from Java-community ... why stop now ?
    Do you believe that the server-side platform its (Windows and languages) are not Earth-size ready ?

    ReplyDelete
    Replies
    1. It is not impossible. But it is a platform, not a library. So it is an elephant job. And not sure if worth it.

      Delete
  10. And one more thing ... have you any idea that microsoft is working on a server side C# version ... ?
    http://joeduffyblog.com/2013/12/27/csharp-for-systems-programming/

    ReplyDelete
  11. Interesting that you say that WebAPI has a future, seeing as all of its predecessors (WCF, ASMX Web Services, etc) were replaced in fairly short order.

    ReplyDelete
    Replies
    1. Thanks Udi for stopping by. The reason I believe it stays is because HTTP is here to stay. For a long time.

      Delete