Distributed Operating Systems: Has the Time Finally Come?

Cloud computing seems to be having finally a surge in popularity. Cloud platform providers like Amazon, Microsoft and Google are adding new services to their portfolio at an increasing pace that honestly I’m starting to find difficult to follow. Their portfolios are starting to shape like complete packages for highly scalable applications. It’s not just virtual machine instances anymore. Each of the major three providers offers solutions for container management, load balancing, data persistence in many forms and shapes, monitoring distributed applications, big data processing, various kinds of communication, serverless architectures etc. A cloud provider is expected to offer all of the infrastructure needed by application developers, so that they can focus only on the business logic. These solution packages are starting to look like some sort of operating systems, distributed operating system.

Truth be told, distributed operating systems have existed for a number of decades. The likes of Amoeba, Mach (an ‘ancestor’ of MacOS), Chorus, DCE and Plan 9 have laid the foundations. These systems were groundbreaking from a computer science point of view, but however fascinating they have never reached any major success. They all mimicked traditional operating systems to make it easier to port existing applications written for a single desktop computer. In my opinion, that may be the reason why they failed. Writing a distributed application is not a trivial task and in essence it is very different from writing an application meant for a single computer.

Today’s cloud platforms take a different approach. They were born from the need to distribute the load of server side applications. They evolved from the ground up providing more and more abstract services driven by the demand rather than reproducing the services offered by a desktop OS. The conditions are different today and there is real demand, almost necessity for these kind of platforms.

A side effect of this bottom-up approach is a lot of unnecessary complexity. Cloud platforms are still just a portfolio of solutions that need to be stitched together, rather than a single platform with many features. Typical cloud applications combine purely cloud solutions like AppEngine or S3 with standard distributed software like Cassandra or Kafka that is installed on VM/Container instances. The difference between the two is in the level of abstraction. Cassandra, Kafka and others are services running on many instances that communicate together and, even if they achieve great scalability and performance, they still need to be installed, connected, configured, upgraded etc. A pure cloud solution hides all those details: there are no individual machines, no configuration files, no installations or versions. There’s just the cloud.

I believe that the future of cloud platforms is going to be a more abstract and homogeneous cloud operating system. Perhaps like Google’s AppEngine or Amazon’s Lambda, but with built in persistence, big data, monitoring but hiding away all the plumbing. Something similar to the Single System Image concept. I also believe that the first provider to realize how to do this without user lock-in will make a revolution in cloud computing.