Kubernetes: The second cloud revolution

This post is reflective of the work outlined in our three-part blog series exploring our vision for a sustainable European cloud. Read part one here, part two here, and part three here.

The real revolution is Cloud Native, but as the Linux of the clouds, Kubernetes will be synonymous with the second cloud revolution.

The first cloud revolution

The promise of “the cloud” (infrastructure as a service – IaaS – in this case) is automation. With compute, storage, and various other primitives accessible through application programming interfaces (APIs), it became possible to automate workloads like MySQL or Kafka on Amazon Web Services (AWS) or Microsoft Azure, and concentrate on your core business.

This promise has certainly been met, but not as expected. Automating these kinds of workloads is hard. It has proven to be so hard that you need the size of a hyperscaler to be able to afford this. We know – we learned the hard way.

The dilemma

The public cloud (AWS, Azure, Google Cloud Platform, etc.) has dramatically increased developer productivity by automating basic primitive components like databases. But, it came at the cost of ceding control to ever expanding tech giants. These public clouds have out-automated everyone, and offer hundreds of services that are indispensable to development teams to compete, by bringing ease-of-use and extreme reliability. Yet, the dependence on these clouds and the enormous share of the revenue they take are painful enough for people to look around.

Software eats everything (including other software)

Docker has introduced a uniform way of packaging software into runnable containers. The adoption of Docker has been staggering. The main reason for this is that it relieves the developer of being dependent on cloud providers. Kubernetes can be crowned as the de facto standard for container orchestration. It is proving to be the home that docker containers have always been looking for. These two have subsequently birthed the Cloud Native movement, taking back control, step-by-step.

Kubernetes Operator Framework is the next huge step forward. It addresses the hard part of automating complex, often stateful workloads (i.e., with data you don’t want to lose).  Software vendors now have a way to distribute their increasingly complex systems.

Taking back control

Take for example Elastic. They develop one of the most useful software stacks at this moment. They want to be open, but they are under attack by hyperscaler(s) with their bottom line under pressure.

Kubernetes (the operator framework, to be precise) is enabling Elastic to distribute their software stack, including the necessary automation to run at scale. We have automated elasticsearch ourselves in the early days of the cloud, and it is really hard. With the Elasticsearch Kubernetes Operator and Elastic joining the Cloud Native Computing Foundation (CNCF), which is part of the Linux Foundation, the dilemma for software engineering teams to move your elasticsearch workload to a managed service offered by a hyperscaler has been resolved.

Creating viable alternatives

Even difficult workloads like machine learning are starting to become as simple to build and operate as Elasticsearch on Kubernetes. The mission of Kubeflow is to “make scaling machine learning (ML) models and deploying them to production as simple as possible, by letting Kubernetes do what it’s great at. With “the community” chipping away at the lack of services available, opportunities are opening up rapidly. A public cloud is still attractive, but software vendors are rapidly automating their systems for the Linux of the clouds (Kubernetes). It is interesting to realise that this re-establishes a software distribution category everyone thought was dead in the water. Enterprise software is back, equipped with hybrid business models of software as a service (SaaS) (on premise). Even local Kubernetes clusters are a viable alternative to a public cloud.

The second cloud revolution

The larger enterprises will be back in control, and can build their on-premise clouds cost effectively. Developers do not really care where they deploy their software, as long as they are free to work with the other systems they need. Smaller companies, who have not developed the deeper pockets needed for their own clouds, can look for local Kubernetes alternatives. Cheaper and greener Kubernetes clouds will be introduced (full disclaimer: we are opening two regions in Europe). The freedom to choose, and the redistribution of revenue will lead to the second cloud revolution. A new wave of innovation will be ushered in, eclipsing what the public clouds have brought us.

This time we own it

And this time we will not be locked in. The Kubernetes community is shepherded by the CNCF. Linux has shown that ownership can be redefined, resulting in impact that is hard to overstate. Kubernetes will be the Linux of the clouds.