The State of Kubernetes 2019

Last year I wrote a post entitled A Trip From the Past to the Future of Kubernetes. In it, I talked about the KVM and AWS versions of our stack and the imminent availability of our Azure release. I also referred to the rollercoaster of the 2 prior years.

Let’s start by revisiting last year’s predictions:

1. Kubernetes will make further steps to become “The Cloud API”

I think we can make a check at that one. The build-up in the last 12 months has been amazing to watch. Kubernetes is now front and center everywhere. Some examples are: IBM buying Redhat and hence Openshift, VMWare buying Heptio and KubeCon growing from hundreds to thousands of attendees. Kubernetes has hit the mainstream. When we talk to companies that are going towards a modern development stack, they talk about Kubernetes. Period.

2. Kubernetes Management and support plays start scaling up and finding their stride

Again…check. We are seeing amazing inbound interest in using our platform to run Kubernetes at scale. Organizations that started playing with K8s a year or two ago, are now going into production. They are finding that this is very much a different beast. Kubernetes is easy to play with but hard to run in production. The question then is if you set up a team or let your cluster be handled by a third party. Your developers will be able to count on the resilience required by their business customers.

3. Kubernetes will die

Again, we hit the nail on the head there. Kubernetes is not enough. As a result, we have been working on our service catalog for almost a year now. Our customers are talking not only about Kubernetes but Istio, Prometheus, Kibana, and more. They want to benefit from these Cloud Native tools but do not want the added overhead of managing them. With Giant Swarm managing them, our customers can focus on the applications required to manage their core business. Bottom line, Managed Kubernetes is becoming the Managed Cloud Native Stack.

Hence, I think, this year, I have to go further out on a limb with my predictions. To see if I succeeded I will revisit them at the end of 2019.

Our Predictions for Cloud Native in 2019:

1. The big ones will double down on Kubernetes

You have already seen that with the big cloud providers as well as other players like IBM and VMWare. The acquisitions of Red Hat and Heptio respectively show that these companies are getting more into the game. These are mostly support plays though and if you look at the managed services market, with companies like Accenture, there are still plenty of companies that service enterprises that will need to increase their capabilities. The cards are being shuffled again, as with all major changes in infrastructure.

The big ones might also be some bigger Fortune 500 companies that decide to rebuild internal platforms on Kubernetes and go their own way based on Open Source tools. Partly due to hiring pressure you will see an increase in Open Source tools around Kubernetes being released from these companies. We cover this same topic below in prediction #5.

2. The Focus will be on Day 2 Operations

Installing Kubernetes is easy, but then what? Our focus on day 2 operations, honing in on security, updates, post mortems, and more has been tested and proven. The big players will be in a race to get newer versions out the fastest with CVEs fixed in production in next to no time. Customers will demand it and adapt their processes to allow for it.

We have talked with many people that were burned here. They are still running Kubernetes 1.8 or lower with no upgrade on the horizon. This leaves them with a gaping security hole, since recent patches have only been released for 1.10 and above.

3. Big Data will move to Kubernetes

You will see big data players support Kubernetes with all their might. Companies will move their production systems away from huge, but old big data stacks. They will also move away from proprietary cloud provider stacks, towards specialized Kubernetes solutions in multi-cluster environments. The distinction between cloud and non-cloud will blur as a side effect but this will take a bit longer than a year.

I recommend keeping an eye on Dotscience.

4. The rise of the control planes

We could not run Kubernetes at the scale we do without running our own control plane in Kubernetes. We use Operators (custom controllers) to manage our tenant clusters. Other players will focus on this, with or without Operators, as it becomes apparent that one cluster to rule them all does not cut it. The Cluster API is one project that needs to be mentioned here.

More clusters will be created at the edge for retail, industrial, and IoT workloads. A cluster per site makes it easier to deploy and update apps to these locations. However, this increase in cluster numbers will make automation even more important.

5. Extending the Kubernetes API

This is a 2-part prediction:

First: More and more apps will be deployed and managed by Custom Resource Definitions (CRDs). Independent Software Vendors (ISVs), especially will provide their software as packages that will either include or use operators to roll-out continuous updates. This software will often be packaged as Helm charts.

Second: Kubernetes providers new and old will mainly extend Kubernetes using CRDs. On the one hand, this will go towards cluster APIs like Giant Swarm or the upstream Cluster API. On the other hand, towards platforms on top of Kubernetes, e.g. ML/AI platforms, PaaS-like offerings, CI/CD pipelines, all as native Kubernetes extensions.

At Giant Swarm we are replacing our API gateway with CRDs and using the Kubernetes API directly. This lets us benefit from Kubernetes features such as RBAC and Admission Controllers. It also makes it easier for our API to interact with our Operators.

This at scale will increase the load on API servers as event managers. As a result, either the community will work on scaling API servers and Etcd itself or resort to out-of-cluster solutions.

In conclusion, Kubernetes is now well entrenched in the mainstream. It is an increased focus both for the big providers and for the big users. Yet, it has by no means plateaued. This means operational challenges at scale on the one hand, and many changes and rapid developments to the ecosystem on the other hand. I am looking forward to your comments with your predictions for 2019.