Before I go into detail on my experiences with Kubernetes, you need to know a few things about me. I’m obsessed with automation and managed solutions. It all started when AWS released Lambda. It was still in Beta, but I fell in love with it immediately.

For a long time I didn’t realise that it was the underlying concept of Lambda that I liked, rather than Lambda itself. You can just have a piece of code and don’t need to worry about anything else, it will just magically work.

After some time, problems with AWS Lambda became apparent as the codebase grew and became more complicated. That’s when my search started for, well, let’s call it “The Tool”. I tried different solutions (Google App Engine Flex and Standard, AWS Elastic Beanstalk, Azure App Service and Functions), but I wasn’t really happy with any of them.

My requirements were:

1. No lock-in to a specific provider
2. Quick and reliable auto scaling
3. High availability with auto repair
4. Low cost
5. Continuous integration to make the development easy
6. Minimum effort needed for managing/supporting it

I know that’s pretty much everything that you could ever want and difficult to achieve, but my main goal is always to get as close to these sub goals as I can. Based on my latest experiment with Kubernetes, I can say that I came closer than ever before to meeting my requirements.

This is what I experienced . . .

1. Lock-in

I’m not saying that Kubernetes can be deployed to any of the big providers or to your on-premise servers by just one single command; but if you decide to change providers it won’t lock you in as badly as, for example, AWS Lambda. You don’t have to rewrite your AWS ‘.ebextensions’ config file to Google’s ‘app.yaml’ and hope it can and will behave the same as your previous configuration.

2. Scaling

Actually, I was very surprised when I started testing our first Kubernetes application. The goal was to build a high performing, reliable cloud computation application for one of our customers. In a nutshell, they had some functionality that was presented to us as a blackbox and we only knew two things about it: that it could be run as a docker image, and we could estimate the resource needed for the computation. Our job was to build the infrastructure around this module.

We used Kubernetes jobs and created a small distributor module to dynamically add new jobs, specifying the resources needed for each new job. The application scaled so smoothly and added new nodes when needed. It also bounded the computation jobs to 100 nodes, which was our upper limit for scaling up during our load testing. And finally, when it started to run out of work, it scaled back to the minimum.

3. High availability

During the test above and other tests, we tried to kill the system in various ways. (Sometimes, even not intentionally, because we hit some limits on Google Cloud). The application was self-healing. There is no better way of saying this. It survived what it could survive, and when the whole cluster died and we restarted it, it picked up any incomplete jobs and restarted them.

4. Low cost

Before Kubernetes our infrastructure usually consisted of at least three virtual machines. One for staging and two more for production to achieve high availability and separate the two environments. With Kubernetes we can run staging and production next to each other on the same VM, because we can separate them using namespaces. This means we can achieve the same functionality with fewer machines.

Furthermore, when using Kubernetes you can specify your resource requirements for each pod/docker image. This way, you can better utilise your VMs. Plus, you can run your workers on the same instances, next to your API layer. On App Engine or on Elastic Beanstalk, you have to create multiple apps to run multiple docker images, which immediately doubles your cost.

Summary/TL;DR

So far, Kubernetes looks like a great tool, that doesn’t lock you in to a specific provider, can easily and smoothly scale when needed, and does a good job of running everything as planned.