Engineering

Our own Kubernetes platform, in two countries

Most agencies rent a managed platform and hope it stays up. We run our own Kubernetes and virtualisation clusters, in Germany and Finland. Here's how it's put together, and why we bother.

Diagram of El Niño's Kubernetes platform: clusters in Falkenstein (Germany) and Helsinki (Finland), with MySQL clusters, Ceph and S3 storage, load balancers and daily backups.

Why run our own platform?

Because we're responsible for the software we build long after it goes live. Running the platform ourselves means we decide how it's set up, we can fix problems at every level, and your data stays where we say it is: in Europe.

Two locations

Our Kubernetes and virtualisation clusters have nodes in two Hetzner data centres: Falkenstein in Germany and Helsinki in Finland. Both are within the EU, so everything falls under the GDPR.

The services on top

Most applications need the same building blocks, so we run them once, properly, and set them up to keep working when a server fails:

  • MySQL clusters for application databases.
  • Ceph and S3 storage for files and uploads.
  • Load balancers in front of the applications.
  • Automatic incremental backups and snapshots.

Applications on Kubernetes

Every application runs as containers on the cluster, described in configuration that lives in the same repository as the code. A few things every application gets by default:

  • Rolling updates. A new version starts next to the old one, and only receives traffic once it's healthy. With more than one instance running, like for this website, there is no downtime during a deploy.
  • HTTPS, automatically. Certificates are requested and renewed by the cluster itself (via Let's Encrypt), and browsers are told to always use HTTPS.
  • Config changes apply themselves. When an application's configuration changes, its containers restart with the new values.
  • Persistent storage for the applications that need it, like the CMS behind this website.

Backups

Application data is backed up every day. Backups are encrypted, kept for at least two weeks and stored somewhere else than the applications themselves: at DigitalOcean Spaces in Amsterdam or Frankfurt.

Monitoring

Grafana shows us how the platform is doing. New Relic and Sentry tell us how the applications are doing, and alert us on errors. That's usually how we know about a problem before a customer does.

Want to know more?

Read how a change gets from a developer's laptop to production in From merge request to production, or see what we offer for hosting and SLAs.