Introduction to Container Orchestration

Docker, Kubernetes, and Mesos, oh my! This guide tackles all three container
orchestration tools to see where their strengths are.

A single container host by itself is like a tree without a forest. That lone host
gives you a useful way to abstract your development and production environments
from what's underneath. But on its own, it gives you only a glimpse of its promise.

The big stories in the container world are now all about orchestration. Docker
Swarm, Kubernetes, Mesos/Marathon and others offer frameworks for managing
container hosts in concert.

But where to start?

In this guide, we'll look at what each of the major container orchestration tools
has to offer and why you'd choose one or the other.

What is Container Orchestration?

For the longest time, deploying an application into production was as much ritual
as it was science.

Deployment inolved ugly bash scripts with as many 'if' statements as there were
corner cases, workarounds and "don't ask why, it just has to be like that"
situations. Coordinating it all was a gnarled sysadmin, and maybe a DBA, who'd
jealously guard and devotedly follow the rites required to get code into
production.

Then came Chef, Puppet, Ansible, and continuous integration and deployment. They
made it easy to standardise testing and deployment. Importantly, once in place,
they allow developers and DevOps people to forget about the detail of what needs to
happen.

Similarly, containers allow us to standardize the environment and abstract away the
specifics of the underlying operating system and hardware. You can think of
container orchestration as doing the same job for the data centre: it allows us the
freedom not to think about what server will host a particular container or how
that container will be started, monitored, and killed.

Container orchestration is the big fight of the moment. While the container format
itself is largely settled, for now, the real differentiation is in how to deploy
and manage those containers.

How to Evaluate
Even though they all do "container orchestration," each solution's approach and
features vary enough that comparing them is best thought of as areas of Venn-
diagram-like crossover.

The three key differentiators to look out for are:

Level of abstraction: Does it deal in containers or services that happen to be

container-based?
Tooling: How do you manage the orchestration and how well does it integrate with
other services?
Architecture: How does it cope at scale and in the face of failure?
It's also worth considering how opinionated each framework is: Are you willing to
buy into an entire philosophy, or do you want something that will unobtrusively
support your existing approach?

Docker Swarm
Swarm is Docker's own container orchestration tool. It uses the standard Docker API
and networking, making it easy to drop into an environment where you're already
working with Docker containers.

Key Concepts
A Swarm consists of manager and worker nodes that run services. Let's look at what
each of them does:

Managers: distribute tasks across the cluster, with one manager orchestrating the
worker nodes that make up the swarm.
Workers: run Docker containers assigned to them by a manager.
Services: an interface to a particular set of Docker containers running across the
swarm.
Tasks: the individual Docker containers running the image, plus commands, needed by
a particular service.
Key-value store: etcd, Consul, or Zookeeper storing the swarm's state and providing
service discoverability.
Things to Look Out For
Swarm is remarkably straightforward to set-up: Use Docker Engine to initalise the
master node(s), and it provides you with a command you can use on each worker node
to add them to the cluster.

Once the swarm is running, you specify services using Docker Compose. When you
bring those services up they are deployed across the hosts of the swarm rather on a
single host. Any networks you define will also work across the swarm.

Swarm is somewhat modular: as mentioned above, you can swap out the kv store, and
there are experiments using Mesos as an alternative scheduler. However, it goes
without saying that by using Swarm, you are buying into the Docker way. For many,
that will be a plus, but it's worth considering that not only is Swarm container-
centric, but it is Docker-centric.

When to Use Docker Swarm

Docker Inc. is investing a lot of effort into Swarm, so things are changing
rapidly. Until now, at least, Swarm has been viewed as suitable for experiments and
smaller scale deployments. That, perhaps, was due to limitations of early Swarm
releases and it remains less proven than Kubernetes and Mesos.

Docker Swarm gives you an easy way to move into container orchestration without
leaving behind the familiarity of existing Docker tools and thinking.

Kubernetes
Kubernetes is based on Google's experience of running workloads at huge scale in
production over the past 15 years. It's not an open sourcing of Borg, their
internal container orchestration system, but draws on lessons Google have learned
from running Borg.

Where Docker Swarm extends single-host Docker, Kubernetes' starting point is the
cluster itself.

Key Concepts
If you're considering Kubernetes, now's the time to think about how willing you are
to step away from the Docker way.

Here's what makes up a Kubernetes cluster:

Master: by default, a single master handles API calls, assigns workloads and
maintains configuration state.
Minions: the servers that run workloads and anything else that's not on the master.
Pods: units of compute power, made up of one or a handful of containers deployed on
the same host, that together perform a task, have a single IP address and flat
networking within the pods.
Services: front end and load balancer for pods, providing a floating IP for access
to the pods that power the service, meaning that changes can happen at the pod
level while maintaining a stable interface.
Replication controllers: responsible for maintaining X replicas of the required
pods.
Labels: key-value tags (e.g. "Name") that you and the system use to identify pods,
replication controllers, and services.
Kubernetes starts big. Whereas Docker Swarm starts at the container and builds out,
Kubernetes starts at the cluster and uses containers almost as an implementation
detail.

Things to Look Out For

Already, you can see that Kubernetes diverges somewhat from what you might be used
to with Docker alone. The CLI and API are different to Docker's, but the momentum
behind Kubernetes means that there's no shortage of language-specific libraries and
other integrations.

Modularity is fundamental to the Kubernetes approach. For example, you can choose
from Flannel, Weave, Calico, and other networking options. You can also swap out
the stock scheduler and use Mesos instead.

That modularity extends to the containers themselves. Kubernetes is not a Docker-

only show. While that might be inconvenient if you're already familiar with Docker
tooling, it does leave open the possibility to use rkt and other container formats.

When to Use Kubernetes

Kubernetes is well-suited to medium-to-large clusters running complex applications.
In particular, if you're already thinking in terms of multiple sets of stateless
microservices, then Kubernetes gives you a framework to establish the rules of
interaction between them and then run the show.

For HA, you'll need some extra work to establish multiple masters.

If you're running stateful apps, such as a database, then, until recently, you'd
have needed to look elsewhere. Let's put aside the debate over whether stateful
apps in containers are a good idea: Kubernetes 1.3 introduces support for "pet"-
like containers that have configuration dependencies and need stateful failover.

The learning curve and set-up will take more effort than Docker Swarm, but the pay-
off comes when you need the flexibility of its modularity and the suitability to
larger scale deployments.

Mesos and Marathon

Apache Mesos pre-dates Docker and is described as a distributed systems kernel. In
other words, it presents a single logical view of multiple computers. In more
concrete terms, it's a cluster manager that makes computing resources available to
frameworks. It's those frameworks that deal with the specifics of what runs on the
Mesos cluster.

Marathon is one such framework and it specialises in running applications,

including containers, on Mesos clusters.

Together, Mesos and Marathon offer an equivalent to Kubernetes, while also allowing
you to run non-containerised workloads alongside containers.

Key Concepts
Mesos operates two levels of scheduling: a Mesos slave reports to the master to say
that it has some free resource. Mesos then offers that resource to a framework,
based on a previously set policy.

The lucky framework then offers that resource to whatever processes it looks after,
again according to an appropriate policy.

There could be multiple frameworks running alongside each other on top of Mesos,
including Marathon for containers, but also Storm or Spark. It's even possible to
use Marathon to deploy other frameworks, but let's leave that to one side.

Here's what makes up our Mesos cluster:

Masters: Zookeeper manages a minimum of three master nodes and enables high
availability by relying on a quorum amongst those nodes.
Slaves: these nodes run the tasks passed down by the framework.
Framework: Mesos itself knows nothing about the workloads, whereas specialist
frameworks decide what to do with the resources offered to them by Mesos.
On top of Mesos, Marathon then offers a highly available framework delivering:

Service discovery: through a dedicated DNS service, as well as other options.

Load balancing: through HAProxy.
Constraint management: to control where in the cluster certain workloads run,
maintain a set level of resources for those workloads, enable rack awareness, and
other constraints.
Applications: the long-running services you want to run on the cluster; may be
Docker containers but can also be other types of workload.
REST API: deploy, alter and destroy workloads.
Things to Look Out For
Both Mesos and Marathon are designed to scale to hundreds of thousands of nodes. If
you have a relatively small cluster, then perhaps the resource and admin overhead
of running such a large system isn't appropriate. It also makes set-up more
complex, especially when compared to Docker Swarm.

Working with Mesos and Marathon will also be somewhat different to either Swarm or
Kubernetes. It's another set of tooling and APIs to learn.

When to use Mesos and Marathon

Mesos is proven at tens of thousands of nodes in production. Project sponsor
Mesosphere says that there are Marathon clusters with thousands of nodes in
production. This is where both Mesos and Marathon gain from being older than either
Kubernetes or Swarm: They've had time to knock off the rough edges and to be proven
in long-running, real-world situations at massive scale.

Mesos and Marathon are, perhaps, a good choice if you need to run non-containerized
workloads alongside containers and you want the reassurance of something that is
proven at very large scale. The trade-off versus Swarm is that it's harder to get
started and there's a steeper learning curve. The trade-off against Kubernetes is
partly more to do with personal appetite for risk, but Kubernetes has been designed
from the start as a container orchestration tool.

Conclusion
The three main options right now vary considerably in implementation and in how you
interact with them. Docker's Swarm gives you the easiest route into orchestrating a
cluster of Docker hosts. Kubernetes is container-centric but focuses less on the
containers themselves and more on deploying and managing services. Mesos with
Marathon promises huge scale but introduces additional complexity.

There are other options out there, too. For example, Rancher and Nomad both offer
different takes on service orchestration.

Ultimately, which option you take will depend on the scale you want to achieve and
which ecosystem you feel most comfortable in.