Application Hosting: Bare Metal, Nomad or Kubernetes?

Application hosting is a complex choice. Do you just install the application and its dependencies on a server, and start the process with the native runtime binary? Are you Dockerizing your application and run a container? Are you using a scheduler like Nomad or Kubernetes?

I was faced with this decision for my lighthouse as a service application. This article explains and discusses the pro and cons of application hosting options. While my choice reflects my application requirements, I’m sure you can learn about the difference options and apply this knowledge when you make a similar decision.

More about lighthouse: This service scans webpages and reports how good they score on terms of SEO, performance and best practices. You can use the scanner here: https://lighthouse.admantium.com/.

This article originally appeared at my blog.

Application Deployment Options

Deployment comes down to a simple core requirement: Make the frontend reachable with a domain name, and make the backend accessible for the frontend.

This sounds easy — and then you think about aspects like TLS encryption, service discovery and communication, fetching log files, fault tolerance, auto-scaling, build pipeline … the list goes on.

So, what are the options? I will discuss three types:

Bare Metal

Required Steps

Frontend

Backend

Deployment

The installation and configuration steps could be automated with Ansible playbooks. Ansible roles for installing Nginx and Docker are available. A custom task would build and push the static HTML to the server. Build and deployment of the backend Docker container can be done with a shell script.

Features

Consul & Nomad

Required Steps

Cluster Management

Frontend

Deployment

Installation steps for the cluster management software would be covered by Ansible playbooks. The Encryption of Consul & Nomad is a semi-automatic process, in which I need to perform rolling updates on the nodes. Installing Docker would be done with Ansible. and Dockerizing the Frontend app, and getting the TLS certificate for Nginx, is a manual process. The workload definition combines deployment, service discovery and health checks. For load balancing, you also need to setup the local nodes to use the Consul DNS server.

Features

Kubernetes

Required Steps

Cluster Management

Frontend

Deployment

Installation would be covered by choosing a Kubernetes distribution1. Dockerizing the frontend app, and installing cert manager is a manual process. Then all the remaining work is to define the Kubernetes resource definitions.

Features

Choice

To make a choice, I considered various aspects: Time to get it running, number of manual tasks, number of features.

At first, I wanted to have the fasted time to market as possible. For me, this would be the bare metal installation. I have Ansible roles for Nginx and Docker installation ready. Pushing HTML and Dockerfiles to the nodes, and even starting the Docker container, can be done with Ansible too. The application would work, but as shown, a lot of features would be missing. Yes, you can define a docker health checks and autostart. Yes, you can setup health monitoring and log scraping. But these involve manual tasks again, and you would have more moving pieces that you need to manage and get experienced with.

Deploying Lighthouse needs more reliability and out-of-the-box features. Is it Nomad & Consul, or Kubernetes?

I have collected experience with Consul & Nomad for my infrastructure@home project. The steps to create a cluster are almost automated with Ansible playbooks. Overall, it works. Yet I had some challenges: Setting up storage volumes, or providing persistent service discovery with DNS on the hosts and between Consul nodes. Also, configuring Traefic or Nginx for ingress traffic was not easy. And there are also unsolved problems: Load-balancing with Nginx between multiple Consul services for example, because DNS resolution for upstream servers is not possible with the open source version of Nginx2.

With Kubernetes, I tried it initially with Docker for Desktop in 2018. Also, at work most applications are hosted with Kubernetes, but we use a given set of Helmcharts for deployment and do not administrate the cluster ourselves. However, a growing interest and steady accumulation of knowledge about the Kubernetes made me curious. The one thing that got me really interested is the K3S with which you get a Kubernetes cluster by running exactly one command on each node! Consider this simplicity against the Consul & Nomad installation. Also, all following tasks will be done with writing resource definitions for Kubernetes. Staying within one abstraction level is a benefit I cannot emphasize enough.

Deployment with K3S

So, in the end I decided to use Kubernetes with K3S. In only 7 days, including improvements of frontend and backend code, Lighthouse is hosted with Kubernetes. I saw how so many details including TLS, DNS, IPs and Ports are automated and abstracted. Yes, the learning curve is steep, and it takes time to absorb how Kubernetes works. F or me, it seems that all the troubles I had before made me knowledgeable to flatten this curve. I’m using Kubernetes, and don’t want to use another option.

Here is a list of these moments to share the enthusiasm with you:

I have just very readable, very declarative resource definitions that provide all of the required features out-of-the box. Take a look at this Deployment declaration.

apiVersion: apps/v1
kind: Deployment
metadata:
name: lighthouse
spec:
replicas: 6
selector:
matchLabels:
app: lighthouse
template:
metadata:
labels:
app: lighthouse
spec:
containers:
- name: lighthouse
image: docker.admantium.com/lighthouse:0.1.4
imagePullSecrets:
- name: registry-secret

Conclusion

Kubernetes is the cluster management software to host the Lighthouse scanner. In only 7 days I set up the cluster on three nodes, created a private Docker registry, and deployed multiple containers running Lighthouse. It’s all there: TLS encryption, automatic health-check, service-discovery, load balancing and auto healing. No additional software on the nodes, no additional config for IP, ports and DNS is necessary. It just works. I highly recommend to make the switch to Kubernetes too.

Footnotes

IT Project Manager & Developer