Kubernetes The Hard Way Vagrant Save Abandoned

A port of Kelsey Hightower's "Kubernetes the Hard Way" tutorial to Vagrant. – By the Kinvolk team.

Project README

Kubernetes The Hard Way (Vagrant)

Vagrant configuration and scripts for a Kubernetes setup, the hard way.

The setup follows https://github.com/kelseyhightower/kubernetes-the-hard-way with the following exceptions:

  • cri-o is used as a container runtime, not cri-containerd
  • The pod-cidr is 10.2${i}.0.0/16, routes are provisioned from scripts/vagrant-setup-routes.bash automatically
  • is the IP of the loadbalancer (haproxy) for HA controllers

Please note that KTHW is a project to learn Kubernetes from bottom up and is not per se a guide to build clusters for production use!

Requirements Host

  • Vagrant (with VirtualBox)
  • Minimum of 7x 512MB of free RAM
  • cfssl, cfssljson and kubectl (scripts/install-tools can be used to download and install the binaries to /usr/local/bin)



To learn Kubernetes from the bottom up, it's recommended to go through KTHW manually. vagrant up gives you three controller and three worker nodes to do that.

The pod-cidr is 10.2${i}.0.0/16, for which the Vagrant nodes have configured routes (see route -n).

The following KTHW parts can/should be skipped:

  • Everything in regard to the frontend loadbalancer
  • Pod network rules are automatically setup via Vagrant

The scripts in scripts/ loosely match the setup steps in KTHW by Hightower and can be used as reference and/or to save typing. See scripts/setup also.

Single script

vagrant destroy -f   # remove previous setup
./scripts/setup      # takes about 5 minutes or more

If everything looks good, continue with "Using the cluster"

Multiple scripts

Remove previously created certificates, tools kubeconfig files:


Generate the required certificates:


Generate the kubeconfig files (as those include copies of the previously generated certificates):


Download required tools and files:


Start the virtual machines (optionally, go drink a coffee or tee):

vagrant up
$ vagrant status
Current machine states:

controller-0              running (virtualbox)
controller-1              running (virtualbox)
controller-2              running (virtualbox)
worker-0                  running (virtualbox)
worker-1                  running (virtualbox)
worker-2                  running (virtualbox)

Setup etcd on the controller nodes:


SSH into the first controller node:

vagrant ssh controller-0

Verify if the etcd is up:

sudo -i
export ETCDCTL_API=3
export endpoints=,,
export cacert=/etc/etcd/ca.pem
export cert=/etc/etcd/kubernetes.pem
export key=/etc/etcd/kubernetes-key.pem
# etcdctl member list --cacert=$cacert --cert=$cert --key=$key --endpoints=$endpoints
6c500a9f4f9113de, started, controller-0,,, false
e206d150eae73959, started, controller-2,,, false
e7e775a3da74a469, started, controller-1,,, false

# etcdctl endpoint health --cacert=$cacert --cert=$cert --key=$key --endpoints=$endpoints is healthy: successfully committed proposal: took = 9.551077ms is healthy: successfully committed proposal: took = 11.362935ms is healthy: successfully committed proposal: took = 13.922045ms

Setup the controller services:


Configure a kubernetes-the-hard-way context on your host, set it as default.


Verify controllers are up and running:

$ kubectl get componentstatuses
NAME                 STATUS    MESSAGE              ERROR
controller-manager   Healthy   ok
scheduler            Healthy   ok
etcd-1               Healthy   {"health": "true"}
etcd-2               Healthy   {"health": "true"}
etcd-0               Healthy   {"health": "true"}

Create ClusterRole's for kubelet API auth:


Setup the worker binaries, services and configuration:


See if the nodes are ready:

$ kubectl get nodes
worker-0   Ready    <none>   53s   v1.20.2
worker-1   Ready    <none>   33s   v1.20.2
worker-2   Ready    <none>   11s   v1.20.2

Using the cluster

Setup DNS add-on

Deploy the DNS add-on and verify it's working:

kubectl apply -f ./manifests/coredns.yaml
kubectl get pods -l k8s-app=coredns -n kube-system
kubectl run busybox --image=busybox:1.28 --command -- sleep 3600
kubectl exec -ti busybox -- nslookup kubernetes

Smoke tests

$ kubectl create -f ./manifests/nginx.yaml
deployment "nginx" created
service "nginx" created
$ NODE_PORT=$(kubectl get svc nginx --output=jsonpath='{range .spec.ports[0]}{.nodePort}')
$ for i in {0..2}; do curl -sS${i}:${NODE_PORT} | awk '/<h1>/{gsub("<[/]*h1>", ""); print $0}'; done
Welcome to nginx!
Welcome to nginx!
Welcome to nginx!

Connect to services from host is the IP range for services. In order to connect to a service from the host, one of the worker nodes (with kube-proxy) must be used as a gateway. Example:

# On Linux
sudo route add -net gw

# On macOS
sudo route -n add -net

Use Traefik loadbalancer

404 page not found

To test traefik is actually doing its job, you can create an ingress rule for the nginx service that you created above:

kubectl apply -f ./manifests/nginx-ingress.yaml
echo " nginx.kthw" | sudo tee -a /etc/hosts
curl nginx.kthw
<!DOCTYPE html>


Contributions are welcome: KTHW Vagrant is meant to be a learning project and testbed for aspiring Kubernetes operators and CKAs (Certified Kubernetes Administrator).

If you want to contribute code or updates, look for the label good first issue.


Error loading config file "/var/log": read /var/log: is a directory

On OSX, KUBECONFIG apparently needs to be set explicitly. ~/.kube/config is a good place and the default on Linux.

Open Source Agenda is not affiliated with "Kubernetes The Hard Way Vagrant" Project. README Source: kinvolk-archives/kubernetes-the-hard-way-vagrant
Open Issues
Last Commit
2 years ago

Open Source Agenda Badge

Open Source Agenda Rating