Lab 1.1. Hello docker

1. Run docker info to confirm docker client and server statuses
2. Run docker run hello-world

Lab 1.2. First container interaction

## view kernel details
uname -r # or `cat /proc/version` or `hostnamectl`
# view os version
cat /etc/*-release # or redhat `/etc/redhat-release`, other unix-based os `/etc/os-release`
# view running processes, see `ps --help`
ps aux
# view processes, alternative to `ps`
ls /proc # to find PID, then
cat /proc/$PID/cmdline

1. Run ps aux to review running processes on your host device
2. Run a busybox container in interactive mode docker run -it busybox
3. Review the container kernel details
4. Review the running processes in the container and note PID
5. Exit the container
6. List running containers
7. List all containers
8. Repeat [2] and exit the container without stopping it
9. List running containers
10. List all containers
11. Delete the containers

# host terminal
ps aux
docker run --name box1 -it busybox
# container terminal
ps aux
uname -r
cat /proc/version
hostnamectl # not found
cat /etc/*-release # not found
busybox | head
exit
# host terminal
docker ps
docker ps -a
docker run --name box2 -it busybox
# container terminal
ctrl+p ctrl+q
# host terminal
docker ps
docker ps -a
docker stop box2
docker rm box1 box2

docker ps showing STATUS of Exited (0) means exit OK, but an Exit STATUS that's not 0 should be investigated docker logs

CTRL+P, CTRL+Q only works when running a container in interactive mode, see how to attach/detach containers for more details

Lab 1.3. Entering and exiting containers

# run container with specified name
docker run -d --name webserver httpd
# run command `date` in a new container
docker run busybox date
# get a "dash" shell to a running container, see `docker exec --help`
docker exec -it $CONTAINER_NAME_OR_ID sh
# get a "bash" shell to a running container
docker exec -it $CONTAINER_NAME_OR_ID bash
# view open ports, the commands below only work if installed in the container
netstat -tupln # see `netstat --help` - t tcp, u udp, p program-names, l listening, n port-numbers-only
ss -tulpn # see `ss --help`, alternative to netstat

1. Run a nginx container
2. List running containers (use another terminal if stuck)
3. Exit the container
4. List running containers
5. Run another nginx container in interactive mode
6. Review container kernel details
7. Review running processes in the container
8. Exit container
9. Run another nginx container in detached mode
10. List running containers
11. Connect a shell to the new container interactively
12. View open ports in the container
13. Exit the container
14. List running containers
15. Delete all containers

# host terminal
docker run --name webserver1 nginx
# host second terminal
docker ps
# host terminal
ctrl+c
docker ps
docker run --name webserver2 -it --rm nginx bash
# container terminal
cat /etc/*-release
ps aux # not found
ls /proc
ls /proc/1 # list processes running on PID 1
cat /proc/1/$PROCESS_NAME
exit
# host terminal
docker run --name webserver3 -d nginx
docker ps
docker exec -it webserver3 bash
# container terminal
netstat -tupln
ss -tulpn
exit
# host terminal
docker ps
docker stop webserver3
docker rm webserver1 webserver2 webserver3

Containers may not always have bash shell, but will usually have the dash shell sh

Lab 1.4. Container arguments

1. Run a busybox container with command sleep 30 as argument, see sleep --help
2. List running containers (use another terminal if stuck)
3. Exit container (note that container will auto exit after 30s)
4. Run another busybox container in detached mode with command sleep 300 as argument
5. List running containers
6. Connect to the container to execute commands
7. Exit container
8. List running containers
9. Run another busybox container in detached mode, no commands
10. List running containers
11. List all containers
12. Delete all containers

# host terminal
docker run --name box1 busybox sleep 30
# host second terminal
docker ps
docker stop box1
# host terminal
docker run --name box2 -d busybox sleep 300
docker ps
docker exec -it box2 sh
# container terminal
exit
# host terminal
docker ps
docker run --name box3 -d busybox
docker ps
docker ps -a
docker stop box2
docker rm box1 box2 box3

The Entrypoint of a container is the init process and allows the container to run as an executable. Commands passed to a container are passed to the container's entrypoint process.

Note that docker commands after $IMAGE_NAME are passed to the container's entrypoint as arguments.
❌ docker run -it mysql -e MYSQL_PASSWORD=hello will pass -e MYSQL_PASSWORD=hello to the container
✔️ docker run -it -e MYSQL_PASSWORD=hello mysql

Lab 1.5. Container ports and IP

1. Run a nginx container with name webserver
2. Inspect the container (use | less to avoid console clutter) and review the State and NetworkSettings fields, quit with q
3. Visit http://$CONTAINER_IP_ADDRESS in your browser (this may not work depending on your envrionment network settings)
4. Run another nginx container with name webserver and exposed on port 80
5. Visit localhost in your browser
6. Delete the containers

# host terminal
docker run -d --name webserver nginx
docker inspect webserver | grep -A 13 '"State"' | less
docker inspect webserver | grep -A 50 '"NetworkSettings"' | less
curl http://$(docker inspect webserver --format "{{.NetworkSettings.IPAddress}}") | less
docker stop webserver
docker rm webserver
docker run -d --name webserver -p 80:80 nginx
curl localhost | less
docker ps
docker ps -a
docker stop webserver
docker rm webserver

Always run containers in detached mode to avoid getting stuck in the container STDOUT

Lab 1.6. Container volumes

1. Create an html/index.html file with some content
2. Run any webserver containers on port 8080 and mount the html folder to the DocumentRoot
   • option nginx DocumentRoot - /usr/share/nginx/html
   • option httpd DocumentRoot - /usr/local/apache2/htdocs
3. Visit localhost:8080
4. List running containers
5. List all containers
6. Delete containers

Lab 1.7. Container environment variables

1. Run a mysql container in detached mode
2. Connect to the container
3. Review the container logs and resolve the output message regarding environment variable
4. Confirm issue resolved by connecting to the container
5. Exit the container
6. List running containers
7. List all containers
8. List all images
9. List all volumes
10. Clean up with docker system prune
11. Check all resources are deleted, containers, images and volumes.

# host terminal
docker run -d --name db mysql
docker exec -it db bash # error not running
docker logs db
docker rm db
docker run -d --name db -e MYSQL_ROOT_PASSWORD=secret mysql
docker ps
docker ps -a
docker image ls
docker volume ls
docker stop db
docker ps # no containers running
docker system prune --all --volumes
docker image ls
docker volume ls

You don't always have to run a new container, we have had to do this to apply new configuration. You can restart an existing container docker ps -a, if it meets your needs, with docker start $CONTAINER

Lab 1.8. Container registries

Explore Docker Hub and search for images you've used so far or images/applications you use day-to-day, like databases, environment tools, etc.

Container images are created with instructions that determine the default container behaviour at runtime. A familiarity with specific images/applications may be required to understand their default behaviours

Lab 2.1. Working with images

1. List all images (if you've just finished lab1.7, run new container to download an image)
2. Inspect one of the images with | less and review the ContainerConfig and Config
3. View the image history
4. Tag the image with the repository localhost and a version
5. List all images
6. View the tagged image history
7. Delete tagged image by ID
8. Lets try that again, delete tagged image by tag

Lab 2.2. Custom images

Although, we can also create an image from a running container using docker commit, we will only focus on using a Dockerfile, which is the recommended method.

Build the below Dockerfile with docker build -t $IMAGE_NAME:$TAG /path/to/Dockerfile/directory, see `docker build --help

# Example Dockerfile
FROM ubuntu
MAINTAINER Piouson
RUN apt-get update && \
    apt-get install -y nmap iproute2 && \
    apt-get clean
ENTRYPOINT ["/usr/bin/nmap"]
CMD ["-sn", "172.17.0.0/16"] # nmap will scan docker network subnet `172.17.0.0/16` for running containers

Lab 2.3. Create image from Dockerfile

See best practices for writing Dockerfile.

# find a package containing an app (debian-based)
apt-file search --regex <filepath-pattern> # requires `apt-file` installation, see `apt-file --help`
apt-file search --regex ".*/sshd$"
# find a package containing an app, if app already installed (debian-based)
dpkg -S /path/to/file/or/pattern # see `dpkg --help`
dpkg -S */$APP_NAME
# find a package containing an app (rpm-based)
dnf provides /path/to/file/or/pattern
dnf provides */sshd

1. Create a Dockerfile based on the following:
   • Base image should be debian-based or rpm-based
   • Should include packages containing ps application and network utilities like ip, ss and arp
   • Should run the nmap process as the ENTRYPOINT with arguments -sn 172.17.0.0/16
2. Build the Dockerfile with repository local and version 1.0
3. List images
4. Run separate containers from the image as follows and review behaviour
   • do not specify any modes
   • in interactive mode with a shell
   • in detached mode, then check the logs
5. Edit the Dockerfile to run the same process and arguments but not as ENTRYPOINT
6. Repeat all three options in [4] and compare the behaviour
7. Clean up

# run ubuntu container to find debian-based packages
docker run -it --rm ubuntu
# container terminal
apt update
apt install -y apt-file
apt-file update
apt-file search --regex "bin/ip$"
apt-file search --regex "bin/ss$"
apt-file search --regex "bin/arp$"
# found `iproute2` and `net-tools`
exit

# alternatively, run fedora container to find rpm-based packages
docker run -it --rm fedora
# container terminal
dnf provides *bin/ip
dnf provides *bin/ss
dnf provides *bin/arp
# found `iproute` and `net-tools`
exit

# host terminal
mkdir test
nano test/Dockerfile

# Dockerfile
FROM alpine
RUN apk add --no-cache nmap iproute2 net-tools
ENTRYPOINT ["/usr/bin/nmap"]
CMD ["-sn", "172.17.0.0/16"]

# host terminal
docker build -t local/alpine:1.0 ./test
docker run --name alps1 local/alpine:1.0
docker run --name alps2 -it local/alpine:1.0 sh
docker run --name alps3 -d local/alpine:1.0
docker log alps3
nano test/Dockerfile

# Dockerfile
FROM alpine
RUN apk add --no-cache nmap iproute2 net-tools
CMD ["/usr/bin/nmap", "-sn", "172.17.0.0/16"]

# host terminal
docker build -t local/alpine:1.1 ./test
docker run --name alps4 local/alpine:1.0
docker run --name alps5 -it local/alpine:1.0 sh
# container terminal
exit
# host terminal
docker run --name alps6 -d local/alpine:1.0
docker log alps6
docker stop alps3 alps5 alps6
docker rm alps1 alps2 alps3 alps4 alps5 alps6
docker image rm local/alpine:1.0 local/alpine:1.1

In most cases, building an image goes beyond a successful build. Some installed packages require additional steps to run containers successfully

Lab 2.4. Containerise your application

See the official language-specific getting started guides which includes NodeJS, Python, Java and Go examples.

1. Bootstrap a frontend/backend application project, your choice of language
2. Install all dependencies and test the app works
3. Create a Dockerfile to containerise the project
4. Build the Dockerfile
5. Run a container from the image exposed on port 8080
6. Confirm you can access the app on localhost:8080

Lab 2.5. Container privileges

1. Display your system's current user
2. Display the current user's UID, GID and group memberships
3. Run a ubuntu container interactively, and in the container shell:
   • display the current user
   • display the current user's UID, GID and group memberships
   • list existing user accounts
   • list existing groups
   • create a file called test-file and display the file ownership info
   • exit the container
4. Run a new ubuntu container interactively with UID 1004, and in the container shell:
   • display the current user
   • display the current user's UID, GID and group memberships
   • exit the container
5. Create a docker image based on ubuntu with a non-root user as default user
6. Run a container interactively using the image, and in the container shell:
   • display the current user
   • display the current user's UID, GID and group memberships
   • exit the container
7. Delete created resources

# host terminal
whoami
id
docker run -it --rm ubuntu
# container terminal
whoami
id
cat /etc/passwd
cat /etc/group
touch test-file
ls -l
ls -ln
exit
# host terminal
docker run -it --rm --user 1004 ubuntu
# container terminal
whoami
id
exit

# test/Dockerfile
FROM ubuntu
RUN groupadd --gid 1000 piouson && useradd --uid 1000 piouson --gid 1000
USER piouson

# host terminal
docker build -t test-image test/
docker run -it --rm test-image
# container terminal
whoami
id
exit
# host terminal
docker image rm test-image

If a containerized application can run without privileges, change to a non-root user
It is recommended to explicitly specify GID/UID when creating a group/user

Task - Docker image

FROM nginx:1.22-alpine
EXPOSE 80

Using docker and the Dockerfile above, build an image with tag bootcamp/nginx:v1 and tag ckad/nginx:latest. Once complete, export a tar file of the image to /home/$USER/ckad-tasks/docker/nginx.tar.
Run a container named web-test from the image bootcamp/nginx:v1 accessible on port 2000, and another container named web-test2 from image ckad/nginx:latest accessible on port 2001. Leave both containers running.

What commands would you use to perform the above operations using podman? Specify these commands on separate lines in file /home/$USER/ckad-tasks/docker/podman-commands

Lab 3.1. Kubernetes in Google Cloud

A local lab setup is covered in chapter 4 with minikube
Skip this lab if you do not currently have a Google Cloud account with Billing enabled

• Signup and Login to console.cloud.google.com
• Use the "Cluster setup guide" to create "My first cluster"
• Connect to the cluster using the "Cloud Shell"
• View existing Kubernetes resources by running kubectl get all

Lab 3.2. Explore Kubernetes API resources via Google Cloud

This lab is repeated in chapter 4 with minikube
Skip this lab if you do not currently have a Google Cloud account with Billing enabled

1. Create an nginx application with three replicas
2. View available resources
3. Delete a pod create
4. View available resources, how many pods left, can you find the deleted pod?
5. List supported API resources
6. Delete the application
7. view available resource
8. Delete the Kubernetes service
9. view available resources
10. If nothing found, allow 5s and try [9] again

kubectl create deploy webserver --image=nginx --replicas=3
kubectl get all
kubectl delete pod $POD_NAME
kubectl get all # new pod auto created to replace deleted
kubectl api-resources
kubectl delete deploy webserver
kubectl get all
kubectl delete svc kubernetes
kubectl get all # new kubernetes service is auto created to replace deleted

Remember to delete Google cloud cluster to avoid charges if you wish to use a local environment detailed in the next chapter

Lab 4.1. Setup minikube and kubectl

1. Confirm minikube running minikube status
2. Create kubectl alias in .bashrc

   printf "
   # minikube kubectl
   alias kubectl='minikube kubectl --'
   " >> ~/.bashrc
   exec bash
   

3. Start using the alias

   kubectl version
   kubectl get all
   

4. Enable kubectl autocompletion, see kubectl completion --help

   echo "source <(kubectl completion bash)" >> ~/.bashrc # macos replace bash with zsh
   exec bash
   

5. The default kubectl edit text editor is vi. To change this:

   export KUBE_EDITOR="nano" # use nano
   export KUBE_EDITOR="vim" # use vim
   

6. Open the Kubernetes dashboard with minikube dashboard
7. Use the Kubernetes Dashboard to deploy a webserver with three replicas
   • visit url provided in browser
   • click on top right plus "+" icon
   • select Create from form
   • enter App name: app, Container image: nginx, Number of pods: 3
   • click Deploy
8. Return to the terminal and delete created resources

   ctrl+c # to terminate dashboard
   kubectl get all
   kubectl delete deploy app
   

9. List Kubernetes clusters with kubectl config get-contexts
10. If you have Kubernetes cluster from both Minikube and Docker Desktop, you can switch between them:

• Set Docker Desktop cluster as current cluster: kubectl config set-context docker-desktop
• Set Minikube cluster as current cluster: kubectl config set-context minikube

Lab 4.2. Explore Kubernetes API resources via Minikube

1. Run an nginx Pod
2. View resources
3. Delete the Pod
4. View resources
5. Repeat Lab 3.2 in Minikube

kubectl run webserver --image=nginx
kubectl get all
kubectl delete pod webserver
kubectl get all # pod gone
# see `lab3.2 solution` for remaining steps

Pods started without a deployment are called Naked Pods - these are not managed by a replicaset, therefore, are not rescheduled on failure, not eligible for rolling updates, cannot be scaled, cannot be replaced automatically.
Although, Naked Pods are not recommended in live environments, they are crucial for learning how to manage Pods, which is a big part of CKAD.

Lab 5.1. Creating Pods

1. Create a Pod with nginx:alpine image and confirm creation
2. Review full details of the Pod in YAML form
3. Display details of the Pod in readable form and review the Node, IP, container start date/time and Events
4. List pods but only show resource names
5. Connect a shell to the Pod and confirm an application is exposed
   • By default, Nginx exposes applications on port 80
   • confirm exposed ports
6. Delete the Pod
7. Review the Pod spec
8. Have a look at the Kubernetes API to determine when pods were introduced

Not all images expose their applications on port 80. Kubernetes doesn't have a native way to check ports exposed on running container, however, you can connect a shell to a Pod with kubectl exec and try one of netstat -tulpn or ss -tulpn in the container, if installed, to show open ports.

Lab 5.2. Managing Pods with YAML file

In the official k8s docs, you will often find example code with a URL, e.g. pods/commands.yaml. The file can be downloaded by appending https://k8s.io/examples to the URL, thus: https://k8s.io/examples/pods/commands.yaml

# download file `pods/commands.yaml`
wget https://k8s.io/examples/pods/commands.yaml
# save downloaded file with a new name `comm.yaml`
wget https://k8s.io/examples/pods/commands.yaml -O comm.yaml
# hide output while downloading
wget -q https://k8s.io/examples/pods/commands.yaml
# view contents of a downloaded file without saving
wget -O- https://k8s.io/examples/pods/commands.yaml
# view contents quietly without saving
wget -qO- https://k8s.io/examples/pods/commands.yaml

1. Generate a YAML file of a busybox Pod that runs the command sleep 60, see create Pod with command and args docs
2. Apply the YAML file.
3. List created resources
4. View details of the Pod
5. Delete the Pod

kubectl run mypod --image=busybox --dry-run=client -o yaml --command -- sleep 60 > lab5-2.yaml
kubectl apply -f lab5-2.yaml
kubectl get pods
kubectl describe pods mypod | less
kubectl delete -f lab5-2.yaml

Some images, like busybox, do not remain in running state by default. An extra command is required, e.g. sleep 60, to keep containers using these images in running state for as long as you need. In the CKAD exam, make sure your Pods remain in running states unless stated otherwise

Lab 5.3. Init Containers

Note that the main container will only be started after the init container enters STATUS=completed

# view logs of pod `mypod`
kubectl logs mypod
# view logs of specific container `mypod-container-1` in pod `mypod`
kubectl logs mypod -c mypod-container-1

1. Create a Pod that logs App is running! to STDOUT
   • use busybox:1.28 image
   • the application should Never restart
   • the application should use a Init Container to wait for 60secs before starting
   • the Init Container should log App is initialising... to STDOUT
   • see init container docs.
2. List created resources and note Pod STATUS
3. View the logs of the main container
4. View the logs of the init container
5. View more details of the Pod and note the State of both containers.
6. List created resources and confirm Pod STATUS
7. Delete Pod

Lab 5.4. Multi-container Pod

1. Create a Pod with 2 containers and a volumne shared by both containers, see multi-container docs.
2. List created resources
3. View details of the Pod
4. Delete the Pod

# lab5-4.yaml
apiVersion: v1
kind: Pod
metadata:
  name: myapp
spec:
  containers:
    - name: myapp-1
      image: busybox:1.28
      volumeMounts:
        - name: logs
          mountPath: /var/log
    - name: myapp-2
      image: busybox:1.28
      volumeMounts:
        - name: logs
          mountPath: /var/log
  volumes:
    - name: logs
      emptyDir: {}

kubectl apply -f lab5-4.yaml
kubectl get pods
kubectl describe pods myapp | less
kubectl logs myapp -c myapp-1
kubectl logs myapp -c myapp-2
kubectl delete -f lab5-4.yaml

Always create single container Pods!

Lab 5.5. Sidecar Containers

Remember you can prepend https://k8s.io/examples/ to any example manifest names from the official docs for direct download of the YAML file

1. Create a busybox Pod that logs date to a file every second
   • expose the logs with a sidecar container's STDOUT to prevent direct access to the main application
   • see example sidecar container manifest https://k8s.io/examples/admin/logging/two-files-counter-pod-streaming-sidecar.yaml
2. List created resources
3. View details of the Pod
4. View the logs of the main container
5. View the logs of the sidecar container
6. Delete created resources

Lab 5.6 Namespaces

You can also follow the admin guide doc for namespaces

Remember you can connect a shell to a Pod with kubectl exec and try one of netstat -tulpn or ss -tulpn in the container, if installed, to show open ports.

1. Create a Namespace myns
2. Create a webserver Pod in the myns Namespace
3. Review created resources and confirm myns Namespace is assigned to the Pod
4. Delete resources created
5. Review the NAMESPACED column of the Kubernetes API resources
6. Review the Namespace object and the Namespace spec

kubectl create ns myns --dry-run=client -o yaml > lab5-6.yaml
echo --- >> lab5-6.yaml
kubectl run mypod --image=httpd:alpine -n myns --dry-run=client -o yaml >> lab5-6.yaml
kubectl apply -f lab5-6.yaml
kubectl get pods
kubectl describe -f lab5-6.yaml | less
kubectl delete -f lab5-6.yaml
kubectl api-resources | less
kubectl explain namespace | less
kubectl explain namespace --recursive | less
kubectl explain namespace.spec | less

Remember that namespaced resources are not visible by default unless the namespace is specified
💡 kubectl get pods - only shows resources in the default namespace
💡 kubectl get pods -n mynamespace - shows resources in the mynamespace namespace

Task - Pods

Imagine a student in the CKAD Bootcamp training reached out to you for assistance to finish their homework. Their task was to create a webserver with a sidecar container for logging in the cow namespace. Find this Pod, which could be located in one of the Namespaces ape, cow or fox, and ensure it is configured as required.

At the end of your task, copy the log file used by the logging container to directory /home/$USER/ckad-tasks/pods/

• Command to setup environment:

  printf '\nlab: environment setup in progress...\n'; echo '{"apiVersion":"v1","items":[{"kind":"Namespace","apiVersion":"v1","metadata":{"name":"fox"}},{"kind":"Namespace","apiVersion":"v1","metadata":{"name":"ape"}},{"kind":"Namespace","apiVersion":"v1","metadata":{"name":"cow"}},{"apiVersion":"v1","kind":"Pod","metadata":{"labels":{"run":"box"},"name":"box","namespace":"ape"},"spec":{"containers":[{"args":["sleep","3600"],"image":"busybox","name":"box"}],"dnsPolicy":"ClusterFirst","restartPolicy":"Always"}},{"apiVersion":"v1","kind":"Pod","metadata":{"labels":{"run":"for-testing"},"name":"for-testing","namespace":"fox"},"spec":{"containers":[{"args":["sleep","3600"],"image":"busybox","name":"for-testing"}],"dnsPolicy":"ClusterFirst","restartPolicy":"Always"}},{"apiVersion":"v1","kind":"Pod","metadata":{"labels":{"run":"webserver"},"name":"webserver","namespace":"fox"},"spec":{"containers":[{"name":"server","image":"ngnx:1.20-alpine","volumeMounts":[{"name":"serverlog","mountPath":"/usr/share/nginx/html"}]},{"name":"logger","image":"busybox:1.28","args":["/bin/sh","-c","while true; do  echo $(date) >> /usr/share/nginx/html/1.log;\n  sleep 30;\ndone\n"],"volumeMounts":[{"name":"serverlog","mountPath":"/usr/share/nginx/html"}]}],"volumes":[{"name":"serverlog","emptyDir":{}}]}}],"metadata":{"resourceVersion":""},"kind":"List"}' | kubectl apply -f - >/dev/null; echo 'lab: environment setup complete!'
  

• Command to destroy environment:

  kubectl delete ns ape cow fox
  

Task - Pods II

In the rat Namespace (create if required), create a Pod named webapp that runs nginx:1.22-alpine image and has env-var NGINX_PORT=3005 which determines the port exposed by the container. The Pod container should be named web and should mount an emptyDir volume to /etc/nginx/templates.
The Pod should have an Init Container named web-init, running busybox:1.28 image, that creates a file in the same emptyDir volume, mounted to /tempdir, with below command:

echo -e "server {\n\tlisten\t\${NGINX_PORT};\n\n\tlocation / {\n\t\troot\t/usr/share/nginx/html;\n\t}\n}" > /tempdir/default.conf.template

Lab 6.1 Troubleshoot failing Pod

1. Create a Pod with mysql image and confirm Pod state
2. Get detailed information on the Pod and review Events (any multiple attempts?), 'State', 'Last State' and their Exit codes.
   • Note that Pod STATES might continue to change for containers in error due to default restartPolicy=Always
3. Review cluster logs for the Pod
4. Apply relevant fixes until you have a mysql Pod in 'Running' state
5. Delete created resources

Lab 6.2. Use port forwarding to access applications

1. Create a webserver Pod
2. List created resources and determine Pod IP address
3. Access the webserver with the IP address (you can use curl)
4. Use port forwarding to access the webserver on http://localhost:5000
5. Terminate port forwarding and delete created resources

Lab 6.3. Set Pod security context

Using the official docs manifest example pods/security/security-context.yaml as base to:

1. Use the official manifest example pods/security/security-context.yaml as base to create a Pod manifest with these security context options:
   • all containers have a logged-in user of UID: 1010, GID: 1020
   • all containers set to run as non-root user
   • mounted volumes for all containers in the pod have group GID: 1110
   • escalating to root privileges is disabled (more on privilege escalation)
2. Apply the manifest file and review details of created pod
3. Review pod details and confirm security context applied at pod-level and container-level
4. Connect an interactive shell to a container in the pod and confirm the following:
   • current user
   • group membership of current user
   • ownership of entrypoint process
   • ownership of the mounted volume /data/demo
   • create a new file /data/demo/new-file and confirm file ownership
   • escalate to a shell with root privileges sudo su
5. Edit the pod manifest file to the following:
   • do not set logged-in user UID/GID
   • do not set root privilege escalation
   • all containers set to run as non-root user
6. Create a new pod with updated manifest
7. Review pod details and confirm events and behaviour
   • what were your findings?
8. Delete created resources
9. Explore the Pod spec and compare the securityContext options available at pod-level vs container-level

Lab 6.4. Working with Jobs

1. Create a Job myjob1 with a suitable image that runs the command echo Lab 6.4. Jobs!
2. List jobs and pods
3. Review the details of myjob1
4. Review the yaml form of myjob1
5. Create another Job myjob2 with a suitable image that runs the command date
6. List jobs and pods
7. Repeat [4] using a manifest file with name myjob3
8. List jobs and pods
9. Delete all jobs created
10. List jobs and pods
11. Edit the manifest file and add the following:
    • 5 pods successfully run the command
    • pods are auto deleted after 30secs
12. Apply the new manifest and:
    • confirm the new changes work as expected
    • note the total number of resources created
    • note the behaviour after 30secs
13. Delete created resources
14. Review the Job spec to understand fields related to working with jobs
15. Review the Kubernetes API Resources to determine when jobs was introduced

Lab 6.5. Working with CronJobs

1. Create a job with a suitable image that runs the date command every minute
2. Review details of the created CronJob
3. Review the YAML form of the created CronJob
4. List created resources and compare results before and after 1 minute
5. Delete created resources
6. Review the CronJob spec to understand fields related to working with cronjobs
7. Review the Job spec of a CronJob and compare this to a standard Job spec
8. Review the Kubernetes API Resources to determine when jobs was introduced

kubectl explain cronjob.spec | less
kubectl explain cronjob.spec.jobTemplate.spec | less
kubectl create cronjob mycj --image=busybox --schedule="* * * * *" -- date
kubectl describe cj mycj | less
kubectl get cj mycj -o yaml | less
kubectl get all
kubectl get pods --watch # watch pods for 60s to see changes
kubectl delete cj mycj # deletes associated jobs and pods!
kubectl api-resources # cronjobs was introduced in batch/v1

All CronJob schedule times are based on the timezone of the kube-controller-manager
Since a CronJob runs a Job periodically, the Job spec auto delete feature ttlSecondsAfterFinished is quite handy

Lab 6.6. Resource limitation

You may use the official container resource example manifest or generate a manifest file with kubectl set resources.

1. Create a Pod with the following spec:
   • runs in dev namespace
   • runs two containers, MongoDB database and webserver frontend
   • restart only on failure, see pod.spec.restartPolicy
   • both containers starts with 0.25 CPU, 64 mebibytes RAM
   • both containers does not exceed 1 CPU, 256 mebibytes RAM
2. List created pods
3. Review pod details and confirm the specified resource quotas are applied
4. Edit the Pod manifest as follows:
   • both containers starts with an insufficient amount RAM, e.g 4 mebibytes
   • both containers does not exceed 8 mebibytes RAM
5. Apply the manifest and review behaviour
6. Review logs for both containers
7. Compare the logs output in [6] to details from kubectl describe
8. Edit the Pod manifest as follows:
   • both containers starts with an amount of RAM equal to host RAM (run cat /proc/meminfo or free -h)
   • both containers starts with an amount CPU equal to host CPU (run cat /proc/cpuinfo or lscpu)
   • both containers does not exceed x2 the amount of host RAM
9. Apply the manifest and review behaviour
10. Delete created resources
11. Review the Pod spec fields related to limits and requests

kubectl create ns dev --dry-run=client -o yaml >> lab6-6.yaml
echo --- >> lab6-6.yaml
# add the contents of the example manifest to lab6-6.yaml and modify accordingly
nano lab6-6.yaml

# lab6-6.yaml
kind: Namespace
metadata:
  name: dev
# etc
---
kind: Pod
metadata:
  name: webapp
  namespace: dev
spec:
  restartPolicy: OnFailure
  containers:
    - image: mongo
      name: database
      resources:
        requests:
          memory: "64Mi"
          cpu: "250m"
        limits:
          memory: "256Mi"
          cpu: 1
    - image: nginx
      name: frontend
      resources: # same as above
# etc

kubectl apply -f lab6-6.yaml
kubectl get pods -n dev
kubectl describe pods webapp -n dev | less
kubectl describe pods webapp -n dev | grep -A 4 -E "Containers:|State:|Limits:|Requests:" | less
nano lab6-6.yaml

# lab6-6.yaml
kind: Pod
spec:
  containers:
    - resources:
        requests:
          memory: "4Mi"
          cpu: "250m"
        limits:
          memory: "8Mi"
          cpu: 1
# etc - use above resources for both containers

kubectl delete -f lab6-6.yaml
kubectl apply -f lab6-6.yaml
kubectl get pods -n dev --watch # watch for OOMKilled | CrashLoopBackOff
kubectl get logs webapp -n dev -c database # not very helpful logs
kubectl get logs webapp -n dev -c frontend
kubectl describe pods webapp -n dev | less # helpful logs - Last State: Terminated, Reason: OutOfMemory (OOMKilled)
kubectl describe pods webapp -n dev | grep -A 4 -E "Containers:|State:|Limits:|Requests:" | less
cat /proc/cpuinfo # check for host memory
cat /proc/meminfo # check for host ram
nano lab6-6.yaml

# lab6-6.yaml
kind: Pod
spec:
  containers:
    - resources:
        requests:
          memory: "8Gi" # use value from `cat /proc/meminfo`
          cpu: 2 # use value from `cat /proc/cpuinfo`
        limits:
          memory: "16Gi"
          cpu: 4
# etc - use above resources for both containers

kubectl delete -f lab6-6.yaml
kubectl apply -f lab6-6.yaml
kubectl get pods -n dev --watch # remains in Pending until enough resources available
kubectl describe pods webapp
kubectl delete -f lab6-6.yaml
kubectl explain pod.spec.containers.resources | less

Remember a multi-container Pod is not recommended in live environments but only used here for learning purposes

Lab 6.7. Resource allocation and usage

This lab requires a Metrics Server running in your cluster, please run minikube addons enable metrics-server to enable Metrics calculation.

# enable metrics-server on minikube
minikube addons enable metrics-server
# list available nodes
kubectl get nodes
# view allocated resources for node and % resource usage for running (non-terminated) pods
kubectl describe node $NODE_NAME
# view nodes resource usage
kubectl top node
# view pods resource uage
kubectl top pod

1. Enable Metrics Server in your cluster
2. What is the cluster Node's minimum required CPU and memory?
3. Create a Pod as follows:
   • image nginx:alpine
   • does not restart, see kubectl explain pod.spec
   • only pulls a new image if not present locally, see kubectl explain pod.spec.containers
   • requires 0.2 CPU to start but does not exceed half of the cluster Node's CPU
   • requires 64Mi memory to start but does not exceed half of the cluster Node's memory
4. Review the running Pod and confirm resources configured as expected
5. Delete created resources

Task - CronJobs

In the boa Namespace, create a Pod that runs the shell command date, in a busybox container, once every hour, regardless success or failure. Job should terminate after 20s even if command still running. Jobs should be automatically deleted after 12 hours. A record of 5 successful Jobs and 5 failed Jobs should be kept. All resources should be named bootcamp, including the container. You may create a new Namespace if required.

At the end of your task, to avoid waiting an hour to confirm all works, manually run the Job from the Cronjob and verify expected outcome.

Task - Resources and Security Context

A client requires a Pod running the nginx:1.21-alpine image with name webapp in the dog Namespace. The Pod should start with 0.25 CPU and 128Mi memory, but shouldn't exceed 0.5 CPU and half of the Node's memory. All processes in Pod containers should run with user ID 1002 and group ID 1003. Containers mustn't run in privileged mode and privilege escalation should be disabled. You may create a new Namespace if required.

When you are finished with the task, the client would also like to know the Pod with the highest memory consumption in the default Namespace. Save the name the Pod in the format <namespace>/<pod-name> to a file /home/$USER/ckad-tasks/resources/pod-with-highest-memory

Lab 7.1. Deploy an app with a replicaset

Deployments can be used to rollout a ReplicaSet which manages the number of Pods. In CKAD you will only work with ReplicaSets via Deployments

1. Create a deployment with three replicas using a suitable image
2. Show more details of the deployment and review available fields:
   • namespace, labels, selector, replicas, update strategy type, pod template, conditions, replicaset and events
3. List all created resources
4. Delete a Pod and monitor results
5. Compare results to using naked Pods (run a pod and delete it)
6. Delete the ReplicaSet with kubectl delete rs $rsName and monitor results
7. Delete created resources
8. Explore the deployment spec
9. Explore the Kubernetes API Resources to determine when deployments and replicasets was introduced

Lab 7.2. Scale deployment

A deployment creates a ReplicaSet that manages scalability. Do not manage replicasets outside of deployments.

1. Create a deployment using the official deployment manifest example controllers/nginx-deployment.yaml
2. List created resources
3. Edit the deployment with kubectl edit and change the namespace to dev
4. Save the editor and confirm behaviour
5. Edit the deployment again using a different editor, change the replicas to 12 and upgrade the image version
6. Save the editor and confirm behaviour, then immediately list all resources and review:
   • deployment status for READY, UP-TO-DATE and AVAILABLE
   • replicaset status for DESIRED, CURRENT and READY
   • pod status for NAME, READY and STATUS
   • compare the ID-suffix in the Pods name to the ReplicaSets name
7. View details of deployment to confirm edit applied, including image change
8. Scale down the deployment back to 3 replicas using kubectl scale and review same in [6]
9. List all resources and confirm scaling applied
10. Delete created resources
11. Edit the apiVersion of the manifest example file to apps/v0
12. Apply the edited manifest and confirm behaviour

Lab 7.3. Working with labels

1. Create a deployment myapp with three replicas using a suitable image
2. List all deployments and their labels to confirm default labels assigned
3. Add a new label pipeline: test to the deployment
4. List all deployments and their labels
5. View more details of the deployment and review labels/selectors
6. View the YAML form of the deployment to see how labels are added in the manifest
7. Verify the default label/selector assigned when you created a new Pod
8. List all resources and their labels filtered by default label of the deployment
9. List all resources and their labels, filtered by new label added, compare with above
10. Remove the default label from one of the pods in the deployment and review behaviour
11. List all pods and their labels
12. List all pods filtered by the default label
13. Delete the deployment
14. Delete the naked Pod from [10]

Lab 7.4. Rolling updates

1. Review the update strategy field under the deployment spec
2. Create a deployment with a suitable image
3. View more details of the deployment
   • by default, how many pods can be upgraded simultaneously during update?
   • by default, how many pods can be created in addition to the number of replicas during update?
4. Create a new deployment with the following parameters:
   • 5 replicas
   • image nginx:1.18
   • additional deployment label update: feature
   • maximum of 2 Pods can be updated simultaneously
   • no more than 3 additional Pod created during updates
5. List all resources filtered by the default label
6. List all resources filtered by the additional label
7. List rollout history for all deployments - how many revisions does the new deployment have?
8. Upgrade/downgrade the image version
9. List all resources specific to the new deployment
10. List rollout history specific to the new deployment - how many revisions?
11. View more details of the deployment and note the image and Events messages
12. Compare the latest change revision of the new deployment's rollout history to the previous revision
13. Revert the new deployment to its previous revision
14. List all resources specific to the new deployment twice or more to track changes
15. List rollout history specific to the new deployment - any new revisions?
16. Scale the new deployment to 0 Pods
17. List rollout history specific to the new deployment - any new revisions?
18. List all resources specific to the new deployment
19. Delete created resources

Lab 7.5. Exploring DaemonSets

DaemonSets can only be created by YAML file, see an official example manifest controllers/daemonset.yaml.

1. Compare the DaemonSet manifest to a Deployment manifest - differences/similarities?
2. Apply the example manifest
3. List all resources and note resources created by the DaemonSet
4. View more details of the DaemonSet
5. Delete created resources
6. Review the Kubernetes API Resources to determine when DaemonSets was introduced
7. List existing DaemonSets in the kube-system namespace and their labels
   • what does Kubernetes use a DaemonSet for?
8. List all resources in the kube-system namespace matching the DaemonSet label
9. Review the DaemonSet spec

Lab 7.6. Resource usage and Autoscaling

Autoscaling is very important in live environments but not covered in CKAD. Visit HorizontalPodAutoscaler Walkthrough for a complete lab on autoscaling.

The lab requires a metrics-server so install one via Minikube if you plan to complete the lab

# list minikube addons
minikube addons list
# enable minikube metrics-server
minikube addons enable metrics-server
# disable minikube metrics-server
minikube addons disable metrics-server

Task - Deployment

Some bootcamp students have been messing with the webapp Deployment for the test environment's webpage in the default Namespace, leaving it broken. Please rollback the Deployment to the last fully functional version. Once on the fully functional version, update the Deployment to have a total of 10 Pods, and ensure that the total number of old and new Pods, during a rolling update, do not exceed 13 or go below 7.

Update the Deployment to nginx:1.22-alpine to confirm the Pod count stays within these thresholds. Then rollback the Deployment to the fully functional version. Before you leave, set the Replicas to 4, and just to be safe, Annotate all the Pods with description="Bootcamp Test Env - Please Do Not Change Image!".

• Command to setup environment:

  printf '\nlab: environment setup in progress...\n'; echo '{"apiVersion":"apps/v1","kind":"Deployment","metadata":{"labels":{"appid":"webapp"},"name":"webapp"},"spec":{"replicas":2,"revisionHistoryLimit":15,"selector":{"matchLabels":{"appid":"webapp"}},"template":{"metadata":{"labels":{"appid":"webapp"}},"spec":{"volumes":[{"name":"varlog","emptyDir":{}}],"containers":[{"image":"nginx:1.12-alpine","name":"nginx","volumeMounts":[{"name":"varlog","mountPath":"/var/logs"}]}]}}}}' > k8s-task-6.yml; kubectl apply -f k8s-task-6.yml >/dev/null; cp k8s-task-6.yml k8s-task-6-bak.yml; sed -i -e 's/nginx:1.12-alpine/nginx:1.13alpine/g' k8s-task-6.yml 2>/dev/null; sed -i '' 's/nginx:1.12-alpine/nginx:1.13alpine/g' k8s-task-6.yml 2>/dev/null; kubectl apply -f k8s-task-6.yml >/dev/null; sleep 1; sed -i -e 's/nginx:1.13alpine/nginx:1.14-alpine/g' k8s-task-6.yml 2>/dev/null; sed -i '' 's/nginx:1.13alpine/nginx:1.14-alpine/g' k8s-task-6.yml 2>/dev/null; kubectl apply -f k8s-task-6.yml >/dev/null; sleep 4; sed -i -e 's/nginx:1.14-alpine/nginx:1.15-alpine/g' k8s-task-6.yml 2>/dev/null; sed -i -e 's/\/var\/logs/\/usr\/share\/nginx\/html/g' k8s-task-6.yml 2>/dev/null; sed -i '' 's/nginx:1.14-alpine/nginx:1.15-alpine/g' k8s-task-6.yml 2>/dev/null; sed -i '' 's/\/var\/logs/\/usr\/share\/nginx\/html/g' k8s-task-6.yml 2>/dev/null; kubectl apply -f k8s-task-6.yml >/dev/null; sleep 2; sed -i -e 's/nginx:1.15-alpine/ngnx:1.16-alpine/g' k8s-task-6.yml 2>/dev/null; sed -i -e 's/\/var\/logs/\/usr\/share\/nginx\/html/g' k8s-task-6.yml 2>/dev/null; sed -i '' 's/nginx:1.15-alpine/ngnx:1.16-alpine/g' k8s-task-6.yml 2>/dev/null; sed -i '' 's/\/var\/logs/\/usr\/share\/nginx\/html/g' k8s-task-6.yml 2>/dev/null; kubectl apply -f k8s-task-6.yml >/dev/null; sleep 4; kubectl apply -f k8s-task-6-bak.yml >/dev/null; sleep 4; kubectl rollout undo deploy webapp --to-revision=5 >/dev/null; kubectl delete $(kubectl get rs --sort-by=".spec.replicas" -oname | tail -n1) >/dev/null; rm k8s-task-6.yml k8s-task-6-bak.yml; echo 'lab: environment setup complete!'
  

• Command to destroy environment:

  kubectl delete deploy webapp
  

Lab 8.1. Connecting applications with services

1. Create a simple deployment with name webserver
2. List created resources
3. List endpoints, and pods with their IPs
   • Can you spot the relationship between the Service, Endpoints and Pods?
4. Create a Service for the deployment, exposed on port 80
5. List created resources and note services fields TYPE, CLUSTER-IP, EXTERNAL-IP and PORT(S)
6. View more details of the Service and note fields IPs, Port, TargetPort and Endpoints
7. View the YAML form of the Service and compare info shown with output in [6]
8. Print the Service env-vars from one of the pods
9. Scale the deployment down to 0 replicas first, then scale up to 2 replicas
10. List all pods and their IPs
11. Print the Service env-vars from one of the pods and compare to results in [3]
12. List endpoints, and pods with their IPs
13. Access the app by the Service: curl $ClusterIP:$Port
14. Access the app by the Service from the container host: minikube ssh then curl $ClusterIP:$Port
15. Run a busybox Pod with a shell connected interactively and perform the following commands:
    • run cat /etc/resolv.conf and review the output
    • run nslookup webserver (service name) and review the output
    • what IPs and/or qualified names do these match?
16. Run a temporary nginx:alpine Pod to query the Service by name:
    • first run kubectl run mypod --rm -it --image=nginx:alpine -- sh
    • then once in container, run curl $SERVICE_NAME:$PORT
    • you should run curl $SERVICE_NAME.$SERVICE_NAMESPACE:$PORT if the Service and the temporary Pod are in separate Namespaces
17. Delete created resources
18. Explore the Service object and the Service spec

Lab 8.2. Connect a frontend to a backend using services

In this lab, we will implement a naive example of a backend-frontend microservices architecture - expose frontend to external traffic with NodePort Service while keeping backend hidden with ClusterIP Service.

Note that live environments typically use Ingress (covered in the next chapter) to expose applications to external traffic

1. Create a simple Deployment, as our backend app, with the following spec:
   • image httpd (for simplicity)
   • name backend
   • has Labels app: backend and tier: webapp
   • has Selectors app: backend and tier: webapp
2. Create a Service for the backend app with the following spec:
   • type ClusterIP
   • port 80
   • same name, Labels and Selectors as backend Deployment
3. Confirm you can access the app by $CLUSTER-IP or $SERVICE_NAME
4. Configure an nginx upstream in nginx/default.conf to redirect traffic for the / route to the backend service

   # nginx/default.conf
   upstream backend-server {
       server backend; # dns service discovery within the same namespace use service name
   }
   
   server {
       listen 80;
   
       location / {
           proxy_pass http://backend-server;
       }
   }
   

5. Create a simple Deployment, as our frontend app, with the following spec:
   • image nginx
   • name frontend
   • has Labels app: webapp and tier: frontend
   • has Selectors app: webapp and tier: frontend
   • Remember that Services target Pods by Selector (the Label Selector of the Service must match the Label of the Pod)
   • mounts the nginx config file to /etc/nginx/conf.d/default.conf (use fullpath $(pwd)/nginx/default.conf)
   • see example hostPath volume mount manifest
6. Create a Service for the frontend app with the following spec:
   • type NodePort
   • port 80
   • same name, Labels and Selectors as frontend Deployment
   • Remember that Services target Pods by Selector
7. Confirm you can access the backend app from the Minikube Node $(minikube ip):NodePort
8. Delete created resources

Task - Service

Create a Pod named webapp in the pig Namespace (create new if required), running nginx:1.20-alpine image. The Pod should have a Annotation motd="Welcome to Piouson's CKAD Bootcamp". Expose the Pod on port 8080.

Task - Service II

A bootcamp student is stuck on a simple task and would appreciate your expertise. Their goal is to create a webapp Deployment running gcr.io/google-samples/node-hello:1.0 image in the bat Namespace, exposed on port 80 and NodePort 32500. The student claims everything was setup as explained in class but still unable to access the application via the Service. Swoop down like a superhero and save the day by picking up where the student left off.

• Command to setup environment:

  printf '\nlab: environment setup in progress...\n'; echo '{"apiVersion":"v1","kind":"List","items":[{"apiVersion":"v1","kind":"Namespace","metadata":{"name":"bat"}},{"apiVersion":"apps/v1","kind":"Deployment","metadata":{"labels":{"appid":"webapp"},"name":"webapp","namespace":"bat"},"spec":{"replicas":2,"selector":{"matchLabels":{"appid":"webapp"}},"template":{"metadata":{"labels":{"appid":"webapp"}},"spec":{"containers":[{"image":"gcr.io/google-samples/node-hello:1.0","name":"nginx"}]}}}},{"apiVersion":"v1","kind":"Service","metadata":{"labels":{"appid":"webapp"},"name":"webapp","namespace":"bat"},"spec":{"ports":[{"port":80,"protocol":"TCP","targetPort":80}],"selector":{"app":"webapp"}}}]}' | kubectl apply -f - >/dev/null; echo 'lab: environment setup complete!'
  

• Command to destroy environment

  kubectl delete ns bat
  

Lab 9.1 Enable Ingress

1. List existing namespaces
2. Enable Ingress on minikube
3. List Namespaces and confirm new Ingress Namespace added
4. List all resources in the Ingress Namespace, including the ingressclass
5. Review the ingress-nginx-controller Service in YAML form, note service type and ports
6. Review the ingressclass in YAML form, is this marked as the default?
7. Review the Ingress spec

Lab 9.2 Understanding ingress

1. Create a Deployment called web using a httpd image
2. Expose the deployment with a Cluster-IP Service called web-svc
3. Create an Ingress called web-ing with a Prefix rule to redirect / requests to the Service
4. List all created resources - what is the value of Ingress CLASS, HOSTS & ADDRESS?
   • think about why the CLASS and HOSTS have such values..
5. Access the app web via ingress curl $(minikube ip)
   • note that unlike Service, a NodePort isn't specified
6. What if we want another application on /test path, will this work? Repeat steps 3-7 to confirm:
   • create a new deployment web2 with image httpd
   • expose the new deployment web2-svc
   • add new Prefix path to existing ingress rule to redirect /test to web2-svc
   • are you able to access the new web2 app via curl $(minikube ip)/test?
   • are you still able to access the old web app via curl $(minikube ip)?
   • what's missing?
7. Let's fix this by adding the correct Annotation to the Ingress config, kubectl edit ingress web-ing:
   fix ingress

   metadata:
     name: web-ing
     annotations:
       nginx.ingress.kubernetes.io/rewrite-target: /
   

8. Try to access both apps via URLs curl $(minikube ip)/test and curl $(minikube ip)
9. Can you access both apps using HTTPS?
10. Review the ingress-nginx-controller by running: kubectl get svc -n ingress-nginx
    • what is the ingress-nginx-controller Service type?
    • what are the ports related to HTTP 80 and HTTPS 443?
11. Can you access both apps via the ingress-nginx-controller NodePorts for HTTP and HTTPS?
12. Delete all created resources

kubectl create deploy web --image=httpd --dry-run=client -oyaml > lab9-2.yml
kubectl apply -f lab9-2.yml
echo --- >> lab9-2.yml
kubectl expose deploy web --name=web-svc --port=80 --dry-run=client -oyaml >> lab9-2.yml
echo --- >> lab9-2.yml
kubectl create ingress web-ing --rule="/*=web-svc:80" --dry-run=client -oyaml >> lab9-2.yml
kubectl apply -f lab9-2.yml
kubectl get deploy,po,svc,ing,ingressclass # CLASS=nginx, HOSTS=*, ADDRESS starts empty then populated later
curl $(minikube ip) # it works
echo --- >> lab9-2.yml
kubectl create deploy web2 --image=httpd --dry-run=client -oyaml > lab9-2.yml
kubectl apply -f lab9-2.yml
echo --- >> lab9-2.yml
kubectl expose deploy web2 --name=web2-svc --port=80 --dry-run=client -oyaml >> lab9-2.yml
KUBE_EDITOR=nano kubectl edit ingress web-ing

Kind: Ingress
spec:
  rules:
  - http:
      paths:
      - path: /
        pathType: Prefix
        ...
      - path: /test
        pathType: Prefix
        backend:
          service:
            name: web2-svc
            port:
              number: 80
# etc

curl $(minikube ip)/test # 404 not found ???
curl $(minikube ip) # it works
KUBE_EDITOR=nano kubectl edit ingress web-ing

Kind: Ingress
metadata:
  name: web-ing
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
# etc

curl $(minikube ip)/test # it works
curl $(minikube ip) # it works
curl https://$(minikube ip)/test --insecure # it works, see `curl --help`
curl https://$(minikube ip) --insecure # it works
kubectl get svc -n ingress-nginx # NodePort, 80:$HTTP_NODE_PORT/TCP,443:$HTTPS_NODE_PORT/TCP
curl $(minikube ip):$HTTP_NODE_PORT
curl $(minikube ip):$HTTP_NODE_PORT/test
curl https://$(minikube ip):$HTTPS_NODE_PORT --insecure
curl https://$(minikube ip):$HTTPS_NODE_PORT/test --insecure
kubectl delete deploy web web2
kubectl delete svc web-svc web2-svc
kubectl delete ingress web-ing web2-ing

Ingress relies on Annotations to specify additional configuration. The supported Annotations depends on the Ingress controller type in use - in this case Ingress-Nginx
Please visit the Ingress-Nginx official Rewrite documentation for more details

Lab 9.3. Simple fanout Ingress

1. Create an Ingress webapp-ingress that:
   • redirects requests for path myawesomesite.com/ to a Service webappsvc:80
   • redirects requests for path myawesomesite.com/hello to a Service hellosvc:8080
   • remember to add the Rewrite Annotation
2. List created resources - compare the value of Ingress HOSTS to the previous lab
3. View more details of the Ingress and review the notes under Rules
4. View the Ingress in YAML form and review the structure of the Rules
5. Create a Deployment webapp with image httpd
6. Expose the webapp Deployment as NodePort with service name webappsvc
7. List all created resources - ingress, service, deployment and other resources associated with the deployment
8. View more details of the Ingress and review the notes under Rules
9. Can you access webapp via the minikube Node curl $(minikube ip) or curl myawesomesite.com?
10. Create a second Deployment hello with image gcr.io/google-samples/hello-app:1.0
11. Expose hello as NodePort with service name hellosvc
12. List newly created resources - service, pods, deployment etc
13. View more details of the Ingress and review the notes under Rules
14. Can you access hello via curl $(minikube ip)/hello or myawesomesite.com/hello?
15. Add an entry to /etc/hosts that maps the minikube Node IP to an hostname $(minikube ip) myawesomesite.com
16. Can you access webapp via curl $(minikube ip) or myawesomesite.com with HTTP and HTTPS
17. Can you access hello via curl $(minikube ip)/hello or myawesomesite.com/hello with HTTP and HTTPS
18. Can you access webapp and hello on myawesomesite.com via the NodePorts specified by the ingress-nginx-controller, webappsvc and hellosvc Services?
19. Delete created resources

Lab 9.4. Multiple hosts ingress

1. Review the IngressClass resource object
2. List the Ingress classes created by the minikube ingress addon
3. Create two Deployments nginx and httpd
4. Expose both deployments as Cluster-IP on port 80
5. Create an Ingress with the following:
   • redirects requests for nginx.yourchosenhostname.com to the nginx Service
   • redirects requests for httpd.yourchosenhostname.com to the httpd Service
   • both rules should use a Prefix path type
6. Review created resources
7. Confirm Ingress PathType and IngressClass
8. Review the IngressClass resource YAML form to determine why it was assigned by default
9. Add an entry to /etc/hosts that maps the minikube Node IP to hostnames below:
   • $(minikube ip) nginx.yourchosenhostname.com
   • $(minikube ip) httpd.yourchosenhostname.com
10. Verify you can access both deployments via their subdomains
11. Delete created resources

Lab 9.5. Declare network policy

You may follow the official declare network policy walkthrough

⚠ A Network Policy will have no effect without a network provider with network policy support (e.g. Calico)
⚠ Minikube Calico plugin might conflict with future labs, so remember to disable Calico after this lab
ℹ You can prepend https://k8s.io/examples/ to example filepaths from the official docs to use the file locally

1. Create a Kubernetes cluster in minikube with Calico enabled
   • delete existing cluster, or create an additional cluster, if Calico is not enabled
2. Confirm Calico is up and running
3. Create a Deployment called webapp using image httpd
4. Expose the Deployment on port 80
5. Review created resources and confirm pods running
6. Create a busybox Pod and connect an interactive shell
7. Run command in the Pod container wget --spider --timeout=1 webapp
8. Limit access to the Service so that only Pods with label tier=frontend have access - see official manifest example service/networking/nginx-policy.yaml
9. View more details of the NetworkPolicy created
10. Create a busybox Pod and connect an interactive shell
11. Run command in the Pod container wget --spider --timeout=1 webapp
12. Create another busybox Pod with label tier=frontend and connect an interactive shell
13. Run command in the Pod container wget --spider --timeout=1 webapp
14. Delete created resources
15. Revert to a cluster without Calico

Task - Ingress

The application is meant to be accessible at ckad-bootcamp.local. Please debug and resolve the issue without creating any new resource.

• Command to setup environment:

  printf '\nlab: environment setup in progress...\n'; echo '{"apiVersion":"v1","kind":"List","items":[{"apiVersion":"v1","kind":"Namespace","metadata":{"name":"bat"}},{"apiVersion":"apps/v1","kind":"Deployment","metadata":{"labels":{"appid":"webapp"},"name":"webapp","namespace":"bat"},"spec":{"replicas":2,"selector":{"matchLabels":{"appid":"webapp"}},"template":{"metadata":{"labels":{"appid":"webapp"}},"spec":{"containers":[{"image":"gcr.io/google-samples/node-hello:1.0","name":"nginx"}]}}}},{"apiVersion":"v1","kind":"Service","metadata":{"labels":{"appid":"webapp"},"name":"webapp","namespace":"bat"},"spec":{"ports":[{"port":80,"protocol":"TCP","targetPort":80}],"selector":{"app":"webapp"}}},{"kind":"Ingress","apiVersion":"networking.k8s.io/v1","metadata":{"name":"webapp","namespace":"bat"},"spec":{"ingressClassName":"ngnx","rules":[{"http":{"paths":[{"path":"/","pathType":"Prefix","backend":{"service":{"name":"webapp","port":{"number":80}}}}]}}]}}]}' | kubectl apply -f - >/dev/null; echo 'lab: environment setup complete!'
  

• Command to destroy environment:

  kubectl delete ns bat
  

Task - Network policy

Given several Pods in Namespaces pup and cat, create network policies as follows:

• Pods in the same Namespace can communicate together

• webapp Pod in the pup Namespace can communicate with microservice Pod in the cat Namespace

• DNS resolution on UDP/TCP port 53 is allowed for all Pods in all Namespaces

• Command to setup environment:

  printf '\nlab: environment setup in progress...\n'; echo '{"apiVersion":"v1","kind":"List","items":[{"apiVersion":"v1","kind":"Namespace","metadata":{"name":"pup"}},{"apiVersion":"v1","kind":"Namespace","metadata":{"name":"cat"}},{"apiVersion":"v1","kind":"Pod","metadata":{"labels":{"server":"frontend"},"name":"webapp","namespace":"pup"},"spec":{"containers":[{"image":"nginx:1.22-alpine","name":"nginx"}],"dnsPolicy":"ClusterFirst","restartPolicy":"Always"}},{"apiVersion":"v1","kind":"Pod","metadata":{"labels":{"server":"backend"},"name":"microservice","namespace":"cat"},"spec":{"containers":[{"image":"node:16-alpine","name":"nodejs","args":["sleep","7200"]}],"dnsPolicy":"ClusterFirst","restartPolicy":"Always"}}]}' | kubectl apply -f - >/dev/null; echo 'lab: environment setup complete!'
  

• Command to destroy environment:

  kubectl delete ns cat pup
  

Lab 10.1. PVs and PVCs

# list PVCs, PVs
kubectl get {pvc|pv|storageclass}
# view more details of a PVC
kubectl decribe {pvc|pv|storageclass} $NAME

1. Create a PV with 3Gi capacity using official docs pods/storage/pv-volume.yaml manifest file as base
2. Create a PVC requesting 1Gi capacity using official docs pods/storage/pv-claim.yaml manifest file as base
3. List created resources
   • What STATUS and VOLUME does the PVC have?
   • Does the PVC use the existing PV and why or why not?
4. What happens when a PV and PVC are created without specifying a StorageClass?
   • repeat steps 1-3 after removing storageClassName from both YAML files
   • what was the results?

Lab 10.2. Configuring Pods storage with PVs and PVCs

The benefit of configuring Pods with PVCs is to decouple site-specific details.

You can follow the official configure a Pod to use a PersistentVolume for storage docs to complete this lab.

1. Create a /mnt/data/index.html file on cluster host minikube ssh with some message, e.g. "Hello, World!"
2. Create a PV with the following parameters, see https://k8s.io/examples/pods/storage/pv-volume.yaml
   • uses hostPath storage
   • allows multiple pods in the Node access the storage
3. Create a Pod running a webserver to consume the storage, see https://k8s.io/examples/pods/storage/pv-pod.yaml
   • uses PVC, see https://k8s.io/examples/pods/storage/pv-claim.yaml
   • image is httpd and default documentroot is /usr/local/apache2/htdocs or /var/www/html
4. Verify all resources created pod,pv,pvc,storageclass, and also review each detailed information
   • review STATUS for PV and PVC
   • did the PVC in [3] bind to the PV in [2], why or why not?
5. Connect to the Pod via an interactive shell and confirm you can view the contents of cluster host file curl localhost
6. Clean up all resources created

# host terminal
minikube ssh
# node terminal
sudo mkdir /mnt/data
sudo sh -c "echo 'Hello from Kubernetes storage' > /mnt/data/index.html"
cat /mnt/data/index.html
exit
# host terminal
echo --- > lab10-2.yaml
wget https://k8s.io/examples/pods/storage/pv-volume.yaml -O- >> lab10-2.yaml
echo --- >> lab10-2.yaml
wget https://k8s.io/examples/pods/storage/pv-claim.yaml -O- >> lab10-2.yaml
echo --- >> lab10-2.yaml
wget https://k8s.io/examples/pods/storage/pv-pod.yaml -O- >> lab10-2.yaml
echo --- >> lab10-2.yaml
nano lab10-2.yaml # edit the final file accordingly
kubectl apply -f lab10-2.yaml
kubectl get pod,pv,pvc,storageclass
kubectl describe pod,pv,pvc,storageclass | less
kubectl exec -it task-pv-pod -- /bin/bash
kubectl delete -f lab10-2.yaml

For further learning, see mounting the same persistentVolume in two places and access control

Task - Persistent volumes

In the kid Namespace (create if required), create a Deployment webapp with two replicas, running the nginx:1.22-alpine image, that serves an index.html HTML document (see below) from the Cluster Node's /mnt/data directory. The HTML document should be made available via a Persistent Volume with 5Gi storage and no class name specified. The Deployment should use Persistent Volume claim with 2Gi storage.

<!-- index.html -->
<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="UTF-8">
    <meta http-equiv="X-UA-Compatible" content="IE=Edge">
    <meta name="viewport" content="width=device-width, initial-scale=1">
    <title>K8s Bootcamp (CKAD)</title>
  </head>
  <body>
    <h1>Welcome to K8s Bootcamp!</h1>
  </body>
</html>

Lab 11.1. Deployment variables

1. Create a db Deployment using mysql image
2. Troubleshoot and fix any deployment issues to get a running STATUS
3. View more details of the Deployment and note where env-var is specified
4. Review the Deployment in YAML form and note how the env-var is specified
5. Create a db Pod with an appropriate environment variable specified
6. Confirm Pod running as expected
7. View more details of the Pod and note where env-var is specified
8. Review the Pod in YAML form and note how the env-var is specified
9. Delete created resources

kubectl create deploy db --image=mysql
kubectl get po --watch # status=containercreating->error->crashloopbackoff->error->etc, ctrl+c to quit
kubectl describe po $POD_NAME # not enough info to find issue, so check logs
kubectl logs $POD_NAME|less # found issue, must specify one of `MYSQL_ROOT_PASSWORD|MYSQL_ALLOW_EMPTY_PASSWORD|MYSQL_RANDOM_ROOT_PASSWORD`
kubectl set env deploy db MYSQL_ROOT_PASSWORD=mysecret
kubectl get po # status=running
kubectl describe deploy db # review deployment env-var format
kubectl get deploy db -oyaml|less # review deployment env-var format
kubectl run db --image=mysql --env=MYSQL_ROOT_PASSWORD=mypwd
kubectl get po # status=running
kubectl describe deploy db # review pod env-var format
kubectl describe deploy,po db | grep -iEA15 "pod template:|containers:" | less # see `grep -h`
kubectl get po db -oyaml|less # review pod env-var format
kubectl delete deploy,po db

Note that you can use Pod fields as env-vars, as well as use container fields as env-vars

Lab 11.2. ConfigMaps as environment variables

1. Create a file.env file with the following content:

   MYSQL_ROOT_PASSWORD=pwd
   MYSQL_ALLOW_EMPTY_PASSWORD=true
   

2. Create a File ConfigMap mycm-file from the file using --from-file option
3. Create an Env-Var ConfigMap mycm-env from the file using --from-env-file option
4. Compare details of both ConfigMaps, what can you find?
5. Compare the YAML form of both ConfigMaps, what can you find?
6. Create manifest files for the two Deployments with mysql image using the ConfigMaps as env-vars:
   • a Deployment called web-file for ConfigMap mycm-file
   • a Deployment called web-env for ConfigMap mycm-env
7. Review both manifest files to confirm if env-vars configured correctly, what did you find?
   • any Deployment with correctly configured env-var?
   • which ConfigMap was used for the working Deployment?
   • Are you aware of the issue here?
8. Create a Deployment with two env-vars from the working ConfigMap
9. Connect a shell to a Pod from the Deployment and run printenv to confirm env-vars
10. Create a Pod with env-vars from the working ConfigMap
    • how will you set the env-vars for the Pod?
11. Confirm Pod running or troubleshoot/fix any issues
12. Connect a shell to the new Pod and run printenv to confirm env-vars
13. Delete all created resources

Lab 11.3. Mounting ConfigMaps

In the previous lab, only the Env-Var ConfigMap worked for our use-case. In this lab we will see how we can use the File ConfigMap.

You may also follow the offical add ConfigMap data to a Volume docs

1. Create a file.env file with the following content:

   MYSQL_ROOT_PASSWORD=pwd
   

2. Create a File ConfigMap mycm from the file and verify resource details
3. Create a manifest file for a Pod with the following:
   • uses mysql image
   • specify an env-var MYSQL_ROOT_PASSWORD_FILE=/etc/config/file.env, see the Docker Secrets section of MYSQL image
   • mount ConfigMap mycm as a volume to /etc/config/, see Populate a volume with ConfigMap
4. Create the Pod and verify all works and env-var set in container
5. Create a html/index.html file with any content
6. Create a ConfigMap from the file and verify resource details
7. Create a webserver deployment with an appropriate image and mount the file to the DocumentRoot via ConfigMap
   • option nginx DocumentRoot - /usr/share/nginx/html
   • option httpd DocumentRoot - /usr/local/apache2/htdocs
8. Connect a shell to the container and confirm your file is being served
9. Delete created resources

echo "MYSQL_ROOT_PASSWORD=pwd" > file.env
kubectl create cm mycm --from-file=file.env --dry-run=client -oyaml > lab11-3.yml
echo --- >> lab11-3.yml
kubectl run mypod --image=mysql --env=MYSQL_ROOT_PASSWORD_FILE=/etc/config/file.env --dry-run=client -oyaml >> lab11-3.yml
wget -qO- https://k8s.io/examples/pods/pod-configmap-volume.yaml | less # copy relevant details to lab11-3.yml
nano lab11-3.yml

kind: Pod
spec:
  volumes:
  - name: config-volume
    configMap:
      name: mycm
  containers:
  - name: mypod
    volumeMounts:
    - name: config-volume
      mountPath: /etc/config
# etc, rest same as generated

kubectl apply -f lab11-3.yml
kubectl get po # mypod in running state
kubectl exec mypod -it -- printenv # shows MYSQL_ROOT_PASSWORD_FILE
# part 2 of lab
mkdir html
echo "Welcome to Lab 11.3 - Part 2" > html/index.html
kubectl create cm webcm --from-file=html/index.html
echo --- >> 11-3.yml
kubectl create deploy webserver --image=httpd --dry-run=client -oyaml > lab11-3.yml
nano lab11-3.yml # copy yaml format above and fix indentation

kind: Deployment
spec:
  template:
    spec:
      volumes:
      - name: config-volume
        configMap:
          name: webcm
      containers:
      - name: httpd
        volumeMounts:
        - name: config-volume
          mountPath: /usr/local/apache2/htdocs

kubectl get deploy,po # note pod name and running status
kubectl exec $POD_NAME -it -- ls /usr/local/apache2/htdocs # index.html
kubectl port-forward pod/$POD_NAME 3000:80 & # bind port 3000 in background
curl localhost:3000 # Welcome to Lab 11.3 - Part 2
fg # bring job to fore-ground, then ctrl+c to terminate
kubectl delete -f lab11-3.yml

Pay attention to the types of ConfigMaps, File vs Env-Var, and also note their YAML form differences

Lab 11.4. Decoding secrets

You may follow the official managing secrets using kubectl docs

1. Review the CoreDNS Pod in the kube-system namespace and determine its serviceAccountName
2. Review the ServiceAccount and determine the name of the Secret in use
3. View the contents of the Secret and decode the value of its keys: ca.crt namespace and token.

Lab 11.5. Secrets as environment variables

Repeat lab 11.2 with secrets

See the official secrets as container env-vars docs

Lab 11.6. Secrets as files

Repeat lab 11.3 with secrets.

See the official using secrets as files docs

Lab 11.7. Secrets as docker registry credentials

1. Create a secret with the details of your docker credentials
2. View more details of the resource created kubectl describe
3. View details of the secret in yaml
4. Decode the contents of the .dockerconfigjson key with jsonpath

See the official create an imagePullSecret docs

Task - ConfigMap and Secrets

The latest Bootcamp cohort have requested a new database in the rig Namespace. This should be created as a single replica Deployment named db running the mysql:8.0.22 image with container named mysql. The container should start with 128Mi memory and 0.25 CPU but should not exceed 512Mi memory and 1 CPU.

The Resource limit values should be available in the containers as env-vars MY_CPU_LIMIT and MY_MEM_LIMIT for the values of the cpu limit and memory limit respectively. The Pod IP address should also be available as env-var MY_POD_IP in the container.

A Secret named db-secret should be created with variables MYSQL_DATABASE=bootcamp and MYSQL_ROOT_PASSWORD="shhhh!" to be used by the Deployment as the database credentials. A ConfigMap named db-config should be used to load the .env file (see below) and provide environment variable DEPLOY_ENVIRONMENT to the Deployment.

# .env
DEPLOY_CITY=manchester
DEPLOY_REGION=north-west
DEPLOY_ENVIRONMENT=staging

Lab 12.1. Liveness probe

Many applications running for long periods of time eventually transition to broken states, and cannot recover except by being restarted. Kubernetes provides liveness probes to detect and remedy such situations.
You may follow the official define a liveness command tutorial to complete this lab.

# get events
kubectl get events
# get events of a specific resource, pod, deployment, etc
kubectl get events --field-selector=involvedObject.name=$RESOURCE_NAME
# watch events for updates
kubectl get events --watch

1. Using the official manifest file pods/probe/exec-liveness.yaml as base, create a Deployment myapp manifest file as follows:
   • busybox image
   • commandline arguments mkdir /tmp/healthy; sleep 30; rm -d /tmp/healthy; sleep 60; mkdir /tmp/healthy; sleep 600;
   • a liveness probe that checks for the presence of /tmp/healthy directory
   • the Probe should be initiated 10secs after container starts
   • the Probe should perform the checks every 10secs

   the container creates a directory /tmp/healthy on startup, deletes the directory 30secs later, recreates the directory 60secs later
   your goal is to monitor the Pod behaviour/statuses during these events, you can repeat this lab until you understand liveness probes

2. Apply the manifest file to create the Deployment
3. Review and monitor created Pod events for 3-5mins
4. Delete created resources

Lab 12.2. Readiness probe

Sometimes, applications are temporarily unable to serve traffic, for example, a third party service become unavailable, etc. In such cases, you don't want to kill the application, but you don't want to send it requests either. Kubernetes provides readiness probes to detect and mitigate these situations. Both readiness probe and liveness probe use similar configuration.

1. Using the official manifest files pods/probe/http-liveness.yaml as base, create a Deployment myapp manifest file as follows:
   • nginx:1.22-alpine image
   • 2 replicas
   • a readiness probe that uses an HTTP GET request to check that the root endpoint / on port 80 returns a success status code
   • the readiness probe should be initiated 3secs after container starts, and perform the checks every 5secs
   • a liveness probe that uses an HTTP GET request to check that the root endpoint / on port 80 returns a success status code
   • the liveness probe should be initiated 8secs after the container is ready, and perform checks every 6secs
2. Apply the manifest file to create the Deployment
3. View more details of the Deployment and:
   • confirm how Probe configuration appear, note values for delay | timeout | period | success | failure, how do you set these values?
   • review Events for probe-related entries
   • note that no Events generated when Probes successful
4. List running Pods
5. View more details of one of the Pods and:
   • confirm both probes are configured
   • review Events for probe-related entries
6. Lets trigger a probe failure to confirm all works, edit the Deployment and change readiness probe port to 8080
7. Review more details of the Deployment and individual Pods and determine how Probe failures are recorded, on Deployment or Pod?
8. Lets overload our Probe configuration, using official manifest pods/probe/tcp-liveness-readiness.yaml as example, edit the Deployment as follows:
   • replace the readiness probe with one that uses TCP socket to check that port 80 is open
   • the readiness probe should be initiated 5secs after container starts, and perform the checks every 10secs
   • replace the liveness probe with one that uses TCP socket to check that port 80 is open
   • the liveness probe should be initiated 15secs after the container starts, and perform checks every 10secs
   • note that these changes trigger a rolling update - chainging parameters within deploy.spec.template
9. Review more details of one of the Pod
10. Delete created resources

Lab 12.3. Protect slow starting containers with startup probe

Sometimes, you have to deal with legacy applications that might require additional startup time on first initialization. In such cases, it can be tricky to set up liveness probe without compromising the fast response to deadlocks that motivated such a probe. The trick is to set up a startup probe with the same command, HTTP or TCP check, with a failureThreshold * periodSeconds long enough to cover the worse case startup time.

1. Using the official manifest files pods/probe/http-liveness.yaml as base, create a Deployment myapp manifest file as follows:
   • nginx:1.22-alpine image
   • 2 replicas
   • a readiness probe that uses an HTTP GET request to check that the root endpoint / on port 80 returns a success status code
   • the readiness probe should be initiated 3secs after container starts, and perform the checks every 5secs
   • a liveness probe that uses an HTTP GET request to check that the root endpoint / on port 80 returns a success status code
   • the liveness probe should be initiated 8secs after the container is ready, and perform checks every 6secs
   • a startup probe with the same command as the liveness probe but checks every 5secs up to a maximum of 3mins
2. Apply the manifest file to create the Deployment
3. View more details of the Deployment and confirm how all Probe configuration appear
4. List running Pods
5. View more details of one of the Pods and confirm how Probe configuration appear
6. Delete created resources

Lab 12.4. Blue/Green deployments

Blue/green deployment is a update-strategy used to accomplish zero-downtime deployments. The current version application is marked blue and the new version application is marked green. In Kubernetes, blue/green deployment can be easily implemented with Services.

1. Create a blue Deployment
   • three replicas
   • image nginx:1.19-alpine
   • on the cluster Node, create an HTML document /mnt/data/index.html with any content
   • mount the index.html file to the DocumentRoot as a HostPath volume
2. Expose blue Deployment on port 80 with Service name bg-svc
3. Verify created resources and test access with curl
4. Create a new green Deployment using [1] as base
   • three replicas
   • use a newer version of the image nginx:1.21-alpine
   • on the cluster Node, create a new HTML document /mnt/data2/index.html with different content
   • mount the index.html file to the DocumentRoot as a HostPath volume
5. Verify created resources and test access with curl
6. Edit bg-svc Service Selector as app=green to redirect traffic to green Deployment
7. Confirm all working okay with curl
8. Delete created resources

Lab 12.5. Canary deployments

Canary deployment is an update strategy where updates are deployed to a subset of users/servers (canary application) for testing prior to full deployment. This is a scenario where Labels are required to distinguish deployments by release or configuration.

1. Create a webserver application
   • three replicas
   • Pod Template label updateType=canary
   • use imagenginx:1.19-alpine
   • create an HTML document index.html with any content
   • mount the index.html file to the DocumentRoot as a ConfigMap volume
2. Expose the Deployment on port 80 with Service name canary-svc
3. Verify created resources and test access with curl
4. Create a new application using [1] as base
   • one replica
   • Pod Template label updateType=canary
   • use a newer version of the image nginx:1.22-alpine
   • create a new HTML document index.html with different content
   • mount the index.html file to the DocumentRoot as a ConfigMap volume
5. Verify created resources and confirm the Service targets both webservers
6. Run multiple curl requests to the IP in [2] and confirm access to both webservers
7. Scale up the new webserver to three replicas and confirm all Pods running
8. Scale down the old webserver to zero and confirm no Pods running
9. Delete created resources

Scaling down to zero instead of deleting provides an easy option to revert changes when there are issues

Task - Probes

You have a legacy application legacy running in the dam Namespace that has a long startup time. Once startup is complete, the /healthz:8080 endpoint returns 200 status. If this application is down at anytime or starting up, this endpoint will return a 500 status. The container port for this application often changes and will not always be 8080.

Create a probe for the existing Deployment that checks the endpoint every 10secs, for a maximum of 5mins, to ensure that the application does not receive traffic until startup is complete. 20 secs after startup, a probe should continue to check, every 30secs, that the application is up and running, otherwise, the Pod should be killed and restarted anytime the application is down.

You do not need to test that the probes work, you only need to configure them. Another test engineer will perform all tests.

• Command to setup environment:

  printf '\nlab: lab environment setup in progress...\n'; echo '{"apiVersion":"v1","kind":"List","items":[{"apiVersion":"v1","kind":"Namespace","metadata":{"name":"dam"}},{"apiVersion":"apps/v1","kind":"Deployment","metadata":{"labels":{"app":"legacy"},"name":"legacy","namespace":"dam"},"spec":{"replicas":1,"selector":{"matchLabels":{"app":"legacy"}},"template":{"metadata":{"labels":{"app":"legacy"}},"spec":{"containers":[{"args":["/server"],"image":"registry.k8s.io/liveness","name":"probes","ports":[{"containerPort":8080}]}],"restartPolicy":"OnFailure"}}}}]}' | kubectl apply -f - >/dev/null; echo 'lab: environment setup complete!'
  

• Command to destroy environment:

  kubectl delete ns dam
  

Task - Zero Downtime Updates

In the hog Namespace, you will find a Deployment named high-app, and a Service named high-svc. It is currently unknown if these resources are working together as expected. Make sure the Service is a NodePort type exposed on TCP port 8080 and that you're able to reach the application via the NodePort.

Create a single replica Deployment named high-appv2 based on high-app.json file running nginx:1.18-alpine.

• Update high-appv2 Deployment such that 20% of all traffic going to existing high-svc Service is routed to high-appv2. The total Pods between high-app and high-appv2 should be 5.
• Next, update high-app and high-appv2 Deployments such that 100% of all traffic going to high-svc Service is routed to high-appv2. The total Pods between high-app and high-appv2 should be 5.

Finally, create a new Deployment named high-appv3 based on high-app.json file running nginx:1.20-alpine with 5 replicas and Pod Template label box: high-app-new.

• Update high-svc Service such that 100% of all incoming traffic is routed to high-appv3.

• Since high-appv2 Deployment will no longer be used, perform a cleanup to delete all Pods related to high-appv2 only keeping the Deployment and ReplicaSet.

• Command to setup environment (also creates high-app.json file):

  printf '\nlab: environment setup in progress...\n'; echo '{"apiVersion":"v1","kind":"List","items":[{"apiVersion":"v1","kind":"Namespace","metadata":{"name":"hog"}},{"apiVersion":"v1","kind":"Service","metadata":{"labels":{"kit":"high-app"},"name":"high-svc","namespace":"hog"},"spec":{"ports":[{"port":8080,"protocol":"TCP","targetPort":8080}],"selector":{"box":"high-svc-child"}}}]}' | kubectl apply -f - >/dev/null; echo '{"apiVersion":"apps/v1","kind":"Deployment","metadata":{"labels":{"kit":"high-app"},"name":"high-app","namespace":"hog"},"spec":{"replicas":4,"selector":{"matchLabels":{"box":"high-app-child"}},"template":{"metadata":{"labels":{"box":"high-app-child"}},"spec":{"containers":[{"image":"nginx:1.15-alpine","name":"nginx","ports":[{"containerPort":80}]}]}}}}' > high-app.json; kubectl apply -f high-app.json  >/dev/null; echo 'lab: environment setup complete!';
  

• Command to destroy environment:

  kubectl delete ns hog
  

Lab 13.1. Using kubectl proxy to access the API

Rather than run kubectl commands directly, we can use kubectl as a reverse proxy to provide the location and authenticate requests. See access the API using kubectl proxy for more details.

You may follow the official accessing the rest api docs

1. Expose the API with kube-proxy
2. Confirm k8s version information with curl
3. Explore the k8s API with curl
4. Create a deployment
5. List the pods created with curl
6. Get details of a specific pod with curl
7. Delete the pod with curl
8. Confirm pod deletion

Lab 13.2. Accessing the API without kubectl proxy

This requires using the token of the default ServiceAccount. The token can be read directly (see lab 11.4 - decoding secrets), but the recommended way to get the token is via the TokenRequest API.

You may follow the official access the API without kubectl proxy docs.

1. Request the ServiceAccount token by YAML. You can also request by kubectl create token $SERVICE_ACCOUNT_NAME on Kubernetes v1.24+.
2. Wait for the token controller to populate the Secret with a token
3. Use curl to access the API with the generated token as credentials

# request token
kubectl apply -f - <<EOF
apiVersion: v1
kind: Secret
metadata:
  name: default-token
  annotations:
    kubernetes.io/service-account.name: default
type: kubernetes.io/service-account-token
EOF
# confirm token generated (optional)
kubectl get secret default-token -o yaml
# use token
APISERVER=$(kubectl config view --minify -o jsonpath='{.clusters[0].cluster.server}')
TOKEN=$(kubectl get secret default-token -o jsonpath='{.data.token}' | base64 --decode)
curl $APISERVER/api --header "Authorization: Bearer $TOKEN" --insecure

Using curl with the --insecure option skips TLS certificate validation

Lab 13.3. Accessing the API from inside a Pod

You may follow the official access the API from within a Pod docs.

From within a Pod, the Kubernetes API is accessible via the kubernetes.default.svc hostname

1. Connect an interactive shell to a container in a running Pod (create one or use existing)
2. Use curl to access the API at kubernetes.default.svc/api with the automounted ServiceAccount credentials (token and certificate)
3. Can you access the Pods list at kubernetes.default.svc/api/v1/namespaces/default/pods?

Lab 13.4. Exploring RBAC

In lab 13.3 we were unable to access the PodList API at kubernetes.default.svc/api/v1/namespaces/default/pods. Lets apply the required permissions to make this work.

1. Create a ServiceAccount and verify
2. Create a Role with permissions to list pods and verify
3. Create a RoleBinding that grants the Role permissions to the ServiceAccount, within the default namespace, and verify
4. Create a "naked" Pod bound to the ServiceAccount
5. Connect an interactive shell to the Pod and use curl to PodList API
6. Can you access the API to get a specific Pod like the one you're running? Hint: Role permissions
7. Can you use a deployment instead of a "naked" Pod?