Kubernetes Security reading list

About exam

Duration2 hours
Number of questions: 15-2015-20
Passing score67%
PrerequisiteCKA
CKS_Curriculum

Setup

Vim:

echo "set ai et sw=2 ts=2 sts=2" > ~/.vimrc

ai = autoindend - Copy indent from current line when starting a new line (typing in Insert mode or when using the “o” or “O” command). et = expandtab - In Insert mode: Use the appropriate number of spaces to insert a . sw = shiftwidth sts = shifttabstop ts = tabspace

To create terminal in vertical split, where you can run k explain po.spec.securityConstraints

:vert term

Domains & Weighting

DomainWeight
* Cluster Setup10%
* Cluster Hardening15%
* System Hardening15%
* Minimize Microservice Vulnerabilities20%
* Supply Chain Security20%
* Monitoring, Logging and Runtime Security20%

Mostly used articles during solving problems by me

Cilium:

Istio:

Cluster setup 10%

topicsnotes
Use Network security policies to restrict cluster level access
Use CIS benchmark to review the security configuration of Kubernetes components (etcd, kubelet, kubedns, kubeapi)
Properly set up Ingress with TLS
Protect node metadata and endpoints
Verify platform binaries before deploying

Cluster Hardening 15%

topicnotes
Use Role Based Access Controls to minimize exposure
Exercise caution in using service accounts e.g. disable defaults, minimize permissions on newly created ones
Restrict access to Kubernetes API
Upgrade Kubernetes to avoid vulnerabilities

System Hardening 15%

topicnotes
Minimize host OS footprint (reduce attack surface)
Using least-privilege identity and access management
Minimize external access to the network
Appropriately use kernel hardening tools such as AppArmor, seccomp

Minimize Microservice Vulnerabilities (20%)

topicnotes
Use appropriate pod security standards
Manage Kubernetes secrets
Understand and implement isolation techniques (multi-tenancy, sandboxed containers, etc.)
Implement Pod-to-Pod encryption (Cilium, Istio)

Supply Chain Security (20%)

topicnotes
Minimize base image footprint
Understand your supply chain (e.g. SBOM, CI/CD, artifact repositories)
Secure your supply chain (permitted registries, sign and validate artifacts, etc.)
Perform static analysis of user workloads and container images (e.g. Kubesec, KubeLinter)

[!NOTE] Checkov is a static code analysis tool for infrastructure as code (IaC) and also a software composition analysis (SCA) tool for images and open source packages.

Monitoring, Logging and Runtime Security

topicnotes
Perform behavioral analytics to detect malicious activities
Detect threats within physical infrastructure, apps, networks, data, users and workloads
Investigate and identify phases of attack and bad actors within the environment
Ensure immutability of containers at runtime
Use Kubernetes audit logs to monitor access

Perform deep analytical investigation and identification of bad actors within the environment

Tips

Falco:

# list all rules
falco -L

Verify checksum of many binaries (search shasum k8s docs):

If you see:

shasum: standard input: no properly formatted SHA checksum lines found

It must be 2 spaces between sha end binary path!!! if you want to compare in one line:

echo "$(cat kube-apiserver.sha256)  kube-apiserver" | shasum -a 256 --check
cat sha.txt
sha file
sha512sum -c sha.txt

Refresh yaml structure using k explain pods.spec:

root@k8s:~# k explain pods.spec | grep -C5 nodeName
    then using the max of of that value or the sum of the normal containers.
    Limits are applied to init containers in a similar fashion. Init containers
    cannot currently be added or removed. Cannot be updated. More info:
    https://kubernetes.io/docs/concepts/workloads/pods/init-containers/

  nodeName      <string>
    NodeName indicates in which node this pod is scheduled. If empty, this pod
    is a candidate for scheduling by the scheduler defined in schedulerName.
    Once this field is set, the kubelet for this node becomes responsible for
    the lifecycle of this pod. This field should not be used to express a desire
    for the pod to be scheduled on a specific node.

root@k8s:~# k explain deployments.spec.template.spec

Falco (supported_fields):

# it can take a bit till Falco displays output, use falco -U/--unbuffered to speed up
falco -U | grep <>

Cri-O

crictl ps

crictl ps -n1 # show only latest running container

crictl ps -id f86cd629e71c

crictl pods -id cab6dafd045d5

crictl pods --name collector1

crictl inspect <CONTAINER ID>

Using the PIDs we can call strace to find Syscalls:

strace -p 14079 | grep -i kill

BOM

bom generate --image registry.k8s.io/kube-apiserver:v1.31.0 --format json --output nginx.spdx.json

Trivy

trivy image --help | grep format
  $ trivy image --format json --output result.json alpine:3.15
  # Generate a report in the CycloneDX format
  $ trivy image --format cyclonedx --output result.cdx alpine:3.15
  -f, --format string              format (table,json,template,sarif,cyclonedx,spdx,spdx-json,github,cosign-vuln) (default "table")
...

openssl

# create new key for new user
openssl genrsa -out myuser.key 2048
openssl req -new -key myuser.key -out myuser.csr

cat <<EOF > CSR.yaml
apiVersion: certificates.k8s.io/v1
kind: CertificateSigningRequest
metadata:
  name: john-developer # add
spec:
  request: $(cat myuser.csr | base64 | tr -d "\n")
  signerName: kubernetes.io/kube-apiserver-client
  usages:
  - client auth
  - digital signature
  - key encipherment
EOF
# View the certificate signing request
openssl req -in new.csr -noout -text

kubectl

copy/paste from kubectl quick ref

alias kx='f() { [ "$1" ] && kubectl config use-context $1 || kubectl config current-context ; } ; f'
alias kn='f() { [ "$1" ] && kubectl config set-context --current --namespace $1 || kubectl config view --minify | grep namespace | cut -d" " -f6 ; } ; f'
# you can list only pods by field selector (e.g. filter by name) a.k.a emulating grep test
# kubectl get pods | grep test
$ kubectl get pods --field-selector=metadata.name=test
NAME   READY   STATUS    RESTARTS   AGE
test   1/1     Running   0          9h

Find all images to scan + metadata.name (based on Jsonpath recursive descent)

# based on jsonpath page
$ kubectl get deploy -o=jsonpath='{range .items[*]}{.metadata.name}{" -> "}{..image}{"\n"}{end}'
deploy -> nginx
image-bouncer-webhook -> kainlite/kube-image-bouncer:latest

# based on kubectl quick ref page also works for pods/deployments

$ kubectl get deploy --output=custom-columns="NAME:.metadata.name,IMAGE:..image"
NAME                    IMAGE
deploy                  nginx
image-bouncer-webhook   kainlite/kube-image-bouncer:latest

Find all SC for pods/containers to troubleshoot:

kubectl get pods -o=custom-columns='N:metadata.name,PSC:..securityContext'

Update kubelet-config:

kubectl -n kube-system edit cm kubelet-config

kubeadm upgrade node phase kubelet-config

systemctl restart kubelet

Upgrade k8s version and components on nodes e.g. from v1.33.4 to v1.34.1:

# show k8s versions
k get node

k drain <node-name> --ignore-daemonsets

kubelet --version
kubeadm version

# not necessary because here kubeadm is already installed in correct version
apt-mark unhold kubeadm
apt-mark hold kubectl kubelet
apt install kubeadm=1.34.1-1.1
apt-mark hold kubeadm

kubeadm upgrade plan

kubeadm upgrade apply v1.34.1

# to check it run again
kubeadm upgrade plan

# kubelet and kubectl

apt update
apt show kubelet | grep 1.34.1
apt install kubelet=1.34.1-1.1 kubectl=1.34.1-1.1

apt-mark hold kubelet kubectl

service kubelet restart
service kubelet status

k get node
k uncordon <node-name>

Footnotes

SELinux

Install containerd and k8s 1.32 on ubuntu 24.04

  1. Create t3.small ec2 instance based on ubuntu, because we need atleast 2GB RAM for k8s.

https://github.com/zealvora/certified-kubernetes-security-specialist/blob/main/domain-1-cluster-setup/kubeadm-install.md

Footnotes

SELinux