Kubernetes Security reading list
About exam
| Duration | 2 hours |
| Number of questions: 15-20 | 15-20 |
| Passing score | 67% |
| Prerequisite | CKA |
| CKS_Curriculum |
Setup
Vim:
echo "set ai et sw=2 ts=2 sts=2" > ~/.vimrc
ai = autoindend - Copy indent from current line when starting a new line (typing in Insert mode or when using the “o” or “O” command).
et = expandtab - In Insert mode: Use the appropriate number of spaces to insert a .
sw = shiftwidth
sts = shifttabstop
ts = tabspace
To create terminal in vertical split, where you can run k explain po.spec.securityConstraints
:vert term
Domains & Weighting
| Domain | Weight |
|---|---|
| * Cluster Setup | 10% |
| * Cluster Hardening | 15% |
| * System Hardening | 15% |
| * Minimize Microservice Vulnerabilities | 20% |
| * Supply Chain Security | 20% |
| * Monitoring, Logging and Runtime Security | 20% |
Mostly used articles during solving problems by me
- Encrypting Confidential Data at Rest
- Auditing
- Configure Service Accounts for Pods
- Apply Pod Security Standards at the Namespace Level
- kubelet
- Configuring each kubelet in your cluster using kubeadm
- Customizing components with the kubeadm API
- Set Kubelet Parameters Via A Configuration File
- Generate Certificates Manually
- https://github.com/bmuschko/cks-study-guide/blob/master/app-b/exam-review-guide.adoc
- Issue a Certificate for a Kubernetes API Client Using A CertificateSigningRequest
- Enforce Pod Security Standards with Namespace Labels
- Distribute Credentials Securely Using Secrets
- JSONPath Support
- kubectl quick reference
Cilium:
Istio:
Cluster setup 10%
| topics | notes |
|---|---|
| Use Network security policies to restrict cluster level access | |
| Use CIS benchmark to review the security configuration of Kubernetes components (etcd, kubelet, kubedns, kubeapi) | |
| Properly set up Ingress with TLS | |
| Protect node metadata and endpoints | |
| Verify platform binaries before deploying |
- Use Network security policies to restrict cluster level access
- Use CIS benchmark to review the security configuration of Kubernetes components (etcd, kubelet, kubedns, kubeapi)
- Ingress objects with security control
- Protect node metadata and endpoints
- Securing a Cluster
- Kube-bench
- Minimize use of, and access to, GUI elements
Cluster Hardening 15%
| topic | notes |
|---|---|
| Use Role Based Access Controls to minimize exposure | |
| Exercise caution in using service accounts e.g. disable defaults, minimize permissions on newly created ones | |
| Restrict access to Kubernetes API | |
| Upgrade Kubernetes to avoid vulnerabilities |
- Use Role Based Access Controls to minimize exposure
- Exercise caution in using service accounts e.g. disable defaults minimize permissions on newly created ones
- Restrict access to Kubernetes API)
- Upgrade Kubernetes to avoid vulnerabilities
- Control anonymous requests to Kube-apiserver
System Hardening 15%
| topic | notes |
|---|---|
| Minimize host OS footprint (reduce attack surface) | |
| Using least-privilege identity and access management | |
| Minimize external access to the network | |
| Appropriately use kernel hardening tools such as AppArmor, seccomp |
- Restrict a Container’s Access to Resources with AppArmor
- Container Runtimes
- Runtime Class
- Restrict a Container’s Syscalls with seccomp
- Using sysctls in a Kubernetes Cluster
- PSA enforces
- Restirct allowed hostpaths
- Access authentication and authorization
Minimize Microservice Vulnerabilities (20%)
| topic | notes |
|---|---|
| Use appropriate pod security standards | |
| Manage Kubernetes secrets | |
| Understand and implement isolation techniques (multi-tenancy, sandboxed containers, etc.) | |
| Implement Pod-to-Pod encryption (Cilium, Istio) |
- Use appropriate pod security standards
- Manage kubernetes secrets
- Understand and implement isolation techniques (multi-tenancy, sandboxed containers, etc.)
- Implement pod to pod encryption (Cilium, Istio)
- Open Policy Agent
- Security Contexts Use container runtime sandboxes in multi-tenant environments (e.g. gvisor, kata containers)
- pod spec
- Kubernetes-API PodSpec
- Pod Security Admission (PSA)
Supply Chain Security (20%)
| topic | notes |
|---|---|
| Minimize base image footprint | |
| Understand your supply chain (e.g. SBOM, CI/CD, artifact repositories) | |
| Secure your supply chain (permitted registries, sign and validate artifacts, etc.) | |
| Perform static analysis of user workloads and container images (e.g. Kubesec, KubeLinter) |
- Secure your supply chain: whitelist allowed image registries, sign and validate images
- Using ImagePolicyWebhook admission Controller
- Use static analysis of user workloads (e.g. kubernetes resources, docker files)
- Scan images for known vulnerabilities
- Aqua security Trivy
- NodeRestriction
- kube-bench installation
- checkov
[!NOTE] Checkov is a static code analysis tool for infrastructure as code (IaC) and also a software composition analysis (SCA) tool for images and open source packages.
Monitoring, Logging and Runtime Security
| topic | notes |
|---|---|
| Perform behavioral analytics to detect malicious activities | |
| Detect threats within physical infrastructure, apps, networks, data, users and workloads | |
| Investigate and identify phases of attack and bad actors within the environment | |
| Ensure immutability of containers at runtime | |
| Use Kubernetes audit logs to monitor access |
- falco
- falco basic rules
- sysdig
- K8s Audit
Perform behavioural analytics of syscall process and file activities at the host and container level to detect malicious activities
- Falco installation guide
- 🚩 Sysdig Falco 101
- 🚩 Falco Helm Chart
- 🚩 Falco Kubernetes helmchart
- 🚩 Detect CVE-2020-8557 using Falco Detect threats within a physical infrastructure, apps, networks, data, users and workloads Detect all phases of attack regardless where it occurs and how it spreads
Perform deep analytical investigation and identification of bad actors within the environment
- Sysdig documentation
- Monitoring Kubernetes with sysdig
- 🚩 CNCF Webinar: Getting started with container runtime security using Falco Ensure immutability of containers at runtime Use Audit Logs to monitor access
Tips
Falco:
# list all rules
falco -L
Verify checksum of many binaries (search shasum k8s docs):
If you see:
shasum: standard input: no properly formatted SHA checksum lines found
It must be 2 spaces between sha end binary path!!! if you want to compare in one line:
echo "$(cat kube-apiserver.sha256) kube-apiserver" | shasum -a 256 --check
cat sha.txt
sha file
sha512sum -c sha.txt
Refresh yaml structure using k explain pods.spec:
root@k8s:~# k explain pods.spec | grep -C5 nodeName
then using the max of of that value or the sum of the normal containers.
Limits are applied to init containers in a similar fashion. Init containers
cannot currently be added or removed. Cannot be updated. More info:
https://kubernetes.io/docs/concepts/workloads/pods/init-containers/
nodeName <string>
NodeName indicates in which node this pod is scheduled. If empty, this pod
is a candidate for scheduling by the scheduler defined in schedulerName.
Once this field is set, the kubelet for this node becomes responsible for
the lifecycle of this pod. This field should not be used to express a desire
for the pod to be scheduled on a specific node.
root@k8s:~# k explain deployments.spec.template.spec
Falco (supported_fields):
# it can take a bit till Falco displays output, use falco -U/--unbuffered to speed up
falco -U | grep <>
Cri-O
crictl ps
crictl ps -n1 # show only latest running container
crictl ps -id f86cd629e71c
crictl pods -id cab6dafd045d5
crictl pods --name collector1
crictl inspect <CONTAINER ID>
Using the PIDs we can call strace to find Syscalls:
strace -p 14079 | grep -i kill
BOM
bom generate --image registry.k8s.io/kube-apiserver:v1.31.0 --format json --output nginx.spdx.json
Trivy
trivy image --help | grep format
$ trivy image --format json --output result.json alpine:3.15
# Generate a report in the CycloneDX format
$ trivy image --format cyclonedx --output result.cdx alpine:3.15
-f, --format string format (table,json,template,sarif,cyclonedx,spdx,spdx-json,github,cosign-vuln) (default "table")
...
openssl
# create new key for new user
openssl genrsa -out myuser.key 2048
openssl req -new -key myuser.key -out myuser.csr
cat <<EOF > CSR.yaml
apiVersion: certificates.k8s.io/v1
kind: CertificateSigningRequest
metadata:
name: john-developer # add
spec:
request: $(cat myuser.csr | base64 | tr -d "\n")
signerName: kubernetes.io/kube-apiserver-client
usages:
- client auth
- digital signature
- key encipherment
EOF
# View the certificate signing request
openssl req -in new.csr -noout -text
kubectl
copy/paste from kubectl quick ref
alias kx='f() { [ "$1" ] && kubectl config use-context $1 || kubectl config current-context ; } ; f'
alias kn='f() { [ "$1" ] && kubectl config set-context --current --namespace $1 || kubectl config view --minify | grep namespace | cut -d" " -f6 ; } ; f'
# you can list only pods by field selector (e.g. filter by name) a.k.a emulating grep test
# kubectl get pods | grep test
$ kubectl get pods --field-selector=metadata.name=test
NAME READY STATUS RESTARTS AGE
test 1/1 Running 0 9h
Find all images to scan + metadata.name (based on Jsonpath recursive descent)
# based on jsonpath page
$ kubectl get deploy -o=jsonpath='{range .items[*]}{.metadata.name}{" -> "}{..image}{"\n"}{end}'
deploy -> nginx
image-bouncer-webhook -> kainlite/kube-image-bouncer:latest
# based on kubectl quick ref page also works for pods/deployments
$ kubectl get deploy --output=custom-columns="NAME:.metadata.name,IMAGE:..image"
NAME IMAGE
deploy nginx
image-bouncer-webhook kainlite/kube-image-bouncer:latest
Find all SC for pods/containers to troubleshoot:
kubectl get pods -o=custom-columns='N:metadata.name,PSC:..securityContext'
Update kubelet-config:
kubectl -n kube-system edit cm kubelet-config
kubeadm upgrade node phase kubelet-config
systemctl restart kubelet
Upgrade k8s version and components on nodes e.g. from v1.33.4 to v1.34.1:
# show k8s versions
k get node
k drain <node-name> --ignore-daemonsets
kubelet --version
kubeadm version
# not necessary because here kubeadm is already installed in correct version
apt-mark unhold kubeadm
apt-mark hold kubectl kubelet
apt install kubeadm=1.34.1-1.1
apt-mark hold kubeadm
kubeadm upgrade plan
kubeadm upgrade apply v1.34.1
# to check it run again
kubeadm upgrade plan
# kubelet and kubectl
apt update
apt show kubelet | grep 1.34.1
apt install kubelet=1.34.1-1.1 kubectl=1.34.1-1.1
apt-mark hold kubelet kubectl
service kubelet restart
service kubelet status
k get node
k uncordon <node-name>
Footnotes
SELinux
Install containerd and k8s 1.32 on ubuntu 24.04
- Create t3.small ec2 instance based on ubuntu, because we need atleast 2GB RAM for k8s.
Related materials
Footnotes
SELinux
https://github.com/stackrox/Kubernetes_Security_Specialist_Study_Guide#cluster-hardening---15
https://github.com/ramanagali/Interview_Guide/blob/main/CKS_Preparation_Guide.md
https://github.com/walidshaari/Certified-Kubernetes-Security-Specialist
https://github.com/zealvora/certified-kubernetes-security-specialist/
videos:
practise:
- killer.sh
- killer Koda
- or install your own cluster https://github.com/killer-sh/cks-course-environment/tree/master
- https://killercoda.com/killer-shell-cks/scenario/apiserver-misconfigured
- https://github.com/ramanagali/Interview_Guide/blob/main/CKS_Preparation_Guide.md
- https://github.com/ViktorUJ/cks/blob/master/tasks/cks/mock/01/README.MD
- https://github.com/ViktorUJ/cks/blob/master/tasks/cks/mock/02/README.MD
- https://github.com/ViktorUJ/cks/blob/master/tasks/cks/mock/03/README.MD
- https://github.com/ViktorUJ/cks/blob/master/tasks/cks/mock/04/README.MD
- killer_shell_a (html dump)
- killer_shell_b (html dump)
- https://killercoda.com/killer-shell-cks/
- https://github.com/kubesimplify/cks-certification/tree/main/falco/newrule