KUBERNETES 101, FOR ENGINEERS WHO ALREADY KNOW DOCKER

You declare the state.
Kubernetes keeps it true.

A practical path from single containers to production clusters, architecture, workloads, networking, storage, security, and the operational patterns in between. 14 waypoints, basic to advanced.

cluster, zsh

      
Waypoint 01 ยท Foundations

Why Kubernetes? Basic

You already know how to build an image, run a container, and wire services together with Compose. This is about the gap between that and running the same workloads reliably across a fleet of machines you don't want to babysit by hand.

Compose is an excellent single-host tool. It stops being enough the moment you need any of the following, and all of them show up quickly in production: a workload spread across more than one machine, automatic recovery when a process or an entire host dies, zero-downtime deploys, and a way to say "I want 5 of these running" and have something else keep that true as machines come and go.

Kubernetes has one real idea, and everything else in this guide is a variation on it: you declare the state you want, and independent control loops continuously reconcile the real world to match it. You don't script "start this container, then that one, then check health, then restart on failure", you write "3 replicas of this Pod spec should exist" and hand it to the API server. Compute, networking, storage, and scaling are all just this same pattern applied to a different resource type.

๐Ÿ“˜
Mental model

Docker runs containers. Kubernetes runs a container runtime, containerd or CRI-O, not the Docker Engine itself since dockershim was removed in 1.24, on every machine in a cluster, and layers a declarative, self-healing scheduling system on top of it.

The vocabulary you're trading up to

Docker / ComposeKubernetesWhat actually changes
docker runPodRarely created directly, a controller creates and supervises it
docker-compose.yml serviceDeployment + ServiceSplit into "what to run" and "how to reach it"
user-defined bridge networkCluster pod network (CNI)Flat network model, every Pod gets a routable IP, no manual linking
named volumePersistentVolume / ClaimStorage is requested abstractly and provisioned by a driver
.env file, -e flagsConfigMap, SecretConfig becomes its own API object, mountable as files or env vars
restart: unless-stoppedlivenessProbe + reconciliationRestarts are health-based, not just exit-code-based
--scale web=3kubectl scale / HPAScaling can be automatic and metric-driven
reverse proxy in front of ComposeService (L4) + Ingress (L7)Routing becomes a first-class, declarative API object
Swarm, or nothingnative schedulingThe scheduler bin-packs Pods across every Node in the cluster

What you're not losing

Your images don't change. Anything that builds into an OCI-compliant image with a Dockerfile runs on Kubernetes unmodified, Kubernetes schedules and supervises containers, it doesn't reinvent what's inside them. If you can build it, Kubernetes can run it; this guide is about everything around the container, not the container itself.

๐Ÿ’ก
How to read this guide

Every manifest here is something you'd hand to kubectl apply -f. Read them the way you'd read a Compose file: as a description of desired state, not a script of steps.

Waypoint 02 ยท Foundations

Cluster Architecture Basic

A cluster is two kinds of machines playing very different roles: a small set of control plane nodes that make decisions, and any number of worker nodes that run your actual containers.

Nothing in the control plane runs your workloads. It exists purely to accept desired state, store it, and continuously push the cluster toward it. If every control plane component vanished, already-running Pods would keep serving traffic, they'd just stop being managed until it came back.

Control Plane
kube-apiserver
The only component every other piece talks to. Validates and persists API objects. Stateless and horizontally scalable.
etcd
Distributed key-value store. The single source of truth for every object in the cluster.
kube-scheduler
Watches for Pods with no assigned Node, picks the best fit based on requests, affinity, and taints.
kube-controller-manager
Runs the reconciliation loops: Node, ReplicaSet, Endpoints, ServiceAccount controllers and more.
cloud-controller-manager
Cloud-specific glue: provisions LoadBalancers, labels Nodes, manages routes. Absent on bare metal.
Worker Node ร— N
kubelet
The agent on every Node. Reads PodSpecs assigned to it and makes the runtime match them.
kube-proxy
Implements the Service abstraction as local iptables or IPVS rules on each Node.
container runtime
containerd or CRI-O, driven via the Container Runtime Interface (CRI). Runs your OCI images.

The reconciliation loop, concretely

Every controller in kube-controller-manager runs the same pattern on an infinite loop: watch the API server for objects it cares about, diff desired state against observed state, and act to close the gap. The ReplicaSet controller is the clearest example:

  1. You create a Deployment with replicas: 3.
  2. The ReplicaSet controller sees a ReplicaSet wanting 3 Pods matching its selector, finds 0 running.
  3. It creates 3 Pod objects and writes them via the API server into etcd.
  4. kube-scheduler notices 3 Pods with no nodeName set, and assigns each one to a Node.
  5. The kubelet on each assigned Node notices a Pod scheduled to it, and tells the container runtime to pull the image and start it.

Delete one of those Pods by hand and the ReplicaSet controller's next watch cycle sees 2 running against a desired 3, and creates a replacement, nobody paged anyone. This loop, repeated for dozens of object types, is the entire engine behind Kubernetes' self-healing reputation. You'll see the same watch/diff/act shape again in autoscalers, in Operators, and in your own controllers if you ever write one.

โš ๏ธ
Common confusion

The Docker Engine is not what runs your containers on a modern cluster. Dockershim was removed in Kubernetes 1.24; kubelet talks to containerd or CRI-O directly over the CRI. Images built with docker build are unaffected, they're standard OCI images either way, and docker build remains a perfectly normal way to produce them.

Waypoint 03 ยท Workloads & Networking

Core Workloads Basic

Everything that runs your code sits on one primitive: the Pod. Everything above it, ReplicaSets, Deployments, StatefulSets, DaemonSets, Jobs, is a different policy for how many Pods should exist, and when.

Pod

A Pod is the smallest deployable unit, one or more containers that share a network namespace (same IP, so they reach each other over localhost) and can share volumes. Containers in a Pod are always scheduled together, on the same Node, and live and die together.

YAML
apiVersion: v1
kind: Pod
metadata:
  name: web-app
  labels:
    app: web-app
spec:
  containers:
    - name: app
      image: myregistry/web-app:1.4.2
      ports:
        - containerPort: 8080
      resources:
        requests:
          cpu: "250m"
          memory: "128Mi"
        limits:
          cpu: "500m"
          memory: "256Mi"
    - name: log-shipper
      image: grafana/promtail:2.9.0
      volumeMounts:
        - name: logs
          mountPath: /var/log/app
  volumes:
    - name: logs
      emptyDir: {}

That second container is a sidecar, it doesn't serve traffic itself, it extends the main container. Three patterns cover almost every multi-container Pod you'll run into:

Sidecar

Runs alongside the main container to extend it: a log shipper, a service-mesh proxy, a config reloader.

Ambassador

Proxies the main container's outbound connections, simplifying how it reaches external or clustered services.

Adapter

Normalizes the main container's output: reformatting logs or metrics into a shape a monitoring system expects.

You rarely create bare Pods, though, if a Pod dies, nothing brings it back. In practice almost everything below manages Pods for you through a shared Pod template, so this spec becomes a piece of a larger object instead of something you apply directly.

ReplicaSet

Keeps a fixed number of Pods matching a label selector running at all times. You'll almost never write one by hand, a Deployment creates and owns one for you, but it's where the "N replicas" guarantee actually lives.

Deployment

A Deployment wraps a ReplicaSet with rollout logic: rolling updates, rollback, and revision history. This is the object you reach for by default for stateless workloads.

YAML
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-app
spec:
  replicas: 3
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0
  selector:
    matchLabels:
      app: web-app
  template:
    metadata:
      labels:
        app: web-app
    spec:
      containers:
        - name: app
          image: myregistry/web-app:1.4.2
          ports:
            - containerPort: 8080

maxSurge: 1, maxUnavailable: 0 means the rollout creates one extra Pod on the new version before removing an old one, zero-downtime, at the cost of briefly running replicas + 1 Pods. Flip the numbers if you'd rather save resources than guarantee full capacity mid-rollout.

BASH
kubectl apply -f deployment.yaml
kubectl rollout status deployment/web-app
kubectl scale deployment/web-app --replicas=5
kubectl rollout undo deployment/web-app
kubectl rollout history deployment/web-app

StatefulSet

For workloads that need a stable identity, databases, queues, anything where "pod 0" and "pod 2" mean different things. StatefulSets give each replica a predictable name (postgres-0, postgres-1, postgres-2), a stable DNS entry, and its own PersistentVolumeClaim that survives rescheduling. Pods are created and terminated in order, one at a time, instead of all at once.

YAML
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: postgres
spec:
  serviceName: postgres
  replicas: 3
  selector:
    matchLabels:
      app: postgres
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
        - name: postgres
          image: postgres:16
          volumeMounts:
            - name: data
              mountPath: /var/lib/postgresql/data
  volumeClaimTemplates:
    - metadata:
        name: data
      spec:
        accessModes: ["ReadWriteOnce"]
        resources:
          requests:
            storage: 10Gi
โš ๏ธ
Not a magic database button

A StatefulSet gives you stable identity and storage, it does not give you replication, failover, or backups. For anything beyond a single-instance dev database, you want an Operator that understands the specific engine (see Waypoint 12).

DaemonSet

Runs exactly one Pod on every Node (or a subset, via a node selector), for node-level agents rather than application replicas. This is where log collectors and metrics exporters live: if you've deployed node-exporter for Prometheus or Promtail for Loki, you've already used a DaemonSet.

YAML
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: node-exporter
spec:
  selector:
    matchLabels:
      app: node-exporter
  template:
    metadata:
      labels:
        app: node-exporter
    spec:
      hostNetwork: true
      containers:
        - name: node-exporter
          image: prom/node-exporter:v1.7.0
          ports:
            - containerPort: 9100

Job & CronJob

A Job runs a Pod to completion, retrying on failure, tracking success, for one-off or batch work. A CronJob wraps a Job with a schedule. Both use restartPolicy: OnFailure or Never, not Always, since the point is to finish, not stay running.

YAML
apiVersion: batch/v1
kind: CronJob
metadata:
  name: db-backup
spec:
  schedule: "0 2 * * *"
  jobTemplate:
    spec:
      template:
        spec:
          containers:
            - name: backup
              image: myregistry/db-backup:1.0
          restartPolicy: OnFailure
๐Ÿ’ก
Choosing a controller

Stateless and horizontally scalable โ†’ Deployment. Needs stable identity or per-replica storage โ†’ StatefulSet. Needs to run on every Node โ†’ DaemonSet. Runs once or on a schedule โ†’ Job / CronJob.

Waypoint 04 ยท Workloads & Networking

Networking Intermediate

Kubernetes networking starts from one rule that's different from Docker's default bridge network: every Pod gets its own routable IP, and every Pod can reach every other Pod directly, on any Node, with no NAT and no manual linking.

That flat network is necessary but not sufficient, Pod IPs are ephemeral, reassigned every time a Pod is rescheduled. You never point one Pod at another Pod's IP directly. You point it at a Service: a stable virtual IP and DNS name in front of whichever Pods currently match a label selector.

Service types

TypeReachable fromTypical use
ClusterIP (default)Inside the cluster onlyService-to-service traffic
NodePortAny Node's IP, on a static portQuick external access, dev/test
LoadBalancerA cloud provider's external LBPublic-facing production traffic
ExternalNameN/A, a DNS CNAMEAliasing an external service into cluster DNS
Headless (clusterIP: None)Direct per-Pod DNS recordsStatefulSets, client-side discovery
YAML
apiVersion: v1
kind: Service
metadata:
  name: web-app
spec:
  type: ClusterIP
  selector:
    app: web-app
  ports:
    - port: 80
      targetPort: 8080

Every field here maps onto a Compose concept you already use: selector replaces the implicit container-name linking Compose gives you for free, and port / targetPort split the Service's own port from the container's, something Compose usually collapses into one.

Ingress

Services operate at L4 (TCP/UDP). For HTTP(S) routing, host- and path-based rules, TLS termination, one entry point fanning out to many Services, you need an Ingress resource plus an Ingress Controller actually running in the cluster to implement it. NGINX Ingress and Traefik are the two most common controllers; if you've used Traefik as a reverse proxy before, the mental model carries over directly, it's doing the same routing job, just driven by Kubernetes objects instead of a static config file.

YAML
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: web-app
  annotations:
    traefik.ingress.kubernetes.io/router.entrypoints: websecure
spec:
  rules:
    - host: app.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: web-app
                port:
                  number: 80
  tls:
    - hosts: ["app.example.com"]
      secretName: app-tls-cert
๐Ÿ“˜
Newer alternative

The Gateway API is gradually superseding Ingress for more expressive routing (traffic splitting, header-based rules) with a role-oriented object model. Ingress isn't deprecated and remains completely fine for straightforward host/path routing.

NetworkPolicy

By default, every Pod can reach every other Pod, the flat network has no segmentation built in. A NetworkPolicy is a firewall rule scoped to Pods: it whitelists traffic to (or from) Pods matching a selector, and requires the cluster's CNI plugin to actually support enforcement. Without one, a NetworkPolicy silently does nothing.

YAML
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-frontend-to-backend
spec:
  podSelector:
    matchLabels:
      app: backend
  policyTypes: ["Ingress"]
  ingress:
    - from:
        - podSelector:
            matchLabels:
              app: frontend
      ports:
        - protocol: TCP
          port: 8080

This says: Pods labeled app: backend accept ingress traffic on 8080 only from Pods labeled app: frontend, everything else is dropped. Same segmentation instinct as VLANs and ACLs on a physical network, just expressed as labels instead of subnets. A default-deny policy plus explicit allows for what actually needs to talk is the standard hardened baseline.

CNI, DNS, and kube-proxy

Three supporting pieces make the network model work in practice:

  • CNI plugin, Calico, Cilium, or Flannel actually implements the flat pod network across Nodes. Calico and Cilium (eBPF-based) also enforce NetworkPolicy; Flannel by itself doesn't.
  • CoreDNS, resolves Service names cluster-wide. web-app.default.svc.cluster.local resolves to the ClusterIP of the web-app Service in the default namespace; the short form web-app works for same-namespace lookups.
  • kube-proxy, turns each Service into local traffic rules on every Node, historically via iptables, increasingly via IPVS for better performance at high Service counts.
Waypoint 05 ยท Data & Config

Storage Intermediate

A container's filesystem disappears when it does. Kubernetes separates "how do I get disk" into three layers, each solving a different problem: ephemeral volumes, persistent volumes, and dynamic provisioning.

Ephemeral volumes

emptyDir is scratch space that lives as long as the Pod does, created empty, deleted when the Pod is, shared between containers in that Pod. It's the equivalent of an anonymous Docker volume rather than a named one: useful for caches or handoff between a sidecar and the main container, useless for anything you need after a restart.

PersistentVolume & PersistentVolumeClaim

A PersistentVolume (PV) is a piece of real storage the cluster knows about, a cloud disk, an NFS share, whatever a backend provides. A PersistentVolumeClaim (PVC) is a request for storage that a Pod actually makes; Kubernetes binds the claim to a matching volume. Pods never reference a PV directly, only a PVC, which is what makes the same manifest portable across clusters with completely different underlying storage.

YAML
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: data-pvc
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: fast-ssd
  resources:
    requests:
      storage: 20Gi
Access modeMeaning
ReadWriteOnce (RWO)Mounted read-write by a single Node at a time
ReadOnlyMany (ROX)Mounted read-only by many Nodes
ReadWriteMany (RWX)Mounted read-write by many Nodes, needs a backend that supports it (NFS; most cloud block storage doesn't)
ReadWriteOncePod (RWOP)Mounted read-write by a single Pod, cluster-wide, stricter than RWO

StorageClass & dynamic provisioning

Rather than an admin pre-creating PVs by hand, a StorageClass describes a "profile" of storage, an SSD tier, a specific cloud disk type, and a provisioner that creates a matching PV on demand the moment a PVC asks for it.

YAML
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: fast-ssd
provisioner: pd.csi.storage.gke.io
parameters:
  type: pd-ssd
reclaimPolicy: Retain
volumeBindingMode: WaitForFirstConsumer

volumeBindingMode: WaitForFirstConsumer delays provisioning until a Pod actually claims the volume, so the disk gets created in the same zone the Pod is scheduled to, skip this and you can end up with a Pod and its volume in different availability zones, unable to attach.

๐Ÿ“˜
CSI

The Container Storage Interface is what makes a provisioner like this pluggable at all, the same pattern as CRI for runtimes and CNI for networking: a standard interface with swappable drivers behind it.

Waypoint 06 ยท Data & Config

Configuration & Secrets Basic

The 12-factor instinct of keeping config out of the image carries over directly, ConfigMaps and Secrets are just API objects instead of a .env file, which means they can be mounted, versioned, and updated independently of the Pod spec.

ConfigMap

YAML
apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
data:
  LOG_LEVEL: "info"
  MAX_CONNECTIONS: "100"

Secret

YAML
apiVersion: v1
kind: Secret
metadata:
  name: db-credentials
type: Opaque
data:
  username: YWRtaW4=
  password: c3VwZXJzZWNyZXQ=
๐Ÿ”’
Base64 is not encryption

Secret values are base64-encoded, not encrypted, anyone with API read access to the Secret can trivially decode them. Enable encryption at rest for etcd, lock access down with RBAC, and for anything sensitive in production consider an external secrets manager (Vault, or the External Secrets Operator syncing from one) rather than trusting the Secret object alone.

Consuming both

Either object can be injected as environment variables or mounted as files, files are the better choice for anything that might rotate, since a mounted Secret updates in place without a Pod restart (the app still needs to notice and reload it).

YAML
envFrom:
  - configMapRef:
      name: app-config
env:
  - name: DB_PASSWORD
    valueFrom:
      secretKeyRef:
        name: db-credentials
        key: password
Waypoint 07 ยท Operations

Scheduling & Resource Management Intermediate

The scheduler's job is bin-packing: fit every Pod onto a Node that can satisfy it. What "satisfy" means is entirely controlled by requests, limits, affinity rules, and taints.

Requests & limits

Requests are what the scheduler guarantees when placing a Pod, a Node is only considered if its unreserved capacity covers the request. Limits are a hard ceiling enforced at runtime: exceed the CPU limit and you get throttled; exceed the memory limit and the container is OOMKilled.

YAML
resources:
  requests:
    cpu: "500m"
    memory: "512Mi"
  limits:
    cpu: "1"
    memory: "1Gi"

Leave requests unset and the scheduler treats the Pod as needing nothing, it can land anywhere, including a Node with no real headroom, and start starving neighbors. Leave limits unset, and a single runaway process can take an entire Node down with it. Neither is a safe default to skip.

Quality of Service classes

How you set requests and limits determines eviction priority when a Node runs low on resources:

ClassHow you get itEviction order
Guaranteedrequests == limits, for every containerEvicted last
BurstableRequests set, but below limits (or limits unset)Evicted after BestEffort
BestEffortNo requests or limits set at allEvicted first

Affinity & anti-affinity

nodeSelector is the blunt instrument, exact label match, nothing more. Affinity is the expressive version: required (hard constraint) or preferred (soft, weighted) rules, against either Node labels or other Pods.

YAML
affinity:
  nodeAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
      nodeSelectorTerms:
        - matchExpressions:
            - key: disktype
              operator: In
              values: ["ssd"]
  podAntiAffinity:
    preferredDuringSchedulingIgnoredDuringExecution:
      - weight: 100
        podAffinityTerm:
          labelSelector:
            matchLabels:
              app: web-app
          topologyKey: kubernetes.io/hostname

That anti-affinity rule tells the scheduler: prefer spreading web-app replicas across different Nodes rather than stacking them on one, so a single Node failure doesn't take out every replica at once.

Taints & tolerations

Affinity pulls Pods toward Nodes; taints push them away unless a Pod explicitly tolerates the taint. This is how you reserve GPU Nodes, or keep workloads off control-plane Nodes, without maintaining an allow-list everywhere else.

BASH
kubectl taint nodes gpu-node-1 dedicated=gpu:NoSchedule
YAML
tolerations:
  - key: "dedicated"
    operator: "Equal"
    value: "gpu"
    effect: "NoSchedule"

PodDisruptionBudget

Affinity and taints control placement; a PodDisruptionBudget (PDB) protects availability during voluntary disruptions, Node drains, cluster upgrades, cluster-autoscaler scale-downs. It doesn't stop involuntary failures like a Node crashing, only operations that ask permission first.

YAML
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: web-app-pdb
spec:
  minAvailable: 2
  selector:
    matchLabels:
      app: web-app

With this in place, kubectl drain refuses to evict a web-app Pod if doing so would drop the running count below 2, it waits, or you intervene, instead of silently taking the service down.

Waypoint 08 ยท Operations

Autoscaling Intermediate

Three independent autoscalers cover three independent questions: how many replicas, how much resource per replica, and how many Nodes.

Horizontal Pod Autoscaler (HPA)

Watches a metric, CPU/memory by default, or a custom metric via an adapter like the Prometheus Adapter, and adjusts replicas on a Deployment or StatefulSet to keep it near a target.

YAML
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: web-app-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: web-app
  minReplicas: 2
  maxReplicas: 10
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 70

This requires the Metrics Server (or a custom metrics pipeline) to already be running, the HPA itself doesn't collect metrics, it just reads them. Requests must be set on the target resource, since utilization is calculated against them, not against limits.

Vertical Pod Autoscaler (VPA)

Adjusts a Pod's requests/limits instead of its replica count, based on observed usage over time. Its default "Auto" mode recreates Pods to apply new values, which makes it awkward to run alongside HPA on the same metric, they'd fight each other. Common pairing: VPA in recommendation-only mode to size requests correctly, HPA doing the actual scaling.

Cluster Autoscaler

Operates one level up: watches for Pods that can't be scheduled anywhere due to insufficient Node capacity and adds Nodes to the underlying cloud node pool, then removes Nodes that have been underutilized for a while and can be safely drained. This is what actually grows and shrinks the fleet the other two autoscalers are scheduling onto.

๐Ÿ’ก
Stacking them

HPA changes replica count โ†’ Cluster Autoscaler notices Pods can't fit โ†’ adds Nodes. That's the standard chain; each layer only needs to understand the one below it.

Waypoint 09 ยท Operations

Security & RBAC Advanced

Every request to the API server, kubectl, a controller, a Pod calling out via its ServiceAccount, passes through authentication, then authorization (almost always RBAC), then admission control before it's persisted.

RBAC

Role-Based Access Control grants verbs (get, list, watch, create, update, delete...) on resources (pods, deployments, secrets...), scoped either to a namespace (Role/RoleBinding) or cluster-wide (ClusterRole/ClusterRoleBinding).

YAML
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  namespace: production
  name: pod-reader
rules:
  - apiGroups: [""]
    resources: ["pods"]
    verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: read-pods
  namespace: production
subjects:
  - kind: ServiceAccount
    name: monitoring-sa
    namespace: production
roleRef:
  kind: Role
  name: pod-reader
  apiGroup: rbac.authorization.k8s.io

Default to Role + RoleBinding. Reach for ClusterRole only when the permission genuinely needs to span namespaces, cluster-wide read access is a much bigger blast radius than most workloads actually need.

ServiceAccounts

The identity a Pod uses to talk to the API server. Every namespace has a default ServiceAccount, auto-mounted into every Pod that doesn't specify one, which means a compromised container gets a live token for free unless you turn that off.

YAML
apiVersion: v1
kind: ServiceAccount
metadata:
  name: web-app-sa
  namespace: production
automountServiceAccountToken: false

Pod Security Admission

Enforces baseline hardening at the namespace level via three standards: privileged (unrestricted), baseline (blocks known privilege escalations), and restricted (heavily locked down, non-root, no privilege escalation, dropped capabilities). Applied as a namespace label, replacing the older PodSecurityPolicy object.

BASH
kubectl label namespace production \
  pod-security.kubernetes.io/enforce=restricted

A short hardening checklist

The recurring pattern in real cluster compromises isn't exotic, it's a handful of settings left at their permissive defaults:

  • Overly broad RBAC, service accounts with cluster-admin or wildcard verbs/resources they don't need
  • hostPath mounts or privileged: true containers, which can allow escaping to the underlying Node
  • Default ServiceAccount tokens auto-mounted into Pods that never call the API
  • No default-deny NetworkPolicy, so lateral movement between namespaces is unrestricted
  • Secrets read broadly instead of scoped to exactly the workloads that need them

Every one of these maps to a control covered earlier in this guide, RBAC, NetworkPolicy, Pod Security Admission, the work is applying them by default instead of only after an incident.

Waypoint 10 ยท Operations

Observability Intermediate

Kubernetes' own health signal is narrow, it knows whether a process is running. Everything about whether that process is actually healthy comes from probes you define, plus a metrics and logging stack you bring yourself.

Probes

ProbeOn failureUse it for
livenessProbeContainer is killed and restartedDetecting deadlocks a process can't recover from itself
readinessProbeRemoved from Service endpoints, not restartedTraffic gating during startup, dependency loss, or overload
startupProbeDelays liveness/readiness checks until it passesSlow-starting apps that would otherwise be killed before finishing boot
YAML
livenessProbe:
  httpGet:
    path: /healthz
    port: 8080
  initialDelaySeconds: 15
  periodSeconds: 20
readinessProbe:
  httpGet:
    path: /ready
    port: 8080
  periodSeconds: 10
startupProbe:
  httpGet:
    path: /healthz
    port: 8080
  failureThreshold: 30
  periodSeconds: 10
โš ๏ธ
The classic footgun

A livenessProbe pointed at an endpoint that depends on a downstream service (a database, a queue) will restart your app every time that dependency blips, turning a transient issue into a crash loop. Liveness should only ever check "is this process itself stuck," never "are my dependencies healthy." That's what readiness is for.

Metrics & the observability stack

The Metrics Server is a lightweight, cluster-internal component that powers kubectl top and feeds the HPA, it's not a monitoring solution, just current CPU/memory snapshots. For anything you'd actually want to graph, alert on, or keep history for, Prometheus, Grafana, and Loki form the de facto standard stack in Kubernetes: Prometheus scrapes metrics (often deployed via the kube-prometheus-stack Helm chart, with a Prometheus Operator managing ServiceMonitor CRDs that tell it what to scrape), Grafana visualizes them, and Loki does the same job for logs. If you've already wired these up outside Kubernetes, the mental model transfers directly, only the scrape-target discovery changes.

BASH
kubectl top nodes
kubectl top pods -n production
Waypoint 11 ยท Ecosystem

Package Management with Helm Intermediate

Helm is to Kubernetes manifests what a package manager is to your OS, a way to template, version, parameterize, and install a bundle of YAML as a single unit called a release.

Charts, values, releases

A chart is a directory of templated manifests plus a schema for the values that fill them in. A release is one installed instance of a chart with a specific set of values, you can install the same chart multiple times under different release names for different environments.

TEXT
mychart/
โ”œโ”€โ”€ Chart.yaml          # name, version, dependencies
โ”œโ”€โ”€ values.yaml         # default values
โ”œโ”€โ”€ templates/
โ”‚   โ”œโ”€โ”€ deployment.yaml
โ”‚   โ”œโ”€โ”€ service.yaml
โ”‚   โ””โ”€โ”€ _helpers.tpl    # reusable template snippets
โ””โ”€โ”€ charts/             # bundled sub-chart dependencies
BASH
helm repo add bitnami https://charts.bitnami.com/bitnami
helm repo update
helm install my-release bitnami/nginx --values values.yaml
helm upgrade my-release bitnami/nginx --set replicaCount=3
helm rollback my-release 1
helm uninstall my-release

Every helm upgrade is itself versioned, which is what makes helm rollback possible, rollout history for an entire bundle of resources, the same idea as kubectl rollout undo scaled up from one Deployment to a whole application.

๐Ÿ“˜
Helm vs Kustomize

Helm templates YAML with a templating language and packages it as a versioned, distributable chart. Kustomize (built into kubectl apply -k) takes the opposite approach, plain YAML overlays and patches, no templating language, no packaging, layering environment-specific diffs on top of a shared base. Reach for Kustomize when you own all the YAML and just need per-environment variation; reach for Helm when you're distributing or consuming a reusable, parameterized package.

Waypoint 12 ยท Ecosystem

Advanced Topics Advanced

The API you've been using, Pods, Deployments, Services, isn't fixed. It's extensible, and that extension mechanism is what most of the Kubernetes ecosystem is actually built on.

Custom Resource Definitions (CRDs)

A CRD registers a brand-new object type with the API server, your own kind, with its own schema, that kubectl get and kubectl apply treat exactly like a built-in resource.

YAML
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
  name: databases.example.com
spec:
  group: example.com
  versions:
    - name: v1
      served: true
      storage: true
      schema:
        openAPIV3Schema:
          type: object
          properties:
            spec:
              type: object
              properties:
                engine:
                  type: string
                version:
                  type: string
                storageSize:
                  type: string
  scope: Namespaced
  names:
    plural: databases
    singular: database
    kind: Database

On its own, a CRD is just a schema, creating a Database object stores it in etcd and does nothing else. It needs a controller watching it to actually mean something.

Operators

An Operator is that controller: a CRD plus a custom control loop that encodes real operational knowledge about running a specific piece of software. Create a Database custom resource and the Operator behind it might provision a StatefulSet, generate credentials into a Secret, schedule backup CronJobs, and handle failover, the same watch/diff/act loop from Waypoint 02, aimed at a problem far more specific than "keep N Pods running." The Prometheus Operator (managing Prometheus, Alertmanager, and ServiceMonitor objects) and cert-manager (issuing and renewing TLS certificates via a Certificate CRD) are two of the most widely deployed examples.

Admission Controllers

Requests hit admission control after authentication and authorization but before being persisted to etcd, the last point where a request can be modified or rejected. Built-in controllers handle things like namespace lifecycle and resource quotas; dynamic admission adds your own logic via webhooks the API server calls out to.

YAML
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingWebhookConfiguration
metadata:
  name: require-resource-limits
webhooks:
  - name: require-limits.example.com
    clientConfig:
      service:
        name: webhook-service
        namespace: platform
        path: "/validate"
    rules:
      - apiGroups: [""]
        apiVersions: ["v1"]
        operations: ["CREATE"]
        resources: ["pods"]
    admissionReviewVersions: ["v1"]
    sideEffects: None

Mutating webhooks run first and can rewrite the object (injecting a sidecar, setting a default); validating webhooks run after and can only accept or reject. Service meshes use exactly this mechanism to auto-inject their sidecar proxy into every new Pod without anyone editing a single Deployment.

Service Mesh, briefly

Istio and Linkerd inject a sidecar proxy (Envoy, for Istio) into every Pod, transparently intercepting all traffic in and out. That buys automatic mutual TLS between services, fine-grained traffic splitting for canary releases, and rich L7 observability, all without touching application code. The trade-off is real: another proxy hop per request, more moving parts to operate, another thing that can misconfigure. Worth adopting when you specifically need zero-trust mTLS or progressive delivery across many services; overkill if NetworkPolicy and an Ingress controller already cover what you need.

Waypoint 13 ยท Reference

kubectl Cheat Sheet Reference

The commands you'll actually type every day, grouped by what you're trying to do.

Cluster & context

kubectl cluster-infoControl plane endpoint and core service addresses
kubectl config get-contextsList known cluster/user/namespace combos
kubectl config use-context my-clusterSwitch active cluster
kubectl config set-context --current --namespace=productionSet the default namespace for the current context

Viewing & inspecting

kubectl get pods -AAll Pods, every namespace
kubectl get all -n productionEvery common object type in a namespace
kubectl get pods -o wideInclude Node and IP columns
kubectl describe pod web-app-7d9f8c-a1b2cFull status, events, and recent history for one object
kubectl get events --sort-by=.lastTimestampCluster events, most recent last

Logs & exec

kubectl logs web-app-7d9f8c-a1b2c -fStream logs from a Pod
kubectl logs web-app-7d9f8c-a1b2c -c app --previousLogs from the last crashed container instance
kubectl exec -it web-app-7d9f8c-a1b2c -- /bin/shInteractive shell inside a running container

Apply & manage

kubectl apply -f deployment.yamlCreate or update from a manifest, declaratively
kubectl diff -f deployment.yamlPreview what apply would change
kubectl delete -f deployment.yamlRemove everything defined in a manifest

Debugging

kubectl port-forward svc/web-app 8080:80Tunnel a local port to a Service
kubectl debug web-app-7d9f8c-a1b2c -it --image=busyboxAttach a debug container to a running Pod
kubectl top pods -n productionLive CPU/memory usage per Pod

Rollouts

kubectl rollout status deployment/web-appWatch a rollout to completion
kubectl rollout undo deployment/web-appRoll back to the previous revision
kubectl rollout restart deployment/web-appForce a rolling restart with no spec change
Waypoint 14 ยท Reference

Best Practices & Next Steps Reference

A short list of defaults that separate a cluster that works from one that works reliably, most of it is just consistently applying what's already in this guide.

Always set requests & limits

Unset requests break scheduling fairness; unset limits let one Pod take down a Node.

Define real liveness/readiness probes

Keep liveness checking the process itself, never a downstream dependency.

Pin image tags

Never :latest in production: you lose the ability to know what's actually running, or to roll back to it.

Namespace by environment or team

Cheap isolation boundary for RBAC, quotas, and NetworkPolicy alike.

Default-deny NetworkPolicy

Then add explicit allows: far easier to reason about than an open mesh with occasional blocks.

PodDisruptionBudgets on anything that matters

So a Node drain or upgrade can't silently take a service below its minimum replica count.

Least-privilege RBAC

Start from nothing and add verbs/resources as needed, not the other way around.

Manage cluster state through Git

ArgoCD or Flux applying from a repo, rather than ad-hoc kubectl apply against production.

Where to go from here

The fastest way to make any of this stick is a disposable local cluster, kind or k3d spin one up in containers in seconds, minikube if you want something closer to a full VM. Rebuild every example in this guide by hand in one before trusting yourself against a real cluster. If you want a structured target to work toward afterward, the CKA (Certified Kubernetes Administrator) exam covers almost exactly the surface area of this guide, hands-on, against a live cluster, under a time limit.