Kubernetes: Keeping Containers Alive at 3 AM

You deploy your app in a container. It crashes at 3 AM. No one is awake to restart it.

Kubernetes solves this. It watches your containers, restarts them when they crash, routes traffic to healthy ones, and scales them up when traffic spikes. You write YAML files describing what you want. Kubernetes makes it happen.

Why Orchestration?

You can run containers manually. Docker works fine. Start a container, map ports, done.

But production is messy. Containers crash. Servers die. Traffic spikes. You need 5 copies of your API but only 2 of your database. You want zero-downtime deployments. Doing this manually means writing bash scripts, setting up monitoring, and hoping nothing breaks while you sleep.

Pods: The Smallest Unit

Kubernetes doesn't manage containers directly. It manages pods.

A pod is one or more containers that run together on the same machine, sharing network and storage. Usually one container per pod, but sometimes you need sidecars: a logging agent, a proxy, something that helps the main container.

Containers in a pod can talk to each other on localhost. They share volumes. They start and stop together. If you need two containers that are tightly coupled, put them in the same pod.

Services: Finding Things in the Cluster

A Service is a stable endpoint that routes traffic to pods. It has a fixed IP and DNS name. Behind that IP, Kubernetes load balances across all matching pods.

Services use selectors to find pods. You label your pods app=backend, and the Service selects all pods with that label. When new pods appear with the label, the Service includes them. When pods die, the Service removes them.

There are different Service types:

• ClusterIP: Internal only. Other pods can reach it, external traffic can't.
• NodePort: Exposes the Service on a port on every node. External traffic can hit any node's IP on that port.
• LoadBalancer: Creates an external load balancer (in cloud environments) with a public IP.

Most services are ClusterIP. You use LoadBalancer for the entry point to your cluster.

Deployments: Rolling Updates Without Downtime

Kubernetes uses rolling updates. It brings up new pods gradually and removes old pods only when new ones are healthy.

You control the speed with two settings:

• maxSurge: How many extra pods to create during the update (e.g., 25% = if you want 10 pods, create up to 12 during rollout)
• maxUnavailable: How many pods can be down during the update (e.g., 25% = at most 2 of 10 can be unavailable)

If the new version fails health checks, the rollout stops. The bad pods never receive traffic. Old pods keep serving. You fix the bug and try again.

What if I deploy a bad version and don't notice until later?

Rollbacks. Kubernetes keeps the previous ReplicaSet around. You run kubectl rollout undo and it reverses the deployment. The old version comes back using the same rolling update process.

Scaling: Reacting to Load

Horizontal Pod Autoscaler (HPA) watches metrics and adjusts replicas automatically.

You set a target metric: "Keep CPU at 70%." The autoscaler does math. If current CPU is 90%, it scales up. If current CPU is 40%, it scales down.

The formula is roughly: desiredReplicas = currentReplicas * (currentMetric / targetMetric). If you have 3 pods at 90% CPU and you want 70%, you get 3 * (90/70) ≈ 4 pods.

Autoscaling works with custom metrics too: requests per second, queue depth, whatever you expose via Prometheus or the metrics API.

Self-Healing: The Cluster Fixes Itself

Kubernetes constantly watches for problems and fixes them.

Pod crashed? Start a new one. Node died? Move the pods to another node. Container is stuck but not technically crashed? Kill it and restart.

Kubernetes uses three types of health checks:

• Liveness probes: Is the container alive? If not, kill and restart it.
• Readiness probes: Is the container ready to serve traffic? If not, remove it from the load balancer but don't kill it.
• Startup probes: Give slow-starting containers more time before liveness checks begin.

A typical liveness probe: HTTP GET to /healthz every 10 seconds. If it fails 3 times in a row, kill the container. The ReplicaSet notices the pod is missing and creates a replacement.

Readiness is different. A container might be alive but temporarily unable to serve traffic: it's warming up caches, waiting for a database, whatever. Readiness probes remove the pod from Service endpoints but leave it running.

Scheduling: Deciding Where Pods Run

The scheduler looks at constraints:

• Does the node have enough CPU and memory?
• Does the pod require specific hardware (GPU, SSD)?
• Are there affinity rules (run near other pods or away from them)?
• Are there taints (nodes rejecting certain pods)?

Node affinity pulls pods toward certain nodes. You can say "prefer nodes with SSDs" or "require nodes in us-east-1." Pod affinity controls relationships between pods: "run my cache pod on the same node as my app pod" or "spread database replicas across nodes so one node failure doesn't kill them all."

Taints and tolerations work the opposite way. A node says "I have taint=gpu:NoSchedule." Only pods that tolerate that taint can run there. This keeps GPU nodes reserved for GPU workloads.

If a pod can't be scheduled (not enough resources, no matching nodes), it stays Pending. You see this in kubectl get pods and have to add capacity or adjust the pod's requirements.

ConfigMaps and Secrets: Configuration Without Rebuilding

ConfigMaps store configuration as key-value pairs. Secrets store sensitive data (passwords, tokens) with base64 encoding and access controls.

You can mount ConfigMaps as files or inject them as environment variables.

envFrom:
  - configMapRef:
      name: app-config

Or mount as a volume:

volumes:
  - name: config
    configMap:
      name: app-config
volumeMounts:
  - name: config
    mountPath: /etc/config

Secrets work the same way but with restricted RBAC. Not everyone can read secrets. In managed Kubernetes, secrets are often encrypted at rest and integrated with cloud KMS.

In Practice

Real clusters combine these pieces.

You write a Deployment that specifies pod template, replica count, and update strategy. The Deployment creates a ReplicaSet that ensures the right number of pods exist. A Service provides a stable endpoint. An HPA watches metrics and scales replicas. ConfigMaps hold non-sensitive config, Secrets hold credentials.

When you deploy:

1. Kubernetes schedules pods onto nodes with enough resources
2. Kubelet on each node starts containers
3. Liveness and readiness probes begin
4. Service endpoints update when pods become ready
5. If pods crash, they restart
6. If nodes die, pods move
7. If traffic spikes, HPA scales up

You don't write bash scripts to do this. You write YAML describing the desired state. The control plane watches actual state and reconciles differences.

Controllers and Reconciliation

The pattern behind all of this is the reconciliation loop.

Every Kubernetes controller watches resources and ensures actual state matches desired state. The Deployment controller watches Deployments and creates ReplicaSets. The ReplicaSet controller watches ReplicaSets and creates Pods. The scheduler watches Pods and assigns them to nodes.

Each controller runs a loop:

1. Watch for changes
2. Compare desired state to actual state
3. Take action to reconcile (create, update, delete resources)
4. Repeat

If you manually delete a pod, the controller doesn't care why it's gone. It just sees "I want 3, I have 2" and creates another. This is why killing pods doesn't permanently remove them. You have to delete the Deployment or scale it to 0.

Summary

Kubernetes manages containers at scale. Instead of manually starting, stopping, and monitoring containers, you describe what you want and Kubernetes maintains it.

Here's what you've learned:

• Pods are the unit of deployment, usually one container per pod
• Services provide stable networking to find pods
• Deployments manage rolling updates and rollbacks
• HPA scales replicas based on metrics
• Self-healing restarts crashed containers and reschedules pods from dead nodes
• The scheduler places pods on nodes based on resources
• ConfigMaps and Secrets separate configuration from code

When someone says "we use Kubernetes," they mean: containers wrapped in Pods, managed by Deployments, exposed via Services, scaled by HPA, scheduled across nodes, with configs in ConfigMaps and credentials in Secrets. All of it watched by controllers that fix problems automatically.

Learning Kubernetes means learning the primitives and how they compose. Pods, Services, Deployments, and ConfigMaps cover 80% of what you need. The rest is solving specific problems: persistent storage with PersistentVolumes, batch jobs with Jobs and CronJobs, cluster networking with Ingress, access control with RBAC.