Module 16 — How Containers Work

Updated 3 September 2026

This is the capstone. Every module before this taught a piece of Linux; a container is those pieces assembled into one idea — and the single most important sentence in modern DevOps is: a container is just a process. Not a tiny virtual machine, not a box, not magic — an ordinary Linux process that the kernel has been told to show a restricted view of the world (namespaces), cap the resources of (cgroups), and give its own root filesystem (an image). Understand that, and Docker, Kubernetes, and "works on my machine" all stop being mysteries. This module builds a container by hand from kernel primitives so you never have to take Docker on faith again.

Legend used throughout: 🧠 concept → 🧩 analogy → 🧪 exercise → ✅ expected result (hidden) → 🎯 interview questions (hidden)

Before you start

This module composes the whole track. You need Module 8 — Processes (a container's main process is PID 1 — signals and zombies matter), Module 2 — The Filesystem (mounts and the root filesystem), Module 4 — Users, Groups, and Permissions (what "root" means — and how container-root differs), Module 13 — Networking (each container gets its own network stack), and the cgroups you met in Module 14 — Logs, Troubleshooting, and Performance. Tools: an Ubuntu 24.04 machine with util-linux (unshare, lsns — already present) and busybox-static for the rootfs demo; Docker optional. Every exercise here is safe — we create namespaces and cgroups that affect only our own throwaway processes.

Goes deeper: the kernel theory beneath this module — how namespaces and cgroups are actually implemented, and the isolation primitives (capabilities, seccomp) that make containers safe — are two full modules in the Operating Systems track: OS 12 (Namespaces, cgroups & Containers) and OS 13 (Isolation: Capabilities & seccomp). This module is the hands-on operator's build-it-yourself view; those are the mechanism.

This page uses Mermaid diagram blocks. Notion shows them as code by default — click the block and set it to Preview to see the diagram. This reminder appears once per page.

Part A — The big idea: a container is just a process

A1. What a container actually is — and how it differs from a VM

Official docs: What is Docker? (Docker Docs) · namespaces(7)

🧠 A virtual machine virtualizes hardware: a hypervisor gives each VM its own emulated CPU, memory, and disks, and each VM boots a complete guest operating system with its own kernel. That's strong isolation, but it's heavy — a full OS per VM, gigabytes of RAM, tens of seconds to boot. A container virtualizes the operating system instead: it is a normal process running directly on the host's kernel, which the kernel has been asked to isolate. No guest kernel, no boot, no emulated hardware — a container "starts" as fast as launching a program, because launching a program is exactly what it is. This is why you can run hundreds of containers on a laptop but only a handful of VMs.

🧠 The kernel gives that process three restrictions, and those three are the container: namespaces control what it can see (its own process list, network, filesystem mounts, hostname), cgroups control what it can use (CPU, memory, PIDs), and an image provides the root filesystem it runs against. Docker, Podman, and containerd are conveniences that assemble these kernel features and ship the image — but the isolation itself is done by the Linux kernel, not by Docker.

Counter-intuitive: from the kernel's point of view there is no "inside the container" — the isolation is an illusion projected at the process, not a wall built around it. People picture a container as a sealed box the process lives in. But run ps on the host and you'll see the container's processes sitting right there as ordinary PIDs, owned by an ordinary UID, schedulable like any other. The kernel is running one shared process table; the container's process simply has a namespaced view that hides everything outside its group and renumbers what's left. Nothing "contains" it — the kernel just answers its questions selectively. This is the whole reason containers are lightweight (no second kernel to lie to it) and the whole reason they're less isolated than VMs (one shared kernel is the only thing standing between a container and the host — Part D).

Real-world analogy — a one-way-mirror interview room

A suspect in an interview room believes the room is the whole world: four walls, one table, no windows. In reality they're in a normal room inside a normal building, and the investigators on the other side of the mirror can see them perfectly, walk in any time, and are using the same air conditioning and power as every other room. The suspect's restricted view is the isolation — not a fortress, just controlled information. From the outside, it's an ordinary room among many.

Where the analogy stops working. The suspect and the investigators still breathe the same air through the same vents — and that shared infrastructure is precisely the container's weakness: they share the host kernel, so a flaw in that shared kernel lets the "suspect" affect the whole building (a container escape), which a true VM's separate foundation would prevent. Also, the room's walls are physical and fixed, whereas a container's "walls" are just the kernel choosing what to reveal, reconfigurable in an instant with a syscall. The analogy captures the view-based isolation but understates how thin — and how software-defined — the boundary really is.

🧪 Exercise A1.1 — Every process already has namespaces (read-only, safe anywhere)

bash

ls -l /proc/self/ns/          # the namespaces THIS shell belongs to — you're already in a full set
# a "container" just means a process whose namespaces differ from the host's; yours match the host's

✅ Expected result — click to reveal

javascript

lrwxrwxrwx 1 root root 0 cgroup -> cgroup:[4026531835]
lrwxrwxrwx 1 root root 0 ipc -> ipc:[4026531839]
lrwxrwxrwx 1 root root 0 mnt -> mnt:[4026531832]
lrwxrwxrwx 1 root root 0 net -> net:[4026531833]
lrwxrwxrwx 1 root root 0 pid -> pid:[4026531836]
lrwxrwxrwx 1 root root 0 pid_for_children -> pid:[4026531836]
lrwxrwxrwx 1 root root 0 time -> time:[4026531834]
lrwxrwxrwx 1 root root 0 time_for_children -> time:[4026531834]
lrwxrwxrwx 1 root root 0 user -> user:[4026531837]
lrwxrwxrwx 1 root root 0 uts -> uts:[4026531838]

What to read out of it:

Every process on Linux already belongs to a namespace of each type — there is no "no namespace." Your shell is in the host's namespaces (the numbers [4026531835]… are the kernel's IDs for them). A container is nothing more than a process placed in new, different namespaces of some of these types.
That's the entire trick demystified: "containerizing" a process means "give it a fresh pid/mnt/net/uts/… namespace instead of sharing the host's." The eight types here (cgroup, ipc, mnt, net, pid, user, uts, and time) are the complete menu the kernel offers — Docker picks from exactly this list. (The pid_for_children and time_for_children entries aren't extra types — they point to the pid/time namespace a newly-forked child will land in, which can differ from the process's own right after it calls unshare.)
Notice these are entries under /proc/self/ — the same /proc process interface from Module 2 — The Filesystem. Namespaces aren't a bolt-on product; they're a core kernel feature exposed through the filesystem like everything else in Linux.

A2. The three primitives — namespaces, cgroups, and the rootfs

Official docs: namespaces(7) · cgroups(7)

🧠 A container is the sum of exactly three kernel mechanisms — memorise this trinity and everything else follows:

🧠 Namespaces = what it can SEE (isolation). Separate views of processes, network, mounts, hostname, users. cgroups = what it can USE (limits & accounting). Caps on CPU, memory, process count, I/O. The root filesystem = what it runs against. An image unpacked into a directory that becomes the container's / via chroot/pivot_root. Give a process new namespaces + a cgroup + a rootfs and exec it — that is a container, with or without Docker.

Diagram — the anatomy of a container (set this block to Preview). One shared kernel; each container is a process with its own view, limits, and filesystem.

Diagram source

flowchart TD
    K["Host Linux kernel (ONE, shared by all)"]
    K --- H["Host processes<br>(host namespaces)"]
    K --- C1["Container A = a process"]
    K --- C2["Container B = a process"]
    C1 --- N1["namespaces: what it SEES<br>(pid/net/mnt/uts/...)"]
    C1 --- G1["cgroup: what it USES<br>(cpu/mem/pids caps)"]
    C1 --- R1["rootfs: image layers<br>as its /"]
    C2 --- N2["namespaces: own view"]
    C2 --- G2["cgroup: own limits"]
    C2 --- R2["rootfs: own image"]

🧪 Exercise A2.1 — Make your own UTS namespace and change its hostname (safe — the change exists only inside the new namespace)

bash

hostname                                                  # the host's hostname
unshare --uts --map-root-user sh -c 'hostname isolated-demo; hostname'   # new UTS namespace, rename inside it
hostname                                                  # host hostname is UNCHANGED — proof of isolation

✅ Expected result — click to reveal

javascript

vm
isolated-demo
vm

What to read out of it:

Inside the new UTS namespace, hostname isolated-demo worked and hostname returned isolated-demo. Back on the host, hostname still says vm. You just did what docker run --hostname does — gave a process its own hostname that the rest of the system can't see and isn't affected by.
unshare is the raw kernel tool (from util-linux) that creates new namespaces and runs a program in them — it's the primitive Docker calls under the hood. --uts asks for a new UTS namespace; --map-root-user sets up a user namespace so you can do this without being real root (more in B2).
This is the single most clarifying container exercise: you isolated one property (the hostname) with one flag, no Docker involved. A real container just does this for several namespace types at once, then adds a cgroup and a rootfs.

Part A — Interview questions

🎯 "What is the difference between a container and a virtual machine?" — asked verbatim in Turing's Docker questions (Q1) and KodeKloud's 2026 set (Q2)

A VM virtualizes hardware: a hypervisor runs each VM with its own full guest OS and kernel, giving strong hardware-level isolation at the cost of weight (GBs of RAM, slow boot). A container virtualizes the OS: it's just a process on the host, sharing the host kernel, isolated with namespaces (what it can see) and cgroups (what it can use). Containers are far lighter and start in milliseconds, but share one kernel, so isolation is weaker than a VM's.

The details that separate candidates: the crisp formulation — "a container is just a process; a VM is a whole machine" — and naming the trade-off explicitly: containers trade isolation strength for density and speed. Mentioning that the shared kernel is both the efficiency win and the security limit (Part D) shows real understanding, not a memorised table.

🎯 "What is a container?" — Turing's Docker set

A container is a standard Linux process (or process tree) running on the host kernel that has been given an isolated view of the system via namespaces, resource limits via cgroups, and its own root filesystem from an image. It packages an application together with all its userspace dependencies (libraries, binaries, files) so it runs the same anywhere the same kernel is available — without carrying a whole guest OS.

The details that separate candidates: resisting the "lightweight VM" phrasing and instead naming the three kernel primitives (namespaces + cgroups + rootfs). Bonus: noting the image bundles userspace but not the kernel, which is exactly why "works on my machine" mostly disappears (same userspace) yet architecture/kernel differences can still bite (Part D).

Part B — Namespaces: what a process can see

B1. The PID namespace — becoming PID 1

Official docs: pid_namespaces(7) · unshare(1)

🧠 The PID namespace gives a process its own process-ID space. The first process in a new PID namespace becomes PID 1 and can only see itself and its descendants — the host's hundreds of processes simply don't exist in its view. This is why, when you docker exec into a container and run ps, you see a handful of processes starting at PID 1, not the host's process list. The container's main process is PID 1 of its namespace.

🧠 That PID-1 role is not cosmetic — it carries the special init responsibilities from Module 8 — Processes: PID 1 must reap zombie children, and the kernel gives PID 1 no default signal handlers. So a process that never expected to be PID 1 (a shell script, a language runtime) may ignore SIGTERM and fail to shut down cleanly, and may leak zombies.

Trap — a naive PID 1 ignores docker stop and leaks zombies, and people blame Docker. When you run docker stop, Docker sends SIGTERM to PID 1 and waits (default 10s) before SIGKILL. But the kernel does not install default signal handlers for PID 1 — so if your container's PID 1 is a process without an explicit SIGTERM handler (many shells, node app.js, etc.), the SIGTERM is silently dropped and your container takes the full 10-second grace period then gets killed every single time — slow, unclean shutdowns and lost in-flight work. Worse, if PID 1 spawns children and doesn't wait() for them, their zombies (Module 8 — Processes) accumulate with no one to reap them. The fix is a tiny init as PID 1 — tini (Docker's --init flag), or an exec-ing entrypoint so your real process becomes PID 1 and handles signals. "Why does my container take 10 seconds to stop?" is this, almost every time.

🧪 Exercise B1.1 — Enter a PID namespace and watch your shell become PID 1 (safe — the namespace vanishes when the shell exits)

bash

echo "host sees this shell as PID $$"                                    # a normal, large PID on the host
sudo unshare --pid --fork --mount-proc sh -c 'echo "in namespace:"; ps -e'   # new PID namespace + private /proc

✅ Expected result — click to reveal

javascript

host sees this shell as PID 2245
in namespace:
    PID TTY          TIME CMD
      1 ?        00:00:00 sh
      2 ?        00:00:00 ps

What to read out of it:

Inside the new PID namespace, ps -e shows just two processes, and the shell is PID 1 — the host's dozens of processes are invisible. That is exactly what ps shows inside a Docker container. You've reproduced the container process view with one unshare call.
--mount-proc is essential and instructive: ps reads /proc, so without a fresh /proc mounted for the new namespace, ps would show the host's processes and ruin the illusion. This is why a container needs its own mnt namespace and a private /proc — namespaces interlock. --fork is needed because PID 1 must be a child of unshare, not unshare itself.
The shell being PID 1 here is the whole "why do I need tini?" lesson made concrete: this PID 1 has no default signal handlers and would have to reap any zombies itself. In a real container, that responsibility is yours to design for.

B2. The other namespaces — mount, net, user — and lsns

Official docs: namespaces(7) · user_namespaces(7)

🧠 A quick tour of the rest of the menu: the mount (mnt) namespace gives a process its own set of mounts — the basis for a container having its own filesystem view (C2). The network (net) namespace gives it its own network stack — its own interfaces, routing table, and ports; this is why two containers can both bind port 80 without colliding (each has its own :80), and it's the foundation the whole Kubernetes/CNI network model is built on (Module 13 — Networking). The uts namespace isolates the hostname (A2); ipc isolates shared-memory/message queues; and the user (user) namespace is the security keystone below.

Counter-intuitive: "root inside the container" can be a completely unprivileged nobody on the host. The user namespace maps UIDs: it can make your process see itself as uid=0 (root) inside the namespace while the kernel treats it as an ordinary, unprivileged UID outside. So a process can be "root" in its container — install packages, own files, bind low ports within its world — yet if it escapes, it lands on the host as a harmless nobody with no real power. This is the mechanism behind rootless containers, and it's one of the strongest isolation improvements of the last decade: container-root ≠ host-root. (It's also why older/misconfigured setups where container-root is host-root, via --privileged or a shared user namespace, are so dangerous — Part D. The deep mechanics are Module 13 — Isolation: Capabilities & seccomp in the OS track.)

🧪 Exercise B2.1 — List namespaces and prove user-namespace root mapping (safe — all in your own throwaway namespaces)

bash

lsns -t pid -t uts | head                                    # namespaces on the system, by type
readlink /proc/self/ns/user                                  # your shell's user-namespace ID
unshare --user --map-root-user sh -c 'id'                    # inside a new user namespace: who am I?
id -u                                                         # ...and back on the host: your real UID

✅ Expected result — click to reveal

javascript

        NS TYPE NPROCS PID USER COMMAND
4026531836 pid      61   2 root kthreadd
4026531838 uts      61   2 root kthreadd

user:[4026531837]

uid=0(root) gid=0(root) groups=0(root)     <- inside the user namespace
1000                                        <- your real host UID, unchanged

What to read out of it:

lsns lists every namespace by kernel ID (NS), with how many processes are in it (NPROCS) and the lowest-PID process. Each container adds rows here — lsns is how you'd see containers from the host as the namespaces they really are.
The punchline is the two id outputs: inside unshare --user --map-root-user, id reports uid=0(root) — you are "root." But id -u on the host still reports your ordinary UID (e.g. 1000). Same process, two identities: privileged inside its namespace, unprivileged outside. That single fact is the heart of container security done right.
readlink /proc/self/ns/user prints the namespace's ID — the same number lsns uses. Comparing these IDs is how you tell whether two processes share a namespace (same ID) or are isolated (different IDs) — the low-level version of "are these in the same container?"

Now imagine this at 500 hosts

The net namespace is where fleet-scale containers get interesting. Each container gets its own network namespace, connected to the host by a veth pair into a bridge (or, at cluster scale, a CNI plugin like Calico/Cilium wiring pods across nodes with real routable IPs). Kubernetes schedules thousands of these; a pod is just a group of containers sharing one network namespace — which is why containers in a pod reach each other on localhost. None of it is new magic: it's the same net namespace from this exercise, multiplied and automated. Understanding that "a pod is a shared net namespace" and "container networking is veth + bridge + iptables/eBPF" is what turns Kubernetes networking from mystery into plumbing you can reason about.

Part B — Interview questions

🎯 "How does Docker achieve isolation between containers?" — a variant of Turing's Q34 ("How does Docker handle container isolation and security?")

Through Linux kernel namespaces and cgroups (plus capabilities/seccomp, Part D). Namespaces isolate what a container sees — a pid namespace so it has its own process tree (its main process is PID 1), a mnt namespace for its own filesystem, a net namespace for its own interfaces and ports, uts for hostname, ipc, and user for UID mapping. cgroups isolate what it can use — CPU, memory, and PID limits so one container can't starve the others. Docker/containerd just orchestrate these kernel features and provide the image.

The details that separate candidates: naming specific namespaces and what each isolates, not just saying "namespaces and cgroups." Mentioning the user namespace (container-root ≠ host-root) and that isolation is enforced by the kernel, not Docker signals you understand the mechanism rather than the marketing.

🎯 "Why does a container need an init process / why does my container take 10 seconds to stop?"

Because the container's main process runs as PID 1, and the kernel treats PID 1 specially: it installs no default signal handlers and expects PID 1 to reap zombie children. docker stop sends SIGTERM then, after a grace period, SIGKILL. If PID 1 is a process with no SIGTERM handler (a bare shell, node app.js), the SIGTERM is ignored, so every stop waits the full grace period and then hard-kills — slow, unclean shutdowns. And un-reaped children become zombies. The fix is a small init as PID 1 (tini / docker run --init), or exec-ing your real app so it becomes PID 1 and handles signals.

The details that separate candidates: connecting it to PID-namespace semantics and the Module 8 — Processes facts about PID 1 (signal defaults + zombie reaping), and knowing the two fixes (tini/--init, or exec in the entrypoint). This is a favourite because it separates people who run containers from people who understand what a container is.

🎯 "What is a user namespace, and why does it matter for security?"

A user namespace maps user/group IDs between the namespace and the host, so a process can be uid=0 (root) inside its container while the kernel treats it as an unprivileged UID on the host. It matters because it decouples "root in the container" from "root on the machine": if a process running as container-root is compromised or escapes, it lands on the host with no real privileges. It's the basis of rootless containers — running the whole container runtime as a normal user.

The details that separate candidates: the exact framing container-root ≠ host-root, and the corollary — that without a user namespace (e.g. --privileged, or bind-mounting sensitive host paths), container-root effectively is host-root, which is why those configurations are dangerous. Tying it to capabilities/seccomp (Module 13 — Isolation: Capabilities & seccomp) rounds it out.

Part C — cgroups and the image: what it can use, what it runs

C1. cgroups — limiting what a container can use

Official docs: cgroups(7)

🧠 Namespaces isolate visibility; cgroups (control groups) isolate resources. A cgroup is a group of processes with limits and accounting attached: maximum memory, a share of CPU time, a cap on the number of PIDs, I/O bandwidth. docker run -m 512m --cpus 1.5 translates directly into a cgroup with a 512 MB memory limit and a CPU quota. If a container tries to exceed its memory limit, the kernel's OOM killer (Module 14 — Logs, Troubleshooting, and Performance) fires inside that cgroup and kills the offending process — the container dies, the host and other containers are unharmed. That per-cgroup OOM is why "my container keeps restarting" so often means "it hit its memory limit."

🧠 There are two cgroup versions. v1 exposes a separate directory hierarchy per controller (/sys/fs/cgroup/memory/, /sys/fs/cgroup/cpu/, …) with files like memory.limit_in_bytes. v2 (the unified hierarchy, now the default on modern systemd distros) puts all controllers in one tree with files like memory.max and cpu.max. Same idea, cleaner layout — know both names, because you'll meet both in the wild.

🧪 Exercise C1.1 — See the cgroup your own shell is already limited by (read-only, safe anywhere)

bash

cat /proc/self/cgroup                                          # which cgroup(s) this process is in
stat -fc %T /sys/fs/cgroup                                     # cgroup2fs = v2 (unified); tmpfs = v1
# read this process's memory limit (v1 path shown; on v2 it's memory.max in the unified tree):
cat /sys/fs/cgroup/memory/$(grep -oP '(?<=memory:).*' /proc/self/cgroup)/memory.limit_in_bytes 2>/dev/null \
  || cat /sys/fs/cgroup/memory.max 2>/dev/null

✅ Expected result — click to reveal

javascript

9:name=systemd:/
4:memory:/process_api/01a06440-.../claude-code-bash
1:cpu:/
0::/

tmpfs

6265896960

What to read out of it:

Your shell is already in cgroups — you don't need Docker to be in one; every process is. The memory: line shows this shell sits in a memory cgroup, and reading memory.limit_in_bytes returns 6265896960 bytes (~5.8 GB) — a real, enforced ceiling. Allocate past it and this process gets OOM-killed while the rest of the machine is fine. That is precisely the mechanism docker run -m uses.
stat -fc %T /sys/fs/cgroup reports the cgroup filesystem type: tmpfs here means this host runs cgroup v1 (per-controller directories); on a modern default you'd see cgroup2fs and read the limit from memory.max in the single unified tree instead. The exercise tries the v1 path first and falls back to the v2 file — one of them prints your limit.
Put A and C together and you have the whole container recipe minus the rootfs: a process, in its own namespaces (Part B, isolation) and its own cgroup (this exercise, limits). Add a root filesystem (C2) and you've built a container by hand.

C2. The image and the root filesystem — chroot, layers, and copy-on-write

Official docs: chroot(2) · What is Docker? (Docker Docs)

🧠 The third primitive is the root filesystem. A container doesn't use the host's /; it uses the image's filesystem as its own /, via chroot (or its stronger successor pivot_root), which changes what a process considers the root directory. An image is that filesystem, built as a stack of read-only layers: most instructions in a Dockerfile (notably RUN, COPY, ADD) add one layer — FROM starts from the base image's existing layers, and metadata-only instructions like CMD/ENV/LABEL add none, and a union/overlay filesystem (OverlayFS) stacks them into a single view. When a container runs, it gets a thin writable layer on top; reads fall through to the read-only layers, writes go to the top layer — copy-on-write.

🧠 Layers are why images are efficient: identical base layers (say, ubuntu:24.04) are stored once and shared across every image and container that uses them, and Docker's build cache rebuilds only the layers whose inputs changed. It's also why ordering your Dockerfile matters — put rarely-changing steps first so the cache survives.

Trap — the writable layer is ephemeral; data written inside a container dies with the container. Because the writable layer is created fresh per container and destroyed when the container is removed, anything your app writes to the container filesystem is gone the moment the container is replaced — and containers are replaced constantly (every deploy, crash, or scale event). The classic disaster: a database container storing its data on the writable layer, wiped on the next docker run. The fix is volumes (or bind mounts) — storage that lives outside the container's lifecycle and is mounted in. Rule of thumb: containers are cattle, not pets — treat their local filesystem as scratch space, and put anything you care about in a volume or external store.

🧪 Exercise C2.1 — Build a root filesystem by hand and chroot into it (safe — a throwaway directory; chroot only affects the child process)

bash

mkdir -p /tmp/tinyroot/bin
cp "$(which busybox)" /tmp/tinyroot/bin/            # busybox = a whole userland in one static binary
ln -sf busybox /tmp/tinyroot/bin/sh                 # give the rootfs a shell and an ls
ln -sf busybox /tmp/tinyroot/bin/ls
sudo chroot /tmp/tinyroot /bin/ls /                 # inside the chroot, "/" IS tinyroot

✅ Expected result — click to reveal

javascript

bin

What to read out of it:

Inside the chroot, the process's / is /tmp/tinyroot — so ls / shows only bin (the one directory we created), not the host's /etc, /home, /usr, etc. You've given a process a completely different root filesystem — the exact job an image does for a real container. A "container image" is just a much bigger version of tinyroot (a full distro's userland instead of one busybox).
This is why "works on my machine" mostly goes away: the image carries its own /usr, /lib, and dependencies, so the app runs against the files it was built with, not whatever happens to be on the host. The one thing the image does not carry is the kernel — that's shared (Part D), which is the residual reason for the occasional "works on my machine, not in prod."
Real runtimes use pivot_root rather than chroot (chroot alone is escapable by a privileged process — chroot was never designed as a security boundary), and they stack the rootfs from read-only image layers via OverlayFS with a writable top layer. But the essence is exactly what you just did: a process whose / is a directory you prepared.

Now imagine this at 500 hosts

Layer sharing is what makes containers deployable at fleet scale. A 900 MB image built on ubuntu:24.04 doesn't ship 900 MB to every node — the base layers are pulled once per node and cached, and only the small app-specific top layers move on each deploy; identical layers are deduplicated across every image on the host. A registry (Docker Hub, ECR, Harbor) is the shared store nodes pull from. This is why image discipline matters at scale — a shared, minimal base layer (or distroless/alpine) pulled once beats a fat unique image pulled 500 times, and it shrinks both deploy time and attack surface (fewer packages = fewer CVEs, tying back to Module 15 — SSH and Security Hardening).

Part C — Interview questions

🎯 "What is the difference between a Docker image and a container?" — KodeKloud 2026 (Q1); Turing asks the image-vs-layer variant (Q19)

An image is a read-only template — a stack of filesystem layers plus metadata (the frozen filesystem and config your app needs). A container is a running (or stopped) instance of an image, with a thin writable layer on top. One image → many containers, each with its own writable layer and its own namespaces/cgroup. The analogy that lands: image is to container as a class is to an object, or a program on disk is to a process.

The details that separate candidates: naming the writable copy-on-write layer as the thing that makes a container distinct from its image, and that it's ephemeral (→ volumes for persistence). For the image-vs-layer variant: a layer is one filesystem diff produced by one build step; an image is the ordered stack of layers, and layers are immutable and shared across images.

🎯 "What is a Dockerfile, and how do image layers and the build cache work?" — KodeKloud 2026 (Q3, Q4)

A Dockerfile is a text file of build instructions read top to bottom (FROM, RUN, COPY, CMD, …); most instructions produce one image layer. Layers stack (via a union filesystem) into the final image. The build cache works per layer: if an instruction and its inputs are unchanged from a previous build, Docker reuses the cached layer instead of rebuilding it — and the cache is invalidated from the first changed instruction downward, so every layer after a change rebuilds too.

The details that separate candidates: the practical consequence — order instructions from least- to most-frequently-changing (e.g. COPY package.json && RUN install before COPY . .) so a code change doesn't bust the dependency-install cache. That single insight is what separates people who've tuned a Dockerfile from people who've only written one.

🎯 "Where does a container store data, and what happens to it when the container is removed?"

By default a container writes to its thin writable layer, which is created with the container and destroyed with it — so that data is lost when the container is removed or replaced (which happens on every deploy). For data that must survive, use a volume (Docker-managed storage outside the container lifecycle) or a bind mount (a host directory mounted in), or write to an external store (a database service, object storage). The writable layer is scratch space, not storage.

The details that separate candidates: framing containers as immutable and disposable ("cattle, not pets") and knowing why the writable layer is ephemeral (copy-on-write top layer over read-only image layers). The classic red flag they're probing for: running a stateful database on the container filesystem with no volume.

Part D — Putting it together, and the security view

D1. From primitives to a runtime — and why "works on my machine" mostly ends

Official docs: What is Docker? (Docker Docs) · namespaces(7)

🧠 Now assemble the whole thing. What docker run actually does, step by step: pull the image (its read-only layers) from a registry; create new namespaces (pid, mnt, net, uts, ipc, and ideally user); set up a cgroup with the requested limits; set the rootfs to the image's layered filesystem via pivot_root; drop capabilities and apply a seccomp profile (D2); and finally exec your process as PID 1 inside all of that. There's no daemon-magic left over — it's the primitives from Parts A–C, orchestrated. Under Docker sits containerd, and under that runc, the small tool that actually makes the namespace/cgroup/rootfs syscalls, following the OCI (Open Container Initiative) spec so any compliant runtime behaves the same.

🧠 This is why "works on my machine" largely disappears: the image bundles the entire userspace your app depends on — its libraries, binaries, config, the whole /usr — so it runs against the same files everywhere. The one thing the image does not carry is the kernel.

Counter-intuitive: a container can't run a different kernel than the host — which is why "Linux containers on a Mac" is secretly a hidden VM. Because containers share the host kernel, a Linux container needs a Linux kernel to run on. So when you run Linux containers on macOS or Windows, Docker Desktop quietly runs a lightweight Linux virtual machine in the background, and your containers run inside that. There's no such thing as a Windows machine running a Linux container "natively" — the shared-kernel model forbids it. This also bounds what containers can fix: they normalize userspace (libraries, files) but a genuine kernel or CPU-architecture difference (an arm64 image on an amd64 host, a syscall your prod kernel lacks) is exactly the residual "works on my machine, not in prod" that images can't paper over — because the kernel is the one layer they don't ship.

D2. Container security — the shared kernel is the boundary

Official docs: capabilities(7) · deep dive: Module 13 — Isolation: Capabilities & seccomp (OS track — Capabilities & seccomp)

🧠 A VM's isolation boundary is the hypervisor and emulated hardware — very hard to cross. A container's boundary is the kernel's own namespace/cgroup/capability enforcement, and every container on a host shares that one kernel. So a kernel vulnerability can let a container escape to the host in a way a VM's hardware boundary would prevent. That's the fundamental trade-off: containers are lighter because they share the kernel, and less isolated because they share the kernel. You harden against it in layers (the same defense-in-depth mindset as Module 15 — SSH and Security Hardening): drop Linux capabilities so container-root can't do most privileged operations, use a user namespace so container-root maps to an unprivileged host UID, apply a seccomp profile to block dangerous syscalls, run a read-only rootfs, and never use --privileged unless you truly mean "give this container the host."

Trap — docker run --privileged throws almost all of it away; a privileged container is effectively root on the host. --privileged grants the container all capabilities, access to all host devices, and lifts seccomp/AppArmor restrictions — it exists for narrow cases (some CI, hardware access) and is routinely abused as a "make the permission error go away" hammer. A process in a privileged container can load kernel modules, read raw disks, and manipulate the host — a compromise of that container is a compromise of the machine, and often the cluster. Treat --privileged (and its cousins: mounting the Docker socket, --cap-add=SYS_ADMIN, mounting host /) as a loud red flag in any review. The safe default is the opposite: drop capabilities you don't need, add back only the few you do.

Real-world analogy — apartments in one building vs detached houses

VMs are detached houses: each has its own foundation, plumbing, and power. A burst pipe or a break-in in one house doesn't touch the others — strong isolation, but you pay to build a whole house every time. Containers are apartments in one building: fast and cheap to add, each with its own locked door and its own meters (namespaces and cgroups), but they share the building's foundation, wiring, and plumbing — the kernel. Lock each door well (drop capabilities, user namespaces, seccomp) and apartments are perfectly safe for the vast majority of tenants. But a flaw in the shared foundation potentially affects every apartment at once, in a way it never could for detached houses.

Where the analogy stops working. Apartment walls are physical and always present; a container's "walls" are the kernel choosing to enforce separation, and a misconfiguration (--privileged, a shared namespace) can remove a wall entirely while everything still looks normal — there's no physical wall to reassure you. And apartments can't be spun up in milliseconds or destroyed and recreated on every deploy; the container's disposability, which the building analogy can't capture, is central to how it's actually operated. The analogy nails the shared-foundation risk but not the software-defined, ephemeral nature of the walls.

Diagram — why a container escape is a host problem and a VM escape isn't (set to Preview).

Diagram source

flowchart TD
    subgraph VMs["Virtual machines"]
        H1["Hypervisor + emulated hardware = hard boundary"]
        H1 --> GK1["Guest kernel A"] --> A1["App A"]
        H1 --> GK2["Guest kernel B"] --> A2["App B"]
    end
    subgraph CTRs["Containers"]
        SK["ONE shared host kernel = the boundary"]
        SK --> C1["Container A (caps dropped, seccomp, userns)"]
        SK --> C2["Container B"]
        SK -.->|"kernel exploit escapes to host"| ESC["Host compromised"]
    end

🧪 Exercise D2.1 — Inspect a process's capabilities (read-only, safe anywhere)

bash

grep -E 'Cap(Prm|Eff|Bnd)' /proc/self/status        # this process's capability sets (hex bitmasks)
capsh --decode=$(grep CapEff /proc/self/status | awk '{print $2}') | head -1   # decode to names (libcap2-bin)
# a hardened container would show a TINY set here; --privileged shows nearly all of them

✅ Expected result — click to reveal

javascript

CapPrm:	000001fffeffffff
CapEff:	000001fffeffffff
CapBnd:	000001fffeffffff

0x000001fffeffffff=cap_chown,cap_dac_override,...,cap_sys_admin,cap_sys_boot,...,cap_bpf,cap_checkpoint_restore

What to read out of it:

Capabilities slice the old all-or-nothing "root" into ~40 separate privileges (CAP_NET_BIND_SERVICE = bind ports <1024, CAP_SYS_ADMIN = a huge grab-bag, CAP_NET_RAW = raw sockets, …). The hex CapEff here decodes to nearly the full set — this shell is effectively unrestricted root. A well-run container drops almost all of these: Docker's default already removes many, and a hardened setup adds --cap-drop=ALL then --cap-add only what's needed.
CapBnd (the bounding set) is the ceiling — capabilities a process can never regain even via setuid. Shrinking the bounding set is exactly how you contain a container: even if something inside becomes "root," it can only do what its remaining capabilities allow. This is the capability half of "container-root ≠ host-root" (the user-namespace half was B2).
Contrast the two extremes: a hardened container's CapEff might decode to just cap_net_bind_service (or nothing), while --privileged leaves this near-full mask in place. Reading /proc/<pid>/status for Cap* is how you audit what a running container can actually do — a real security check, grounded in Module 13 — Isolation: Capabilities & seccomp.

Part D — Interview questions

🎯 "Are containers more or less secure than VMs, and why?"

Less isolated than VMs, by design. A VM's boundary is the hypervisor and emulated hardware — crossing it is very hard, because each VM has its own kernel. A container's boundary is the shared host kernel's namespace/cgroup/capability enforcement, so a kernel vulnerability can enable a container escape that a VM would contain. In exchange, containers are dramatically lighter and faster. You narrow the gap with layers: drop capabilities, user namespaces (container-root ≠ host-root), seccomp syscall filtering, read-only rootfs, no --privileged, and keeping the host kernel patched (Module 15 — SSH and Security Hardening).

The details that separate candidates: stating the trade-off honestly (shared kernel = lighter and weaker), naming concrete hardening (capabilities/seccomp/userns, not just "use Docker securely"), and noting that because the kernel is shared, host patching is container security — the CVE that matters most may be in the kernel, not the image. For extreme isolation, mention gVisor/Kata (user-space kernel or micro-VM per container).

🎯 "What does docker run --privileged do, and when should you use it?"

It grants the container all Linux capabilities, access to all host devices, and lifts the default seccomp/AppArmor confinement — effectively root on the host. A process in a privileged container can load kernel modules, read raw block devices, and manipulate host resources, so a compromise of it is a compromise of the machine. You should use it almost never — only for narrow, trusted cases like certain CI runners or hardware/GPU access, and even then prefer scoping down (--cap-add the specific capability, --device the specific device) instead of the blanket flag.

The details that separate candidates: knowing it's not a single switch but a bundle (all caps + all devices + relaxed seccomp), reaching for least-privilege alternatives (--cap-add, --device, specific mounts) rather than the hammer, and flagging mounting the Docker socket into a container as a similar "you just gave it the host" mistake. It's the container version of least privilege from Module 15 — SSH and Security Hardening.

🎯 "An image runs fine on your laptop but crashes on the production server. The Dockerfile hasn't changed. What could it be?"

Since the image bundles userspace but shares the host kernel, suspect the things the image doesn't carry. Top candidates: CPU architecture mismatch (an arm64 image built on an Apple-silicon laptop running on amd64 prod, or vice-versa — check docker image inspect/uname -m; use multi-arch builds); a kernel version/feature difference (a syscall or kernel feature present on your newer laptop kernel but not prod's — surfaces via seccomp denials or ENOSYS); different resource limits (prod has a tight memory cgroup → OOM-killed, Part C); missing volumes/config/secrets that exist on your laptop's bind mounts but not in prod; or host-dependent bind mounts / device access.

The details that separate candidates: immediately reasoning from "what isn't in the image?" — kernel and architecture first — rather than re-checking the Dockerfile. It proves you understand the shared-kernel model: images normalize userspace, not the kernel, the arch, or the runtime resource limits.

Part E — Toolkit

E1. Production practice — symptoms and fixes

Symptom	What is really happening	What to run	The fix / next step
Container restarts every few minutes	It's hitting its memory cgroup limit and being OOM-killed, then restarted	docker inspect <c> --format '{{.State.OOMKilled}}'; docker logs; docker stats	Raise -m/--memory or fix the leak; the cgroup OOM ≠ a host OOM (Part C, Untitled)
docker stop takes ~10s then hard-kills; unclean shutdown	PID 1 has no SIGTERM handler, so SIGTERM is ignored until the SIGKILL timeout	Check what PID 1 is (docker top <c>); is the entrypoint a bare shell?	Use --init/tini, or exec the real app in the entrypoint so it becomes signal-handling PID 1 (Part B)
Data vanished after a redeploy	App wrote to the ephemeral writable layer, destroyed with the container	docker inspect <c> → look for Mounts; is there a volume?	Mount a volume/bind mount for stateful paths; never store DB data on the container FS (Part C)
Container needs --privileged "to work"	It wants one capability or device, and --privileged is being used as a shortcut	Read the error; identify the exact capability/device needed	--cap-add=<CAP> or --device=<dev> for the specific need; never blanket --privileged (Part D)
Image runs on laptop, fails in prod (Dockerfile unchanged)	Something the image doesn't carry differs: kernel, CPU arch, cgroup limits, mounts	uname -m / docker image inspect (arch); dmesg/seccomp logs; prod memory limit	Multi-arch build; match/patch host kernel; align resource limits & mounted config (Part D)
ps inside container shows your app as PID 1	Normal — it's the first process in a new PID namespace	docker exec <c> ps -e	Nothing to fix; just design PID 1 for signals + zombie reaping (Part B)

E2. Capstone — four realistic tickets

Attempt each before opening the answer. These are the container scenarios that come up in real on-call and interviews.

🎫 Ticket 1 — "Our API container gets killed and restarted every few minutes in prod, but runs fine locally. Logs just stop mid-request. What's going on?"

This is a per-container OOM kill. "Runs fine locally" + "killed in prod" + "logs stop abruptly" is the signature of hitting a memory cgroup limit that exists in prod but not on your laptop. Confirm: docker inspect <container> --format '{{.State.OOMKilled}}' → true is the smoking gun; docker logs will show the process just ending (SIGKILL leaves no graceful message); docker stats or the orchestrator's metrics show memory climbing to the limit then resetting. On Kubernetes, kubectl describe pod shows Reason: OOMKilled and the restart count.

Why prod not local: locally you ran with no -m, so the container could use all host RAM; prod sets --memory/a K8s limits.memory (as it should). The container's memory cgroup enforces that cap and the kernel OOM-kills inside the cgroup (Part C, Module 14 — Logs, Troubleshooting, and Performance) — the host and neighbours are fine, only your process dies.

Fix: decide whether the limit is too low or the app leaks. Profile actual usage (docker stats, a heap profiler); if the working set genuinely needs more, raise the limit; if it grows unbounded, it's a leak — fix it, because raising the limit just delays the kill. Also make sure the app has a little headroom below the limit (the kernel kills at the limit, not at "nicely warned"). Strong answer names OOMKilled, the cgroup mechanism, and the leak-vs-undersized fork in the road.

🎫 Ticket 2 — "Every deploy, our container takes exactly 10 seconds to stop, and in-flight requests get dropped. Nobody set a 10-second anything."

PID 1 is ignoring SIGTERM. docker stop (and Kubernetes pod termination) sends SIGTERM, waits a grace period (Docker default 10s), then sends SIGKILL. "Exactly 10 seconds, every time" means the SIGTERM is being ignored and the process only dies on the forced SIGKILL — dropping whatever it was doing. The cause (Part B): the container's PID 1 has no default signal handlers, and your entrypoint is probably a shell (sh -c "node app.js") or an app that doesn't trap SIGTERM, so the signal lands on a PID 1 that does nothing with it.

Confirm: docker top <container> — is PID 1 sh/bash wrapping your app, rather than your app itself? If so, the shell is PID 1 and your app is its child, never receiving the signal.

Fix, two options: (1) exec your app in the entrypoint (exec node app.js) so it replaces the shell and becomes PID 1, receiving SIGTERM directly — then handle it to drain connections and exit. (2) Add a real init: docker run --init (injects tini) or bake tini in, so PID 1 is an init that forwards signals and reaps zombies. Then implement graceful shutdown in the app (stop accepting new requests, finish in-flight, exit) so termination is clean well within the grace window. Strong answer ties the 10s to the SIGTERM→grace→SIGKILL sequence and PID-1 signal semantics, and gives both the exec and tini fixes.

🎫 Ticket 3 — "We redeployed the app and all the data in our containerized Postgres is gone. It was there yesterday."

The database was writing to the container's ephemeral writable layer, and the redeploy replaced the container. A container's writable layer is created with the container and destroyed when it's removed (Part C); a deploy does exactly that — docker rm the old, docker run the new — so anything on the container filesystem is wiped. Confirm: docker inspect <old-or-new-container> and look at Mounts — if Postgres's /var/lib/postgresql/data is not backed by a volume or bind mount, the data lived on the writable layer and is unrecoverable from the removed container.

Immediate: if the old container was only stopped not removed, its writable layer may still exist (docker ps -a, docker start/commit to extract) — grab the data now. If it was removed, the data is gone (restore from backup — and this is why backups are non-negotiable for stateful workloads).

Prevent: mount a named volume (or bind mount, or use an external/managed database) for the data path: docker run -v pgdata:/var/lib/postgresql/data .... Volumes live outside the container lifecycle, so redeploys don't touch them. Broader lesson (Part C): treat containers as disposable ("cattle, not pets") and keep all state in volumes or external stores — arguably, run stateful databases on managed services or with a StatefulSet + persistent volumes rather than a bare container at all. Strong answer names the ephemeral writable layer, checks Mounts, and prescribes volumes + the stateless-container principle.

🎫 Ticket 4 — "A team's Dockerfile/compose runs the container with --privileged and mounts /var/run/docker.sock. They say it's the only way their tool works. Sign-off?"

Do not sign off as-is — this is two of the most dangerous container patterns at once. --privileged grants all capabilities + all host devices + relaxed seccomp — effectively root on the host (Part D); mounting the Docker socket lets the container tell the host's Docker daemon to start new containers (including a privileged one mounting host /), which is a well-known full host takeover path. A compromise of this one container is a compromise of the whole node, and on a shared cluster, potentially far more.

What to do instead — least privilege (the Module 15 — SSH and Security Hardening mindset applied to containers): find out exactly what the tool needs. Needs one capability? --cap-drop=ALL --cap-add=<THE_ONE>. Needs a specific device (GPU, /dev/fuse)? --device=<that> only. Needs to build images? Use a rootless/daemonless builder (buildah, kaniko, BuildKit rootless) instead of the host Docker socket. Needs to orchestrate containers? Give it a scoped API credential, not raw socket access. Add --read-only rootfs and a seccomp profile. If after all that it genuinely needs privilege, isolate it — a dedicated, non-shared node, or a stronger sandbox (gVisor/Kata, a micro-VM), never on the general fleet.

Sign-off answer: "Not with --privileged + docker.sock. Tell me the specific operation that's failing and I'll scope it to the exact capability/device/credential. If it truly needs host-level privilege, it runs isolated from everything else, not on shared infra." Strong answer identifies both foot-guns, explains the escape path, and drives to least-privilege alternatives rather than a flat 'no'.

E3. Documentation reference

Concept	Primitive / command	Reference
Isolation (what a process sees)	unshare, nsenter, lsns, /proc/<pid>/ns/	namespaces(7) · unshare(1)
PID namespace / init	unshare --pid --fork --mount-proc; tini	pid_namespaces(7)
User namespace / rootless	unshare --user --map-root-user	user_namespaces(7)
Resource limits (what it can use)	/proc/<pid>/cgroup, /sys/fs/cgroup/; docker run -m --cpus	cgroups(7)
Root filesystem / image	chroot, pivot_root; OverlayFS layers	chroot(2) · What is Docker?
Privilege confinement	/proc/<pid>/status (Cap*), capsh --decode, --cap-drop, seccomp	capabilities(7)

E4. Self-assessment

You've got this module — and the container half of any DevOps interview — if you can answer these aloud, no peeking:

Complete the sentence and defend it: "A container is just a ___." How does that differ from a VM?
Name the three kernel primitives that make a container, and what each one does.
What does a PID namespace do? Why is the container's main process PID 1, and why does that matter for docker stop and zombies?
What is a user namespace, and why does "container-root ≠ host-root" matter for security?
Why can two containers both bind port 80 without conflicting?
What is a cgroup? Walk through what happens when a container exceeds its memory limit.
What is an image made of? Explain layers, the union filesystem, copy-on-write, and the writable layer's lifetime.
Where should a container store data that must survive a redeploy, and why not the container filesystem?
Why are containers less isolated than VMs? List three concrete hardening measures and what --privileged undoes.
An image works on your laptop but not in prod, same Dockerfile — what do you check first, and why?

E5. Sources

Documentation (every link verified to resolve):

namespaces(7) · pid_namespaces(7) · user_namespaces(7) · cgroups(7) · capabilities(7) · unshare(1) · chroot(2) · What is Docker? (Docker Docs)

Interview questions researched from (published/updated within the last two years):

Turing — 100+ Docker Interview Questions (2025) — "What is a container?", Q1 container vs VM, Q19 image vs layer, Q34 isolation & security. KodeKloud — Docker Interview Questions (2026) — Q1 image vs container, Q2 container vs VM ("just a process… namespaces + cgroups… shared kernel"), Q3 Dockerfile, Q4 layers & build cache (updated Jul 2026).

Verification note: all commands were run on Ubuntu 24.04 (util-linux unshare/lsns, busybox-static) in a clean container. The PID-namespace-as-PID-1, UTS-hostname isolation, user-namespace root-mapping, cgroup memory limit, capability sets, and by-hand chroot rootfs are real output. This host runs cgroup v1 (so limits read from memory.limit_in_bytes); modern systemd defaults are cgroup v2 (memory.max) — both are noted where they differ.

You've reached the end of the track.

Sixteen modules ago this started at "what even is Linux." You now understand the machine from the kernel boundary up: the filesystem, permissions, and the shell; find, grep, sed, and awk; processes, packages, scripting, and systemd; storage, networking, logs, and performance; SSH and hardening; and finally containers — which, you now know, are just the earlier fifteen modules (a process, its namespaces, its cgroup, its filesystem, its network, its privileges) assembled into one idea. When an interviewer says "a container is just a process," you can prove it with unshare and a chroot. When they say "the server is slow," you have a method. That's the difference between memorising commands and understanding the system. Go back to the Operating Systems track for the theory beneath any module you want to go deeper on — and good luck in the interview.

Spotted a mistake or want something added? Send me a note.

🐳Module 16 — How Containers Work

Part A — The big idea: a container is just a process

A1. What a container actually is — and how it differs from a VM

A2. The three primitives — namespaces, cgroups, and the rootfs

Part A — Interview questions

Part B — Namespaces: what a process can see

B1. The PID namespace — becoming PID 1

B2. The other namespaces — mount, net, user — and lsns

Part B — Interview questions

Part C — cgroups and the image: what it can use, what it runs

C1. cgroups — limiting what a container can use

C2. The image and the root filesystem — chroot, layers, and copy-on-write

Part C — Interview questions

Part D — Putting it together, and the security view

D1. From primitives to a runtime — and why "works on my machine" mostly ends

D2. Container security — the shared kernel is the boundary

Part D — Interview questions

Part E — Toolkit

E1. Production practice — symptoms and fixes

E2. Capstone — four realistic tickets

E3. Documentation reference

E4. Self-assessment

E5. Sources

Module 16 — How Containers Work