Module 5 — The Shell, Properly

Updated 2 September 2026

Module 5 — The Shell, Properly

You have been using bash for four modules. This one opens the machine: how it expands what you type before anything runs, where it finds commands, how programs report success, and how their inputs and outputs plumb together. Every later module — and most shell bugs you will ever debug — stand on these mechanics.

Legend used throughout: 🧠 concept → 🧩 analogy → 🧪 exercise → ✅ expected result (hidden) → 🎯 interview questions (hidden)

Before you start

You need Modules Module 1 — What Linux IsModule 4 — Users, Groups, and Permissions — especially: word splitting (2), .bashrc (3), and chmod +x with ./ (4). Nothing new to install. Sandbox: mkdir ~/shellwork && cd ~/shellwork.

This page uses Mermaid diagram blocks. Notion shows them as code by default — click the block and set it to Preview to see the diagram. This reminder appears once per page.

Part A — Variables, the environment, and your shell's setup

A1. Shell variables — named holes the shell fills in

🧠 name=value creates a variable; $name anywhere in a later command is replaced by its value before the command runs — the swap you first saw with $SHELL in Module 1, now yours to define. Two rules of the grammar: no spaces around = (the exercise below shows why, memorably), and variable names are case-sensitive with an all-caps convention reserved, informally, for environment variables (A2).

The professional habit that falls out immediately: when unsure what a command will actually receive, put echo in front of it and read the expanded line. Expansion happens the same way either way — echo just shows you the result instead of acting on it.

Real-world analogy — the fill-in-the-blank form letter

A form letter reads "Dear , your order # has shipped". The mail-merge fills every blank from a list, and only then is the letter sent. $name is a blank; the shell is the mail-merge; the command is sent only after every blank is filled.

Where the analogy stops working. Mail-merge software refuses to send with an unfilled blank. The shell fills an unset variable with nothing — silently — and sends anyway. The letter goes out reading "Dear , your order # has shipped". Half of Part B and one capstone ticket exist because of this exact behaviour.

🧪 Exercise A1.1 — Define, expand, inspect
bash
cd ~/shellwork
target=web-01               # define (no spaces around =)
echo $target                # expand it
echo deploying to $target   # expansion happens inside any command line
echo target                 # no $: just the word itself
Expected result — click to reveal
javascript
web-01
deploying to web-01
target

What to read out of it:

  • The $ is the whole trigger: with it, the shell substituted; without it, the literal word passed through. Programs never see $target — they see whatever it expanded to, which is why debugging starts with "what did the shell actually pass?"
  • This variable lives only in this shell. Open a second terminal and echo $target — empty. Not shared, not saved: closing the terminal deletes it. Persistence is A3's business.
🧪 Exercise A1.2 — The spaces mistake, on purpose (fails, famously)
bash
name = "web-01"    # looks reasonable; is not
Expected result — an error, on purpose — click to reveal
javascript
bash: name: command not found

What to read out of it:

  • With spaces, the shell read this as: run a command called name with arguments = and web-01. Word splitting — Module 2's lesson — applied before any notion of "assignment" could. There is no special assignment parser watching for your intent; there is only the grammar.
  • File this error's shape away: command not found naming something you meant as a variable is nearly always the spaces-around-equals slip.

A2. The environment — variables that travel to children

Official docs: environ(7) · env(1)

🧠 Every command you run starts a child process — a separate program (Module 8 makes this precise) that receives a copy of its parent's environment: the subset of variables marked for export. A plain name=value stays private to your shell; export name promotes it into the environment, and from then on every child — and their children — receives a copy at start. Inspect the whole environment with env (or printenv); you have already met its residents: HOME, PATH, SHELL, USER, LANG.

This is the configuration channel of modern operations: services read database URLs, log levels, and feature flags from environment variables — which is why understanding the copy semantics below is not trivia.

Counter-intuitive: the environment flows one way, at one moment. A child gets a snapshot copy at launch. Changes in the parent afterwards never reach a running child; changes a child makes never reach the parent — no exceptions, no mechanism. This is also, at last, the full answer to Module 2's puzzle of why cd must be a builtin: a separate cd program could only change its own copy of "where am I", then exit, changing nothing.
Real-world analogy — the onboarding packet

Every new hire receives a photocopy of the company handbook as it exists on their first day. Notes they scribble in their copy change nobody else's; revisions to the master reach only future hires.

Where the analogy stops working. Real companies circulate updates to old employees. Processes have no circulation whatsoever — a service started in January runs on January's environment until restarted, which is precisely why "I changed the variable but the app ignores it" tickets end with a restart (Module 11).

🧪 Exercise A2.1 — Watch export draw the line
bash
greeting=hello
bash -c 'echo "child sees: $greeting"'   # bash -c: run one command in a CHILD shell
export greeting
bash -c 'echo "child sees: $greeting"'   # again, after export
Expected result — click to reveal
javascript
child sees: 
child sees: hello

What to read out of it:

  • Line one: the child expanded $greeting to nothing — the variable existed only in the parent. Line two: after export, the copy travelled. That empty first line is the unset-variable silence from A1's analogy, live.
  • Note the single quotes around the child's command — they kept the parent from expanding $greeting prematurely (Part B explains exactly this). The child did its own expansion, which was the point of the experiment.

A3. Startup files — where settings become permanent

🧠 Variables die with the shell. Permanence comes from bash's startup files — scripts it runs on launch, and the proper home of Module 3's dotfiles. The practical Ubuntu picture: ~/.bashrc runs for every interactive shell — put exports, aliases, and customizations there; ~/.profile runs at login and, on Ubuntu, sources .bashrc — so .bashrc is the one file to edit in this track. After editing, apply to the current shell with source ~/.bashrc (new terminals pick it up automatically).

Real-world analogy — the morning checklist

A barista opening the café runs the laminated checklist: grinder on, till counted, chairs down. Every shift starts identically because the list runs every time. .bashrc is your shell's opening checklist.

Where the analogy stops working. The café has one checklist; bash has several (.profile, .bashrc, and friends), keyed to how the shell was started — login versus not, interactive versus script. The full matrix is legendarily fiddly; the working rule "customize .bashrc, let .profile source it" covers Ubuntu daily life, and scripts (Module 10) deliberately read none of them.

🧪 Exercise A3.1 — Make one setting immortal
bash
echo 'export EDITOR=nano' >> ~/.bashrc   # append the setting (>> from Module 3, now official)
source ~/.bashrc                          # apply to THIS shell
echo $EDITOR
Expected result — click to reveal
javascript
nano

What to read out of it:

  • EDITOR is a real convention: programs that need to open an editor for you (git, crontab, visudo) launch whatever it names. You just configured every future tool at once — the point of environment variables in one example.
  • Open a new terminal and check echo $EDITOR there too: the checklist ran, the setting held. That round trip — edit, source, verify, new-terminal verify — is the standard ritual for every .bashrc change.

A4. Aliases — your personal abbreviations

🧠 alias ll='ls -alF' teaches your shell that ll means that longer command; alias alone lists what's defined; unalias ll removes one. Aliases expand only at the start of a command line, only in interactive shells — put the keepers in .bashrc. Ubuntu pre-defines a few (ll among them, usually).

Trap — aliases do not exist in scripts. Scripts run in non-interactive shells, which skip alias expansion (and .bashrc). The ll that works at your prompt is command not found inside a script — and, more subtly, an alias like rm='rm -i' that protects you interactively provides zero protection to any script. Muscle memory trained on protective aliases is a liability on machines that lack them; train on the real commands.
Real-world analogy — speed dial

Alias is speed dial: 1 means "call home", saving you the full number a dozen times a day. Nobody else's phone has your speed dial, and dialing 1 from the office switchboard calls something else entirely.

Where the analogy stops working. Speed dial fails loudly — a wrong number answers. An alias absent fails as command not found, but an alias present with different meaning on someone else's machine executes their meaning silently. On shared servers, check before trusting: type ll tells you exactly what a name expands to (C1 formalizes type).

🧪 Exercise A4.1 — Define, use, inspect
bash
alias ll='ls -alF'
ll
type ll
Expected result — click to reveal
javascript
total 8
drwxrwxr-x  2 zaeem zaeem 4096 Sep  2 16:02 ./
drwxr-x--- 15 zaeem zaeem 4096 Sep  2 16:02 ../
ll is aliased to `ls -alF'

What to read out of it (owner, link counts and dates are your machine's own):

  • -alF decoded with four modules of vocabulary: all files (dotfiles included), long format, and -F appends type markers — / for directories. The . and .. entries from Module 2, present as promised.
  • type answered "what IS this name" — for an alias, it shows the expansion. Module 1 taught type for builtins vs programs; aliases are the third answer it gives.

Part A — Interview questions

🎯 "What are the environmental variables?" — asked verbatim in Turing's 100+ Linux interview questions (2025); WeCreateProblems (2026) words it "Explain the use of environment variables in Linux."

Named values a process receives as a snapshot copy from its parent at launch — the standard configuration channel of Unix. Set with name=value, promoted into the environment with export, inspected with env/printenv. The system's own residents: PATH (command search), HOME, USER, SHELL, LANG. Modern operations run on them — database URLs, API keys, log levels — because they configure a process without touching files or code.

The details that separate candidates: the copy-at-launch semantics and both one-way consequences (parent changes don't reach running children — hence restart-after-config-change; child changes don't reach the parent — hence cd is a builtin); plus one security note: environment variables of a process are visible to root and, historically, to more — secrets belong in secret stores, with env vars as the delivery leg.

🎯 "What is the purpose of the alias command?" — asked verbatim in WeCreateProblems' 100+ Linux Commands Interview Questions (2026)

Defines a personal shorthand the interactive shell expands before running: alias ll='ls -alF'. Listed with alias, removed with unalias, made permanent in .bashrc. Good for abbreviating your own high-frequency commands; inspected with type name when behaviour surprises.

The details that separate candidates: the two boundaries — aliases don't exist in scripts (non-interactive shells skip them) and don't travel to other machines; and the safety-alias critique: alias rm='rm -i' trains muscle memory that betrays you on every machine without it — seniors mention this unprompted.

Part B — Quoting and globbing: controlling expansion

B1. Double quotes — one word, blanks still filled

🧠 Module 2 showed word splitting eating a filename with spaces; A1 showed variables expanding. Quoting is how you control both. Double quotes do two things at once: hold their contents together as one word (splitting suppressed), while still allowing $variable and $(command) expansion inside. The professional default is simple and worth adopting today, wholesale: always double-quote variable expansions"$file", not $file. Unquoted expansions get re-split on spaces after expansion, which is the single largest class of shell bugs in existence.

Real-world analogy — the padded envelope

Double quotes are a padded envelope: whatever goes in travels as one parcel, however many spaces it contains. The envelope is transparent to the mail-merge — blanks inside still get filled — but the parcel never splits in transit.

Where the analogy stops working. An envelope is optional packaging. In shell, unpackaged parcels are actively cut apart at every space by the machinery — quoting is not politeness, it is the difference between one argument and several, and the cutting happens silently on the way to the command.

🧪 Exercise B1.1 — The bug, then the fix
bash
cd ~/shellwork
file="release notes.txt"
touch "$file"          # quoted: creates ONE file
ls -l $file            # unquoted: what does ls receive?
ls -l "$file"          # quoted: and now?
Expected result — first ls fails, on purpose — click to reveal
javascript
ls: cannot access 'release': No such file or directory
ls: cannot access 'notes.txt': No such file or directory
-rw-rw-r-- 1 zaeem zaeem 0 Sep  2 16:05 'release notes.txt'

What to read out of it (owner and dates are your machine's own):

  • The unquoted $file expanded to release notes.txt and was then split: ls received two arguments, and Module 2's two-error signature appeared — same disease, now with the mechanism fully visible: expansion first, splitting second, command last.
  • The quoted version delivered one argument. Note ls itself quoting the name in its output — modern coreutils flags awkward names back at you.
  • Re-read Module 2's D3 with today's eyes: everything there was this section, experienced before it could be explained.

B2. Single quotes — nothing expands

🧠 Single quotes suppress everything: no splitting, no $ expansion, no substitution — the contents pass through byte-for-byte. Use them when the text must survive literally: strings containing $, sed/awk programs (Modules 6–7 depend on this constantly), anything destined for a different interpreter than your shell. One hard grammar rule: there is no escaping inside single quotes — a single-quoted string simply cannot contain a single quote; switch to double quotes for those.

Real-world analogy — "quote me exactly"

Double quotes are relaying a message: "tell them the meeting is at [whatever time the calendar says]" — blanks filled in transit. Single quotes are a court transcript: every character verbatim, blanks and all, no interpretation permitted.

Where the analogy stops working. A human transcriber still fixes obvious slips. Single quotes fix nothing and interpret nothing — which is exactly their value: when the recipient is another program with its own $ syntax (awk, notably), only a verbatim handoff prevents your shell from eating the other program's grammar.

🧪 Exercise B2.1 — The two quote types, side by side
bash
user=zaeem
echo "logged in as $user"
echo 'logged in as $user'
echo "it costs \$5"          # backslash: escape ONE character inside double quotes
Expected result — click to reveal
javascript
logged in as zaeem
logged in as $user
it costs $5

What to read out of it:

  • Same line, three treatments: expanded, verbatim, and selectively-escaped. The decision procedure, permanently: want expansion → double quotes; want nothing touched → single quotes; want one exception → backslash it inside double quotes.
  • Line two is not a failure — printing literal $user is exactly what documentation, config templates, and code destined for other interpreters need.

B3. Globbing — patterns the shell expands into filenames

🧠 * matches any run of characters, ? exactly one, [abc]/[0-9] one from a set. The mechanism matters more than the symbols: the shell expands the pattern into a list of matching filenames before the command runsls *.log never shows ls a star; ls receives app.log db.log, already resolved. Three consequences to internalize: dotfiles are excluded (Module 2's backup-missed-the-dotfiles bug, explained at last); a pattern matching nothing is passed through literally, star and all (bash's default — the error then comes from the command, complaining about a file literally named *.pdf); and since expansion precedes execution, echo before any glob shows you precisely what a destructive command would receive.

Counter-intuitive: wildcards are not implemented by commands. rm, ls, cp contain no pattern code at all — they receive plain filename lists, courtesy of the shell. This is why quoting a glob (ls "*.log") disables it (the command receives a literal star), and why a directory with a million matching files breaks commands with Argument list too long: the shell tried to pass a million arguments. The command was never going to "handle the pattern" — there is no pattern by the time it runs.
Real-world analogy — the stencil over the shelf

A glob is a stencil held against the shelf: every label showing through is picked, and the picked items — not the stencil — go into the requisition. The warehouse worker filling the order never sees the stencil.

Where the analogy stops working. A stencil matching nothing yields an empty requisition a human would question. Bash hands over the stencil itself as if it were an item name — a command then hunts for a file literally called *.pdf. Useful error, weird mechanism; scripts guard against it explicitly (Module 10's nullglob).

🧪 Exercise B3.1 — Watch expansion happen (last line fails on purpose)
bash
cd ~/shellwork
touch app.log db.log notes.txt
echo *.log         # echo shows what ANY command would receive
echo *
ls *.pdf           # a pattern with no matches
Expected result — an error, on purpose — click to reveal
javascript
app.log db.log
app.log db.log notes.txt release notes.txt
ls: cannot access '*.pdf': No such file or directory

What to read out of it:

  • The echos display the post-expansion command line — the exact arguments any command in that position would receive. echo + glob is the free dry run for every rm/mv/cp you will ever aim at a pattern; use it before the real thing, always.
  • Line two carries a subtle horror: release notes.txt (B1's file) appears as two space-separated words indistinguishable from separate names. Expansion output is a split word list, not a display listing — globs and spaced filenames interact viciously, another reason the professional convention avoids spaces in names.
  • The failure line: no match, so ls received the literal string *.pdf and looked for that file. Read the quotes in the error — they are showing you exactly what it received.
Now imagine this at 500 hosts. Cleanup jobs love globs, and globs love surprises: on the one host where the pattern matches nothing, the job processes a literal * name or errors; on the host with 800,000 matching files, it dies with Argument list too long. Fleet-grade file selection uses find (Module 6), which streams matches instead of expanding them into one giant command line — same intent, mechanism that scales.

Part B — Interview questions

🎯 "How can you use wildcards in Linux commands?" — asked verbatim in WeCreateProblems' 100+ Linux Commands Interview Questions (2026)

* (any run), ? (one character), [a-z]/[abc] (one from a set) — written in any argument position: ls *.log, rm report-2026-0?.csv, cp deploy-[ab].conf backup/. The shell — not the command — expands the pattern into matching filenames before execution; dotfiles are excluded unless the pattern itself starts with a dot.

The details that separate candidates: stating who expands (the shell), with a consequence or two — quoting disables matching; empty matches pass the literal pattern through; huge matches overflow the argument list — and the dry-run habit of previewing any destructive glob with echo first. Recall-level answers list the symbols; systems-level answers explain the mechanism.

🎯 Corpus note — quoting questions

Published question banks barely touch quoting — none of the sets used in this track ask about single-versus-double quotes directly. Yet quoting failures are a favourite live probe: interviewers show a broken rm $file or an awk one-liner mangled by double quotes and ask "why?". Prepare by being able to narrate B1.1's failure — expansion, then splitting, then execution — without notes; that narration is the answer to every quoting question they can construct.

Part C — Finding commands, and hearing how they finished

C1. PATH — the shell's search route

🧠 Type date and the shell must find a program by that name. The route is fixed and fast:

Diagram source
flowchart TD
    A["You type a name"] --> B{"Alias?"}
    B -->|"yes"| C["Expand alias, restart"]
    B -->|"no"| D{"Shell builtin?"}
    D -->|"yes"| E["Run inside the shell"]
    D -->|"no"| F{"Cached location<br>(hash table)?"}
    F -->|"yes"| G["Run cached program"]
    F -->|"no"| H["Walk PATH directories<br>in order, first hit wins"]
    H -->|"found"| I["Cache it, run it"]
    H -->|"nothing"| J["command not found"]

PATH is an environment variable: a colon-separated directory list, searched left to right, first match wins. A typical user's PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin (often with ~/.local/bin prepended). Module 1's type and its hashed note, Module 4's ./script requirement — all of it is this one mechanism. Extend it in .bashrc: export PATH="$HOME/bin:$PATH" — prepend your directory, keep the rest.

Counter-intuitive: the current directory is deliberately NOT on the PATH. That is why ./hello.sh needs the ./. The reason is security, and it is a good story: with . on the PATH, anyone could drop a malicious program named ls into /tmp, and the next admin to type ls there would run it. The two extra characters you type forever are the price of that attack not working.
Real-world analogy — the parts-supplier call list

The workshop keeps an ordered list of suppliers taped by the phone. Need a part? Call down the list, first supplier who stocks it wins, and the clerk pencils a note remembering who had it (the hash cache). "We don't stock that anywhere" is command not found.

Where the analogy stops working. A clerk might try a supplier not on the list in a pinch. The shell never will — a program in an unlisted directory does not exist to bare-name lookup, however plainly you can see it with ls. The list is not advice; it is the entire universe of names.

🧪 Exercise C1.1 — Read your route, then step off it (second half fails on purpose)
bash
echo $PATH
type date               # where on the route is date?
cd ~/perms              # Module 4's sandbox, home of hello.sh
hello.sh                # bare name — not on the route
./hello.sh              # explicit address
Expected result — an error, on purpose — click to reveal
javascript
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
date is /usr/bin/date
bash: hello.sh: command not found
hello from my first script

(Your PATH will differ in detail — tool installers love adding entries; type date may say hashed if you've run date this session, Module 1's footnote.)

What to read out of it:

  • Walk date's lookup along your own PATH string, directory by directory, to /usr/bin — make the mechanism concrete once and it is yours forever.
  • The failure is the PATH working as designed: ~/perms is not on the route, so the bare name does not exist, while ./ bypasses the route entirely with an explicit address. command not found for something you can see now has a precise diagnosis: right file, unlisted directory.

C2. Exit codes — how programs say how it went

🧠 Every command finishes with a number: the exit code0 for success, anything else for failure. The shell stores the last one in $?. This is the machine-readable verdict behind Module 1's "silence is success": humans read absence-of-output; automation reads the number. Codes you have already caused, now with their names: 1/2 — general errors (the program chooses), 126 — found but not executable (Module 4's pre-chmod script), 127 — not found (every typo since Module 1).

Codes power the shell's conditional operators, the foundation of all automation: a && b — run b only if a succeeded; a || b — run b only if a failed; a ; b — run b regardless. Read && as "and then", || as "or else".

Real-world analogy — the inspection tag

Every job leaving a workshop bay gets a tag: green 0, or a red tag bearing a reason code. The next station reads the tag, not the workpiece: green → proceed (&&); red → side-track to rework (||). Nobody re-inspects; the tag is trusted.

Where the analogy stops working. Tags are hung by the worker who did the job — and programs, like workers, can tag wrong. A badly written script that ignores its own failures exits 0, and everything downstream trusts the lie. Module 10's set -e discipline exists to keep your own tags honest; distrust of others' tags is why deploy pipelines verify results independently.

🧪 Exercise C2.1 — Read verdicts, then chain on them
bash
ls notes.txt       # cd ~/shellwork first if needed
echo $?
ls missing.txt
echo $?
mkdir -p /tmp/demo && echo "created" || echo "failed"
cp missing.txt /tmp/ && echo "copied" || echo "copy failed"
Expected result — click to reveal
javascript
notes.txt
0
ls: cannot access 'missing.txt': No such file or directory
2
created
cp: cannot stat 'missing.txt': No such file or directory
copy failed

What to read out of it:

  • 0 then 2: the invisible verdict, made visible. One catch worth learning now: $? holds only the most recent verdict — even your echo $? overwrote it (with echo's own 0). Capture it immediately if you need it: status=$?.
  • The chains: && fired the success branch; on the failing cp, && was skipped and || fired. The a && b || c idiom reads like if/else and mostly behaves like one (the true conditional arrives in Module 10).
  • ls chose 2 for "serious trouble" — each program defines its own nonzero vocabulary; only 0 has a universal meaning.

Part C — Interview questions

🎯 "What is the significance of the PATH variable?" — asked verbatim in WeCreateProblems' 100+ Linux Commands Interview Questions (2026)

PATH is the colon-separated, ordered directory list the shell walks to resolve bare command names — first match wins, misses everywhere mean command not found. It is why installed-but-elsewhere tools "don't exist", why ./script is needed for the current directory (which is excluded, deliberately, for security), and why the same command name can resolve differently for different users or in cron. Extended per-user via .bashrc (export PATH="$HOME/bin:$PATH").

The details that separate candidates: search order as an attack/override surface (prepending a directory shadows system commands — used both by installers and by attackers); the security story for excluding .; and the operational classic — cron and other non-interactive contexts carry a minimal PATH, which is why "works in my terminal, fails in the scheduler" is a PATH bug until proven otherwise.

🎯 Corpus note — exit codes

No published set used in this track asks about exit codes directly — and yet $?, &&/||, and "how does your pipeline know the deploy failed?" come up in nearly every live DevOps interview, usually inside a scripting or CI question. Be able to state cold: 0 is success and the only universal code; $? holds the last verdict and is immediately perishable; 126 vs 127; and that automation trusts codes, not output — then connect it to set -e when scripting comes up. The absence from question banks makes fluency here more distinguishing, not less.

Part D — Plumbing: streams, redirection, and pipes

D1. Three streams, and re-aiming them

🧠 Every process is born holding three open channels, numbered: 0 stdin (input, usually your keyboard), 1 stdout (results), 2 stderr (errors and diagnostics). Separating 1 from 2 is the design insight of Unix plumbing: results and complaints travel different wires, so each can be aimed independently:

  • > file — send stdout to file (created or overwritten); >> file — append instead.
  • 2> file — send stderr somewhere; 2>> appends.
  • 2>&1 — send stderr to wherever stdout currently points (order matters: > log 2>&1).
  • > /dev/null — the kernel's bottomless discard device, for output you have decided not to want.
  • < file — feed a file to stdin.
Trap — > truncates before the command runs. The shell opens (and empties) the target file first, then starts the command. Two consequences: a failed command still leaves the file emptied, and the classic self-destruction sort data.txt > data.txt empties data.txt before sort reads a byte — the file is simply gone. Redirect to a new name, or learn sponge/temp-file patterns in Module 10.
Real-world analogy — the conveyor and the reject chute

A machine on the factory floor has a product conveyor (stdout) and a reject chute (stderr). Redirection is re-aiming them: conveyor into a shipping crate (> file), chute into the incinerator (2>/dev/null), or both into one bin for later sorting (> file 2>&1). The machine itself is unmodified — plumbing is arranged around it.

Where the analogy stops working. Re-aim a physical chute mid-shift and subsequent output follows. Shell redirections are plumbed once, before startup, by left-to-right snapshot — which is why 2>&1 > log (aim chute at conveyor's current target: the terminal; then move conveyor) differs from > log 2>&1 (move conveyor to log; aim chute at it). The order bug produces "why is stderr still on my screen" tickets weekly, worldwide.

🧪 Exercise D1.1 — Split the streams, then merge them
bash
cd ~/shellwork
ls notes.txt missing.txt                       # both streams to the terminal, interleaved
ls notes.txt missing.txt > out.txt 2> err.txt  # separated
cat out.txt
cat err.txt
ls notes.txt missing.txt > all.txt 2>&1        # merged
cat all.txt
ls missing.txt 2>/dev/null                     # complaint, discarded
echo $?
Expected result — click to reveal
javascript
ls: cannot access 'missing.txt': No such file or directory
notes.txt
notes.txt
ls: cannot access 'missing.txt': No such file or directory
ls: cannot access 'missing.txt': No such file or directory
notes.txt
2

What to read out of it:

  • One command, two files, cleanly separated: results in out.txt, complaint in err.txt. This is how every scheduled job should be plumbed (a Module 11 refrain).
  • In the merged file, the error may appear before the result — the two streams buffer differently, so interleaving order is not guaranteed. Real logs have this texture; timestamps, not position, order events.
  • The discard line printed nothing — but $? still says 2. Silencing a stream does not silence the verdict: automation built on exit codes keeps working with output thrown away. Discard output only when you have decided the code is what you act on.

D2. Pipes — plugging programs into each other

🧠 a | b connects a's stdout directly to b's stdin — no file in between, no disk touched; the data flows through kernel memory. This is the composition mechanism the small-tools philosophy (Module 1, Part D) was designed around: each tool does one thing; pipes make sentences from the words. You have borrowed it twice (| head); now it is formally yours. Two mechanics worth knowing from day one: all programs in a pipeline run concurrently (b starts consuming while a still produces — a pipeline of huge inputs needs no huge memory), and stderr is not piped — complaints from every stage land on your terminal, bypassing the plumbing.

Real-world analogy — machines in a line

A bottling line: the filler feeds the capper feeds the labeler, product flowing continuously — nobody waits for "all bottles filled" before capping starts. Each machine was built independently; the conveyors between them make it a line.

Where the analogy stops working. Factory conveyors can buffer pallets on the floor. A pipe holds only a small kernel buffer: when a downstream stage stalls, upstream stages block — which is why somecommand | less uses no memory on gigabytes (production is paused while you read), and why one slow stage sets the whole pipeline's pace.

🧪 Exercise D2.1 — Compose your existing vocabulary
bash
ls /etc | wc -l                # how many entries in /etc?
ls -l /var/log | head -5       # long listing, first five lines only
tail -20 ~/.bash_history | wc -l   # how much recent history? (a brand-new account may not have this file yet — see the result)
Expected result — click to reveal
javascript
228
total 808
lrwxrwxrwx 1 root root     39 May  8 16:23 README -> ../../usr/share/doc/systemd/README.logs
-rw-r--r-- 1 root root  45911 Sep  2 14:22 alternatives.log
drwxr-xr-x 2 root root   4096 Sep  2 14:22 apt
-rw-r--r-- 1 root root  61229 Apr 10 02:20 bootstrap.log
20

What to read out of it (counts are your machine's own):

  • Every command here was taught modules ago; the pipe is the only new ingredient, and suddenly questions ("how many?", "just the top") become one-liners. This is the inflection point of the whole track: from running commands to composing them.
  • Note head receiving ls -l's output mid-production and cutting it off at five lines — ls is then simply terminated early. Concurrency, visible.
  • If the last line instead printed tail: cannot open ... No such file or directory and then 0: nothing is broken — bash writes the history file at logout, so a brand-new account has not created one yet. Log out, back in, and it exists. (Note wc still printed a count: the pipe carried tail's empty stdout while the error took the stderr bypass.)
  • Modules 6 and 7 supply the great pipeline citizens — grep, sort, awk — and this section is why they compose.

D3. Command substitution — output as input

🧠 $(command) runs the command and pastes its output into the surrounding line — the last of the shell's big expansions, and the glue for "use this result inside that command". You met it once on faith in Module 4 (chown $(whoami)); the mechanics: runs in a subshell, output captured, trailing newline stripped, result subject to word splitting unless double-quoted — so the B1 rule applies with full force: "$(command)".

Real-world analogy — the sub-errand

Filling a form that asks for today's exchange rate, you pause, make a phone call, write the answer into the blank, and submit. $(...) is the phone call: a sub-errand whose answer lands in the blank mid-sentence.

Where the analogy stops working. You would notice a phone call answering with three paragraphs and summarize. The shell pastes whatever came back — multi-line output and all — then word-splits it into the sentence unless quoted. The blank does not judge; it holds what it is given.

🧪 Exercise D3.1 — Results inside commands
bash
echo "there are $(ls | wc -l) files here"
backup="notes-$(date +%F).txt"       # date +%F prints YYYY-MM-DD
cp notes.txt "$backup"
ls notes-*
Expected result — click to reveal
javascript
there are 7 files here
notes-2026-09-02.txt

What to read out of it (count and date are your machine's own):

  • A pipeline ran inside a string, its answer landing mid-sentence. Substitution composes with everything — including pipes.
  • The dated-backup idiom is used verbatim across the industry: timestamped copies before risky changes, built from Module 2's cp, Module 1's date, and today's $(...). You now construct file names from live data — carefully quoted.

Part D — Interview questions

🎯 "What is meant by PIPE in Linux?" — asked verbatim in Turing's 100+ Linux interview questions (2025); WeCreateProblems (2026) words it "What is a pipe, and how is it used in Linux?"

| connects one command's stdout to the next command's stdin through a kernel buffer — no intermediate file, no disk. It is the composition operator of the small-tools philosophy: ls -l | grep conf | wc -l chains three single-purpose tools into an answer. Stages run concurrently, memory use stays flat regardless of data volume, and stderr bypasses the pipe entirely, arriving at the terminal.

The details that separate candidates: concurrency (not batch hand-off) and its corollary — a stalled consumer blocks the producer; stderr's separate path; and the exit-code subtlety: a pipeline's status is the last command's, so failing-cmd | tee log reports success — the pipefail fix belongs in every professional script (Module 10 installs it).

🎯 "What is redirection?" — asked verbatim in Turing's 100+ Linux interview questions (2025); WeCreateProblems (2026) asks "How do you redirect output to a file?"

Re-aiming a process's standard streams before it runs: > stdout to a file (truncating), >> appending, 2> stderr, 2>&1 stderr to stdout's current target, < file as stdin, /dev/null as the discard. Canonical job form: command >> /var/log/job.log 2>&1. The shell sets up all plumbing left-to-right before executing, which is why 2>&1's position matters.

The details that separate candidates: the truncate-before-run trap (sort f > f destroys f); the > log 2>&1 vs 2>&1 > log ordering, explained via the snapshot mechanism rather than recited; and separating streams on purpose — results parseable, errors visible — as a design habit, not an exam answer.

🎯 Corpus note — command substitution

Not asked directly in any recent published set consulted — but it appears inside the answers to scripting questions everywhere ("store the output of a command in a variable"). Own the one-liner: $(cmd) captures output, trailing newline stripped, quote it or it word-splits. Backtick syntax cmd is the legacy spelling — recognize it in old scripts, write the modern form.

Part E — Toolkit

Official docs: Part E is reference material — every source it draws on is linked in the E3 documentation table below.

E1. Production practice — symptoms and fixes

SymptomWhat is really happeningWhat to runThe fix
Works in your terminal; command not found in automationAutomation contexts (cron, systemd) provide a minimal PATH, and non-interactive shells read no .bashrc — your PATH additions and aliases don't exist theretype thecommand interactively · compare echo $PATH in both contextsAbsolute paths in automation, or set PATH explicitly at the top of the job
A variable "loses its value" in a subshell or scriptIt was never exported — children got no copybash -c 'echo $VAR' as a probeexport VAR; for services, set it in the service's own config (Module 11)
Command mangles a filename into two, or acts on wrong filesUnquoted expansion re-split on spaces; or a glob expanded beyond intentPrefix the line with echo and read what it becomesDouble-quote every expansion; dry-run globs with echo before destructive use
A log or data file is suddenly empty after a "failed" command> truncated it before the command ran — failure came after the damagels -l timestamps; reconstruct the redirectionNever redirect a file onto itself; write to a new name, move after success (&&)
"I redirected everything to the log but errors still hit the screen"2>&1 placed before > — stderr aimed at the terminal snapshotRe-read the line left to right, narrating each plumbing stepOrder it > file 2>&1
Pipeline "succeeded" but produced garbage/nothingAn early stage failed; the pipeline's exit code is the last stage'sCheck stderr (it bypassed the pipe) · echo $? per stage while debuggingset -o pipefail in scripts (Module 10); verify outputs, not just codes

E2. Capstone — four tickets

Work these like real tickets: read the ticket, write the commands and the explanation you would send, then open the worked answer. Everything needed was taught in this module.
🎫 Ticket 1 — "The nightly job says command not found; the same line works when I paste it"

Ticket text: "Our scheduled backup job logs backup-tool: command not found. Pasting the identical line into a terminal works perfectly. The tool is installed in /opt/backup/bin. Explain and fix."

Worked answer: the terminal shell read .bashrc, which prepends /opt/backup/bin to PATH; the scheduler's non-interactive shell reads no such file and carries a minimal PATH — so the bare name resolves in one context and not the other. Same command, different universes of names. Prove it: log echo "$PATH" from inside the job and compare. Fix, in order of preference: call the tool by absolute path (/opt/backup/bin/backup-tool) — automation should not depend on name resolution; or set PATH=/opt/backup/bin:/usr/bin:/bin explicitly at the top of the job. The general law, worth writing on the wall: interactive convenience (PATH edits, aliases, .bashrc) does not exist in automation.

🎫 Ticket 2 — "Cleanup script wiped the wrong directory"

Ticket text: "A script ran `rm -rf ${STAGING_DIR}` and deleted the working directory it was launched from, not the staging area. STAGING_DIR is set in the engineer's .bashrc and 'has always worked'. Postmortem the mechanism and the guards."*

Worked answer: mechanism — the script ran non-interactively, read no .bashrc, so STAGING_DIR was unset; the expansion produced nothing, leaving plain rm -rf *, which faithfully wiped whatever directory the script happened to be launched from (A1's form letter, sent with its blank unfilled). Had it been written rm -rf $STAGING_DIR/, the empty expansion would have produced rm -rf / — refused by GNU rm's --preserve-root failsafe, but a script should never get to lean on that final guard. Guards, all of which belong in Module 10's script prologue: quote and require the variable ("${STAGING_DIR:?not set}" — the :? form aborts with a message when unset; meet it properly in Module 10), set -u to make unset expansion fatal, never end an rm path with a bare expansion, and dry-run with echo in review. The .bashrc dependency was the root cause: configuration for automation lives in the automation, never in a human's dotfiles.

🎫 Ticket 3 — "The deploy log is empty but the deploy clearly ran (and failed)"

Ticket text: "Runbook line: deploy.sh 2>&1 > /var/log/deploy.log. The deploy failed; the log contains only the success chatter, and the errors were lost with the closed terminal. Why?"

Worked answer: plumbing order. Left to right: 2>&1 first — aim stderr at stdout's current target, the terminal; then > /var/log/deploy.log — move stdout to the log, stderr staying where it was pointed. Result: chatter logged, errors to a terminal nobody kept. The fix is transposition: deploy.sh > /var/log/deploy.log 2>&1 (stdout to log first, then stderr to the same place). Better runbook practice: split them — >> deploy.log 2>> deploy.err — so failures are findable without scrolling chatter, and use >> so reruns never truncate history. The interviewable sentence: redirections are snapshots evaluated left to right, not live links.

🎫 Ticket 4 — "Report generator overwrote its own input"

Ticket text: "An engineer 'cleaned up' a data file with sort data.csv > data.csv and the file is now zero bytes. They insist sort must be buggy. Explain, recover, prevent."

Worked answer: not sort's doing — the shell's. Before sort launched, the shell opened data.csv for stdout redirection, truncating it to zero; sort then read an already-empty file and correctly wrote nothing. The program never had a chance to see the data. Recovery: backups or re-generation — the bytes are gone (Module 3's shredder rule). Prevention: write to a new name and swap on success — sort data.csv > data.sorted && mv data.sorted data.csv — which also leaves the original intact if sort fails (the && earning its keep); Module 7 adds sort -o data.csv data.csv, sort's own safe in-place flag, and the general lesson stands: the shell sets up all plumbing before the command runs.

E3. Documentation reference

TopicAuthoritative sourceVerified link
Everything in this moduleThe Bash Reference Manual (variables, quoting, expansion, PATH, redirection, pipelines)Bash Reference Manual
The environmentenviron(7), env(1)environ(7) · env(1)
Glob patternsglob(7)glob(7)
The discard devicenull(4)null(4)

E4. Self-assessment

Answer out loud, without notes. The section number tells you where to re-read.

  1. name = value fails with command not found. Narrate exactly how the shell parsed it. (A1)
  2. A variable is visible in your shell but empty in every child. One word names the fix — and what are the two one-way rules of the environment? (A2)
  3. Why must cd be a builtin? Answer using this module's model, not Module 2's assertion. (A2)
  4. Which file do you edit for permanent settings on Ubuntu, how do you apply it without a new terminal, and why do scripts ignore it? (A3–A4)
  5. State the quoting decision procedure in three clauses, and why "$file" is the professional default. (B1–B2)
  6. Who expands *.log — and derive two consequences of that answer (quoted glob; million-file directory). (B3)
  7. A command you can see with ls is command not found by bare name. Diagnose, and explain why . is off the PATH. (C1)
  8. What do exit codes 0, 126, and 127 mean, why is $? perishable, and what do && and || read as? (C2)
  9. Plumb from memory: stdout and stderr to the same log file, appending. Then explain why the reversed order fails. (D1)
  10. sort f > f empties f. Whose fault, and what is the safe pattern? (D1, Ticket 4)
  11. Two facts about pipes that surprise beginners: when do stages run, and which stream escapes the pipe? (D2)
  12. Build, from memory, a dated backup filename using command substitution — with correct quoting. (D3)

E5. Sources

Interview-question sources used in this module (fetched and quoted verbatim during research, September 2026):

Turing — 100+ Linux Interview Questions (2025) · WeCreateProblems — 100+ Linux Commands Interview Questions (2026) · Adaface — 96 Linux Commands interview questions (September 2024, pipes/redirection material).

Corpus honesty note: shell mechanics are underrepresented in published question banks — pipes, redirection, environment variables and PATH appear (cited above), but quoting, exit codes, and command substitution are asked almost exclusively live, inside scripting scenarios. Where this page marks a "corpus note" instead of a source, that is the honest state of the published record: the material is interview-critical but tested by demonstration, not by quiz.

All documentation links on this page were fetched and confirmed reachable on 2 September 2026.

Next: you can compose commands. Time to meet the two greatest things to compose — Module 6 — Finding Things: find and grep.
Spotted a mistake or want something added? Send me a note.