Module 5 — The Shell, Properly
Updated 2 September 2026
You have been using bash for four modules. This one opens the machine: how it expands what you type before anything runs, where it finds commands, how programs report success, and how their inputs and outputs plumb together. Every later module — and most shell bugs you will ever debug — stand on these mechanics.
Legend used throughout: 🧠 concept → 🧩 analogy → 🧪 exercise → ✅ expected result (hidden) → 🎯 interview questions (hidden)
You need Modules Module 1 — What Linux Is–Module 4 — Users, Groups, and Permissions — especially: word splitting (2), .bashrc (3), and chmod +x with ./ (4). Nothing new to install. Sandbox: mkdir ~/shellwork && cd ~/shellwork.
Part A — Variables, the environment, and your shell's setup
A1. Shell variables — named holes the shell fills in
🧠 name=value creates a variable; $name anywhere in a later command is replaced by its value before the command runs — the swap you first saw with $SHELL in Module 1, now yours to define. Two rules of the grammar: no spaces around = (the exercise below shows why, memorably), and variable names are case-sensitive with an all-caps convention reserved, informally, for environment variables (A2).
The professional habit that falls out immediately: when unsure what a command will actually receive, put echo in front of it and read the expanded line. Expansion happens the same way either way — echo just shows you the result instead of acting on it.
A form letter reads "Dear , your order # has shipped". The mail-merge fills every blank from a list, and only then is the letter sent. $name is a blank; the shell is the mail-merge; the command is sent only after every blank is filled.
Where the analogy stops working. Mail-merge software refuses to send with an unfilled blank. The shell fills an unset variable with nothing — silently — and sends anyway. The letter goes out reading "Dear , your order # has shipped". Half of Part B and one capstone ticket exist because of this exact behaviour.
🧪 Exercise A1.1 — Define, expand, inspect
cd ~/shellwork
target=web-01 # define (no spaces around =)
echo $target # expand it
echo deploying to $target # expansion happens inside any command line
echo target # no $: just the word itself✅ Expected result — click to reveal
web-01
deploying to web-01
targetWhat to read out of it:
- The $ is the whole trigger: with it, the shell substituted; without it, the literal word passed through. Programs never see $target — they see whatever it expanded to, which is why debugging starts with "what did the shell actually pass?"
- This variable lives only in this shell. Open a second terminal and echo $target — empty. Not shared, not saved: closing the terminal deletes it. Persistence is A3's business.
🧪 Exercise A1.2 — The spaces mistake, on purpose (fails, famously)
name = "web-01" # looks reasonable; is not✅ Expected result — an error, on purpose — click to reveal
bash: name: command not foundWhat to read out of it:
- With spaces, the shell read this as: run a command called name with arguments = and web-01. Word splitting — Module 2's lesson — applied before any notion of "assignment" could. There is no special assignment parser watching for your intent; there is only the grammar.
- File this error's shape away: command not found naming something you meant as a variable is nearly always the spaces-around-equals slip.
A2. The environment — variables that travel to children
🧠 Every command you run starts a child process — a separate program (Module 8 makes this precise) that receives a copy of its parent's environment: the subset of variables marked for export. A plain name=value stays private to your shell; export name promotes it into the environment, and from then on every child — and their children — receives a copy at start. Inspect the whole environment with env (or printenv); you have already met its residents: HOME, PATH, SHELL, USER, LANG.
This is the configuration channel of modern operations: services read database URLs, log levels, and feature flags from environment variables — which is why understanding the copy semantics below is not trivia.
Every new hire receives a photocopy of the company handbook as it exists on their first day. Notes they scribble in their copy change nobody else's; revisions to the master reach only future hires.
Where the analogy stops working. Real companies circulate updates to old employees. Processes have no circulation whatsoever — a service started in January runs on January's environment until restarted, which is precisely why "I changed the variable but the app ignores it" tickets end with a restart (Module 11).
🧪 Exercise A2.1 — Watch export draw the line
greeting=hello
bash -c 'echo "child sees: $greeting"' # bash -c: run one command in a CHILD shell
export greeting
bash -c 'echo "child sees: $greeting"' # again, after export✅ Expected result — click to reveal
child sees:
child sees: helloWhat to read out of it:
- Line one: the child expanded $greeting to nothing — the variable existed only in the parent. Line two: after export, the copy travelled. That empty first line is the unset-variable silence from A1's analogy, live.
- Note the single quotes around the child's command — they kept the parent from expanding $greeting prematurely (Part B explains exactly this). The child did its own expansion, which was the point of the experiment.
A3. Startup files — where settings become permanent
🧠 Variables die with the shell. Permanence comes from bash's startup files — scripts it runs on launch, and the proper home of Module 3's dotfiles. The practical Ubuntu picture: ~/.bashrc runs for every interactive shell — put exports, aliases, and customizations there; ~/.profile runs at login and, on Ubuntu, sources .bashrc — so .bashrc is the one file to edit in this track. After editing, apply to the current shell with source ~/.bashrc (new terminals pick it up automatically).
A barista opening the café runs the laminated checklist: grinder on, till counted, chairs down. Every shift starts identically because the list runs every time. .bashrc is your shell's opening checklist.
Where the analogy stops working. The café has one checklist; bash has several (.profile, .bashrc, and friends), keyed to how the shell was started — login versus not, interactive versus script. The full matrix is legendarily fiddly; the working rule "customize .bashrc, let .profile source it" covers Ubuntu daily life, and scripts (Module 10) deliberately read none of them.
🧪 Exercise A3.1 — Make one setting immortal
echo 'export EDITOR=nano' >> ~/.bashrc # append the setting (>> from Module 3, now official)
source ~/.bashrc # apply to THIS shell
echo $EDITOR✅ Expected result — click to reveal
nanoWhat to read out of it:
- EDITOR is a real convention: programs that need to open an editor for you (git, crontab, visudo) launch whatever it names. You just configured every future tool at once — the point of environment variables in one example.
- Open a new terminal and check echo $EDITOR there too: the checklist ran, the setting held. That round trip — edit, source, verify, new-terminal verify — is the standard ritual for every .bashrc change.
A4. Aliases — your personal abbreviations
🧠 alias ll='ls -alF' teaches your shell that ll means that longer command; alias alone lists what's defined; unalias ll removes one. Aliases expand only at the start of a command line, only in interactive shells — put the keepers in .bashrc. Ubuntu pre-defines a few (ll among them, usually).
Alias is speed dial: 1 means "call home", saving you the full number a dozen times a day. Nobody else's phone has your speed dial, and dialing 1 from the office switchboard calls something else entirely.
Where the analogy stops working. Speed dial fails loudly — a wrong number answers. An alias absent fails as command not found, but an alias present with different meaning on someone else's machine executes their meaning silently. On shared servers, check before trusting: type ll tells you exactly what a name expands to (C1 formalizes type).
🧪 Exercise A4.1 — Define, use, inspect
alias ll='ls -alF'
ll
type ll✅ Expected result — click to reveal
total 8
drwxrwxr-x 2 zaeem zaeem 4096 Sep 2 16:02 ./
drwxr-x--- 15 zaeem zaeem 4096 Sep 2 16:02 ../
ll is aliased to `ls -alF'What to read out of it (owner, link counts and dates are your machine's own):
- -alF decoded with four modules of vocabulary: all files (dotfiles included), long format, and -F appends type markers — / for directories. The . and .. entries from Module 2, present as promised.
- type answered "what IS this name" — for an alias, it shows the expansion. Module 1 taught type for builtins vs programs; aliases are the third answer it gives.
Part A — Interview questions
🎯 "What are the environmental variables?" — asked verbatim in Turing's 100+ Linux interview questions (2025); WeCreateProblems (2026) words it "Explain the use of environment variables in Linux."
Named values a process receives as a snapshot copy from its parent at launch — the standard configuration channel of Unix. Set with name=value, promoted into the environment with export, inspected with env/printenv. The system's own residents: PATH (command search), HOME, USER, SHELL, LANG. Modern operations run on them — database URLs, API keys, log levels — because they configure a process without touching files or code.
The details that separate candidates: the copy-at-launch semantics and both one-way consequences (parent changes don't reach running children — hence restart-after-config-change; child changes don't reach the parent — hence cd is a builtin); plus one security note: environment variables of a process are visible to root and, historically, to more — secrets belong in secret stores, with env vars as the delivery leg.
🎯 "What is the purpose of the alias command?" — asked verbatim in WeCreateProblems' 100+ Linux Commands Interview Questions (2026)
Defines a personal shorthand the interactive shell expands before running: alias ll='ls -alF'. Listed with alias, removed with unalias, made permanent in .bashrc. Good for abbreviating your own high-frequency commands; inspected with type name when behaviour surprises.
The details that separate candidates: the two boundaries — aliases don't exist in scripts (non-interactive shells skip them) and don't travel to other machines; and the safety-alias critique: alias rm='rm -i' trains muscle memory that betrays you on every machine without it — seniors mention this unprompted.
Part B — Quoting and globbing: controlling expansion
B1. Double quotes — one word, blanks still filled
🧠 Module 2 showed word splitting eating a filename with spaces; A1 showed variables expanding. Quoting is how you control both. Double quotes do two things at once: hold their contents together as one word (splitting suppressed), while still allowing $variable and $(command) expansion inside. The professional default is simple and worth adopting today, wholesale: always double-quote variable expansions — "$file", not $file. Unquoted expansions get re-split on spaces after expansion, which is the single largest class of shell bugs in existence.
Double quotes are a padded envelope: whatever goes in travels as one parcel, however many spaces it contains. The envelope is transparent to the mail-merge — blanks inside still get filled — but the parcel never splits in transit.
Where the analogy stops working. An envelope is optional packaging. In shell, unpackaged parcels are actively cut apart at every space by the machinery — quoting is not politeness, it is the difference between one argument and several, and the cutting happens silently on the way to the command.
🧪 Exercise B1.1 — The bug, then the fix
cd ~/shellwork
file="release notes.txt"
touch "$file" # quoted: creates ONE file
ls -l $file # unquoted: what does ls receive?
ls -l "$file" # quoted: and now?✅ Expected result — first ls fails, on purpose — click to reveal
ls: cannot access 'release': No such file or directory
ls: cannot access 'notes.txt': No such file or directory
-rw-rw-r-- 1 zaeem zaeem 0 Sep 2 16:05 'release notes.txt'What to read out of it (owner and dates are your machine's own):
- The unquoted $file expanded to release notes.txt and was then split: ls received two arguments, and Module 2's two-error signature appeared — same disease, now with the mechanism fully visible: expansion first, splitting second, command last.
- The quoted version delivered one argument. Note ls itself quoting the name in its output — modern coreutils flags awkward names back at you.
- Re-read Module 2's D3 with today's eyes: everything there was this section, experienced before it could be explained.
B2. Single quotes — nothing expands
🧠 Single quotes suppress everything: no splitting, no $ expansion, no substitution — the contents pass through byte-for-byte. Use them when the text must survive literally: strings containing $, sed/awk programs (Modules 6–7 depend on this constantly), anything destined for a different interpreter than your shell. One hard grammar rule: there is no escaping inside single quotes — a single-quoted string simply cannot contain a single quote; switch to double quotes for those.
Double quotes are relaying a message: "tell them the meeting is at [whatever time the calendar says]" — blanks filled in transit. Single quotes are a court transcript: every character verbatim, blanks and all, no interpretation permitted.
Where the analogy stops working. A human transcriber still fixes obvious slips. Single quotes fix nothing and interpret nothing — which is exactly their value: when the recipient is another program with its own $ syntax (awk, notably), only a verbatim handoff prevents your shell from eating the other program's grammar.
🧪 Exercise B2.1 — The two quote types, side by side
user=zaeem
echo "logged in as $user"
echo 'logged in as $user'
echo "it costs \$5" # backslash: escape ONE character inside double quotes✅ Expected result — click to reveal
logged in as zaeem
logged in as $user
it costs $5What to read out of it:
- Same line, three treatments: expanded, verbatim, and selectively-escaped. The decision procedure, permanently: want expansion → double quotes; want nothing touched → single quotes; want one exception → backslash it inside double quotes.
- Line two is not a failure — printing literal $user is exactly what documentation, config templates, and code destined for other interpreters need.
B3. Globbing — patterns the shell expands into filenames
🧠 * matches any run of characters, ? exactly one, [abc]/[0-9] one from a set. The mechanism matters more than the symbols: the shell expands the pattern into a list of matching filenames before the command runs — ls *.log never shows ls a star; ls receives app.log db.log, already resolved. Three consequences to internalize: dotfiles are excluded (Module 2's backup-missed-the-dotfiles bug, explained at last); a pattern matching nothing is passed through literally, star and all (bash's default — the error then comes from the command, complaining about a file literally named *.pdf); and since expansion precedes execution, echo before any glob shows you precisely what a destructive command would receive.
A glob is a stencil held against the shelf: every label showing through is picked, and the picked items — not the stencil — go into the requisition. The warehouse worker filling the order never sees the stencil.
Where the analogy stops working. A stencil matching nothing yields an empty requisition a human would question. Bash hands over the stencil itself as if it were an item name — a command then hunts for a file literally called *.pdf. Useful error, weird mechanism; scripts guard against it explicitly (Module 10's nullglob).
🧪 Exercise B3.1 — Watch expansion happen (last line fails on purpose)
cd ~/shellwork
touch app.log db.log notes.txt
echo *.log # echo shows what ANY command would receive
echo *
ls *.pdf # a pattern with no matches✅ Expected result — an error, on purpose — click to reveal
app.log db.log
app.log db.log notes.txt release notes.txt
ls: cannot access '*.pdf': No such file or directoryWhat to read out of it:
- The echos display the post-expansion command line — the exact arguments any command in that position would receive. echo + glob is the free dry run for every rm/mv/cp you will ever aim at a pattern; use it before the real thing, always.
- Line two carries a subtle horror: release notes.txt (B1's file) appears as two space-separated words indistinguishable from separate names. Expansion output is a split word list, not a display listing — globs and spaced filenames interact viciously, another reason the professional convention avoids spaces in names.
- The failure line: no match, so ls received the literal string *.pdf and looked for that file. Read the quotes in the error — they are showing you exactly what it received.
Part B — Interview questions
🎯 "How can you use wildcards in Linux commands?" — asked verbatim in WeCreateProblems' 100+ Linux Commands Interview Questions (2026)
* (any run), ? (one character), [a-z]/[abc] (one from a set) — written in any argument position: ls *.log, rm report-2026-0?.csv, cp deploy-[ab].conf backup/. The shell — not the command — expands the pattern into matching filenames before execution; dotfiles are excluded unless the pattern itself starts with a dot.
The details that separate candidates: stating who expands (the shell), with a consequence or two — quoting disables matching; empty matches pass the literal pattern through; huge matches overflow the argument list — and the dry-run habit of previewing any destructive glob with echo first. Recall-level answers list the symbols; systems-level answers explain the mechanism.
🎯 Corpus note — quoting questions
Published question banks barely touch quoting — none of the sets used in this track ask about single-versus-double quotes directly. Yet quoting failures are a favourite live probe: interviewers show a broken rm $file or an awk one-liner mangled by double quotes and ask "why?". Prepare by being able to narrate B1.1's failure — expansion, then splitting, then execution — without notes; that narration is the answer to every quoting question they can construct.
Part C — Finding commands, and hearing how they finished
C1. PATH — the shell's search route
🧠 Type date and the shell must find a program by that name. The route is fixed and fast:
Diagram source
flowchart TD
A["You type a name"] --> B{"Alias?"}
B -->|"yes"| C["Expand alias, restart"]
B -->|"no"| D{"Shell builtin?"}
D -->|"yes"| E["Run inside the shell"]
D -->|"no"| F{"Cached location<br>(hash table)?"}
F -->|"yes"| G["Run cached program"]
F -->|"no"| H["Walk PATH directories<br>in order, first hit wins"]
H -->|"found"| I["Cache it, run it"]
H -->|"nothing"| J["command not found"]PATH is an environment variable: a colon-separated directory list, searched left to right, first match wins. A typical user's PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin (often with ~/.local/bin prepended). Module 1's type and its hashed note, Module 4's ./script requirement — all of it is this one mechanism. Extend it in .bashrc: export PATH="$HOME/bin:$PATH" — prepend your directory, keep the rest.
The workshop keeps an ordered list of suppliers taped by the phone. Need a part? Call down the list, first supplier who stocks it wins, and the clerk pencils a note remembering who had it (the hash cache). "We don't stock that anywhere" is command not found.
Where the analogy stops working. A clerk might try a supplier not on the list in a pinch. The shell never will — a program in an unlisted directory does not exist to bare-name lookup, however plainly you can see it with ls. The list is not advice; it is the entire universe of names.
🧪 Exercise C1.1 — Read your route, then step off it (second half fails on purpose)
echo $PATH
type date # where on the route is date?
cd ~/perms # Module 4's sandbox, home of hello.sh
hello.sh # bare name — not on the route
./hello.sh # explicit address✅ Expected result — an error, on purpose — click to reveal
/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
date is /usr/bin/date
bash: hello.sh: command not found
hello from my first script(Your PATH will differ in detail — tool installers love adding entries; type date may say hashed if you've run date this session, Module 1's footnote.)
What to read out of it:
- Walk date's lookup along your own PATH string, directory by directory, to /usr/bin — make the mechanism concrete once and it is yours forever.
- The failure is the PATH working as designed: ~/perms is not on the route, so the bare name does not exist, while ./ bypasses the route entirely with an explicit address. command not found for something you can see now has a precise diagnosis: right file, unlisted directory.
C2. Exit codes — how programs say how it went
🧠 Every command finishes with a number: the exit code — 0 for success, anything else for failure. The shell stores the last one in $?. This is the machine-readable verdict behind Module 1's "silence is success": humans read absence-of-output; automation reads the number. Codes you have already caused, now with their names: 1/2 — general errors (the program chooses), 126 — found but not executable (Module 4's pre-chmod script), 127 — not found (every typo since Module 1).
Codes power the shell's conditional operators, the foundation of all automation: a && b — run b only if a succeeded; a || b — run b only if a failed; a ; b — run b regardless. Read && as "and then", || as "or else".
Every job leaving a workshop bay gets a tag: green 0, or a red tag bearing a reason code. The next station reads the tag, not the workpiece: green → proceed (&&); red → side-track to rework (||). Nobody re-inspects; the tag is trusted.
Where the analogy stops working. Tags are hung by the worker who did the job — and programs, like workers, can tag wrong. A badly written script that ignores its own failures exits 0, and everything downstream trusts the lie. Module 10's set -e discipline exists to keep your own tags honest; distrust of others' tags is why deploy pipelines verify results independently.
🧪 Exercise C2.1 — Read verdicts, then chain on them
ls notes.txt # cd ~/shellwork first if needed
echo $?
ls missing.txt
echo $?
mkdir -p /tmp/demo && echo "created" || echo "failed"
cp missing.txt /tmp/ && echo "copied" || echo "copy failed"✅ Expected result — click to reveal
notes.txt
0
ls: cannot access 'missing.txt': No such file or directory
2
created
cp: cannot stat 'missing.txt': No such file or directory
copy failedWhat to read out of it:
- 0 then 2: the invisible verdict, made visible. One catch worth learning now: $? holds only the most recent verdict — even your echo $? overwrote it (with echo's own 0). Capture it immediately if you need it: status=$?.
- The chains: && fired the success branch; on the failing cp, && was skipped and || fired. The a && b || c idiom reads like if/else and mostly behaves like one (the true conditional arrives in Module 10).
- ls chose 2 for "serious trouble" — each program defines its own nonzero vocabulary; only 0 has a universal meaning.
Part C — Interview questions
🎯 "What is the significance of the PATH variable?" — asked verbatim in WeCreateProblems' 100+ Linux Commands Interview Questions (2026)
PATH is the colon-separated, ordered directory list the shell walks to resolve bare command names — first match wins, misses everywhere mean command not found. It is why installed-but-elsewhere tools "don't exist", why ./script is needed for the current directory (which is excluded, deliberately, for security), and why the same command name can resolve differently for different users or in cron. Extended per-user via .bashrc (export PATH="$HOME/bin:$PATH").
The details that separate candidates: search order as an attack/override surface (prepending a directory shadows system commands — used both by installers and by attackers); the security story for excluding .; and the operational classic — cron and other non-interactive contexts carry a minimal PATH, which is why "works in my terminal, fails in the scheduler" is a PATH bug until proven otherwise.
🎯 Corpus note — exit codes
No published set used in this track asks about exit codes directly — and yet $?, &&/||, and "how does your pipeline know the deploy failed?" come up in nearly every live DevOps interview, usually inside a scripting or CI question. Be able to state cold: 0 is success and the only universal code; $? holds the last verdict and is immediately perishable; 126 vs 127; and that automation trusts codes, not output — then connect it to set -e when scripting comes up. The absence from question banks makes fluency here more distinguishing, not less.
Part D — Plumbing: streams, redirection, and pipes
D1. Three streams, and re-aiming them
🧠 Every process is born holding three open channels, numbered: 0 stdin (input, usually your keyboard), 1 stdout (results), 2 stderr (errors and diagnostics). Separating 1 from 2 is the design insight of Unix plumbing: results and complaints travel different wires, so each can be aimed independently:
- > file — send stdout to file (created or overwritten); >> file — append instead.
- 2> file — send stderr somewhere; 2>> appends.
- 2>&1 — send stderr to wherever stdout currently points (order matters: > log 2>&1).
- > /dev/null — the kernel's bottomless discard device, for output you have decided not to want.
- < file — feed a file to stdin.
A machine on the factory floor has a product conveyor (stdout) and a reject chute (stderr). Redirection is re-aiming them: conveyor into a shipping crate (> file), chute into the incinerator (2>/dev/null), or both into one bin for later sorting (> file 2>&1). The machine itself is unmodified — plumbing is arranged around it.
Where the analogy stops working. Re-aim a physical chute mid-shift and subsequent output follows. Shell redirections are plumbed once, before startup, by left-to-right snapshot — which is why 2>&1 > log (aim chute at conveyor's current target: the terminal; then move conveyor) differs from > log 2>&1 (move conveyor to log; aim chute at it). The order bug produces "why is stderr still on my screen" tickets weekly, worldwide.
🧪 Exercise D1.1 — Split the streams, then merge them
cd ~/shellwork
ls notes.txt missing.txt # both streams to the terminal, interleaved
ls notes.txt missing.txt > out.txt 2> err.txt # separated
cat out.txt
cat err.txt
ls notes.txt missing.txt > all.txt 2>&1 # merged
cat all.txt
ls missing.txt 2>/dev/null # complaint, discarded
echo $?✅ Expected result — click to reveal
ls: cannot access 'missing.txt': No such file or directory
notes.txt
notes.txt
ls: cannot access 'missing.txt': No such file or directory
ls: cannot access 'missing.txt': No such file or directory
notes.txt
2What to read out of it:
- One command, two files, cleanly separated: results in out.txt, complaint in err.txt. This is how every scheduled job should be plumbed (a Module 11 refrain).
- In the merged file, the error may appear before the result — the two streams buffer differently, so interleaving order is not guaranteed. Real logs have this texture; timestamps, not position, order events.
- The discard line printed nothing — but $? still says 2. Silencing a stream does not silence the verdict: automation built on exit codes keeps working with output thrown away. Discard output only when you have decided the code is what you act on.
D2. Pipes — plugging programs into each other
🧠 a | b connects a's stdout directly to b's stdin — no file in between, no disk touched; the data flows through kernel memory. This is the composition mechanism the small-tools philosophy (Module 1, Part D) was designed around: each tool does one thing; pipes make sentences from the words. You have borrowed it twice (| head); now it is formally yours. Two mechanics worth knowing from day one: all programs in a pipeline run concurrently (b starts consuming while a still produces — a pipeline of huge inputs needs no huge memory), and stderr is not piped — complaints from every stage land on your terminal, bypassing the plumbing.
A bottling line: the filler feeds the capper feeds the labeler, product flowing continuously — nobody waits for "all bottles filled" before capping starts. Each machine was built independently; the conveyors between them make it a line.
Where the analogy stops working. Factory conveyors can buffer pallets on the floor. A pipe holds only a small kernel buffer: when a downstream stage stalls, upstream stages block — which is why somecommand | less uses no memory on gigabytes (production is paused while you read), and why one slow stage sets the whole pipeline's pace.
🧪 Exercise D2.1 — Compose your existing vocabulary
ls /etc | wc -l # how many entries in /etc?
ls -l /var/log | head -5 # long listing, first five lines only
tail -20 ~/.bash_history | wc -l # how much recent history? (a brand-new account may not have this file yet — see the result)✅ Expected result — click to reveal
228
total 808
lrwxrwxrwx 1 root root 39 May 8 16:23 README -> ../../usr/share/doc/systemd/README.logs
-rw-r--r-- 1 root root 45911 Sep 2 14:22 alternatives.log
drwxr-xr-x 2 root root 4096 Sep 2 14:22 apt
-rw-r--r-- 1 root root 61229 Apr 10 02:20 bootstrap.log
20What to read out of it (counts are your machine's own):
- Every command here was taught modules ago; the pipe is the only new ingredient, and suddenly questions ("how many?", "just the top") become one-liners. This is the inflection point of the whole track: from running commands to composing them.
- Note head receiving ls -l's output mid-production and cutting it off at five lines — ls is then simply terminated early. Concurrency, visible.
- If the last line instead printed tail: cannot open ... No such file or directory and then 0: nothing is broken — bash writes the history file at logout, so a brand-new account has not created one yet. Log out, back in, and it exists. (Note wc still printed a count: the pipe carried tail's empty stdout while the error took the stderr bypass.)
- Modules 6 and 7 supply the great pipeline citizens — grep, sort, awk — and this section is why they compose.
D3. Command substitution — output as input
🧠 $(command) runs the command and pastes its output into the surrounding line — the last of the shell's big expansions, and the glue for "use this result inside that command". You met it once on faith in Module 4 (chown $(whoami)); the mechanics: runs in a subshell, output captured, trailing newline stripped, result subject to word splitting unless double-quoted — so the B1 rule applies with full force: "$(command)".
Filling a form that asks for today's exchange rate, you pause, make a phone call, write the answer into the blank, and submit. $(...) is the phone call: a sub-errand whose answer lands in the blank mid-sentence.
Where the analogy stops working. You would notice a phone call answering with three paragraphs and summarize. The shell pastes whatever came back — multi-line output and all — then word-splits it into the sentence unless quoted. The blank does not judge; it holds what it is given.
🧪 Exercise D3.1 — Results inside commands
echo "there are $(ls | wc -l) files here"
backup="notes-$(date +%F).txt" # date +%F prints YYYY-MM-DD
cp notes.txt "$backup"
ls notes-*✅ Expected result — click to reveal
there are 7 files here
notes-2026-09-02.txtWhat to read out of it (count and date are your machine's own):
- A pipeline ran inside a string, its answer landing mid-sentence. Substitution composes with everything — including pipes.
- The dated-backup idiom is used verbatim across the industry: timestamped copies before risky changes, built from Module 2's cp, Module 1's date, and today's $(...). You now construct file names from live data — carefully quoted.
Part D — Interview questions
🎯 "What is meant by PIPE in Linux?" — asked verbatim in Turing's 100+ Linux interview questions (2025); WeCreateProblems (2026) words it "What is a pipe, and how is it used in Linux?"
| connects one command's stdout to the next command's stdin through a kernel buffer — no intermediate file, no disk. It is the composition operator of the small-tools philosophy: ls -l | grep conf | wc -l chains three single-purpose tools into an answer. Stages run concurrently, memory use stays flat regardless of data volume, and stderr bypasses the pipe entirely, arriving at the terminal.
The details that separate candidates: concurrency (not batch hand-off) and its corollary — a stalled consumer blocks the producer; stderr's separate path; and the exit-code subtlety: a pipeline's status is the last command's, so failing-cmd | tee log reports success — the pipefail fix belongs in every professional script (Module 10 installs it).
🎯 "What is redirection?" — asked verbatim in Turing's 100+ Linux interview questions (2025); WeCreateProblems (2026) asks "How do you redirect output to a file?"
Re-aiming a process's standard streams before it runs: > stdout to a file (truncating), >> appending, 2> stderr, 2>&1 stderr to stdout's current target, < file as stdin, /dev/null as the discard. Canonical job form: command >> /var/log/job.log 2>&1. The shell sets up all plumbing left-to-right before executing, which is why 2>&1's position matters.
The details that separate candidates: the truncate-before-run trap (sort f > f destroys f); the > log 2>&1 vs 2>&1 > log ordering, explained via the snapshot mechanism rather than recited; and separating streams on purpose — results parseable, errors visible — as a design habit, not an exam answer.
🎯 Corpus note — command substitution
Not asked directly in any recent published set consulted — but it appears inside the answers to scripting questions everywhere ("store the output of a command in a variable"). Own the one-liner: $(cmd) captures output, trailing newline stripped, quote it or it word-splits. Backtick syntax cmd is the legacy spelling — recognize it in old scripts, write the modern form.
Part E — Toolkit
E1. Production practice — symptoms and fixes
| Symptom | What is really happening | What to run | The fix |
|---|---|---|---|
| Works in your terminal; command not found in automation | Automation contexts (cron, systemd) provide a minimal PATH, and non-interactive shells read no .bashrc — your PATH additions and aliases don't exist there | type thecommand interactively · compare echo $PATH in both contexts | Absolute paths in automation, or set PATH explicitly at the top of the job |
| A variable "loses its value" in a subshell or script | It was never exported — children got no copy | bash -c 'echo $VAR' as a probe | export VAR; for services, set it in the service's own config (Module 11) |
| Command mangles a filename into two, or acts on wrong files | Unquoted expansion re-split on spaces; or a glob expanded beyond intent | Prefix the line with echo and read what it becomes | Double-quote every expansion; dry-run globs with echo before destructive use |
| A log or data file is suddenly empty after a "failed" command | > truncated it before the command ran — failure came after the damage | ls -l timestamps; reconstruct the redirection | Never redirect a file onto itself; write to a new name, move after success (&&) |
| "I redirected everything to the log but errors still hit the screen" | 2>&1 placed before > — stderr aimed at the terminal snapshot | Re-read the line left to right, narrating each plumbing step | Order it > file 2>&1 |
| Pipeline "succeeded" but produced garbage/nothing | An early stage failed; the pipeline's exit code is the last stage's | Check stderr (it bypassed the pipe) · echo $? per stage while debugging | set -o pipefail in scripts (Module 10); verify outputs, not just codes |
E2. Capstone — four tickets
🎫 Ticket 1 — "The nightly job says command not found; the same line works when I paste it"
Ticket text: "Our scheduled backup job logs backup-tool: command not found. Pasting the identical line into a terminal works perfectly. The tool is installed in /opt/backup/bin. Explain and fix."
Worked answer: the terminal shell read .bashrc, which prepends /opt/backup/bin to PATH; the scheduler's non-interactive shell reads no such file and carries a minimal PATH — so the bare name resolves in one context and not the other. Same command, different universes of names. Prove it: log echo "$PATH" from inside the job and compare. Fix, in order of preference: call the tool by absolute path (/opt/backup/bin/backup-tool) — automation should not depend on name resolution; or set PATH=/opt/backup/bin:/usr/bin:/bin explicitly at the top of the job. The general law, worth writing on the wall: interactive convenience (PATH edits, aliases, .bashrc) does not exist in automation.
🎫 Ticket 2 — "Cleanup script wiped the wrong directory"
Ticket text: "A script ran `rm -rf ${STAGING_DIR}` and deleted the working directory it was launched from, not the staging area. STAGING_DIR is set in the engineer's .bashrc and 'has always worked'. Postmortem the mechanism and the guards."*
Worked answer: mechanism — the script ran non-interactively, read no .bashrc, so STAGING_DIR was unset; the expansion produced nothing, leaving plain rm -rf *, which faithfully wiped whatever directory the script happened to be launched from (A1's form letter, sent with its blank unfilled). Had it been written rm -rf $STAGING_DIR/, the empty expansion would have produced rm -rf / — refused by GNU rm's --preserve-root failsafe, but a script should never get to lean on that final guard. Guards, all of which belong in Module 10's script prologue: quote and require the variable ("${STAGING_DIR:?not set}" — the :? form aborts with a message when unset; meet it properly in Module 10), set -u to make unset expansion fatal, never end an rm path with a bare expansion, and dry-run with echo in review. The .bashrc dependency was the root cause: configuration for automation lives in the automation, never in a human's dotfiles.
🎫 Ticket 3 — "The deploy log is empty but the deploy clearly ran (and failed)"
Ticket text: "Runbook line: deploy.sh 2>&1 > /var/log/deploy.log. The deploy failed; the log contains only the success chatter, and the errors were lost with the closed terminal. Why?"
Worked answer: plumbing order. Left to right: 2>&1 first — aim stderr at stdout's current target, the terminal; then > /var/log/deploy.log — move stdout to the log, stderr staying where it was pointed. Result: chatter logged, errors to a terminal nobody kept. The fix is transposition: deploy.sh > /var/log/deploy.log 2>&1 (stdout to log first, then stderr to the same place). Better runbook practice: split them — >> deploy.log 2>> deploy.err — so failures are findable without scrolling chatter, and use >> so reruns never truncate history. The interviewable sentence: redirections are snapshots evaluated left to right, not live links.
🎫 Ticket 4 — "Report generator overwrote its own input"
Ticket text: "An engineer 'cleaned up' a data file with sort data.csv > data.csv and the file is now zero bytes. They insist sort must be buggy. Explain, recover, prevent."
Worked answer: not sort's doing — the shell's. Before sort launched, the shell opened data.csv for stdout redirection, truncating it to zero; sort then read an already-empty file and correctly wrote nothing. The program never had a chance to see the data. Recovery: backups or re-generation — the bytes are gone (Module 3's shredder rule). Prevention: write to a new name and swap on success — sort data.csv > data.sorted && mv data.sorted data.csv — which also leaves the original intact if sort fails (the && earning its keep); Module 7 adds sort -o data.csv data.csv, sort's own safe in-place flag, and the general lesson stands: the shell sets up all plumbing before the command runs.
E3. Documentation reference
| Topic | Authoritative source | Verified link |
|---|---|---|
| Everything in this module | The Bash Reference Manual (variables, quoting, expansion, PATH, redirection, pipelines) | Bash Reference Manual |
| The environment | environ(7), env(1) | environ(7) · env(1) |
| Glob patterns | glob(7) | glob(7) |
| The discard device | null(4) | null(4) |
E4. Self-assessment
Answer out loud, without notes. The section number tells you where to re-read.
- name = value fails with command not found. Narrate exactly how the shell parsed it. (A1)
- A variable is visible in your shell but empty in every child. One word names the fix — and what are the two one-way rules of the environment? (A2)
- Why must cd be a builtin? Answer using this module's model, not Module 2's assertion. (A2)
- Which file do you edit for permanent settings on Ubuntu, how do you apply it without a new terminal, and why do scripts ignore it? (A3–A4)
- State the quoting decision procedure in three clauses, and why "$file" is the professional default. (B1–B2)
- Who expands *.log — and derive two consequences of that answer (quoted glob; million-file directory). (B3)
- A command you can see with ls is command not found by bare name. Diagnose, and explain why . is off the PATH. (C1)
- What do exit codes 0, 126, and 127 mean, why is $? perishable, and what do && and || read as? (C2)
- Plumb from memory: stdout and stderr to the same log file, appending. Then explain why the reversed order fails. (D1)
- sort f > f empties f. Whose fault, and what is the safe pattern? (D1, Ticket 4)
- Two facts about pipes that surprise beginners: when do stages run, and which stream escapes the pipe? (D2)
- Build, from memory, a dated backup filename using command substitution — with correct quoting. (D3)
E5. Sources
Turing — 100+ Linux Interview Questions (2025) · WeCreateProblems — 100+ Linux Commands Interview Questions (2026) · Adaface — 96 Linux Commands interview questions (September 2024, pipes/redirection material).
Corpus honesty note: shell mechanics are underrepresented in published question banks — pipes, redirection, environment variables and PATH appear (cited above), but quoting, exit codes, and command substitution are asked almost exclusively live, inside scripting scenarios. Where this page marks a "corpus note" instead of a source, that is the honest state of the published record: the material is interview-critical but tested by demonstration, not by quiz.
All documentation links on this page were fetched and confirmed reachable on 2 September 2026.