Module 15 — Debugging, Testing, and Production Scripts
Updated 3 September 2026
🧠 concept → 🧩 analogy → 🧪 exercise → ✅ expected result (hidden) → 🎯 interview questions (hidden)
Part A — Debugging: making bash narrate
A1. set -x: the running commentary
set -x (or invoking with bash -x script.sh) makes bash print every command after expansion, before execution, prefixed with +, on stderr. That "after expansion" is the whole value: you see what the command became — variables filled in, quotes resolved, splits performed — which is exactly the layer where twelve modules of bugs live. The prefix is customizable through the variable PS4, and one upgrade earns its place in every toolbox: file-and-line stamps.
export PS4='+ ${BASH_SOURCE##*/}:${LINENO}: '
set -xScope tracing like a scalpel, not a floodlight: set -x before the suspect region, set +x after — a 400-line script traced end-to-end is noise, ten traced lines are a diagnosis.
Where the analogy stops working. Flight recorders are sealed against the crew. Your trace is not sealed against your secrets: set -x prints expanded commands, so curl -H "Authorization: Bearer $TOKEN" writes the token into stderr — and into whatever log captures it. Production rule: set +x around secret-handling regions, always, and treat trace output as sensitive (Module 7's read -rs discipline extends to traces).
🧪 Exercise 15.1 — the narrated run
cd ~/bash-course
nano tracedemo.sh # six lines:#!/bin/bash
export PS4='+ ${BASH_SOURCE##*/}:${LINENO}: '
set -x
name="Ada"
greeting="Hello, $name"
echo "$greeting"chmod +x tracedemo.sh
./tracedemo.sh✅ Expected result — click to reveal
+ tracedemo.sh:4: name=Ada
+ tracedemo.sh:5: greeting='Hello, Ada'
+ tracedemo.sh:6: echo 'Hello, Ada'
Hello, AdaWhat to read out of it: three trace lines (stderr) and one output line (stdout) — the channels stay separate, so ./tracedemo.sh 2>trace.log files the narration away from the results (Module 4, still paying dividends). Look at line 5's trace: greeting='Hello, Ada' — the variable already expanded, quotes shown as bash re-quotes them. When a quoting bug strikes, this view is the argument-boundary probe (Module 2 D1) applied to every line at once. The PS4 stamps turn "somewhere" into tracedemo.sh:5.
A2. bash -n: the parse-only pass, and the debugging ladder
bash -n script.sh parses without executing: syntax errors (the unclosed if, the missing fi) surface instantly, with zero commands run — the only completely safe check you can give an untrusted or half-edited script. (bash -v echoes lines as read, pre-expansion — occasionally useful for spotting what bash even sees; -x remains the workhorse.) The professional ladder, cheapest rung first: ShellCheck (static, catches whole bug classes before any run) → bash -n (parses?) → probe prints (printf '[%s]\n' at the suspect value — Module 2) → scoped set -x (narrate the suspect region) → bats (Part B: pin the fix so it stays fixed).
🧪 Exercise 15.2 — catch a syntax error without running anything
The file is broken on purpose.
cd ~/bash-course
printf '#!/bin/bash\nif true; then\n echo hi\n' > broken.sh # note: no "fi"
bash -n broken.sh ; echo "syntax-verdict: $?"
bash -n tracedemo.sh && echo "tracedemo parses"✅ Expected result — click to reveal (contains a deliberate break)
broken.sh: line 4: syntax error: unexpected end of file
syntax-verdict: 2
tracedemo parsesWhat to read out of it: the missing fi was reported at end of file — where bash finally gave up waiting for it, not where you forgot it; syntax errors point at the discovery site, and the cause is usually above (unclosed if/do/quote). Verdict 2 = bash's own usage-error convention (Module 3). Nothing in broken.sh executed — which is why bash -n belongs in pre-commit hooks and CI (paired with ShellCheck, which would also have said more).
Interview questions — Part A
🎯 "How will you debug a shell script?" — asked verbatim at InterviewBit; Edureka asks "How to debug the problems encountered in the shell script/program?" and "Can you write a script to portray how set –x works?"; KnowledgeHut asks "How do you debug a shell script?"
The direct answer, as the ladder: ShellCheck first (static analysis names bug classes); bash -n for pure syntax; probe prints (printf '[%s]\n' "$var") at suspect values; set -x / bash -x for a narrated run — scoped with set +x, upgraded with a file:line PS4; and an ERR trap ($LINENO, $BASH_COMMAND — Module 14) for unattended runs where nobody watches traces.
Going deeper: say what -x actually shows (commands after expansion) and why that matters (quoting/splitting bugs become visible); demonstrate the Edureka Q59 shape on demand — the same loop run plain and under -x, narrating the difference.
The details that separate candidates: the secrets caveat (traces print expanded tokens — set +x around credential handling); PS4's exported-function nuance kept simple (set it inside the script before set -x — env-passed PS4 is unreliable); and positioning the ladder as cost-ordered — static before dynamic, cheap before invasive — which is the debugging philosophy interviewers actually probe for.
Part B — bats: tests that keep scripts fixed
B1. Your first test file
bats (Bash Automated Testing System) runs test files where each @test "name" { … } block is a bash snippet that passes if every command in it succeeds — Module 3's verdicts, promoted into a test framework. Its helper run command executes a command and captures $status (the verdict) and $output (stdout+stderr) for you to assert against. The one structural requirement is Module 8's template paying off: tests source your script to reach its functions, so the script must be source-safe — definitions plus a guarded main. And the guard has a sharp edge worth learning from a real failure:
🧪 Exercise 15.3 — a testable script, and its tests
cd ~/bash-course
nano greet.sh # a source-safe script:#!/bin/bash
greet() {
[ "$#" -eq 1 ] || return 2
echo "Hello, $1"
}
if [[ "${BASH_SOURCE[0]}" == "$0" ]]; then
greet "$@"
finano greet.bats # the tests:#!/usr/bin/env bats
setup() {
source ./greet.sh
}
@test "greets a user by name" {
run greet Ada
[ "$status" -eq 0 ]
[ "$output" = "Hello, Ada" ]
}
@test "rejects a missing argument" {
run greet
[ "$status" -eq 2 ]
}chmod +x greet.sh
bats greet.bats
./greet.sh Ada # still a working script, too✅ Expected result — click to reveal
1..2
ok 1 greets a user by name
ok 2 rejects a missing argument
Hello, AdaWhat to read out of it: two tests, two ok lines (the 1..2 header is TAP, the test-output standard bats speaks — CI systems parse it natively). Note what got tested: the happy path and the failure contract — return 2 on bad usage is part of greet's interface (Module 3's codes-as-API), and now it cannot regress silently. The final line proves the same file still executes normally: source-safe and runnable are not in tension; they are the template.
🧪 Exercise 15.4 — watch a test fail (that's the point of having them)
The test fails on purpose.
cd ~/bash-course
nano greet2.bats # one wishful test:#!/usr/bin/env bats
setup() { source ./greet.sh; }
@test "greeting includes an exclamation mark" {
run greet Ada
[ "$output" = "Hello, Ada!" ]
}bats greet2.bats ; echo "bats-verdict: $?"✅ Expected result — click to reveal (contains a deliberate failure)
1..1
not ok 1 greeting includes an exclamation mark
# (in test file greet2.bats, line 7)
# `[ "$output" = "Hello, Ada!" ]' failed
bats-verdict: 1What to read out of it: not ok, the file, the line, the exact assertion that failed — and bats itself exited 1 (a trustworthy verdict, so CI gates on it with zero glue). This is the Module 8 refactor-with-regression-test habit industrialized: change greet, run bats, and the tests — not your memory — say whether behavior held. When you fix a production bug from now on, the professional move is: write the failing test first, watch it go not ok, fix, watch it go ok, commit both.
Interview questions — Part B
🎯 "How do you test shell scripts?" — asked in DevOps screens with growing frequency; no major published list carries a canonical wording yet (the bats-core docs are the de facto reference), so treat the topic as the question
The direct answer: structure scripts as functions with a guarded main (source-safe — Module 8's template); unit-test the functions with bats (run fn args, assert on $status and $output); lint with ShellCheck in the same CI job; and smoke-test the script end-to-end against a scratch environment (mktemp -d fixtures — Module 14).
Going deeper: what makes bash hard to test — side effects everywhere — and how the track's habits counter it: pure-ish functions that print results (Module 8's channels), dependency points injectable via variables (CURL=${CURL:-curl}), scratch dirs per test in bats' setup/teardown. Name TAP output as the CI integration story.
The details that separate candidates: the source-guard's && gotcha (source's exit status = the failed test's — tests die in setup; if-form fixes it) — a war story that proves hands-on testing; and honest scope judgment: bats for logic and contracts, not for re-testing tar or curl — "I test my code, not the coreutils" is the sentence that lands.
Part C — Cron: where scripts meet the empty room
C1. The schedule, and the environment that isn't yours
crontab -e edits your schedule; each line is five time fields then a command: minute (0-59), hour (0-23), day of month (1-31), month (1-12), day of week (0-7, both 0 and 7 = Sunday) — 30 2 * * * /opt/scripts/backup.sh runs nightly at 02:30. That part everyone learns in ten minutes. What fills the rest of a career is the environment: cron runs your command in a nearly empty room — SHELL is /bin/sh (not bash! — Module 1's dialect trap, institutionalized: the crontab(5) man page says so), PATH is minimal (typically just /usr/bin:/bin — your login PATH, aliases, and bashrc do not exist here), no terminal attached ([ -t 0 ] says robot — Module 7), and your working directory is your home, not your project. The classic consequences: command not found (127) for tools that "work fine when I run it" (Module 3's E2 table row, now with its natural habitat), prompts that hang or drain stdin (Module 7 Ticket 4), and [[ ]]-syntax errors when a bash script gets run by sh.
The defensive crontab entry, therefore, carries its own weather: absolute paths, explicit logging, and a bash invocation if there's any doubt:
30 2 * * * /usr/bin/flock -n /var/lock/backup.lock /opt/scripts/backup.sh >> /var/log/backup.log 2>&1— Module 14's lock, Module 4's append-both-channels, absolute everything. (System-wide schedules also live in /etc/crontab and /etc/cron.d with an extra user field — reading before editing applies.)
Where the analogy stops working. A confused cleaner leaves a note or calls. Cron's failure channel is easy to miss entirely: output is mailed to a local mailbox almost nobody reads, and an un-redirected failing job can fail silently for months. The fix is structural, not attentional: every entry ends in >> logfile 2>&1 (or the script self-logs — Module 14 C1), and jobs that matter get monitored for absence — a freshness check on their output, because "the cleaner stopped coming" is otherwise invisible.
🧪 Exercise 15.5 — simulate the empty room without waiting for 02:30
env -i /bin/sh -c 'echo "PATH in a bare env: $PATH"; echo "home: $HOME"; date'✅ Expected result — click to reveal
PATH in a bare env: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
home:
Thu Sep 3 17:50:22 +08 2026What to read out of it (your date differs): env -i (run with an empty environment) is the desk-side simulator of cron's room. The PATH shown is not yours — it is the shell's own built-in fallback (dash supplies one when the environment carries none; some shells supply less, and then even date becomes not found, which is precisely what 127-in-cron feels like). The empty home: line is real: no HOME in a bare environment, so scripts that write to ~/… aim at nothing. Test any cron-bound script under env -i /bin/sh -c '…' and its hidden assumptions — PATH, HOME, your aliases, your agent — fall out at your desk instead of at 02:30.
Interview questions — Part C
🎯 "What is the Crontab?" and "How many fields are present in a crontab file and what does each field specify?" — both asked verbatim at Edureka (their companion: "What are the two files of crontab command?")
The direct answer: crontab (cron table) is the per-user schedule the cron daemon executes, managed by the crontab command (-e edit, -l list). Six fields per entry: minute, hour, day-of-month, month, day-of-week — then the command (Edureka counts the command as the sixth field). Access control: cron.allow / cron.deny list who may use crontab.
Going deeper: field syntax fluency — * any, */5 every five, 1-5 ranges, 1,15 lists; day-of-week 0 and 7 both Sunday; and the operational reading that DOM and DOW together mean either matches (a genuine standards quirk worth flagging when both are set).
The details that separate candidates: the environment story (SHELL=/bin/sh, minimal PATH, no TTY, home as cwd) delivered unprompted — because "what is crontab" is really the warm-up for "why does your script fail there"; plus the modern framing that serious scheduling increasingly lives in systemd timers (logs in the journal, dependencies, jitter) while crontab remains the lingua franca — knowing both is the senior posture.
🎯 "My script works in my shell but fails from cron — walk me through it." — the single most reliable scenario question in Linux ops screens; the mechanics are documented in crontab(5) rather than any question list
The direct answer, as the checklist: capture the evidence first (>> /tmp/job.log 2>&1 on the entry, temporarily); then the usual suspects in probability order — PATH (bare command names → 127; fix: absolute paths or set PATH in the crontab), shell (sh, not bash → [[/arrays syntax errors; fix: proper shebang and direct execution, or /bin/bash script), working directory (relative paths aim at $HOME; fix: cd in the script or absolute paths), prompts/stdin (no terminal; Module 7's rules), and missing agent/env (SSH keys, tokens your login shell exports).
Going deeper: the reproduction tool — env -i /bin/sh -c 'your command' — brings the failure to your desk instead of waiting for the schedule; and the differential question "what does my login shell provide that cron doesn't?" as the organizing principle for the whole class.
The details that separate candidates: naming the silent failure mode too (no redirect → output to unread local mail → months of quiet failure) and its structural fix (log everything, monitor for freshness/absence); and the discipline of testing cron jobs via cron once (a one-minute temporary schedule) rather than declaring victory from an interactive run — the empty room is the only honest test environment.
Part D — The capstone script: fifteen modules in forty lines
D1. safe-backup.sh, read like a veteran
This script is real: it was written for this module, run, broken once, fixed, and passed through ShellCheck clean. Every line traces to a module — read it with the annotations, then build it.
#!/bin/bash
# safe-backup.sh — archive a directory, keep N days, refuse to overlap.
# usage: safe-backup.sh SOURCE_DIR DEST_DIR (KEEP_DAYS optional, default 7)
set -euo pipefail # M14: strict mode
readonly SCRIPT_NAME="${0##*/}" # M8 template + M9 trim
readonly KEEP_DAYS="${KEEP_DAYS:-7}" # M9: documented tunable
log() { echo "[$(date '+%F %T')] [$SCRIPT_NAME] $*" >&2; } # M4: diagnostics on stderr
die() { log "ERROR: $*"; exit 1; } # M8: the ONE exiting helper
trap 'log "failed at line $LINENO: $BASH_COMMAND (exit $?)"' ERR # M14: articulate death
main() {
[ "$#" -eq 2 ] || { echo "usage: $SCRIPT_NAME SOURCE_DIR DEST_DIR" >&2; exit 2; } # M7+M3
local src="$1" dest="$2" # M8: local discipline
[ -d "$src" ] || die "no such source dir: $src" # M5: guards first
[ -d "$dest" ] || die "no such dest dir: $dest"
[[ "$KEEP_DAYS" =~ ^[0-9]+$ ]] || die "KEEP_DAYS must be a number, got: $KEEP_DAYS" # M5
exec 9>"$dest/.backup.lock" # M14: single instance
flock -n 9 || { log "another backup is running; skipping"; exit 0; }
tmp=$(mktemp -d) # deliberately NOT local: the EXIT trap fires after main returns
trap 'rm -rf "$tmp"' EXIT # M13: acquire, trap, work
local stamp; stamp=$(date +%Y%m%d-%H%M%S) # M8: split declaration (SC2155)
local archive="$dest/backup-$stamp.tar.gz"
log "archiving $src"
tar -czf "$tmp/work.tar.gz" -C "$src" . # work in scratch first
[ -s "$tmp/work.tar.gz" ] || die "archive is empty" # M5: evidence, not verdicts alone
mv "$tmp/work.tar.gz" "$archive" # atomic-ish publish
log "wrote $archive ($(wc -c < "$archive") bytes)" # M4: anonymous-stdin wc
log "pruning archives older than $KEEP_DAYS days"
find "$dest" -maxdepth 1 -name 'backup-*.tar.gz' -mtime +"$KEEP_DAYS" -print0 \
| xargs -0 -r rm -- # M11: armored pipeline
log "done"
}
main "$@" # M8: the only top-level actionThree lines deserve their war stories. The un-local tmp: the first draft used local tmp — and died at exit with tmp: unbound variable, because the EXIT trap fires after main returns, when locals are gone, and set -u made the ghost loud (Modules 8+13+14 interacting — exactly the kind of cross-module bug this track exists to make legible). The scratch-then-move: tar writes into the temp dir and the finished archive is mved into place, so a mid-run death leaves no half-written archive where monitoring looks — the EXIT trap sweeps the scratch instead (Module 3 Ticket 1's green-and-empty backup, structurally impossible here). The -s check: verdicts gate, evidence certifies — the archive must exist and contain bytes before it is published.
Diagram source
flowchart TD
A["main invoked"] --> B["usage + input guards<br>exit 2 on violation"]
B --> C{"flock -n<br>acquired?"}
C -->|"no"| D["log skip, exit 0<br>twin on duty"]
C -->|"yes"| E["mktemp -d<br>+ EXIT trap"]
E --> F["tar into scratch"]
F --> G{"archive has<br>bytes? (-s)"}
G -->|"no"| H["die — ERR trap logs<br>line + command"]
G -->|"yes"| I["mv into dest<br>(publish)"]
I --> J["prune old:<br>find -print0 | xargs -0"]
J --> K["exit 0 —<br>EXIT trap sweeps scratch"]
H --> K2["nonzero exit —<br>EXIT trap STILL sweeps"]🧪 Exercise 15.6 — build it, run it, break it
cd ~/bash-course
nano safe-backup.sh # the script above, exactly
chmod +x safe-backup.sh
mkdir -p srcdata destdata && echo "hello" > srcdata/file.txt
./safe-backup.sh srcdata destdata ; echo "verdict: $?"
ls destdata
./safe-backup.sh ; echo "usage-verdict: $?"
./safe-backup.sh /nope destdata ; echo "die-verdict: $?"
shellcheck safe-backup.sh && echo "shellcheck: clean"✅ Expected result — click to reveal (contains deliberate failures)
[2026-09-03 17:48:17] [safe-backup.sh] archiving srcdata
[2026-09-03 17:48:17] [safe-backup.sh] wrote destdata/backup-20260903-174817.tar.gz (153 bytes)
[2026-09-03 17:48:17] [safe-backup.sh] pruning archives older than 7 days
[2026-09-03 17:48:17] [safe-backup.sh] done
verdict: 0
backup-20260903-174817.tar.gz
usage: safe-backup.sh SOURCE_DIR DEST_DIR
usage-verdict: 2
[2026-09-03 17:48:18] [safe-backup.sh] ERROR: no such source dir: /nope
die-verdict: 1
shellcheck: cleanWhat to read out of it (timestamps and byte count yours; note plain ls doesn't list the dotfile .backup.lock, only the archive): a clean run with verdict 0 and evidence on disk; a usage failure speaking the Module 3/7 convention (stderr, exit 2); a guard failure through die (logged, exit 1); and a linter with nothing to say. Now break it on purpose — export KEEP_DAYS=banana, point it at an unreadable source, SIGTERM it mid-tar — and watch fifteen modules of defenses each catch their own failure class. That exploration is the final exercise of the track.
Interview questions — Part D
🎯 "Walk me through a production-quality shell script you've written." — the closing question of countless DevOps interviews; unpublished by nature, and the entire track is the preparation
The direct answer is a structure, delivered while sketching: strict mode and why; the ERR/EXIT traps and what each catches; input guards with honest exit codes; the lock (and the skip-is-not-an-error decision); scratch-then-publish so monitoring never sees half-work; armored cleanup; logging with timestamps to stderr; and tests + ShellCheck in CI keeping it all true next quarter.
Going deeper: pick one or two war stories from the details — the trap-vs-local scoping interaction, the publish-by-mv atomicity choice, the lock-skip logging decision — because interviewers at this stage are listening for scars, not syntax.
The details that separate candidates: narrating failures by class ("this line exists because of the empty-backup class, this one because of the overlap class") — which demonstrates the mental model the whole track built: production bash is not clever commands; it is defaults inverted, verdicts honored, boundaries armored, cleanup guaranteed, and everything readable at 3 a.m.
Part E — Production
E1. 🏭 Production practices
Debugging is ladder-ordered and secrets-aware. ShellCheck → bash -n → probes → scoped set -x with a file:line PS4 — and set +x fences every credential-touching region, because traces are logs and logs leak.
Scripts ship with tests, and CI runs them. bats files beside the scripts, TAP into the pipeline, ShellCheck in the same job; a bugfix lands as failing-test-then-fix, so the bug's return is impossible, not unlikely.
Scripts are source-safe by construction — functions plus the if-form main guard — because testability, library reuse (Module 8), and interactive debugging all hang off that one property.
Cron entries carry their own weather: absolute paths, explicit >> log 2>&1, flock at the crontab level, and a one-minute temporary schedule as the honest smoke test. Anything that matters is also monitored for absence — output freshness, not just exit codes.
Every unattended script is tested in the empty room (env -i /bin/sh -c …) before it meets the scheduler — PATH, HOME, TTY and agent assumptions surface at the desk.
The capstone's shape is the deliverable: guards → lock → scratch → work → verify evidence → publish → prune, with ERR/EXIT traps underneath. New operational scripts start from it, not from an empty buffer.
E2. Production-practice table
| Symptom | What is really happening | What to run | The fix |
|---|---|---|---|
| "Works in my shell, 127 from cron" | cron's minimal PATH lacks the tool; your login PATH was doing the finding | env -i /bin/sh -c 'the command' — reproduce at your desk | Absolute paths (or set PATH in the crontab); test via a one-minute schedule |
| Bash-only syntax errors, but only under cron | cron's SHELL is /bin/sh — dash ran your bashisms (Module 1's trap, scheduled) | head -1 the script; check how the crontab invokes it | Proper shebang + execute the file directly (kernel honors the shebang) |
| Cron job failing for months, nobody knew | No redirection — output went to unread local mail; no absence monitoring | grep the crontab for entries lacking >>; check mail spool existence | >> log 2>&1 on every entry; freshness checks on expected outputs |
| A token appeared in the job log | set -x traced an expanded credential into stderr, which the log captured | Search logs for the leak's shape; rotate the credential now | set +x fences around secret handling; treat traces as sensitive output |
| bats: every test errors in setup with source ... failed | The script's [[ … ]] && main guard left a false verdict as source's exit status | Run source ./script.sh; echo $? in a bash — nonzero confirms | The if-form guard: if [[ "${BASH_SOURCE[0]}" == "$0" ]]; then main "$@"; fi |
| Syntax error reported at the last line of a long script | bash -n/bash hit end-of-file still waiting for a fi/done/closing quote opened far above | bash -n after each structural edit; bisect by commenting halves | Editor highlighting + ShellCheck (which usually names the unclosed construct's line) |
E3. 🎓 Capstone — four tickets from the queue
🎓 Ticket 1 — "The new hire added set -x to the deploy script 'for observability' and left it on. Today's incident review found the registry token printed in the CI logs, which contractors can read."
Diagnosis. A1's sealed-recorder caveat, realized: -x prints commands after expansion, so docker login -p "$TOKEN" traced the secret in clear text into stderr, which CI dutifully archived. Observability was the right instinct; the floodlight was the wrong tool.
Work the steps: confirm the exposure window (when -x landed → now), find every affected log, and rotate the token first — the log is already copied somewhere you can't see.
Fix and prevention: replace always-on tracing with the ladder: ERR trap + timestamped logging for ambient observability (Module 14), scoped set -x/set +x fences for investigations, and a secrets rule in review: any set -x within sight of credential handling is a finding. Bonus hardening: pass secrets via stdin or files rather than argv at all — argv is visible to ps (Module 12), a second leak this ticket narrowly missed.
🎓 Ticket 2 — "We fixed the same off-by-one in the log-pruner three times this year. Each fix worked; each later refactor reintroduced it."
Diagnosis. A missing regression test — Part B's exact use case. The bug class is stable (boundary condition in date math); the protection was human memory, which refactors reset. Three fixes, zero tests: the team kept buying the same bandage.
Work the steps: write the bats test first, encoding the boundary: given files aged N-1, N, N+1 days (build them with touch -d, in a mktemp -d fixture), assert exactly which survive pruning. Watch it fail against the current regressed code (not ok — the bug, pinned). Fix. Watch ok.
Fix and prevention: commit test and fix together; wire bats into CI so the fourth reintroduction cannot merge. Then the retrospective habit: any bug fixed twice earns a test on the second fix, mandatory — the third fix should be a CI refusal, not an incident.
🎓 Ticket 3 — "Monthly report cron produced nothing this month. The log shows the job started, then: report.sh: line 12: [[: not found. Line 12 hasn't changed all year — and the job ran fine for eleven months."
Diagnosis. The scheduled dialect trap (E2 row 2): something changed how the script is invoked, not the script — a migrated crontab entry now reads sh /opt/report.sh (or the entry's command got wrapped), so dash executes a bash script and dies at the first [[ (Module 5's dialect line). Eleven good months mean the old entry executed it directly, shebang honored (Module 1 B2).
Work the steps: diff the crontab against backup/VCS; the invocation is the change. Confirm with sh /opt/report.sh by hand — same error, same line.
Fix and prevention: restore direct execution (/opt/report.sh, executable bit set — Module 1) and keep crontabs in version control so "nothing changed" is checkable, not folkloric. The transferable habit: when a stable script breaks, interrogate the invocation and environment before the code — Module 1's three run-methods never stop mattering.
🎓 Ticket 4 — "Graduation exercise: the team adopts your safe-backup.sh fleet-wide. Ops asks three questions before sign-off: how do we know it ran? how do we know the backup is good? and what happens the night the disk fills mid-run?"
Diagnosis. Not a bug — a design review, and every answer is already in the track. Ran? Its log lines are timestamped and machine-parseable; add a freshness monitor on the newest backup-*.tar.gz (E1: monitor absence, not just verdicts). Good? The -s check guards non-emptiness; a real restore test (extract to mktemp scratch, compare a sentinel file) upgrades evidence to proof — schedule it weekly. Disk fills? tar fails mid-write → set -e stops the script → ERR trap logs line and command → EXIT trap removes the scratch — and because publishing was a final mv, no partial archive ever appeared in dest; the previous night's archive still stands.
Work the steps: demonstrate the disk-full path safely: KEEP_DAYS=7 ./safe-backup.sh srcdata /tmp/tiny-tmpfs against a deliberately small tmpfs mount, and read the failure transcript against the flowchart in D1.
Fix and prevention: ship it with the answers written down — the header comment gains three lines (monitoring hook, restore-test schedule, failure semantics). A script whose failure story can be explained to ops in three sentences is what "production-ready" has meant all along. That is the track.
E4. Documentation reference
| Topic | Authoritative reference | What you'll find there |
|---|---|---|
| -x, -n, -v and friends | Bash manual — The Set Builtin | Every debugging-relevant shell flag, precisely defined |
| PS4, LINENO, BASH_SOURCE | Bash manual — Bash Variables | The variables that turn traces and traps into located reports |
| bats | bats-core documentation | Tutorial, writing tests, setup/teardown, gotchas, CI usage |
| The linter | ShellCheck | Paste-in checker and the wiki behind every SCxxxx cited in this track |
| Crontab format and environment | crontab(5) — man7.org | The five fields, SHELL=/bin/sh default, environment-setting lines |
E5. Self-assessment
Answer from memory, out loud or on paper. Every answer is in this module — and behind it, the track.
- What exactly does set -x print, at which pipeline stage of Module 2's rewriting, and on which channel?
- Write the PS4 that stamps file and line, and say where it must be set to work reliably.
- Why is always-on tracing a secrets incident waiting to happen, and what is the fencing discipline?
- What does bash -n do and not do — and why is it the only fully safe check for an untrusted script?
- Recite the debugging ladder in cost order and justify the ordering.
- In bats: what does run capture, and what two variables do assertions read?
- Why does the &&-form source guard break bats setup, and what is the robust spelling?
- What is TAP, and why does bats' exit code make CI integration trivial?
- List the five cron environment differences that break interactive-tested scripts, with one fix each.
- Write the defensive crontab entry for a nightly job, naming the module behind each fragment.
- In safe-backup.sh: why is tmp deliberately not local, why scratch-then-mv, and why check -s after tar?
- Ops asks "how do we know it ran, how do we know it's good, what if it dies mid-run?" — answer for any script you now write.
E6. Sources
InterviewBit — Top Shell Scripting Interview Questions — "How will you debug a shell script?" (no publication date shown on page)
Edureka — Top 60 Shell Scripting Interview Questions and Answers — "How to debug the problems encountered in the shell script/program?", "Can you write a script to portray how set –x works?", "What is the Crontab?", "How many fields are present in a crontab file and what does each field specify?", "What are the two files of crontab command?" (page updated Dec 9, 2024)
KnowledgeHut — Shell Scripting Interview Questions and Answers — "How do you debug a shell script?" (no publication date shown on page)
A note on the corpus: debugging and crontab are richly represented in published lists; shell-script testing is not — bats circulates through its own documentation and engineering-blog folklore rather than interview-question lists, and the testing question above is labeled accordingly. The works-in-shell-fails-in-cron scenario is asked constantly and published rarely; same labeling. No questions were invented. All expected-result outputs — including the full capstone script and both bats runs — were produced by actually running them on Ubuntu 24.04 / bash 5.2 / Bats 1.10.0; timestamps, byte counts, and PIDs are flagged as machine-dependent.