Module 11 — Text Processing in Pipelines
Updated 3 September 2026
🧠 concept → 🧩 analogy → 🧪 exercise → ✅ expected result (hidden) → 🎯 interview questions (hidden)
Part A — grep, and enough regex to be dangerous
A1. grep beyond the verdict
Module 3 used grep -q purely for its exit code. Full grep prints every matching line — it is the fleet's search engine, and four options do most of the daily work: -i (ignore case), -v (invert: lines that do not match), -c (count matching lines instead of printing them), -n (prefix line numbers). It reads files by name or, as a pipe segment, its stdin — same tool, both mouths.
🧪 Exercise 11.1 — build the lab, then search it
cd ~/bash-course && mkdir -p textlab && cd textlab
cat > access.log <<'EOF'
203.0.113.9 - GET /index.html 200
198.51.100.7 - GET /api/users 200
203.0.113.9 - POST /api/login 401
192.0.2.44 - GET /index.html 200
203.0.113.9 - POST /api/login 401
198.51.100.7 - GET /api/orders 500
192.0.2.44 - GET /style.css 200
203.0.113.9 - POST /api/login 401
198.51.100.7 - GET /api/orders 500
203.0.113.9 - GET /admin 403
EOF
grep 401 access.log
grep -c 401 access.log
grep -v 200 access.log | wc -l✅ Expected result — click to reveal
203.0.113.9 - POST /api/login 401
203.0.113.9 - POST /api/login 401
203.0.113.9 - POST /api/login 401
3
6What to read out of it: ten lines of "traffic," and grep carved out exactly the three failed logins — then -c counted them without printing (count of matching lines, remember, not occurrences). The -v pipeline inverts: 6 lines are not clean 200s — a number that reads as "error volume" at a glance. This little log is deliberately fake (RFC 5737 documentation IPs); every remaining exercise mines it, so keep the textlab directory.
A2. The regex survival kit
grep's patterns are regular expressions — a different, more powerful language than Module 2's globs, and the difference bites exactly once per career: in regex, * does not mean "anything" (that's .*) and . is not a literal dot (that's \.). The survival kit, seven pieces: ^ start of line · $ end of line · . any one character · [abc]/[0-9] one character from a set · * zero-or-more of the previous thing · + one-or-more (needs grep -E, the extended dialect) · a|b alternatives (also -E). That kit covers the vast majority of operational grepping, and it is the same language [[ =~ ]] spoke in Module 5 — that recipe (^[0-9]+$: start, digits, one-or-more, end) now reads as a sentence.
Where the analogy stops working. A sketch artist knows which features matter. Regex has no judgment: . matches any character including the dot you meant literally, so 192.0.2.44 as a pattern also matches 192x0y2z44. In operational grepping over IPs and versions, escape your dots (192\.0\.2\.44) — and when a colleague's grep matches "too much," the unescaped dot is suspect number one.
🧪 Exercise 11.2 — sketches against the log
cd ~/bash-course/textlab
grep -E 'POST|500' access.log
grep -c '^203' access.log
grep '00$' access.log | wc -l✅ Expected result — click to reveal
203.0.113.9 - POST /api/login 401
203.0.113.9 - POST /api/login 401
198.51.100.7 - GET /api/orders 500
203.0.113.9 - POST /api/login 401
198.51.100.7 - GET /api/orders 500
5
6What to read out of it: the -E alternation pulled two different kinds of trouble (POSTs and 500s) in one pass, in file order. ^203 counted 5 lines starting with the suspicious IP — all its traffic, not just its failures (cross-check: 11.3's top-talkers will say 5 203.0.113.9 too) — the anchor doing real work, since "203" might appear anywhere in a URL. 00$ matched line-endings: the four 200s and two 500s — six lines ending in 00, and a reminder that anchors match your intent only when you think about what else satisfies them.
Interview questions — Part A
🎯 "Explain the use of grep in shell scripting. Write a script that searches for a specific word in a file and prints the matching lines." — asked verbatim at FinalRound AI; Zero To Mastery asks "How do you use grep in Bash?"
The direct answer: grep pattern file prints matching lines; scripted with arguments and guards: grep -- "$1" "$2" behind Module 7 validation. Options that make it operational: -i, -v, -c, -n, -r (recurse a directory), -E (extended regex).
Going deeper: the -- (end of options, Module 9's mv armor) protects against patterns that begin with -; and know your exit codes cold — 0 match, 1 no match, 2 error (Module 3's model citizen) — because the script's verdict is often the deliverable, not the printed lines.
The details that separate candidates: pattern vs literal — -F treats the pattern as fixed text (faster, and safe for user-supplied strings full of regex characters); word-boundary precision with -w (searching for error without matching usererror); and the streams discipline — grep's own complaints go to stderr, so 2>/dev/null on a recursive grep hides permission noise deliberately, a scoped and commented exception to Module 4's chainsaw rule.
Part B — The column tools: cut, sort, uniq, tr
B1. cut: one column, please
cut -d' ' -f1 — delimiter space, field one — slices a column out of every line. It is dumb in a useful way: no patterns, no logic, just "split on this one character, give me field N" (multiple fields: -f1,3). Its dumbness is also its boundary: multiple consecutive delimiters are separate empty fields to cut, so ragged whitespace defeats it — that job belongs to awk (Part C), whose default splitting collapses whitespace runs.
B2. sort and uniq: the counting machine
sort orders lines; uniq collapses adjacent duplicates, and uniq -c counts each run as it collapses. Chain them and you get the single most famous pipeline in operations — the top-talkers pattern: cut -d' ' -f1 access.log | sort | uniq -c | sort -rn | head — extract the key, group it (sort), count the groups (uniq -c), rank by count (sort -rn: reverse, numeric), take the leaders. Every "top IPs / top URLs / top errors" question is this one pipeline with a different field number.
🧪 Exercise 11.3 — top talkers, and the trap un-sprung
The second pipeline miscounts on purpose.
cd ~/bash-course/textlab
cut -d' ' -f1 access.log | sort | uniq -c | sort -rn
cut -d' ' -f1 access.log | uniq -c | head -5 # ← no sort first✅ Expected result — click to reveal (contains a deliberate miscount)
5 203.0.113.9
3 198.51.100.7
2 192.0.2.44
1 203.0.113.9
1 198.51.100.7
1 203.0.113.9
1 192.0.2.44
1 203.0.113.9What to read out of it: the correct machine says 203.0.113.9 owns half the traffic — one line per IP, ranked. The sortless version shows the same IP four separate times, each run of length 1 counted separately — the interleaving in the log became fragmentation in the report. If a top-talkers output ever shows the same key on multiple lines, the missing sort is the diagnosis, instantly. (uniq's counts are right-aligned in a width-7 column — that leading whitespace is normal and trims away with awk or sed 's/^ *//' when it matters.)
B3. tr: the character transformer
tr maps characters to characters on its stdin (it takes no filenames — pipe or redirect into it): tr 'A-Z' 'a-z' lowercases a stream (the stream-sized sibling of Module 9's ${var,,}), tr -s ' ' squeezes runs of a character to one (the pre-treatment that makes ragged whitespace safe for cut), tr -d '\r' deletes carriage returns (Module 1's CRLF surgery, stream edition), and tr '\n' ',' turns a list into a line.
🧪 Exercise 11.4 — normalize a messy line
echo "ERROR disk full" | tr 'A-Z' 'a-z' | tr -s ' '✅ Expected result — click to reveal
error disk fullWhat to read out of it: two transformations, one readable pipeline — case folded, triple space squeezed. This normalize-then-process shape (tr first, then cut/sort/grep on clean input) is how professionals make the dumb-but-fast tools reliable on real-world text instead of reaching for heavier machinery.
Interview questions — Part B
🎯 "How do you find the top N IP addresses (or URLs, or errors) in a log file?" — the most-asked pipeline question in DevOps interviews, in countless phrasings; the components appear verbatim across FinalRound AI's grep/sed/awk set
The direct answer: cut -d' ' -f1 access.log | sort | uniq -c | sort -rn | head -10 — extract, group, count, rank, take ten. Adjust -f (or an awk {print $N}) for URLs, status codes, user agents.
Going deeper: narrate why each stage exists — especially the first sort (uniq's adjacency requirement) and the -n on the ranking sort (numeric, not dictionary) — because interviewers deliberately hand you the broken versions and watch. Constant memory, streaming, gigabytes-friendly (Module 4's conveyor property) is worth one sentence.
The details that separate candidates: the awk alternative awk '{count[$1]++} END {for (k in count) print count[k], k}' | sort -rn | head — one pass, no pre-sort, and you can explain when it wins (huge cardinality: no full-file sort) and its cost (memory proportional to distinct keys); plus sort -u as the "just dedupe, don't count" shortcut. Owning both spellings and the trade-off is a strong-hire signal on this classic.
🎯 "Write a shell script that counts the number of lines in a given file." — asked verbatim at FinalRound AI
The direct answer: wc -l < "$1" — with Module 7's guards around it ([ "$#" -eq 1 ], [ -f "$1" ], usage to stderr, exit 2).
Going deeper: the < "$1" redirect (versus wc -l "$1") prints the bare number without the filename (Module 4's anonymous-stdin fact) — exactly what a script wants to capture into a variable: count=$(wc -l < "$1"). Mention the edge everyone forgets: a file without a trailing newline undercounts by that last fragment (wc counts newline characters — Module 6's guard exists for the same reason).
The details that separate candidates: counting matching lines without wc at all (grep -c pattern file) and counting across many files (wc -l *.log gives per-file plus total; cat *.log | wc -l gives one number) — the follow-ups this question exists to set up.
Part C — sed and awk: the editors
C1. sed: find-and-replace for streams
sed (stream editor) applies editing commands to every line flowing through. Ninety percent of its real-world use is two commands. Substitute: sed 's/old/new/' file replaces the first match per line; add the g flag — s/old/new/g — for every match per line (per-line first-vs-all, exactly like Module 9's single-vs-double slash). The pattern is regex, so A2's kit applies, dots and all. Delete: sed '/pattern/d' drops matching lines — /^$/d (lines where start-of-line meets end-of-line: blank ones) is the classic.
By default sed prints the edited stream and touches nothing; sed -i edits the file in place, and deserves respect: no dry run shows you what -i did after the fact. The professional rhythm is Module 9's: run without -i, read the plan, then add -i (or -i.bak, which keeps the original as file.bak).
🧪 Exercise 11.5 — edit a config, carefully
cd ~/bash-course/textlab
printf 'server=old-db.internal\nbackup=old-db.internal\n\ncache=redis-1\n' > app.conf
sed 's/old-db/new-db/' app.conf # preview: nothing is written
sed '/^$/d' app.conf # preview: blank line dropped
sed -i.bak 's/old-db/new-db/g' app.conf
grep -c new-db app.conf ; grep -c old-db app.conf.bak✅ Expected result — click to reveal
server=new-db.internal
backup=new-db.internal
cache=redis-1
server=old-db.internal
backup=old-db.internal
cache=redis-1
2
2What to read out of it: two previews first — the substitution (blank line intact) and the deletion (old names intact: each preview shows only its own edit, because neither wrote anything). Then the real edit with a safety net: the final counts prove app.conf now holds 2 new names and the .bak still holds the 2 old ones. Roll-back is mv app.conf.bak app.conf — one command, because you paid one -i.bak in advance.
C2. awk: the field machine
awk sees every line as fields — $1, $2, … $NF (last field), split on whitespace-runs by default or on anything you name with -F,. Its programs are condition { action } pairs run per line, and four shapes cover most operational work:
awk -F, '{print $1, $3}' commits.csv # project columns (ragged-whitespace-proof)
awk -F, '{sum += $3} END {print "total:", sum}' commits.csv # accumulate; END runs after the last line
awk -F, '$3 > 40 {print $1}' commits.csv # filter by a FIELD'S VALUE — numerically!
awk '{count[$NF]++} END {for (s in count) print s, count[s]}' access.log # count-by-keyNote what the third shape just did: $3 > 40 compares numbers — grep cannot do that (a regex has no idea 9 < 40 < 100), and it is the single most common reason a pipeline graduates from grep to awk. The $ here means "field," not bash's "variable" — which is why awk programs ride inside single quotes (Module 2's solid wall, keeping bash's hands off awk's $).
Where the analogy stops working. A spreadsheet's columns are declared once, in the header. awk re-splits every line independently — a line with a missing field silently shifts every later column left, and $3 quietly reads what used to be $4's data. Real logs have ragged lines; production awk either validates (NF == 5 { … }) or accepts occasional garbage — the temp never warns you the row was short.
🧪 Exercise 11.6 — cells, sums, filters, tallies
cd ~/bash-course/textlab
printf 'alice,frontend,42\nbob,backend,17\ncarol,frontend,58\n' > commits.csv
awk -F, '{print $1, $3}' commits.csv
awk -F, '{sum += $3} END {print "total:", sum}' commits.csv
awk -F, '$3 > 40 {print $1 " is busy (" $3 ")"}' commits.csv
awk '{count[$NF]++} END {for (s in count) print s, count[s]}' access.log | sort✅ Expected result — click to reveal
alice 42
bob 17
carol 58
total: 117
alice is busy (42)
carol is busy (58)
200 4
401 3
403 1
500 2What to read out of it: projection, aggregation, numeric filtering, and a group-by — a fair slice of SQL, one line each. The last pipeline is the status-code dashboard of your fake log (note the trailing | sort: awk's count-by uses an associative array, and its iteration order is as unordered as bash's — Module 10's lesson, portable across languages). And notice total: 117 — awk does real decimal arithmetic where bash truncates (Module 6 A2's promised escape hatch, delivered).
Interview questions — Part C
🎯 "Write a shell script that uses sed to replace all occurrences of a word in a file." — asked verbatim at FinalRound AI; Hirist asks "How do you delete blank lines from a file using a script?"
The direct answer: sed -i 's/oldword/newword/g' "$file" — the g for every occurrence per line, -i for in-place. Blank lines: sed -i '/^$/d' "$file".
Going deeper: the professional wrapper — preview without -i first, or use -i.bak for an instant rollback; guard the file's existence (Module 5); and mind the delimiter trick: replacing paths means s|/old/path|/new/path| — sed accepts any delimiter after s, sidestepping a forest of escaped slashes.
The details that separate candidates: knowing what g actually scopes (per line, not per file — without it you replace the first hit on each line, which on config files often looks fine); that the pattern side is regex (a literal dot needs \. — replacing 1.2.3 without escaping also matches 1x2y3); and the ops caveat that sed -i rewrites the whole file (new inode — running processes holding the old file still see old content; the log-rotation implications land in Module 15).
🎯 "Explain how to use awk in shell scripting. Write a script that processes a CSV file and prints specific columns." — asked verbatim at FinalRound AI; Hirist asks "How do you use awk in shell scripting?"; Edureka's Q18 uses awk '{print $3}'
The direct answer: awk -F, '{print $1, $3}' file.csv — -F names the delimiter, $N are fields, and condition { action } pairs filter and transform: awk -F, '$3 > 100 {print $1}'.
Going deeper: the four working shapes (project, accumulate with END, filter numerically, count-by-key with arrays) and the two boundary facts — awk's default splitting collapses whitespace runs (why it beats cut on ragged output like ps or ls -l), and naive -F, breaks on real-world CSV with quoted commas ("Smith, Ada" — at which point the honest answer is a real CSV parser, not more awk).
The details that separate candidates: passing shell values in properly — awk -v threshold="$t" '$3 > threshold' — instead of quote-juggling; NR (line number) and NF (field count) fluency, including NF==0 and NF!=expected as data-quality guards; and the judgment sentence interviewers reward: grep finds lines, sed edits text, awk reasons about columns — choose by the shape of the question.
Part D — find and xargs: pipelines that act on files
D1. find: a query language for the filesystem
find dir tests walks a directory tree and prints every path passing its tests: find /var/log -name '*.log' (note the quoted glob — it must reach find as a pattern, not be expanded by bash first; Module 2's quoting reflex protecting a new customer), find . -mtime +30 (modified more than 30 days ago — the cleanup workhorse), -type f/-type d, combined freely. Where a glob sees one directory, find sees the whole tree.
xargs turns a stream of names into command arguments: find … | xargs ls -l runs ls with all the found paths at once (batching automatically when there are thousands). And here the module's oldest enemy returns for one last fight: xargs splits its input on whitespace by default — a found path containing a space arrives as two broken arguments. The armored idiom, used verbatim across production everywhere: find … -print0 | xargs -0 … — find ends each path with an unprintable NUL character (the one character no filename can contain), xargs splits only on NUL, and no filename can break it, ever.
🧪 Exercise 11.7 — the armored pipeline, and the broken one
The second pipeline breaks on purpose.
cd ~/bash-course/textlab
mkdir -p tmpfiles
touch tmpfiles/a.tmp "tmpfiles/report draft.tmp" tmpfiles/keep.txt
find tmpfiles -name '*.tmp' -print0 | xargs -0 ls -l
find tmpfiles -name '*.tmp' | xargs ls -l # ← whitespace splitting✅ Expected result — click to reveal (contains a deliberate break)
-rw-r--r-- 1 zaeem zaeem 0 Sep 3 17:26 tmpfiles/a.tmp
-rw-r--r-- 1 zaeem zaeem 0 Sep 3 17:26 tmpfiles/report draft.tmp
ls: cannot access 'tmpfiles/report': No such file or directory
ls: cannot access 'draft.tmp': No such file or directory
-rw-r--r-- 1 zaeem zaeem 0 Sep 3 17:26 tmpfiles/a.tmpWhat to read out of it (owner and dates yours): the armored version listed both files, space and all. The naked version shattered report draft.tmp into two phantom paths — the double-error signature you first met in Module 2 B1, now produced by a different splitter (xargs's, not bash's) with the same root cause and the same lesson: boundaries must travel with the data. -print0 | xargs -0 is the only spelling that belongs in a script; now swap ls -l for rm -- in your head and feel why.
D2. Choosing the tool — the decision
Diagram source
flowchart TD
A["Text problem<br>in a pipeline"] --> B{"What is the<br>question about?"}
B -->|"which LINES<br>match?"| C["grep<br>-i -v -c -E"]
B -->|"one COLUMN,<br>clean delimiter"| D["cut -d -f"]
B -->|"columns, math,<br>ragged spacing"| E["awk<br>condition { action }"]
B -->|"EDIT the text<br>(replace, delete lines)"| F["sed s/// and /d<br>preview, then -i.bak"]
B -->|"CHARACTERS<br>(case, squeeze, CRLF)"| G["tr"]
B -->|"group + count<br>+ rank"| H["sort | uniq -c | sort -rn"]
B -->|"FILES matching<br>criteria, then act"| I["find -print0<br>| xargs -0"]
B -->|"follow a live log"| J["tail -f"]Interview questions — Part D
🎯 "How do you use xargs in Bash?" — asked verbatim at Zero To Mastery
The direct answer: xargs reads items from stdin and appends them as arguments to a command — find . -name '*.tmp' | xargs rm — batching thousands of names into few command invocations.
Going deeper: lead with the safety rule — whitespace splitting breaks spacey filenames, so the production spelling is find … -print0 | xargs -0 end to end; then the shaping options: -n 1 (one argument per invocation), -I{} (place the argument mid-command: xargs -I{} mv {} /archive/), -P N (parallel invocations — instant bounded fan-out), and -r (run nothing on empty input — without it some xargs run the command once with no arguments).
The details that separate candidates: explaining why xargs exists at all (the kernel limits a command line's total size — rm * on a million files dies with "Argument list too long"; xargs batches under the limit); and the contrast with find -exec cmd {} + (no pipe, no splitting problem, but no -P parallelism) — choosing between them with reasons is the senior move.
🎯 "Write a shell script that monitors a log file for new entries and prints them in real-time." — asked verbatim at FinalRound AI
The direct answer: tail -f /var/log/app.log — print the file's tail, then keep following as lines are appended. Filtered live view: tail -f app.log | grep --line-buffered ERROR.
Going deeper: -f follows the file descriptor, so when log rotation moves the file aside, plain -f keeps watching the old ghost — tail -F (capital) re-opens by name and survives rotation; that single letter is the production difference. The --line-buffered on grep matters too: pipes buffer in blocks by default (Module 4's buffering aside), so without it "real-time" arrives in 4KB lumps.
The details that separate candidates: knowing this is a watch, not a monitor — production monitoring ships logs to an aggregator and alerts on patterns; tail -f is the debugging tool you outgrow (saying so, then still wielding it fluently, is exactly the right posture); and the loop-alternative for polling metrics rather than logs (while sleep 5; do …; done — Module 6) when "new entries" aren't line-appended.
Part E — Production
E1. 🏭 Production practices
find-to-action pipelines are NUL-armored end to end. -print0 | xargs -0 (or find -exec … {} +) — a filename nobody vetted must not be able to change what a cleanup deletes.
In-place edits pay for insurance. sed -i.bak (or edit a copy and mv it over), preceded by a preview run; config edits at fleet scale go through config management, with sed reserved for one-offs and emergencies — and even then previewed.
Counting pipelines are reviewed for the two silent corruptions: the missing pre-sort before uniq -c, and the missing -n on the ranking sort. Both produce plausible wrong numbers, so both are checklist items, not memory items.
Literal searches use grep -F, user-supplied patterns are treated as hostile, and -- ends options before any variable argument (grep, rm, mv alike).
awk is fed clean field expectations. NF-guards on ragged input; -v var="$value" for shell values (never quote-splicing); and a comment naming the expected column layout, because $3 is only meaningful relative to a format someone once promised.
Long pipelines are built incrementally and kept readable. One stage at a time, verified at each | (the debugging habit this module trained); in scripts, broken across lines with trailing | so the diff shows which stage changed.
E2. Production-practice table
| Symptom | What is really happening | What to run | The fix |
|---|---|---|---|
| Top-talkers report shows the same key on several lines | No sort before uniq -c — adjacency requirement violated, runs fragmented | Compare with and without the pre-sort on the same input | sort | uniq -c — always paired; or the awk count-by |
| Ranking puts 9 above 10, 99 above 100 | Final sort is lexicographic — dictionary order on digits | printf '9\n10\n' | sort vs sort -n | sort -rn for count rankings; sort -V for versions |
| Cleanup deleted the wrong files — names split at spaces | Naked find | xargs: whitespace splitting broke paths into fragments | Rerun the find with ls -l in place of the action; watch the double errors | -print0 | xargs -0, or find -exec … {} + |
| grep matches lines it obviously shouldn't (IPs, versions) | Unescaped . in the pattern matching any character | Test the pattern against a crafted counter-example (192x0y2z44) | Escape literal dots, or use grep -F for fixed strings |
| sed -i "worked" but the running service still reads old config | -i wrote a new file (new inode); the process holds the old one open | ls -i the file before/after; check the service's open files | Reload/restart the service after in-place edits; know rotation semantics (Module 15) |
| awk column extraction returns wrong data on some lines | Ragged lines re-split independently — short rows shift every field left | awk 'NF != 5 {print NR": "$0}' file — list the malformed rows | Guard with NF checks; fix the producer, or route bad rows to a reject file |
E3. 🎓 Capstone — four tickets from the queue
🎓 Ticket 1 — "On-call asks for 'top 5 IPs hitting us with 401s, last night's log.' The intern's answer: grep 401 access.log | cut -d' ' -f1 | uniq -c | head -5 — and the numbers look weirdly flat, lots of 1s and 2s."
Diagnosis. Two of B2's corruptions in one line: no sort before uniq -c (interleaved IPs fragment into little runs — hence the flat 1s and 2s) and no sort -rn after it (so head -5 takes the first five fragments, not the top five talkers). The grep and cut stages are fine; the counting machine is missing both its sorts.
Work the steps: insert the stages one at a time, checking at each |: grep 401 access.log | cut -d' ' -f1 | sort | uniq -c | sort -rn | head -5 — on the textlab log the 401 leader is 203.0.113.9 with 3, now counted as one line.
Fix and prevention: the corrected one-liner, plus the review rule from E1: uniq -c never appears without a sort on each side of it (grouping before, ranking after). Interviewers run this exact trap; so does 2 a.m.
🎓 Ticket 2 — "A teammate ran our documented cleanup — find /srv/uploads -name '*.tmp' -mtime +7 | xargs rm — and it deleted a file called budget from a user's folder. The user's actual junk file was named budget 2024.tmp."
Diagnosis. D1's whitespace shatter with rm attached: budget 2024.tmp split into budget and 2024.tmp; rm deleted a real file named budget (and failed harmlessly on 2024.tmp). The pipeline was documented — the bug was institutionalized, waiting for the first spacey filename.
Work the steps: reproduce safely in a scratch dir with ls -l standing in for rm (Exercise 11.7's exact experiment); watch the shatter.
Fix and prevention: find /srv/uploads -name '*.tmp' -mtime +7 -print0 | xargs -0 rm -- — NUL armor plus option-terminating --. Then fix the documentation (the runbook taught the broken form) and add the E1 checklist line so review catches the next naked xargs. Restore the user's file from backup — and note which module taught you to check the backup actually has bytes in it.
🎓 Ticket 3 — "Someone 'fixed' database hosts across configs with sed -i 's/db1.internal/db2.internal/g' *.conf. Now a config that mentioned db1-internal.example.com — note the dash — reads db2.internalexample.com… wait, no: db1-internal became… actually what exactly happened here?"
Diagnosis. The unescaped-dot regex (A2) inside an in-place edit (C1): the pattern db1.internal matches db1-internal too (. = any character), so db1-internal.example.com → db2.internalexample.com? No — walk it precisely: the match covers db1-internal (11 chars: d b 1 any i n t e r n a l), replaced by db2.internal, yielding db2.internal.example.com — a different wrong host that happens to resolve nowhere. The ticket's confusion is part of the lesson: regex accidents produce outputs humans misread.
Work the steps: replay on a sample line without -i: echo 'db1-internal.example.com' | sed 's/db1.internal/db2.internal/' — read the actual output instead of reasoning from memory (this module's core habit: preview, don't predict).
Fix and prevention: escape the dot (s/db1\.internal/db2.internal/g) or anchor more context; always preview before -i; and keep -i.bak insurance so the misedit is a mv away from undone. For fleet-wide config changes, this is the moment to say "config management, not sed" out loud.
🎓 Ticket 4 — "Capacity review needs 'average response size per endpoint' from a space-separated log where field 7 is bytes and field 5 is the path. The intern's draft pipes cut into a bash loop with $((sum+=…)) and takes 40 minutes on one day's log."
Diagnosis. A column-math problem solved with line-by-line shell arithmetic: millions of loop iterations in bash (plus Module 9's fork-tax if anything spawns per line), for a job that is one awk pass. The 40 minutes is the diagnosis: per-line shell processing of bulk data is the anti-pattern; awk exists precisely here — and bash arithmetic would also truncate the average (Module 6's integer-only fact).
Work the steps: the one-liner: awk '{sum[$5]+=$7; n[$5]++} END {for (p in sum) printf "%s %.1f\n", p, sum[p]/n[p]}' access.log | sort — two associative arrays, one pass, decimal division, sorted output (Module 10's ordering rule).
Fix and prevention: replace the loop; time both versions and attach the ratio to the review (data beats argument — Module 9 Ticket 4's habit). The general law for the runbook: bash orchestrates; awk/sort/uniq compute. When a shell loop is doing arithmetic on every line of a big file, the pipeline has been assembled inside out.
E4. Documentation reference
| Topic | Authoritative reference | What you'll find there |
|---|---|---|
| grep and its regex | grep(1) — man7.org | Options, BRE/ERE dialects, exit statuses — the full search engine |
| Columns and counting | cut(1) · sort(1) · uniq(1) · tr(1) | Field slicing; -n/-V/-h sort modes; the adjacency warning in uniq's own words; character surgery |
| The editors | sed(1) — man7.org · gawk(1) — man7.org | sed's command set and addressing; awk's full pattern-action language |
| Filesystem pipelines | find(1) — man7.org · xargs(1) — man7.org | Tests (-name, -mtime, -type), -print0; xargs -0, -P, -I, -n |
| Following logs | tail(1) — man7.org | -f vs -F, byte/line modes — the rotation-survival distinction |
E5. Self-assessment
Answer from memory, out loud or on paper. Every answer is in this module.
- grep -c counts what, exactly — and how would you count total occurrences instead?
- Recite the seven-piece regex kit, and explain what ^ERROR .*timeout$ matches, piece by piece.
- Why does the pattern 192.0.2.44 overmatch, what's the precise fix, and what's the fixed-string alternative?
- Build the top-talkers pipeline from memory and justify both sorts.
- What does uniq actually collapse, in its own man page's words — and what do the wrong numbers look like when you forget?
- When does cut fail on whitespace data, and which two tools (one per style) handle it instead?
- sed 's/a/b/' vs sed 's/a/b/g' — scope of each; and what does -i.bak buy you, exactly?
- Write awk one-liners for: print columns 1 and 3 of a CSV; sum column 3; rows where column 3 > 40; count-by-last-field.
- Why do awk programs ride in single quotes, and how do you pass a bash variable in properly?
- Why is find | xargs unsafe, what is the armored spelling, and what character makes the armor work?
- What problem does xargs fundamentally exist to solve, and what do -n, -I, -P each shape?
- tail -f vs tail -F — which survives log rotation, and why does the other watch a ghost?
E6. Sources
FinalRound AI — 25 Essential Shell Scripting Interview Questions — "Explain the use of grep in shell scripting. Write a script that searches for a specific word in a file and prints the matching lines.", "Write a shell script that uses sed to replace all occurrences of a word in a file.", "Explain how to use awk in shell scripting. Write a script that processes a CSV file and prints specific columns.", "Write a shell script that counts the number of lines in a given file.", "Write a shell script that monitors a log file for new entries and prints them in real-time." (page updated April 2, 2025)
Hirist — Top 30+ Shell Scripting Interview Questions and Answers — "How do you use awk in shell scripting?", "How do you delete blank lines from a file using a script?" (published Jul 22, 2025; last modified Dec 31, 2025)
Zero To Mastery — Bash Interview Prep — "How do you use grep in Bash?", "How do you use awk and sed in Bash?", "How do you use xargs in Bash?" (published June 18, 2026)
Edureka — Top 60 Shell Scripting Interview Questions — Q18's awk '{print $3}' column extraction (page updated Dec 9, 2024)
A note on the corpus: this module's tools are the best-covered interview territory in all of shell scripting — the published corpus is deep, and the composite questions (top-talkers, log summaries) are asked in virtually every DevOps screen even where no list prints one canonical wording; the two composite questions above are labeled accordingly. No questions were invented. All expected-result outputs were produced by running the commands on Ubuntu 24.04 (GNU coreutils/grep/sed/gawk) against the module's fixed fake log; file owners and timestamps are flagged as machine-dependent where shown.