Module 06 — The Resolver Side of a Linux Host

Updated 20 August 2026

Module 06 · The Resolver Side of a Linux Host

Every module so far used dig. Your applications do not use dig, and they do not take the same path. This module is about the layer that has silently shaped every result in this track — and about the single most confusing sentence in DNS support: "dig gives the right answer but the app connects somewhere else."

🧠 concept → 🧪 exercise → ✅ expected result (hidden) → 🎯 interview questions (answers hidden)

Prerequisite: Modules 01–04. You need /etc/resolv.conf and the stub resolver (01 B2), the trailing dot and FQDNs (01 A3), TTLs and caching (03), and the 5-second timeout mentioned in 01 B2.

Part A · The path an application actually takes

A1 · getaddrinfo, not dig

The analogy. Think of you and a taxi driver given the same destination.

You look it up on your phone and read off the coordinates. The driver has his own route, his own shortcuts, and a note on the dashboard about a road closure you know nothing about.

You will both talk confidently about "the way there" and mean different things. That is dig and your application — and when their answers disagree, neither is lying.

Official docs: getaddrinfo(3) · nsswitch.conf(5) · resolv.conf(5)

When curl, a browser or a Java process needs an address, it does not speak DNS. It calls a C library function — getaddrinfo() — and hands over a name. Everything after that is the library's business.

Diagram source

flowchart TD
    APP["curl / python / java<br>calls getaddrinfo"] --> NSS["/etc/nsswitch.conf<br>which sources, in which order?"]
    NSS -->|"files"| HOSTS["/etc/hosts<br>NO DNS INVOLVED"]
    NSS -->|"dns"| STUB["stub resolver"]
    STUB --> RC["/etc/resolv.conf<br>nameserver · search · ndots"]
    RC --> QUERY["one or MORE queries<br>search list expansion"]
    QUERY --> SERVER["the configured resolver"]
    DIG["dig"] -.->|"skips nsswitch<br>skips /etc/hosts<br>skips the search list"| SERVER
    style HOSTS fill:#ef4444,color:#fff
    style DIG fill:#f59e0b,color:#fff
    style NSS fill:#8b5cf6,color:#fff

dig is a DNS tool. getaddrinfo is a name resolution tool, and DNS is only one of the things it consults. That is the entire source of the confusion, and it is not a bug in either.

Three differences matter, and each one is a real production incident:

dig never reads /etc/hosts. An entry there overrides DNS for the application and is invisible to dig
dig does not use the search list unless you pass +search, so it may send a completely different name than your application does
dig does not sort or filter the results, while getaddrinfo applies address-selection rules — including preferring IPv6

So "dig works but the app doesn't" is not a contradiction. It is a clue, and it points at this module.

A2 · nsswitch.conf — the switchboard

The analogy. Think of the order in which you look for someone's phone number.

First the note stuck to your desk. Then the company directory. Then you ask around.

Whichever answers first wins, and you stop looking — so the note on your desk quietly outranks the official directory, forever, and nobody maintaining that directory can see it.

Official docs: nsswitch.conf(5) · hosts(5)

One line decides everything:

bash

grep '^hosts' /etc/nsswitch.conf

plain text

hosts:          files dns

Source	What it means
files	/etc/hosts. Consulted first, and a match ends the search
dns	The stub resolver, i.e. /etc/resolv.conf
myhostname	systemd module: resolves the local hostname and localhost without any file
resolve	Talks to systemd-resolved over its own socket, bypassing /etc/resolv.conf entirely
mdns4_minimal	Multicast DNS for .local. Note [NOTFOUND=return] after it — it stops the search

Order is policy, and the order is left to right. files dns means a /etc/hosts entry always beats DNS — which is how you pin a hostname during a migration, and also how a forgotten line from a debugging session sends production traffic to a decommissioned server for six months.

The line to look for on modern Ubuntu and Fedora is resolve:

plain text

hosts: files resolve [!UNAVAIL=return] dns

That says: try /etc/hosts, then talk to systemd-resolved directly over its socket, and only fall back to the classic dns path if resolved is unavailable. On such a host, /etc/resolv.conf may not be consulted at all — which is why editing it has no effect and why C1 exists.

A3 · /etc/hosts — the override that beats DNS

The analogy. Think of that sticky note with someone's old number on it.

It has no expiry date, nobody else has a copy, and you will trust it over the official directory every single time.

It was correct on the day you wrote it. It was written during a crisis eight months ago. And the person auditing the directory will find nothing wrong, because they cannot see your desk.

Official docs: hosts(5) · nsswitch.conf(5)

Module 01 A1 called /etc/hosts "the file DNS replaced". It was never removed, it is still consulted first, and it has no TTL, no expiry and no authority — it is true until a human edits it.

🧪 Exercise A3.1 — Make dig and your application disagree, on purpose

bash

# an entry for a name that does NOT exist in DNS
echo "203.0.113.77  demo-host.lab.internal demo-host" | sudo tee -a /etc/hosts

# what the APPLICATION sees
getent hosts demo-host.lab.internal ; echo "getent rc=$?"

# what DNS says
dig +short demo-host.lab.internal A ; echo "dig rc=$?"

✅ Expected result — click to reveal

plain text

$ getent hosts demo-host.lab.internal
203.0.113.77    demo-host.lab.internal demo-host
getent rc=0

$ dig +short demo-host.lab.internal A
getent rc=0     <- dig printed NOTHING

Two tools, one name, opposite answers — and both are correct.

getent found 203.0.113.77 because it walked the nsswitch path, hit files first, and matched /etc/hosts. DNS was never consulted. That is what curl, your browser and every application on the box will do.

dig printed nothing because it asked DNS directly, and this name does not exist in DNS at all.

Reverse the polarity and you have the incident. Put a wrong address in /etc/hosts for a name that does exist in DNS. Now dig returns the correct production address, the application connects to the wrong one, and every DNS check you run comes back clean. Engineers have spent entire days on that, because the tool they trust is structurally blind to the cause.

So the rule is: getent hosts is the diagnostic that matches reality. dig tells you what DNS says. getent tells you what the application will get. When they disagree, the answer is in /etc/hosts or nsswitch.conf, and you have found it in two commands.

Clean up: sudo sed -i '/demo-host/d' /etc/hosts

🎯 Interview questions — The resolution path

Q. dig returns the right address but the application connects to the wrong one. What is happening?

They are not taking the same path. dig speaks DNS directly; the application calls getaddrinfo(), which follows /etc/nsswitch.conf — normally /etc/hosts first, then DNS.

So the usual cause is an /etc/hosts entry overriding DNS, which dig is structurally incapable of seeing. Other causes: a search domain being appended so the application queries a different name; systemd-resolved being consulted over its own socket rather than via /etc/resolv.conf; or an application-level cache holding an old result.

The command I would run to settle it in one step is getent hosts <name>, because that follows exactly the path the application takes. dig tells you what DNS says; getent tells you what the app will get. When those two disagree you already know the fault is local, and you have narrowed it to two files.

Q. What is nsswitch.conf and why does its order matter?

It configures which sources the C library consults for each kind of lookup, and in what order. For hostnames the sources are typically files (/etc/hosts), dns, and on systemd hosts resolve, which talks to systemd-resolved over a socket.

Order is policy: files dns means a hosts entry always wins over DNS.

The modern subtlety worth raising: on Ubuntu and Fedora the line is usually hosts: files resolve [!UNAVAIL=return] dns. That resolve entry bypasses /etc/resolv.conf completely — so on those hosts editing resolv.conf can have no effect at all, and reading it tells you nothing about where queries are really going. That single detail explains a large share of "I changed the DNS server and nothing happened" tickets.

Part B · resolv.conf in full

B1 · nameserver — order, timeout, attempts

The analogy. Think of three phone numbers you ring in order, waiting five seconds each.

If the first number is dead, every single call still takes five seconds longer than it should — and then succeeds. Nobody reports that as a fault. They report that the office feels slow.

And if all three are dead, you have spent thirty seconds before giving up, not five.

Official docs: resolv.conf(5) · getaddrinfo(3)

Module 01 B2 introduced nameserver lines and warned that they are failover, not load balancing. Here is the arithmetic.

Option	Default and effect
nameserver <ip>	Up to 3 are used. Tried strictly in order
options timeout:n	5 seconds. How long to wait for each server before moving on
options attempts:n	2. How many full passes over the whole list
options rotate	Round-robin the list per process, turning failover into crude balancing
options single-request	Send the A and AAAA queries sequentially instead of in parallel

Do the multiplication, because it is the number that shows up as "the app is slow".

With the defaults — 3 nameservers, timeout:5, attempts:2 — a name that resolves nowhere costs 3 × 5 × 2 = 30 seconds before getaddrinfo returns a failure. Not 5 seconds. Thirty.

And if only the first nameserver is dead, every single lookup pays 5 seconds before succeeding against the second. To a user, a health check or a load balancer, that is indistinguishable from an outage — while every DNS server involved reports itself perfectly healthy.

This is why timeout:1 attempts:2 is standard practice in containers, where a fast failure is far more useful than a slow success. The default was designed for 1987 link speeds and has never been revisited.

B2 · search and domain

The analogy. Think of saying "Ahmad in Finance" instead of his full name and company.

Inside the office everyone knows what you mean, because the rest is assumed. Say the same thing outside the building and it means nothing at all.

The search list is that assumed remainder — and every guess it tries is a real question that someone has to answer "no" to.

Official docs: resolv.conf(5) — search and domain · RFC 9499 — relative name

Module 01 A3 said a name without a trailing dot is relative, and something may add labels to it before querying. search is that something.

plain text

search corp.example.com example.com

means: for a name judged relative, try <name>.corp.example.com, then <name>.example.com, then the bare name.

search is a list; domain is the obsolete single-entry version of the same thing. If both appear, the last one in the file wins — which is a classic surprise when a DHCP client appends domain to a file that already had search.

And the search list multiplies your query count. Every unsuccessful suffix is a real query with a real round trip. A three-entry search list turns one lookup into up to four.

B3 · ndots — the rule, and the arithmetic

The analogy. Think of the rule your office phone uses to decide whether the number you dialled is an internal extension or an outside line.

"Fewer than five digits means internal" is a sensible rule — until the threshold is set so high that every outside number gets tried as three different extensions first, on every call, all day.

That is ndots, and dialling the outside prefix explicitly — the trailing dot — skips the whole guessing game.

Official docs: resolv.conf(5) — options ndots:n · RFC 9499

ndots decides whether a name is treated as relative or absolute, and it is the most consequential four-letter option in DNS.

The rule, exactly: count the dots in the name as written. If that count is less than ndots, the name is treated as relative — the search list is tried first, and the bare name only if all suffixes fail. If the count is ndots or more, the bare name is tried first.

The default is ndots:1. So admin-vfo (0 dots) goes through the search list; seamless.se (1 dot) is queried as-is first.

A trailing dot short-circuits the whole thing — an absolute name is never expanded. That is Module 01 A3's payoff, and it is the fastest way to prove a problem is search-list-related.

🧪 Exercise B3.1 — Watch the search list fire, and watch dig refuse to play along

bash

sudo cp /etc/resolv.conf /tmp/resolv.bak
printf 'nameserver 8.8.8.8\nsearch seamless.se\noptions ndots:2\n' | sudo tee /etc/resolv.conf

echo "--- 'admin-vfo' has 0 dots, below ndots:2 -> search list FIRST ---"
getent hosts admin-vfo

echo "--- dig does NOT use the search list by default ---"
dig +short admin-vfo A ; echo "  ^ empty"

echo "--- dig +search DOES ---"
dig +search +short admin-vfo A

echo "--- 'www.seamless.se' has 2 dots -> tried absolute first ---"
getent hosts www.seamless.se

sudo cp /tmp/resolv.bak /etc/resolv.conf

✅ Expected result — click to reveal

plain text

--- 'admin-vfo' has 0 dots, below ndots:2 -> search list FIRST ---
185.64.27.150   admin-vfo.seamless.se

--- dig does NOT use the search list by default ---
  ^ empty

--- dig +search DOES ---
185.64.27.150

--- 'www.seamless.se' has 2 dots -> tried absolute first ---
52.77.52.233    seamless.se www.seamless.se

Look at the first two results together. This is the whole module in four lines of output.

getent hosts admin-vfo resolved to 185.64.27.150 — and note what it printed as the canonical name: admin-vfo.seamless.se. The library appended the search domain and queried a name you never typed.

dig admin-vfo returned nothing at all, because dig sent the bare label admin-vfo to the root of the namespace, where it does not exist.

Same string, same machine, same second — opposite results. Neither tool is wrong. They are answering different questions, and if you were debugging this with dig alone you would conclude the name does not exist while the application resolves it perfectly.

dig +search reproduces the library's behaviour, and it is the option to reach for whenever a short name is involved.

The third case shows the other half of the rule. www.seamless.se has two dots, which meets ndots:2, so it was tried absolute first and matched immediately — no search-list queries wasted.

Now the production consequence. Every relative name costs one extra failed query per search-list entry. On a busy host with a three-entry search list that is three wasted round trips on every single lookup — and Module 08 shows the Kubernetes default of ndots:5, where the arithmetic becomes genuinely painful.

🎯 Interview questions — resolv.conf

Q. What does ndots do?

It sets the threshold for treating a name as absolute. If a name contains fewer dots than ndots, the resolver tries the search-list suffixes first and the bare name last. At ndots or more, it tries the name as given first.

Default is 1, so host is expanded and host.example.com is not. A trailing dot makes a name absolute regardless and skips expansion entirely.

Why it matters operationally: every search-list entry that fails is a real query and a real round trip. Raising ndots — as Kubernetes does, to 5 — means ordinary names like api.example.com are expanded through the whole search list before the correct absolute query is even attempted, which multiplies query volume and adds latency to every external lookup a pod makes.

Q. How long does a failing DNS lookup take on a default Linux host?

Up to 30 seconds — three nameservers × a 5-second timeout × 2 attempts. Not 5 seconds, which is what people assume.

And if only the first nameserver is dead, every lookup still costs a full 5 seconds before succeeding against the second, so the symptom is uniform slowness rather than errors.

The point that lands: at scale, DNS failures almost never present as "DNS is down". They present as "the application is slow", because the stub waits out its timeouts and then succeeds. That is why timeout:1 attempts:2 is standard in containers — a fast failure is more useful than a slow success — and why latency, not error rate, is the metric that catches DNS problems.

Part C · The local caching layers

C1 · 127.0.0.53, and why editing resolv.conf does nothing

The analogy. Think of the internal switchboard extension.

Dialling it works perfectly. But if someone asks "which outside line did that call go out on?", the extension number cannot tell you — it names the operator, not the line.

And rewriting the extension on your desk phone changes nothing, because the operator decides which line to use, not you.

Official docs: systemd-resolved.service(8) · resolvectl(1) · nsswitch.conf(5)

Module 01 B2 promised an explanation for this file:

plain text

# This is /run/systemd/resolve/stub-resolv.conf managed by man:systemd-resolved(8).
# Do not edit.
nameserver 127.0.0.53
options edns0 trust-ad
search .

127.0.0.53 is a listener on your own machine, run by systemd-resolved. It is a caching stub that sits between the C library and the real upstream resolvers — and the real upstream servers are not in this file. They come from DHCP, from a VPN, from netplan or from systemd-resolved's own configuration.

Three consequences follow, and all three are daily support tickets:

cat /etc/resolv.conf cannot tell you where your queries go. It names a local process, not a server
Editing the file usually does nothing — it is regenerated, and the comment says so
On a host with resolve in nsswitch.conf, applications bypass this file entirely and talk to resolved over a socket, so even a successful edit would not affect them

systemd-resolved also does per-interface split DNS, and that is the part that surprises people. Connect a VPN and resolved can route *.corp.internal to the VPN's nameservers while everything else goes to your home router. That is genuinely useful and completely invisible in /etc/resolv.conf.

It is also why "it works on the VPN and not off it" is a resolved question, not a DNS-server question — and why the tool in C2 exists.

C2 · resolvectl — the command that tells the truth

The analogy. Think of asking the switchboard operator directly which line she used.

She will also tell you something the phone on your desk never could: that calls to the head-office prefix go out on a completely different line from everything else. That is split DNS, and it is invisible in every file you would think to check.

Official docs: resolvectl(1) · systemd-resolved.service(8)

bash

resolvectl status          # which servers, per interface, plus split-DNS routing
resolvectl query NAME      # resolve the way the SYSTEM does, not the way dig does
resolvectl statistics      # cache hits and misses
resolvectl flush-caches    # empty the local cache

🧪 Exercise C2.1 — Find the resolver your queries actually reach

bash

cat /etc/resolv.conf | head -4
resolvectl status | grep -A6 'Link'
resolvectl query seamless.se

✅ Expected result — click to reveal

plain text

$ cat /etc/resolv.conf | head -4
# This is /run/systemd/resolve/stub-resolv.conf managed by man:systemd-resolved(8).
# Do not edit.
nameserver 127.0.0.53
options edns0 trust-ad

$ resolvectl status | grep -A6 'Link'
Link 2 (wlp3s0)
    Current Scopes: DNS
         Protocols: +DefaultRoute +LLMNR -mDNS -DNSOverTLS
Current DNS Server: 192.168.1.1
       DNS Servers: 192.168.1.1
        DNS Domain: ~.

Link 5 (tun0)
    Current Scopes: DNS
         Protocols: -DefaultRoute +LLMNR -mDNS -DNSOverTLS
Current DNS Server: 10.8.0.1
       DNS Servers: 10.8.0.1
        DNS Domain: corp.internal

$ resolvectl query seamless.se
seamless.se: 52.77.52.233                      -- link: wlp3s0
-- Information acquired via protocol DNS in 12.4ms.
-- Data is authenticated: no

This output is from a laptop with a VPN attached — the shape to recognise, not a capture from a container.

/etc/resolv.conf said 127.0.0.53 and told you nothing. resolvectl status told you everything:

Two links, two different DNS servers. 192.168.1.1 on wifi, 10.8.0.1 on the VPN
DNS Domain: ~. on the wifi link — the ~. means "route everything here by default"
DNS Domain: corp.internal on tun0 — anything ending in corp.internal goes to the VPN's resolver instead. This is split DNS, configured automatically, invisible in every file you would think to look at
resolvectl query prints which link answered — -- link: wlp3s0. That single annotation resolves most "works on VPN / doesn't work off VPN" tickets in one command

The habit to build. On any systemd host, resolvectl status replaces cat /etc/resolv.conf as the first command. And when a name behaves differently from what dig reports, resolvectl query shows the system's answer including which interface's servers produced it.

And note Data is authenticated: no — that is DNSSEC validation status, and Module 07 explains when it says yes.

C3 · dnsmasq, containers, and the layers you did not ask for

The analogy. Think of an onion of middlemen you never hired.

Your message passes through a local assistant, then a building operator, then the company switchboard, then the outside line — and any one of them can be holding an old note.

The innermost layer is the worst, because it is inside the application itself, where none of your tools can see.

Official docs: resolv.conf(5) · nsswitch.conf(5) · BIND 9 — forwarding

Layer	Where you meet it
systemd-resolved	127.0.0.53 on Ubuntu, Fedora, Arch. Caches, does split DNS
dnsmasq	Home routers, older desktops, libvirt, and many Kubernetes node setups
Docker's embedded DNS	127.0.0.11 inside a container on a user-defined network. Resolves container names
Kubernetes resolv.conf	Injected into every pod, with a five-entry search list and ndots:5. Module 08
Application caches	Inside the process, invisible to every tool on this page

The application cache is the one that defeats everything you have learned. A JVM with the default networkaddress.cache.ttl security setting caches a successful lookup for the lifetime of the process. No TTL you publish, no cache you flush and no resolver you reconfigure has any effect on it.

Node.js, Go and Python each have their own behaviour, and connection-pooling libraries add another layer by holding open sockets to an address resolved long ago.

So when DNS is demonstrably correct at every layer and one application still connects to the old address, the remaining suspect is the application itself — and the fix is a restart or a configuration change in the runtime, not anything in DNS. Knowing when to stop looking at DNS is a real skill.

Part D · Field recipes

D1 · dig works, the application does not

The analogy. Think of two people sent to the same address with different maps.

The argument about who is right is a waste of time. The useful question is which map each of them used, because that is where the wrong turn is written down.

Each command below eliminates one map.

Official docs: getent(1) · resolvectl(1) · nsswitch.conf(5)

Diagram source

flowchart TD
    S["dig is right,<br>the app is wrong"] --> G["getent hosts NAME"]
    G -->|"getent ALSO right"| APP["not name resolution.<br>App cache, connection pool,<br>or the app is not using<br>this name at all"]
    G -->|"getent WRONG"| H{"grep the name<br>in /etc/hosts"}
    H -->|"found"| FIX1["/etc/hosts override<br>remove it"]
    H -->|"not found"| N{"check nsswitch.conf<br>hosts: line"}
    N -->|"has 'resolve'"| RV["systemd-resolved<br>resolvectl status<br>check split DNS per link"]
    N -->|"files dns"| SR{"is the name relative?<br>fewer dots than ndots"}
    SR -->|"yes"| SEARCH["search-list expansion<br>compare dig +search<br>with plain dig"]
    SR -->|"no"| SRV["different resolver:<br>compare dig @configured-ns<br>with the app's view"]
    style FIX1 fill:#ef4444,color:#fff
    style APP fill:#8b5cf6,color:#fff
    style RV fill:#f59e0b,color:#fff

🧪 Exercise D1.1 — The four-command triage

bash

NAME=api.internal.example.com

echo "1. what DNS says"          ; dig +short "$NAME"
echo "2. what the SYSTEM says"   ; getent hosts "$NAME"
echo "3. is it overridden?"      ; grep -n "${NAME%%.*}" /etc/hosts /etc/nsswitch.conf
echo "4. which resolver, really"; resolvectl query "$NAME" 2>/dev/null || cat /etc/resolv.conf

✅ Expected result — click to reveal

There is no single expected output — the value is in which of the four disagree. Read them as a decision table:

1 and 2 agree, app still wrong → name resolution is fine. The fault is inside the application: a cached result, a connection pool holding an old socket, or configuration pointing at a different name entirely. Stop debugging DNS.

1 right, 2 wrong → the C library is finding something else. Command 3 finds it: an /etc/hosts entry, or an nsswitch.conf order you did not expect.

1 empty, 2 works → search-list expansion. The application is querying a longer name than you are. Confirm with dig +search.

1 and 2 both right for you, wrong for the app's host → you are on different resolvers. Command 4 shows which, including per-link split DNS on a VPN.

Why to run all four rather than guessing. Each eliminates a layer, and the pair that disagrees names the layer. Four commands, and you can say which file on which host is responsible — which is a handover, not a hypothesis.

D2 · Prove which resolver is really being used

The analogy. Think of the staff rota versus the CCTV footage.

The rota says who was supposed to be on duty. The footage shows who actually was. When the two disagree, nobody argues with the footage.

Configuration files are the rota. tcpdump is the footage.

bash

# systemd hosts - the authoritative answer
resolvectl status | grep -E 'Link|Current DNS Server|DNS Domain'

# any host - what the stub is configured with
grep -E '^(nameserver|search|options)' /etc/resolv.conf

# what nsswitch will actually consult, in order
grep '^hosts' /etc/nsswitch.conf

# watch a real query leave the box, and see where it goes
sudo timeout 5 tcpdump -n -i any 'udp port 53' &
sleep 1 ; getent hosts seamless.se > /dev/null ; wait

tcpdump is the tiebreaker and it cannot be argued with. Configuration files describe intent; the packet capture shows the destination address the query was actually sent to, and the name that was actually in it. When someone insists the resolver is 10.0.0.1 and the capture shows 192.168.1.1, the discussion is over.

It also makes search-list expansion visible — you will see the failed name.suffix1, name.suffix2 queries go out before the successful one, which is the cheapest possible proof of the latency problem in B3.

Part E · Putting it together

E1 · How this all fits — the complete picture

Diagram source

flowchart TD
    APP["application<br>getaddrinfo NAME"] --> NSS["/etc/nsswitch.conf"]
    NSS -->|"1. files"| HOSTS["/etc/hosts<br>MATCH ENDS THE SEARCH<br>no TTL, no expiry"]
    NSS -->|"2. resolve"| RSD["systemd-resolved socket<br>bypasses resolv.conf<br>per-link split DNS"]
    NSS -->|"3. dns"| STUB["stub resolver<br>reads /etc/resolv.conf"]
    STUB --> ND{"dots in name<br>< ndots ?"}
    ND -->|"yes - relative"| SL["try search suffixes FIRST<br>one query EACH<br>bare name LAST"]
    ND -->|"no - absolute"| BARE["query the name as given"]
    SL --> NS1["nameserver 1<br>wait up to timeout:5"]
    BARE --> NS1
    NS1 -->|"no reply"| NS2["nameserver 2<br>another 5s"]
    NS2 -->|"no reply"| NS3["nameserver 3, then<br>repeat the whole list<br>attempts:2 -> up to 30s"]
    DIG["dig"] -.->|"skips ALL of this"| NS1
    APPC["application cache<br>JVM: forever by default"] -.->|"can override everything"| APP
    style HOSTS fill:#ef4444,color:#fff
    style APPC fill:#ef4444,color:#fff
    style DIG fill:#f59e0b,color:#fff
    style SL fill:#f59e0b,color:#fff

Four ideas, and every confusing DNS ticket you will ever get comes from one of them.

dig and your application do not take the same path. dig speaks DNS; applications call getaddrinfo, which consults /etc/hosts first and may query a different name entirely.
/etc/hosts silently beats DNS, has no expiry, and is invisible to every DNS tool.
ndots decides whether your name gets suffixes appended — and every failed suffix is a real round trip.
127.0.0.53 means the file is not telling you where queries go. resolvectl status is.

E2 · Production practice

Habit	Why
Run getent hosts alongside dig, always	dig shows what DNS says; getent shows what the application will get. The disagreement is the diagnosis
Grep /etc/hosts before believing any DNS result	It overrides DNS, has no TTL, and no DNS tool can see it. Debug entries outlive the debugging by months
On systemd hosts, use resolvectl status, not cat /etc/resolv.conf	127.0.0.53 is a local process. The real upstreams and any per-link split DNS are only visible via resolvectl
Never hand-edit a resolv.conf that says "Do not edit"	It is regenerated, and on resolve-enabled hosts applications never read it anyway
Use FQDNs with a trailing dot in configuration files	Skips search-list expansion entirely — fewer queries, lower latency, and no dependence on ndots
Set timeout:1 attempts:2 in containers	The defaults cost up to 30 seconds on a failure. A fast failure beats a slow success
Keep the search list short	Every entry is a wasted round trip on every relative lookup, on every host, forever
Treat "the app is slow" as a possible DNS symptom	At scale DNS faults present as latency, not errors, because the stub waits out timeouts and then succeeds
Know when to stop blaming DNS	If dig and getent agree and the app still misbehaves, it is an application cache or connection pool
Reach for tcpdump -n 'udp port 53' to settle arguments	Files describe intent; the capture shows the destination and the actual name queried

E3 · Capstone exercise

Reproduce every failure in this module deliberately, then explain each from the output alone. No scrolling back.

Brief. On a Linux host you control, produce and explain all seven:

Make dig and getent return different addresses for the same name. Explain which is "right".
Make dig return nothing for a name that getent resolves perfectly. Explain the mechanism, and give the dig option that reproduces the library's behaviour.
Show the exact worst-case time a failing lookup can take on default settings, and derive it.
On a systemd host, show that /etc/resolv.conf cannot answer "which resolver am I using", and show what can.
Demonstrate that a name with a trailing dot bypasses the search list, using packet-level evidence.
Give the four-command triage for "dig works, the app doesn't", and say which layer each command eliminates.
Name the one failure in this module that none of your commands can see, and how you would confirm it.

✅ Model answer — attempt it first, then click

1. /etc/hosts override.

bash

echo "203.0.113.77  demo-host.lab.internal" | sudo tee -a /etc/hosts
getent hosts demo-host.lab.internal   # 203.0.113.77
dig +short demo-host.lab.internal     # nothing

Neither is "right" — they answer different questions. dig reports what DNS contains. getent reports what the application will receive, and the application is what users experience. For a support ticket, getent is the authoritative view.

2. Search-list expansion.

bash

printf 'nameserver 8.8.8.8\nsearch seamless.se\noptions ndots:2\n' | sudo tee /etc/resolv.conf
getent hosts admin-vfo        # 185.64.27.150  admin-vfo.seamless.se
dig +short admin-vfo          # nothing
dig +search +short admin-vfo  # 185.64.27.150

admin-vfo has 0 dots, which is below ndots:2, so the library appends seamless.se and queries a name you never typed. dig sent the bare label to the root, where it does not exist. dig +search is the option that reproduces library behaviour.

3. Thirty seconds. 3 nameservers × timeout:5 × attempts:2 = 30. And with only the first server dead, every successful lookup still costs 5 seconds. The failure mode is latency, not errors — which is why nobody reports it as a DNS problem.

bash

cat /etc/resolv.conf      # nameserver 127.0.0.53   <- a local process, tells you nothing
resolvectl status         # the real per-link servers, plus split-DNS domains

127.0.0.53 is systemd-resolved on this machine. The real upstreams come from DHCP, VPN or netplan and appear nowhere in the file. On a host whose nsswitch.conf includes resolve, applications do not read the file at all.

5. Packet-level evidence, which is the part the question is really asking for:

bash

sudo tcpdump -n -i any 'udp port 53' &
getent hosts admin-vfo       # -> query for admin-vfo.seamless.se
getent hosts admin-vfo.      # -> query for admin-vfo.  only

The capture shows the suffix being appended in the first case and not in the second. The trailing dot makes the name absolute, so ndots never applies — Module 01 A3's payoff, now visible on the wire.

6. The triage, and what each rules out:

plain text

dig +short NAME        -> what DNS contains        (eliminates: the zone / the record)
getent hosts NAME      -> what the app will get    (eliminates: everything above the C library)
grep NAME /etc/hosts   -> local override           (eliminates: the file that beats DNS)
resolvectl query NAME  -> which resolver + link    (eliminates: wrong server / split DNS)

The pair that disagrees names the layer. If 1 and 2 agree and the app is still wrong, name resolution is not the problem.

7. The application-level cache. A JVM with default settings caches a successful lookup for the life of the process; connection pools hold sockets opened against an address resolved long ago. No tool on this page can see inside a process, so dig, getent, resolvectl and tcpdump will all be clean while the application keeps using the old address.

Confirm it by restarting the process and seeing whether the behaviour changes, and by checking the runtime's DNS cache setting — for the JVM, networkaddress.cache.ttl.

The five things most people miss:

Requirement 1's real answer — that neither tool is wrong. Ranking them is the mistake; knowing what each one measures is the skill
dig +search in requirement 2. Most people conclude the name does not exist and stop
Deriving 30 seconds rather than saying "5" in requirement 3, and noting the symptom is latency
Packet-level proof in requirement 5. Asserting that the trailing dot works is not the same as showing the query that was actually sent
Requirement 7 at all. Knowing when to stop blaming DNS is a genuine skill, and the application cache is the layer that punishes people who cannot

E4 · Official documentation

Link	Covers
resolv.conf(5)	nameserver, search, domain, and every options flag including ndots, timeout, attempts, rotate
nsswitch.conf(5)	The switchboard, the source list, and the [NOTFOUND=return] action syntax
hosts(5)	The file that beats DNS
getaddrinfo(3)	What applications actually call, and the address-selection rules that reorder results
getent(1)	The command-line window onto the same path the application takes
resolvectl(1) · systemd-resolved.service(8)	Per-link servers, split DNS, cache statistics, and the 127.0.0.53 stub
RFC 9499 — DNS Terminology	Stub resolver, relative name, FQDN — the vocabulary this module leans on

How to read these efficiently.

man 5 resolv.conf is the single highest-value page in this module. It is short, complete, and it is the definitive statement of the ndots rule that everyone paraphrases incorrectly.

Read the getaddrinfo(3) NOTES section, not the whole page. That is where the behaviour that surprises people lives.

man nsswitch.conf for the action syntax — [NOTFOUND=return] and friends. It looks cryptic and takes two minutes to learn.

The offline route, and it is complete here. Every reference in this module is a man page on the machine itself: man 5 resolv.conf, man 5 nsswitch.conf, man 5 hosts, man 3 getaddrinfo, man 1 resolvectl. On a locked-down host with no internet, you already have the entire documentation set for this subject.

E5 · Self-assessment

1. Why can dig and an application disagree about the same name?

They take different paths. dig speaks DNS directly. Applications call getaddrinfo(), which follows /etc/nsswitch.conf — usually /etc/hosts first, then DNS — and may append search-list suffixes, so it can query a different name entirely.

dig also ignores /etc/hosts completely and does not apply the address-selection ordering that getaddrinfo does.

The command that matches application behaviour is getent hosts.

2. State the ndots rule precisely.

Count the dots in the name as written. Fewer than ndots → treat it as relative: try each search-list suffix first, bare name last. ndots or more → try the name as given first.

Default is 1. A trailing dot makes the name absolute and skips expansion entirely, regardless of ndots.

3. How long can a failed lookup take on default settings, and why is that a problem?

Up to 30 seconds: 3 nameservers × timeout:5 × attempts:2.

It is a problem because it presents as latency, not failure. A single dead nameserver costs 5 seconds on every lookup that then succeeds, so users report "slow", monitoring shows healthy DNS servers, and nobody suspects DNS.

In containers, timeout:1 attempts:2 is standard for exactly this reason.

4. Your /etc/resolv.conf says nameserver 127.0.0.53. What do you actually know?

Only that systemd-resolved is running locally and the stub listener is on that address. You do not know which upstream resolvers are used — they come from DHCP, a VPN or netplan and are not in the file.

You also cannot fix anything by editing the file: it is regenerated, and if nsswitch.conf contains resolve, applications bypass it and talk to resolved over a socket.

resolvectl status is the command that answers the question.

5. What is split DNS as systemd-resolved implements it?

Per-interface routing of queries by domain. A VPN link can claim corp.internal while the wifi link keeps ~. — the default route for everything else — so corporate names go to the VPN's resolver and the rest go to your router.

It is invisible in /etc/resolv.conf and visible in resolvectl status as the DNS Domain field on each link.

It is why "works on the VPN, fails off it" is a resolved question rather than a DNS-server question.

6. Why do FQDNs with a trailing dot belong in configuration files?

They are absolute, so ndots never applies and the search list is never expanded. That means exactly one query instead of up to one-per-suffix-plus-one, on every lookup, on every host.

It also makes behaviour identical everywhere — a relative name resolves differently depending on the search list of whatever host the config lands on, which is a class of bug that disappears entirely with a trailing dot.

7. dig and getent agree, and the application still connects to the wrong address. Now what?

Name resolution is not the problem, and continuing to debug DNS is wasted effort.

The remaining suspects are inside the process: an application-level DNS cache — a JVM with default settings caches successful lookups for the life of the process — or a connection pool holding sockets opened against an address resolved long ago.

Confirm by restarting the process, and check the runtime's cache setting, e.g. networkaddress.cache.ttl.

8. What does hosts: files resolve [!UNAVAIL=return] dns mean?

Try /etc/hosts first. Then ask systemd-resolved over its socket. [!UNAVAIL=return] means: if resolved answered at all — even with "not found" — stop here and return that; only fall through to the classic dns path if resolved was actually unavailable.

The practical consequence is that /etc/resolv.conf is effectively unused on such a host, which is why editing it changes nothing.

9. How do you settle an argument about which resolver a host is really using?

tcpdump -n -i any 'udp port 53' while triggering a lookup. Configuration files describe intent; the capture shows the destination address the query actually went to, and the exact name that was in it.

It also makes search-list expansion visible — you see the failed suffixed queries go out before the successful one, which is direct evidence of the latency cost.

10. Why is /etc/hosts dangerous in production?

It silently overrides DNS, has no TTL or expiry, is per-host so machines can disagree, and is invisible to every DNS diagnostic tool.

A pin added during an incident is correct that day and wrong forever after, and the next engineer's dig output will look perfect while the application connects somewhere else.

If you must use it, treat it as configuration: managed by your config-management tool, with an expiry date and a ticket, never edited by hand on one box.

Next — Module 07 · DNSSEC.

You have seen ad, cd and do flags, an RRSIG in a +trace, a DS record at the .se delegation, and secure: no in rndc zonestatus — all deferred to "Module 07". Now you sign the lab.internal zone from Module 05, build the chain of trust, validate it with delv, and break it on purpose to produce the SERVFAIL that DNSSEC failures always disguise themselves as.

📚 Sources for the interview questions

The nsswitch.conf, /etc/hosts override, getent-versus-dig divergence and the complete ndots demonstration in B3.1 were run for real on 20 August 2026 against seamless.se, with glibc resolution and dig 9.18. The resolvectl status output in C2.1 is labelled in place as the shape from a VPN-attached laptop rather than a capture, because the environment this was written in runs no systemd-resolved.

Documentation verified directly: resolv.conf(5), nsswitch.conf(5), hosts(5), getaddrinfo(3), resolvectl(1), and RFC 9499.

Question selection cross-referenced against publicly published 2026 DNS and networking interview question sets:

Top 25 DNS Interview Questions and Answers for 2026 — nitizsharma.com — what causes DNS resolution failure, how to flush DNS cache, split-horizon DNS, DNS resolver types
Top 30 Most Common DNS Interview Questions — Verve AI — troubleshooting DNS and essential commands, likely scenario questions
How to Troubleshoot DNS Like a Pro Using dig and nslookup — OpsCheck
50 Must-Prepare Networking Interview Questions for DevOps Engineers

Answers were rewritten and deepened rather than reproduced. Published sets answer "how do you troubleshoot DNS" with a list of commands; almost none mention that dig and the application take different paths, which is the single most common reason a DNS investigation goes nowhere — and the one thing an interviewer who has run production systems will be listening for.

Spotted a mistake or want something added? Send me a note.

🐧Module 06 — The Resolver Side of a Linux Host

Part A · The path an application actually takes

A1 · getaddrinfo, not dig

A2 · nsswitch.conf — the switchboard

A3 · /etc/hosts — the override that beats DNS

🎯 Interview questions — The resolution path

Part B · resolv.conf in full

B1 · nameserver — order, timeout, attempts

B2 · search and domain

B3 · ndots — the rule, and the arithmetic

🎯 Interview questions — resolv.conf

Part C · The local caching layers

C1 · 127.0.0.53, and why editing resolv.conf does nothing

C2 · resolvectl — the command that tells the truth

C3 · dnsmasq, containers, and the layers you did not ask for

Part D · Field recipes

D1 · dig works, the application does not

D2 · Prove which resolver is really being used

Part E · Putting it together

E1 · How this all fits — the complete picture

E2 · Production practice

E3 · Capstone exercise

E4 · Official documentation

E5 · Self-assessment

Module 06 — The Resolver Side of a Linux Host