⚡Module 08 — Execution Strategies & Performance at Scale

Tuesday 17 August 2026  09:14:02 +0800 (0:00:06.812)       0:00:06.812 ******
===============================================================================
Gathering Facts --------------------------------------------------------- 6.81s
Install base packages --------------------------------------------------- 4.22s
Deploy nginx configuration ---------------------------------------------- 0.94s
Ensure nginx is running -------------------------------------------------- 0.61s
Create application user -------------------------------------------------- 0.44s
Set sysctl values -------------------------------------------------------- 0.38s
...
Playbook run took 0 days, 0 hours, 0 minutes, 19 seconds

Gathering Facts is the single most expensive item, and you did not write it. That is the pattern on almost every real playbook, and it is why Part C exists.

Read it as a ratio, not a number. Here 6.81s of 19s — 36% of the run — went on collecting data before doing anything. On 200 hosts with forks: 5 that ratio holds and the absolute number becomes minutes, on every run, including every CI-triggered one.

Do this before changing any setting. Raising forks when the bottleneck is a single slow task achieves nothing; disabling fact gathering in a play that genuinely needs facts breaks it. profile_tasks costs nothing and tells you which lever to pull.

Enable profile_tasks and timer callbacks, which give per-task wall clock and a slowest-tasks summary, then read the ratio rather than the raw numbers.

On most real playbooks the largest single item is Gathering Facts, followed by connection setup — not the tasks. So the first questions are whether the play needs facts at all, whether forks is still at the default 5, and whether pipelining is on.

Measure first. Raising forks when one slow task is the bottleneck changes nothing.

forks=5,  pipelining=off   real  0m38.412s
forks=50, pipelining=off   real  0m6.883s
forks=50, pipelining=on    real  0m4.201s

5      <- pipelining off: mkdir, copy, chmod, execute, cleanup
2      <- pipelining on: essentially just execute

Two separate effects, and it is worth keeping them distinct.

forks removed the queueing — 38s to 6.9s, roughly a 5.5x improvement, because hosts stopped waiting in line. This is pure parallelism and does nothing for a single host.

Pipelining removed work per host — a further ~40%, by collapsing five SSH operations per task into two. This helps even on one host, and it compounds with task count: a 50-task playbook saves 150 SSH operations per host.

Which to reach for: forks when you have many hosts and few tasks; pipelining when you have many tasks regardless of host count. In practice, both.

The number of hosts worked on in parallel; default 5, which turns a 200-host run into 40 sequential rounds. 25–50 is normal.

The limit is the control node — each fork is a Python process holding an SSH connection, so CPU, RAM and the open-file-descriptor limit bound it. Going to several hundred needs a properly sized control node and a raised ulimit -n. The managed nodes are not the constraint.

It feeds the module to the remote Python over the already-open SSH session instead of creating a temp directory, copying a file, executing and cleaning up — roughly five SSH operations per task down to two, about a 40% reduction.

It is off by default because it requires requiretty to be disabled in /etc/sudoers on the managed nodes, and not every distribution image ships that way. With become in use and requiretty on, tasks fail with sudo: sorry, you must have a tty to run sudo.

The SSH ControlPersist socket path, built from user, host and port, has exceeded the operating system's Unix socket path limit — common with long cloud DNS hostnames.

Fix by shortening control_path_dir (for example /tmp/.acp) or using a shorter control_path template such as %(directory)s/%%h-%%r. The error message never mentions hostnames, which is what makes it memorable once you have hit it.

Use modules that accept lists rather than looping — package: name: [a, b, c] is one transaction, a loop is three. Enable pipelining to collapse the per-task SSH operations. Keep ControlPersist multiplexing on so the TCP and TLS handshake happens once per host rather than per task.

And structurally: fewer, larger tasks beat many small ones, because the round trip is the cost, not the work.

full gather              real  0m6.812s
gather_subset network    real  0m2.944s
caching, cold            real  0m6.903s
caching, warm            real  0m1.402s

/tmp/afacts/:
db01  web01  web02  web03

Three distinct wins, and they stack.

gather_subset cut the gather itself roughly in half by skipping hardware, which probes block devices and mounts.

Caching removed the gather entirely on the second run — 6.9s to 1.4s. That is the setting that matters for a fleet you run against repeatedly, and gathering = smart is what makes it take effect: without it, Ansible gathers regardless and the cache is only written, never read.

Open one of the cache files — cat /tmp/afacts/web01 | python3 -m json.tool | head — and you will see the full fact dictionary as JSON. That is exactly what hostvars['web01'] reads from on a cached run, which is also why caching fixes the hostvars problem from Module 02 Exercise B4.2 without a preliminary gathering play.

⚠️ Now change something on a host — add an IP, for instance — and re-run. The playbook still reports the old value until fact_caching_timeout expires or you pass --flush-cache. That is the trade, and you should be able to state it before someone asks.

Fact caching with gathering = smart — removes the gather entirely on repeat runs, which is the biggest win for a fleet you run against regularly.

gather_subset — collect only what you need; excluding hardware typically halves it, since that subset probes block devices and mounts.

gather_facts: false — for plays that reference no ansible_* variable at all.

And measure first with profile_tasks, because Gathering Facts is usually the single largest item but not always.

It gathers facts only for hosts that have no cached facts. Without it, gathering stays implicit and Ansible gathers at every play start regardless — so the cache is written but never read, and you get no speed-up at all while believing caching is enabled.

explicit is the third option: never gather unless a play asks.

Stale data. A host gains memory or an interface and the playbook keeps using yesterday's values until the cache expires — and the playbook looks correct throughout, which makes it a genuinely unpleasant bug.

Mitigate with a fact_caching_timeout matched to how fast the estate changes, --flush-cache where correctness outweighs speed, and by not caching facts that drive security decisions.

It suits large, read-heavy fleets; it is wrong for provisioning runs where the machine is changing underneath you.

hostvars['db01']['ansible_facts'] is undefined unless db01's facts were gathered in this run — the Module 02 trap. With caching enabled, previously gathered facts are available, so a play targeting only web can read db01's address without a preliminary fact-gathering play.

The caveat is the same: those cached values may be stale.

linear — the default:

TASK [Uneven work] ***********************
ok: [web02]        <- 1s
ok: [web03]        <- 1s
ok: [db01]         <- 1s
ok: [web01]        <- 8s

TASK [Announce] **************************       <- nothing starts until 8s
ok: [web02] => {"msg": "done on web02"}
...
real  0m9.2s

free:

TASK [Uneven work] ***********************
ok: [web02]
TASK [Announce] **************************
ok: [web02] => {"msg": "done on web02"}     <- web02 finished the WHOLE play at 1s
TASK [Announce] **************************
ok: [web03] => {"msg": "done on web03"}
TASK [Uneven work] ***********************
ok: [web01]                                  <- web01 still on task 1 at 8s
real  0m8.4s

Read the interleaving in the free output. Task headers repeat out of order because each host is running its own copy of the play independently. Three hosts finished completely while web01 was still on task 1.

The wall-clock saving here is small (9.2s → 8.4s) and that is the honest lesson — free cannot beat the slowest host, it only stops the others waiting. The benefit grows with the number of tasks and the variance between hosts, and it is zero when all hosts take the same time.

What you gave up is larger than what you gained, unless the work is genuinely independent: there is now no point in the play where you can rely on all hosts having reached the same state.

linear — the default, with a barrier between tasks so every host completes task N before any starts N+1. Required for orchestration.

free — no barrier; each host races through the whole play independently. For independent, order-insensitive work such as patching or log collection.

host_pinned — a worker takes one host and finishes it before moving on; useful with many hosts and few forks.

debug — drops into an interactive debugger on failure.

The trade with free is not just speed: you lose every cross-host ordering guarantee, so "migrate the database then restart the app tier" stops holding.

forks is global parallelism — how many hosts Ansible works on at once, across the whole run.

serial batches the entire play — each batch completes every task before the next batch begins. This is how rolling deploys are built.

throttle limits concurrency for one specific task, for a rate-limited API or licence server, while the rest of the play runs at full speed. It can only ever reduce concurrency below forks, never raise it.

async: 7200 with poll: 30 — Ansible starts the task in the background on the target and polls for completion, so no single SSH session has to stay open for two hours.

For parallelising independent long work, poll: 0 fires and forgets, and you collect results later with async_status plus until: result.finished.

Caveats worth volunteering: poll: 0 loses the outcome entirely unless you check; async does not work with control-node action plugins; and the job state lives in ~/.ansible_async/ on the target, so a reboot destroys it.

It runs once per batch, not once per play — so with serial: 10 across 100 hosts it executes ten times.

That surprises people writing database migrations. If you need genuinely once, combine run_once with delegate_to a fixed host and guard it, or move it into a separate play that targets a single host before the rolling play begins.

Step 1 — measure. Never tune first.

ANSIBLE_CALLBACKS_ENABLED=profile_tasks,timer,cgroup_perf_recap \
  ansible-playbook site.yml 2>&1 | tail -30

Suppose it shows: Gathering Facts 14m, Install packages 9m, Security scan 18m, everything else under a minute.

Step 2 — ansible.cfg:

[defaults]
forks             = 50                    # requirement 3
gathering         = smart                 # requirement 2
fact_caching      = jsonfile
fact_caching_connection = /tmp/ansible_facts
fact_caching_timeout    = 7200
callbacks_enabled = profile_tasks, timer

[ssh_connection]
pipelining       = True
ssh_args         = -o ControlMaster=auto -o ControlPersist=60s
control_path_dir = /tmp/.acp             # long cloud hostnames

ulimit -n 8192                            # requirement 3 - BEFORE raising forks

Step 3 — the playbook:

---
- name: Tuned fleet playbook
  hosts: all
  serial: [1, 10, "25%"]                  # requirement 7
  max_fail_percentage: 0                  # requirement 7
  gather_facts: true
  gather_subset:
    - "!all"
    - "!min"
    - network                             # requirement 2 - keep network, drop hardware

  tasks:
    - name: Start the security scan in the background    # requirement 5
      ansible.builtin.command: /opt/security/scan.sh
      async: 1800
      poll: 0
      register: scan_job
      changed_when: false

    - name: Install packages in ONE transaction          # requirement 4
      ansible.builtin.package:
        name:
          - nginx
          - git
          - htop
        state: present

    - name: Register with the licence server             # requirement 6
      ansible.builtin.uri:
        url: https://licence.internal/register
        method: POST
      throttle: 5
      changed_when: false

    - name: Now collect the scan result                  # requirement 5
      ansible.builtin.async_status:
        jid: "{{ scan_job.ansible_job_id }}"
      register: scan_result
      until: scan_result.finished
      retries: 60
      delay: 30

The seven decisions:

1. profile_tasks first — requirement 1. Without it you might have "optimised" the 40-second tasks and left the 18-minute scan blocking.
2. gather_subset: ["!all", "!min", "network"] — requirement 2. Drops hardware, which is the expensive subset, while keeping the network facts templates use. Plus gathering = smart with caching, so repeat runs skip gathering entirely.
3. forks: 50 with ulimit -n 8192 raised first — requirement 3. The limit is the control node's CPU, RAM and file descriptors, not the targets, and raising forks without raising the fd limit makes it slower with no error.
4. package: name: [list] — requirement 4. One transaction, one round trip, replacing three.
5. async: 1800, poll: 0 at the start, async_status at the end — requirement 5. The scan runs while the other tasks proceed, and the result is still collected, so a failure is not silently discarded.
6. throttle: 5 — requirement 6. Only that task is limited; the rest of the play stays at forks: 50.
7. serial: [1, 10, "25%"] with max_fail_percentage: 0 — requirement 7. Canary of one, then progressive batches, halting on the first failure.

The mistake to avoid: serial and forks interact. With serial: 10 only ten hosts are in the batch, so forks: 50 has nothing extra to do — the effective parallelism is min(forks, batch size). That is fine and expected here, since the batching is the safety requirement and the forks setting matters for the larger later batches.

Verify:

ANSIBLE_CALLBACKS_ENABLED=profile_tasks,timer ansible-playbook site.yml --limit web01
ansible-playbook site.yml --list-hosts
ansible-playbook site.yml --check --diff --limit web01

Gathering Facts — a full extra module execution per host, often 30%+ of the run — followed by connection setup. Rarely the tasks themselves.

That is why "my playbook is slow" almost never means "my tasks are slow", and why you profile before tuning.

The control node: CPU, RAM and open file descriptors, since each fork is a Python process holding an SSH connection. Raise ulimit -n before going past roughly 50.

The symptom of overshooting is the run getting slower with no error, which cgroup_perf_recap will show as control-node saturation.

It requires requiretty disabled in sudoers on the managed nodes, and not every image ships that way — with become in use it fails with sudo: sorry, you must have a tty to run sudo.

Check with an ad-hoc grep of /etc/sudoers and /etc/sudoers.d/ across the fleet before enabling it globally.

Without it, gathering stays implicit and Ansible gathers at every play start regardless — the cache is written but never read, so you get no speed-up while believing caching is on.

Stale facts, producing bugs where the playbook looks entirely correct. Manage with a fact_caching_timeout matched to how fast the estate changes, --flush-cache when correctness matters more than speed, and by not caching facts that drive security decisions.

Every cross-host ordering guarantee. Without the barrier, "migrate the database then restart the app tier" no longer holds — a fast app server can restart against an unmigrated database.

And the wall-clock gain is bounded by the slowest host; free only stops the others waiting.

forks is global parallelism across the run. serial batches the entire play so each batch completes every task before the next begins. throttle caps concurrency for one task, and can only reduce below forks.

poll: 0 never learns the outcome unless you follow up with async_status. async does not work with action plugins that run on the control node. And the job state lives in ~/.ansible_async/ on the target, so async_status must target the same host and a reboot destroys it.

Ten — once per batch, not once per play. For genuinely once, put it in a separate play targeting a single host before the rolling play, or combine run_once with delegate_to and a guard.

Thousands of hosts where push saturates the control node; hosts unreachable inbound (NAT, customer sites, ephemeral instances); or when you want continuous enforcement.

You lose orchestration entirely — every host is independent — and centralised visibility, since you must ship logs yourself. That is usually decisive, which is why most teams scale the control node or move to AWX instead.

Profile → fact strategy → fact caching → forks (with ulimit) → pipelining → fewer round trips → async/throttle for specific tasks.

Facts and caching usually deliver more than forks and pipelining combined, and almost everyone tries forks first.

Behaviour verified against the current official strategies documentation and configuration reference.

Question selection cross-referenced against publicly published 2026 Ansible interview question sets:

• Spacelift — 50+ Top Ansible Interview Questions & Answers for 2026
• GeeksforGeeks — Top 50+ Ansible Interview Questions and Answers
• Vinsys — Top 30 Ansible Interview Questions and Answers 2026
• K21 Academy — Ansible Interview Questions & Answers 2026
• Hirist — Top 25+ Ansible Interview Questions and Answers 2026

Answers were rewritten and deepened rather than reproduced — published versions are usually correct but shallow, and the added operational detail is what differentiates a candidate in the room.