Module 12 — Ansible Tower, AWX & Automation Platform

Updated 20 August 2026

Module 12 · AWX / Ansible Automation Platform

Running Ansible as a service rather than from laptops and CI jobs — with RBAC, credential management, scheduling, an audit trail, and self-service for people who should never touch YAML.

🧠 concept → 🧪 exercise → ✅ expected result (hidden) → 🎯 interview questions (answers hidden)

Prerequisite: Modules 01–11.


Part A · Why AWX exists

A1 · The problems it solves

The analogy. Think of a hotel reception desk. The keys live in a cabinet behind the desk, not in the guests' pockets. You are handed a card that opens your room and nothing else, and it stops working the morning you check out. The desk keeps a record of who was given which card and when. And housekeeping can get into your room every day without ever being handed a master key.

AWX is that reception desk. Your playbooks are completely unchanged — what it adds is the key cabinet (credential storage), the room-specific cards (RBAC), the expiry (revocation), and the register (the audit trail). Every feature in this module is one of those four things.

Problem with laptops + CIWhat AWX provides
Anyone with the repo and SSH keys can run anythingRBAC — who may run which template against which inventory
"Who ran that, and when?"Audit trail — every job, its output, its inputs, its operator
Vault passwords and SSH keys on individual machinesCredential store — encrypted, injected at run time, never displayed
Two people run the same playbook simultaneouslyJob queueing and concurrency control
Cron on someone's laptopScheduling with visibility and history
Support staff cannot run automation without YAMLSurveys — a web form that becomes extra vars
"It works on my machine"Execution Environments — one pinned container runtime
The honest framing for an interview: "AWX is not faster or more capable than ansible-playbook — it runs exactly the same engine. What it adds is everything around the run: access control, credentials, scheduling, queueing, an audit trail, and a UI and API for people who should not be handed SSH keys. You adopt it when the constraint stops being 'can we automate this' and becomes 'who is allowed to, and can we prove what happened'."

A2 · Ansible Tower, AWX and Automation Controller — the same thing, three names

Read this section carefully if you have heard of "Ansible Tower" but not "AWX". That is the normal situation, and the naming is genuinely confusing because the product has been renamed twice. They are not three competing tools — they are one lineage.

The timeline

plain text
2013   Ansible Tower released — the commercial web UI for Ansible.
       Closed source, sold by AnsibleWorks, then Red Hat after the 2015 acquisition.

2017   Red Hat open-sources Tower's codebase as AWX.
       AWX becomes the free UPSTREAM project; Tower remains the supported downstream product.

2021   With Ansible Automation Platform 2.0, "Ansible Tower" is renamed
       "automation controller". The product did not change. The name did.

Today  AWX (free, upstream, community-supported)
         -> automation controller (supported, subscription)
              -> one component of Ansible Automation Platform (AAP)
The analogy that makes it click, and the one Red Hat itself uses:

AWX is to Automation Controller what Fedora is to RHEL.

Same codebase lineage. AWX moves fast, is free, and is where features land first. Automation Controller is the stabilised, supported, subscription version. You learn one and you know the other — the UI, the object model and the API are the same.

Why you have heard of Tower and not AWX

NameWho says it, and why
Ansible TowerWas the commercial product for eight years, so it is the name in job adverts, in most training material, in enterprise procurement, and in the vocabulary of anyone who worked with it before 2021. This is the name an interviewer will most likely use
AWXThe free upstream. Mostly encountered by people who self-host it, or who evaluate before buying. Rarely appears in job adverts
automation controllerThe current correct name. Used in current Red Hat documentation, and by teams already on AAP 2.x
AAPThe whole suite — controller plus Private Automation Hub, Execution Environments and Event-Driven Ansible
What to say when an interviewer says "Tower".

Do not correct them, and do not pretend it is a different product. Answer the question, and fold the current naming in naturally:

"Yes — Tower is what's now called automation controller since AAP 2.0; I've used AWX, which is the same codebase as the free upstream. In it I'd solve that with a job template plus a survey and Execute-only RBAC..."

That single sentence does three things: it answers them, it shows you know the current naming, and it signals you have hands-on exposure rather than only product-brochure familiarity. Correcting an interviewer's vocabulary without answering their question achieves the opposite.

One practical difference worth knowing, beyond the name. AWX is not supported for production by Red Hat — it has no support contract, no long-term-support releases, and it upgrades frequently with occasional breaking changes.

Plenty of organisations run it in production anyway and accept that. But "we run AWX in production" and "we run automation controller in production" are different risk statements, and an interviewer at an enterprise may well probe which one you mean.

The old naming, restated as a table

NameWhat it is
AWXThe free, open-source upstream project. Community-supported, fast-moving
Automation ControllerThe supported downstream product — formerly called Ansible Tower
Ansible Automation Platform (AAP)The whole Red Hat suite: Controller + Private Automation Hub + Execution Environments + Event-Driven Ansible
So: Tower → automation controller is a rename, not a new product. AWX is the free upstream of that same codebase. AAP is the suite it belongs to.

If you only remember one line: "Tower is now called automation controller, and AWX is its free upstream — like Fedora to RHEL."


Part B · The object model

B1 · How the pieces fit

Diagram source
flowchart TD
    ORG["Organization<br>the top-level boundary"] --> PRJ["Project<br>a Git repo of playbooks"]
    ORG --> INV["Inventory<br>static or a plugin source"]
    ORG --> CRED["Credentials<br>SSH key, vault password, cloud"]
    ORG --> TEAM["Teams and Users<br>with Roles"]
    PRJ --> JT["Job Template<br>playbook + inventory + credentials"]
    INV --> JT
    CRED --> JT
    JT --> SURV["Survey<br>a web form for extra vars"]
    JT --> SCHED["Schedule<br>cron-like, with history"]
    JT --> JOB["Job<br>one run, with full output"]
    JT --> WF["Workflow<br>chains templates with<br>success/failure branches"]
    TEAM -.->|"RBAC grants"| JT
    style JT fill:#DDD6FE,stroke:#7C3AED,stroke-width:2px
    style JOB fill:#D1FAE5,stroke:#059669
ObjectWhat it holds
OrganizationThe tenancy boundary. Everything else belongs to one
ProjectA Git repository and branch. AWX clones it before each run if you enable "update on launch"
InventoryHosts and groups — typed in, or from an inventory source (the Module 09 plugins)
CredentialEncrypted secrets by type: Machine (SSH), Vault, AWS, Git, Registry
Job Template⭐ The runnable unit — project + playbook + inventory + credentials + options
SurveyA form attached to a template; answers become extra vars
Workflow TemplateA graph of templates with on-success / on-failure / always branches
ScheduleRecurring execution with history and next-run visibility
Execution EnvironmentThe container image the job runs inside — Module 06 Part D2
Instance GroupWhich execution nodes run the job — for network isolation or capacity

B2 · Credentials — the part that matters most

The analogy. Think of valet parking. You hand over the key and the valet drives your car — that is the whole point of the arrangement. But they cannot take the key home, they cannot have a copy cut, and you get it back at the end of the night. They have use of the car without ownership of the key.

AWX credentials work exactly like that: once saved, a secret can be used by a job and never displayed again, not even to an administrator. That separation — permission to run without permission to see — is the single strongest argument for AWX, and it is the one thing neither a laptop nor a CI environment variable can honestly give you.

AWX credentials are write-only from the UI. Once saved, a secret can be used by a job but never displayed again, not even to an administrator. That is the core security property, and it is what lets you give someone permission to run a template without giving them the SSH key or vault password it uses.

That separation — execute rights without read rights on the secret — is the thing laptops and CI variables cannot give you, and it is the strongest single argument for AWX.

Credential typeInjected as
MachineSSH private key or password, plus become method and password
VaultThe vault password, optionally with a vault ID — Module 07 Part B2
Source ControlGit credentials for cloning the project
Cloud (AWS/Azure/GCP)Environment variables for inventory sources and cloud modules
CustomYour own fields, injected as env vars, extra vars or a file
Credential plugins go one step further — AWX fetches the secret from HashiCorp Vault, CyberArk, Azure Key Vault or AWS Secrets Manager at job launch, so it is not stored in AWX at all. That is the Module 07 client-script idea, provided as a first-class feature.

B3 · RBAC

RoleCan
AdminEverything on that object, including deleting it
Execute⭐ Launch a job template — without seeing its credentials
UseAttach this inventory or credential to a template you own
ReadView the object and its job history
ApproveApprove a paused workflow node
The canonical AWX design question is: "let the support desk restart a service in production, without giving them production access."

The answer is a job template that runs a narrowly scoped playbook, with a survey offering only the permitted service names as a multiple-choice field, a limit restricting the hosts, and the support team granted Execute on that one template and nothing else. They can run it; they cannot read the credentials, edit the playbook, change the inventory, or run anything else. Every launch is attributed and logged.

That answer demonstrates RBAC, surveys, least privilege and auditability in four sentences.

🧪 Exercise B3.1 — Run AWX locally and build a job template
bash
# AWX on kind - the standard local evaluation path
kind create cluster
kubectl apply -k "github.com/ansible/awx-operator/config/default?ref=2.19.1"
kubectl -n awx get pods -w

# Retrieve the generated admin password
kubectl -n awx get secret awx-demo-admin-password \
  -o jsonpath="{.data.password}" | base64 --decode; echo

kubectl -n awx port-forward svc/awx-demo-service 8080:80

Then in the UI, in this order:

plain text
1. Credentials  -> add a Machine credential with your SSH key
2. Projects     -> add a Git project pointing at your playbook repo
3. Inventories  -> add an inventory, then an inventory SOURCE (aws_ec2 or a file)
4. Templates    -> add a Job Template: project + playbook + inventory + credential
5. Launch it
Expected result — click to reveal
plain text
NAME                        READY   STATUS    RESTARTS
awx-demo-postgres-13-0      1/1     Running   0
awx-demo-task-7c9f...       4/4     Running   0
awx-demo-web-6d8b...        3/3     Running   0

After launching, the job output pane shows exactly the same text ansible-playbook produces:

plain text
PLAY [Configure web tier] ******************************************
TASK [Gathering Facts] *********************************************
ok: [web01]
TASK [Ensure nginx is installed] ***********************************
changed: [web01]
PLAY RECAP *********************************************************
web01 : ok=4  changed=2  unreachable=0  failed=0

That identical output is the point worth internalising. AWX runs the same ansible-playbook, in an Execution Environment container, against the same inventory. It adds no capability to the run itself.

What it added is around the run: the job has a number, an operator, a timestamp, the exact project commit SHA, the inventory used, and searchable output — all retained. Open the job's Details tab and you can see precisely which revision of the repository ran, which answers "what was deployed on Tuesday?" definitively.

Note the three pods. web serves the UI and API, task runs the dispatcher and job scheduler, postgres stores everything. Jobs execute in short-lived EE containers spawned by the task pod — which is why an Execution Environment is a prerequisite rather than an optional extra.

🎯 Interview questions — AWX concepts

Q. What does AWX give you that ansible-playbook does not?

Nothing about the run itself — it executes the same engine. It adds everything around the run: RBAC over who may launch what against which inventory, a credential store where secrets are write-only and never displayed, an audit trail of every job with its operator and the exact commit SHA, scheduling with history, job queueing and concurrency control, surveys giving non-YAML users a web form, and Execution Environments for a consistent runtime.

You adopt it when the constraint shifts from "can we automate this" to "who is allowed to, and can we prove what happened".

Q. AWX, Tower, Automation Controller, AAP — how do they relate?

One lineage, three names. Ansible Tower was the commercial web UI from 2013. In 2017 Red Hat open-sourced its codebase as AWX, which became the free upstream. In 2021, with AAP 2.0, Tower was renamed automation controller — a rename, not a new product. Ansible Automation Platform is the whole suite: controller plus Private Automation Hub, Execution Environments and Event-Driven Ansible.

The analogy Red Hat uses: AWX is to automation controller what Fedora is to RHEL — same lineage, upstream moves fast and is unsupported, downstream is stabilised and subscription-supported.

Worth adding: AWX has no Red Hat production support, so "we run AWX in production" and "we run automation controller" are different risk statements.

Q. An interviewer asks about "Ansible Tower". How do you handle the naming?

Answer the question, and fold the current naming in naturally rather than correcting them:

"Yes — Tower is what's now called automation controller since AAP 2.0; I've used AWX, which is the same codebase upstream. In it I'd solve that with a job template plus a survey and Execute-only RBAC..."

That answers them, shows you know the current terminology, and signals hands-on exposure. Correcting their vocabulary instead of answering achieves the opposite — and "Tower" remains the name in most job adverts, so it is not really wrong.

Q. Let the support desk restart a production service without giving them production access. How?

A job template running a narrowly scoped playbook, with a survey offering only the permitted service names as a multiple-choice field and a limit restricting the hosts. Grant the support team Execute on that one template only.

They can launch it; they cannot view the credentials, edit the playbook, change the inventory, or run anything else. Every launch is attributed, timestamped and logged.

That is least privilege, RBAC, surveys and auditability in one design.

Q. How are credentials protected in AWX?

They are encrypted at rest and write-only from the UI — once saved, a secret can be used by a job but never displayed again, not even to an administrator.

That is what allows execute rights without read rights on the underlying secret, which is the security property laptops and CI variables cannot provide.

Credential plugins go further, fetching from HashiCorp Vault, CyberArk or a cloud secret store at launch time so the secret is never stored in AWX at all.


Part C · Workflows, surveys and scheduling

C1 · Workflow templates

The analogy. Think of a document that needs a manager's signature. The form travels from desk to desk, and at one particular desk it stops — and stays stopped, for as long as it takes, until a named person signs it. The signature is recorded: who signed, and when. Very often that record is the entire reason the step exists.

An approval node is that desk. It pauses a workflow indefinitely until someone with the right permission acts, and it logs their identity — which puts change control inside the automation rather than beside it in a ticketing system that nobody can prove was followed.

Diagram source
flowchart LR
    A["Build artifact"] -->|"on success"| B["Deploy to staging"]
    A -->|"on failure"| F["Notify #alerts"]
    B -->|"on success"| C["Run smoke tests"]
    B -->|"on failure"| R["Rollback staging"]
    C -->|"on success"| D["APPROVAL NODE<br>a human must click"]
    D -->|"approved"| E["Deploy to production"]
    E -->|"always"| G["Update the CMDB"]
    style D fill:#FEF3C7,stroke:#D97706,stroke-width:2px
    style E fill:#DDD6FE,stroke:#7C3AED,stroke-width:2px
The approval node is the feature worth naming. A workflow can pause indefinitely until someone with the Approve role clicks, and the approval is recorded with their identity and timestamp.

That is a change-management gate inside the automation rather than beside it in a ticketing system — and it is a strong answer to "how do you handle change control for automated production changes?"

Workflow featurePurpose
on-success / on-failure / alwaysBranch on the outcome of the previous node
Approval nodePause until a human with Approve permission acts
Convergence nodeWait for several parallel branches before continuing
set_stats artifacts⭐ Pass data from one job to the next in the workflow
Nested workflowsA workflow node can itself be a workflow
yaml
# Passing data between workflow nodes - the mechanism people miss
- name: Record the built version for later nodes
  ansible.builtin.set_stats:
    data:
      built_version: "{{ app_version }}"
      artifact_url: "{{ build_result.url }}"
set_stats is how workflow nodes communicate. Values set this way become extra vars in every subsequent node of the workflow. Without it, each job template is isolated and you end up passing state through a file or an external store — which is the awkward workaround people reach for when they do not know set_stats exists.

C2 · Surveys

The analogy. Think of a form with a dropdown versus a form with a blank box. A dropdown offering three choices can only ever produce one of three answers. A blank box can produce anything at all, including things the person who designed the form never imagined. That is harmless on a suggestions card and genuinely dangerous on an instruction the building acts upon.

Survey answers arrive as extra vars — which, from Module 02, is the very top of the precedence list and beats everything in the playbook. So a free-text survey field is effectively a way for the user to set any variable to any value. That is why multiple-choice should be your default, and why an assert at the top of the playbook is the right guard on the fields that must stay free-text.

plain text
Survey field types:  text · textarea · password · integer · float ·
                     multiple choice (single) · multiple choice (multiple)

Each field maps to a variable name, and answers arrive as EXTRA VARS
— precedence level 22, so they beat everything (Module 02 Part C1).
Survey answers are extra vars, which means they win over everything in your playbook. A free-text survey field is therefore a way for a user to override any variable your playbook defines.

Use multiple-choice wherever the set of valid answers is knowable, mark secret fields as password type so they are masked in the job output, and validate anything free-text with an assert at the top of the playbook — the Module 03 Part C1 pattern. A survey is an input boundary and should be treated like one.

C3 · Scheduling and notifications

plain text
Schedules   — cron-like, with timezone handling, next-run visibility,
              and full job history for every firing
Notifications — Slack, email, webhook, PagerDuty; attach to a template
              on start / success / failure
Why this beats cron on a control node: a cron job that fails silently is invisible until someone notices the consequence. An AWX schedule that fails produces a notification, a red job in the dashboard, retained output, and a history you can point at. Same playbook, entirely different operational posture.

Part D · Automating AWX itself

D1 · Configuration as code

The analogy. Think of a house extension built without plans. It works. People live in it happily for years. Then it burns down and you discover nobody can rebuild it — because it never existed as a design at all, only as a series of decisions somebody made on site and never wrote down. And the photographs are not plans; they show you what it looked like, not how it was put together.

An AWX that has been configured by clicking for two years is that extension. Managing it with the awx.awx collection from Git is having the plans: reproducible in a new cluster, reviewable as a diff before it changes, and genuinely rebuildable after a disaster.

Clicking through a UI is not reproducible. The awx.awx collection manages AWX with Ansible.

yaml
# awx-config.yml
---
- name: Configure AWX
  hosts: localhost
  gather_facts: false
  vars:
    controller_host: "https://awx.internal"
    controller_username: admin
    controller_password: "{{ vault_awx_password }}"

  tasks:
    - name: Ensure the project exists
      awx.awx.project:
        name: Platform Automation
        organization: Engineering
        scm_type: git
        scm_url: https://git.internal/ansible/platform.git
        scm_branch: main
        scm_update_on_launch: true
        scm_clean: true
        controller_host: "{{ controller_host }}"
        controller_username: "{{ controller_username }}"
        controller_password: "{{ controller_password }}"

    - name: Ensure the job template exists
      awx.awx.job_template:
        name: Deploy Web Tier
        job_type: run
        organization: Engineering
        project: Platform Automation
        playbook: playbooks/site.yml
        inventory: Production
        credentials:
          - Production SSH
          - Production Vault
        execution_environment: platform-ee-1.4.2
        ask_limit_on_launch: true
        become_enabled: true
        survey_enabled: true
        survey_spec:
          name: Deployment options
          description: Choose what to deploy
          spec:
            - question_name: Application version
              variable: app_version
              type: text
              required: true
            - question_name: Target environment
              variable: target_env
              type: multiplechoice
              choices: ["staging", "production"]
              required: true
        controller_host: "{{ controller_host }}"
        controller_username: "{{ controller_username }}"
        controller_password: "{{ controller_password }}"
This is the answer that separates a user of AWX from an engineer of it. "AWX configuration is itself code — projects, inventories, credentials, job templates, surveys and RBAC are all managed with the awx.awx collection from a Git repository, applied by a pipeline. So the platform is reproducible, reviewable in a diff, and restorable after a disaster — rather than a hand-clicked state nobody can rebuild."

The alternative — a UI configured by hand over two years — is a single point of failure that no backup fully captures.

bash
ansible-galaxy collection install awx.awx
export CONTROLLER_HOST=https://awx.internal
export CONTROLLER_USERNAME=admin
export CONTROLLER_PASSWORD=...            # or CONTROLLER_OAUTH_TOKEN
ansible-playbook awx-config.yml

D2 · The API and CLI

bash
pip install awxkit

awx login                                        # obtain a token
awx job_templates list                           # ⭐
awx job_templates launch "Deploy Web Tier" --monitor      # ⭐ launch and follow
awx job_templates launch "Deploy Web Tier" \
    --extra_vars '{"app_version":"2.4.1"}' --monitor
awx jobs list --status failed                    # ⭐
awx jobs stdout 12345                            # ⭐ retrieve a job's output
awx export --organization Engineering > org.json # ⭐ backup as JSON
awx import < org.json

# Or straight to the REST API
curl -sH "Authorization: Bearer $TOKEN" \
  https://awx.internal/api/v2/job_templates/ | jq '.results[].name'
Everything in the UI is the API. AWX's UI is a client of its own REST API, so anything you can click you can automate — which is what makes awx.awx and awxkit possible, and what lets a pipeline launch a job template and wait for the result.
🧪 Exercise D2.1 — Launch a job from CI and gate on the result
bash
pip install awxkit
export CONTROLLER_HOST=https://awx.internal
export CONTROLLER_OAUTH_TOKEN="$AWX_TOKEN"

awx job_templates launch "Deploy Web Tier" \
  --extra_vars '{"app_version": "2.4.1"}' \
  --monitor \
  --filter status,id
echo "exit code: $?"

awx jobs list --status failed --order_by '-finished' --count 5
Expected result — click to reveal
plain text
------Starting Standard Out Stream------

PLAY [Configure web tier] ******************************************
TASK [Gathering Facts] *********************************************
ok: [web01]
TASK [Ensure nginx is installed] ***********************************
changed: [web01]

PLAY RECAP *********************************************************
web01 : ok=4  changed=2  unreachable=0  failed=0

------End of Standard Out Stream--------

status: successful
id: 4821
exit code: 0

And on a failure:

plain text
status: failed
id: 4822
exit code: 1

--monitor is what makes this usable from CI. Without it, launch returns immediately with a job ID and exit code 0 — so your pipeline reports success while the deployment is still running, and never learns whether it worked. With --monitor, awxkit streams the output and exits non-zero if the job fails, so the CI job fails correctly.

This is the pattern for combining CI and AWX rather than choosing between them. CI does what it is good at — lint, Molecule, build, gate on a merge. AWX does what it is good at — hold the credentials, enforce RBAC, run the change, keep the audit trail. The pipeline triggers AWX rather than holding production SSH keys itself, which means a compromised CI runner cannot reach production directly.

Use an OAuth token scoped to that one template, not admin credentials, so the pipeline can launch exactly one thing.

🎯 Interview questions — Workflows and automation

Q. What is a workflow template and when do you need one?

A graph of job templates connected by on-success, on-failure and always branches — so a single launch can build, deploy to staging, smoke-test, pause for approval, deploy to production, and always update the CMDB.

You need one when a change is several jobs with conditional paths, or when it requires a human gate. The approval node pauses indefinitely until someone with Approve permission acts, and records who approved and when — change management inside the automation rather than beside it.

Q. How do you pass data between jobs in a workflow?

ansible.builtin.set_stats in the earlier job. Values set that way become extra vars in every subsequent node of the workflow.

Without it each job template is isolated and people resort to writing a file or using an external store, which is the workaround that signals someone has not met set_stats.

Q. What is the risk with surveys?

Survey answers arrive as extra vars — precedence level 22 — so they override everything in the playbook. A free-text field is effectively a way for the user to set any variable.

Mitigate with multiple-choice fields wherever the valid set is knowable, password-type fields for secrets so they are masked in output, and an assert block validating free-text input at the top of the playbook. A survey is an input boundary and deserves input validation.

Q. How do you manage AWX configuration itself?

As code, with the awx.awx collection — projects, inventories, credentials, job templates, surveys, schedules and RBAC all declared in a Git repository and applied by a pipeline.

That makes the platform reproducible, reviewable in a diff and restorable after a disaster. A UI hand-configured over two years is a single point of failure no backup fully captures.

awxkit and the REST API cover the operational side — launching, monitoring, exporting.

Q. AWX or CI/CD — which should run your playbooks?

Both, for different jobs. CI does lint, Molecule, build and merge gating. AWX holds the credentials, enforces RBAC, runs the change and keeps the audit trail.

The pipeline triggers AWX with awx job_templates launch --monitor, using a token scoped to that one template — so --monitor makes the CI job fail when the deployment fails, and a compromised CI runner still cannot reach production directly because it never holds the SSH keys.


Part E · Putting it together

E1 · Production practice

HabitWhy
Manage AWX with awx.awx from Git, not by clickingA hand-configured UI is unreproducible and unrestorable
Grant Execute on specific templates, never Admin broadlyExecute without read on credentials is the whole point of AWX
Multiple-choice survey fields wherever the valid set is knowableFree text becomes an extra var and overrides everything
assert at the top of any survey-driven playbookA survey is an input boundary; validate it
Credential plugins fetching from a real secret store at launchThe secret never lives in AWX at all
Pin the Execution Environment per job templateOtherwise a rebuilt EE silently changes what your job runs
scm_update_on_launch plus a pinned branch or tagJobs run known code; the job record captures the exact commit
Approval nodes before any production workflow nodeAttributed, timestamped change control inside the automation
Notifications on failure for every scheduleA silent cron failure is invisible until the consequence appears
CI triggers AWX; CI does not hold production keysA compromised runner cannot reach production directly
--monitor on any CI-triggered launchWithout it the pipeline reports success while the job is still running
awx export on a schedule, stored outside AWXConfiguration backup independent of the database backup

E2 · Capstone exercise

Attempt this without looking anything up. It exercises RBAC, surveys, workflows, configuration-as-code and CI integration together.

Brief. Design and declare an AWX setup that:

  1. Lets the support desk restart one of three named services in production, with no production access of any kind
  2. Ensures they cannot pass an arbitrary service name
  3. Requires a named engineer to approve anything touching production configuration
  4. Passes the built version from a build job to the deploy job automatically
  5. Is entirely reproducible from Git, not clicked
  6. Can be launched by CI such that the pipeline fails when the deployment fails
  7. Never stores the production SSH key in either CI or AWX's own database
Model answer — attempt it first, then click
yaml
# awx/support-desk.yml   -- requirements 1, 2, 5
---
- name: Declare the support-desk capability
  hosts: localhost
  gather_facts: false
  module_defaults:
    group/awx.awx.controller:
      controller_host: "{{ controller_host }}"
      controller_oauthtoken: "{{ vault_awx_token }}"

  tasks:
    - name: Job template - restart a permitted service
      awx.awx.job_template:
        name: "Support - Restart Service"
        organization: Engineering
        project: Platform Automation
        playbook: playbooks/restart-service.yml
        inventory: Production
        credentials: ["Production SSH", "Production Vault"]
        execution_environment: "platform-ee-1.4.2"
        become_enabled: true
        ask_limit_on_launch: false                 # requirement 1 - they cannot widen scope
        survey_enabled: true
        survey_spec:
          name: Which service?
          spec:
            - question_name: Service
              variable: target_service
              type: multiplechoice                 # requirement 2 - NOT free text
              choices: ["nginx", "app-api", "worker"]
              required: true

    - name: Grant the support team Execute on that template ONLY
      awx.awx.role:
        team: Support Desk
        role: execute                              # requirement 1
        job_template: "Support - Restart Service"
        state: present
yaml
# playbooks/restart-service.yml   -- requirement 2, defence in depth
---
- name: Restart a permitted service
  hosts: web
  become: true
  tasks:
    - name: Reject anything not on the allow-list
      ansible.builtin.assert:
        that:
          - target_service in ['nginx', 'app-api', 'worker']
        fail_msg: "target_service '{{ target_service }}' is not permitted"

    - name: Restart it
      ansible.builtin.service:
        name: "{{ target_service }}"
        state: restarted
yaml
# awx/deploy-workflow.yml   -- requirements 3, 4, 5
    - name: Build job records the version
      awx.awx.job_template:
        name: "Build Artifact"
        project: Platform Automation
        playbook: playbooks/build.yml
        inventory: Build Nodes
      # playbooks/build.yml ends with:
      #   - ansible.builtin.set_stats:
      #       data: {built_version: "{{ app_version }}"}    <- requirement 4

    - name: Production deploy workflow
      awx.awx.workflow_job_template:
        name: "Deploy to Production"
        organization: Engineering
        ask_variables_on_launch: true

    - name: Workflow nodes
      awx.awx.workflow_job_template_node:
        workflow_job_template: "Deploy to Production"
        unified_job_template: "{{ item.tpl }}"
        identifier: "{{ item.id }}"
        approval_node: "{{ item.approval | default(omit) }}"
      loop:
        - {id: build,   tpl: "Build Artifact"}
        - {id: approve, approval: {name: "Approve production deploy",
                                    description: "Engineer sign-off required",
                                    timeout: 86400}}          # requirement 3
        - {id: deploy,  tpl: "Deploy Web Tier"}
yaml
# Requirement 7 - the SSH key never lives in AWX's database
    - name: Credential sourced from HashiCorp Vault at launch time
      awx.awx.credential:
        name: "Production SSH"
        organization: Engineering
        credential_type: "Machine"
        inputs:
          username: deploy
        state: present
      # then attach a credential INPUT SOURCE so ssh_key_data is
      # fetched from HashiCorp Vault when the job launches
bash
# Requirement 6 - CI triggers AWX and gates on the outcome
awx job_templates launch "Deploy to Production" \
  --extra_vars '{"app_version": "'"$CI_COMMIT_TAG"'"}' \
  --monitor
# --monitor exits non-zero when the job fails, so the pipeline fails too

The seven decisions:

  1. Execute role on one template, ask_limit_on_launch: false — requirement 1. They can launch it and nothing else, and cannot widen the host scope.
  2. multiplechoice survey, not text — requirement 2. Survey answers are extra vars at precedence 22; a free-text field would let them set any variable in the playbook.
  3. The assert in the playbook as well — defence in depth. The survey constrains the UI; the assert constrains the API, which someone with the token could call directly.
  4. An approval node before the deploy node — requirement 3, with a timeout so it does not hang forever. The approval is recorded with identity and timestamp.
  5. set_stats at the end of the build playbook — requirement 4. Those values become extra vars in every later workflow node automatically.
  6. --monitor on the CI launch — requirement 6. Without it the pipeline exits 0 immediately and never learns the outcome.
  7. A credential input source pointing at HashiCorp Vault — requirement 7. AWX holds a reference, not the key; CI holds only a scoped OAuth token. Neither system stores the production SSH key.

Verify:

bash
ansible-playbook awx/support-desk.yml
awx job_templates list --name "Support - Restart Service"
awx roles list --team "Support Desk"
awx workflow_job_templates list
awx job_templates launch "Support - Restart Service" \
  --extra_vars '{"target_service":"redis"}' --monitor   # must FAIL on the assert

That last command is the proof that requirement 2 actually holds at the API layer, not just in the web form.


E3 · Command reference — everything from this module

Commands from Module 12. ⭐ marks genuinely daily-use.

Running AWX locally

bash
kind create cluster
kubectl apply -k "github.com/ansible/awx-operator/config/default?ref=2.19.1"
kubectl -n awx get pods -w                                        # ⭐ watch it come up
kubectl -n awx get secret awx-demo-admin-password \
  -o jsonpath="{.data.password}" | base64 --decode; echo          # ⭐ admin password
kubectl -n awx port-forward svc/awx-demo-service 8080:80          # ⭐
kubectl -n awx logs -f deployment/awx-demo-task                   # dispatcher logs

awxkit CLI

bash
pip install awxkit
export CONTROLLER_HOST=https://awx.internal
export CONTROLLER_OAUTH_TOKEN="$AWX_TOKEN"                        # ⭐ scoped token

awx login                                                          # obtain a token
awx job_templates list                                             # ⭐
awx job_templates launch "Deploy Web Tier" --monitor               # ⭐ CI-safe
awx job_templates launch "Deploy Web Tier" \
    --extra_vars '{"app_version":"2.4.1"}' --monitor               # ⭐
awx jobs list --status failed --order_by '-finished' --count 5     # ⭐
awx jobs stdout 4821                                               # ⭐ retrieve output
awx jobs monitor 4821
awx workflow_job_templates list
awx projects update "Platform Automation" --monitor                # ⭐ force an SCM pull
awx inventory_sources update "AWS Production" --monitor            # ⭐ refresh inventory
awx export --organization Engineering > org.json                   # ⭐ config backup
awx import < org.json
awx roles list --team "Support Desk"                               # ⭐ who can do what

The REST API directly

bash
curl -sH "Authorization: Bearer $TOKEN" \
  https://awx.internal/api/v2/job_templates/ | jq '.results[].name'

curl -sX POST -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
  -d '{"extra_vars":{"app_version":"2.4.1"}}' \
  https://awx.internal/api/v2/job_templates/12/launch/ | jq .id

curl -sH "Authorization: Bearer $TOKEN" \
  https://awx.internal/api/v2/jobs/4821/stdout/?format=txt

Configuration as code — the awx.awx modules

yaml
awx.awx.organization         # ⭐ the tenancy boundary
awx.awx.project              # ⭐ a Git repo + branch
awx.awx.inventory            # ⭐
awx.awx.inventory_source     # ⭐ an aws_ec2 or other plugin source
awx.awx.host / awx.awx.group
awx.awx.credential           # ⭐
awx.awx.credential_type      # custom credential shapes
awx.awx.job_template         # ⭐ THE runnable unit, incl. survey_spec
awx.awx.workflow_job_template
awx.awx.workflow_job_template_node   # ⭐ incl. approval nodes
awx.awx.schedule             # ⭐
awx.awx.notification_template
awx.awx.user / awx.awx.team
awx.awx.role                 # ⭐ RBAC grants
awx.awx.execution_environment
awx.awx.export / awx.awx.import      # ⭐ bulk config in and out
yaml
# Passing data between workflow nodes
- ansible.builtin.set_stats:                 # ⭐ becomes extra vars downstream
    data:
      built_version: "{{ app_version }}"
The CI-to-AWX handoff, in one line worth memorising:
bash
awx job_templates launch "<template>" --extra_vars '{...}' --monitor

--monitor streams the output and exits non-zero on failure, which is what makes the pipeline fail correctly. Without it you get exit 0 and a green build for a deployment that is still running — or has already failed.


E4 · Official documentation

LinkCovers
AWX documentationInstallation, concepts, administration
AWX Operator installationDeploying AWX on Kubernetes
awx.awx collectionEvery module for managing AWX as code
Red Hat Automation Platform docsThe supported product — Controller, Hub, EDA
ansible.builtin.set_statsPassing data between workflow nodes
awxkit / AWX CLIThe command-line client
Execution EnvironmentsBuilding the images AWX jobs run inside
Event-Driven AnsibleThe AAP component that triggers automation from events

E5 · Self-assessment

1. What does AWX add, and what does it not?

It adds nothing to the run itself — it executes the same ansible-playbook. It adds RBAC, a write-only credential store, an audit trail with the exact commit SHA, scheduling with history, queueing, surveys, and Execution Environments.

You adopt it when the constraint becomes "who is allowed to run this, and can we prove what happened".

2. AWX, Tower, Automation Controller, AAP?

AWX is the free upstream project. Ansible Tower was the supported product, renamed Automation Controller. AAP is the full suite — Controller plus Private Automation Hub, Execution Environments and Event-Driven Ansible.

3. What is the core security property of AWX credentials?

They are write-only — usable by a job but never displayable again, not even to an admin. That enables execute rights without read rights on the secret, which laptops and CI variables cannot provide.

Credential plugins go further, fetching from HashiCorp Vault or a cloud store at launch so the secret is never in AWX at all.

4. Design: support desk restarts a production service with no production access.

A narrowly scoped job template, a multiple-choice survey listing only permitted services, ask_limit_on_launch: false, and the team granted Execute on that template alone. Plus an assert in the playbook so the constraint also holds at the API layer.

They launch it; they cannot see credentials, edit the playbook, change the inventory or run anything else — and every launch is attributed.

5. Why are surveys a security consideration?

Answers arrive as extra vars at precedence 22, overriding everything in the playbook. A free-text field lets the user set any variable.

Use multiple-choice wherever possible, password type for secrets so they are masked, and validate free text with assert at the top of the playbook.

6. How do you pass data between workflow nodes?

set_stats in the earlier job — those values become extra vars in every subsequent node. Without it each template is isolated and people resort to files or external stores.

7. What is an approval node and why does it matter?

A workflow node that pauses indefinitely until someone with the Approve role acts, recording their identity and timestamp.

It puts change control inside the automation rather than beside it in a ticket, and it is the direct answer to "how do you handle change management for automated production changes?"

8. How should AWX itself be configured?

As code, with the awx.awx collection from a Git repository applied by a pipeline — projects, inventories, credentials, templates, surveys, schedules and RBAC.

A UI hand-clicked over two years is unreproducible and unrestorable. Add scheduled awx export for a config backup independent of the database.

9. Should CI or AWX run your playbooks?

Both, for different jobs. CI lints, tests with Molecule, builds and gates merges. AWX holds credentials, enforces RBAC, runs the change and keeps the audit trail.

CI triggers AWX with a scoped token, so a compromised runner never holds production SSH keys.

10. Why is --monitor essential on a CI-triggered launch?

Without it, awx job_templates launch returns immediately with exit code 0 — the pipeline goes green while the deployment is still running, and never learns whether it succeeded.

--monitor streams the output and exits non-zero when the job fails.


Next — Module 13 · Troubleshooting & Debugging.

A systematic method for diagnosing failures — reading errors correctly, the verbosity ladder, the debugger, and the failure patterns that recur.

📚 Sources for the interview questions

Behaviour verified against the current AWX documentation and the awx.awx collection reference.

Question selection cross-referenced against publicly published 2026 Ansible interview question sets:

Answers were rewritten and deepened rather than reproduced — published versions are usually correct but shallow, and the added operational detail is what differentiates a candidate in the room.

Spotted a mistake or want something added? Send me a note.