delegate_to and run_once

Scenario: After every deploy the load balancer has no backends, but each web server has grown its own private copy of backends.conf. The release announcement is also posted once per web server instead of once per release.

New words, in plain English

• delegate_to - run this task on a DIFFERENT host, while keeping the current host's variables.
• run_once - run this task only once for the whole play, not once per host.
• throttle - limit how many hosts run this particular task concurrently.
• ansible_play_hosts - the list of hosts still active in the current play.

Some work belongs to a host other than the one being configured. Registering a web server with a load balancer is a change to the load balancer, driven by facts about the web server.

delegate_to: lb01 runs the task on lb01 while inventory_hostname and all the current host's variables stay as they are. That is the crucial property: the task executes elsewhere but still knows which host it is acting on behalf of. Without it, the task writes to each web server's own filesystem - which is exactly the scenario.

run_once: true makes a task execute a single time for the whole play rather than once per host. Correct for anything that is a property of the release rather than of a host: an announcement, a database migration, a cache purge.

The two combine, and the combination is the standard orchestration idiom: delegate_to: lb01 plus run_once: true performs one action, on the load balancer, for the entire rollout.

There is a concurrency trap worth naming. Several hosts delegating a write to the same file on the same target will race, and lines get lost. Either serialise with throttle: 1, or - better - do it once with run_once and build the whole list from ansible_play_hosts.

Analogy: delegate_to is phoning the front desk to say that room 302 is ready. The message is about room 302, but the work happens at the desk. Sending every room to write in the ledger simultaneously is how entries get overwritten.

A worked example

# WRONG - writes to each web server's own filesystem
- ansible.builtin.lineinfile:
    path: /etc/lb/backends.conf
    line: "{{ inventory_hostname }}:8081"

# RIGHT - one task, on the load balancer, for the whole play.
# Building the full list avoids any concurrent-write race.
- name: Publish the backend pool to the load balancer
  ansible.builtin.copy:
    dest: /etc/lb/backends.conf
    content: |
      {% for h in ansible_play_hosts %}
      {{ h }}:{{ hostvars[h].webapp_port | default(8081) }}
      {% endfor %}
  delegate_to: lb01
  run_once: true

# Once per RELEASE, not once per host
- name: Announce the release
  ansible.builtin.lineinfile:
    path: /var/log/announce.log
    line: "release {{ app_release }} deployed"
    create: true
  delegate_to: lb01
  run_once: true

# If you must have each host act on the shared target, serialise it
- ansible.builtin.lineinfile:
    path: /etc/lb/backends.conf
    line: "{{ inventory_hostname }}:8081"
  delegate_to: lb01
  throttle: 1

delegate_facts: true stores any facts gathered by a delegated task against the delegate rather than the current host - useful when you delegate a setup task to discover something about a machine outside the play.

A point that catches people: run_once picks the first host in the current batch. Under serial, that means once per batch, not once per play. If a task must genuinely happen once for the entire rollout, put it in a separate play targeting a single host, or in pre_tasks before the batched play begins.

delegate_to: localhost is the common special case: calling an API, sending a notification, or writing a report on the control node itself. local_action is the older shorthand for the same thing.

Warning: Under serial, run_once executes once per BATCH, not once per play. A migration that must run exactly once needs its own unbatched play.

Goal: Put this to work in the ansible-delegate-run-once lab. Open /labs/ansible, pick ansible-delegate-run-once, and fix the real broken project - Ansible really does SSH into four managed hosts and converge them.