What this usually means
Ansible decides a task is 'changed' when it detects a difference between the current state and desired state. For modules like 'template' or 'copy', the comparison is often a checksum or content diff. If the task always reports 'changed', either the detection mechanism is flawed (e.g., the module doesn't support idempotency natively) or the actual state on the host is being modified in a way Ansible doesn't recognize as already correct. Common root causes include: using 'shell' or 'command' without 'creates'/'removes' or proper 'changed_when' logic; template files that differ in trailing newlines or whitespace; modules that rely on external diff tools with inconsistent output; or a race condition where another process alters the state between check and apply.
The first ten minutes — establish facts before touching code.
- 1Run ansible-playbook with -v (verbose) to see the diff output for the failing task — look for unexpected differences like trailing newlines or whitespace.
- 2Check if the task uses 'shell' or 'command' module: these always report 'changed' unless you set 'creates', 'removes', or a 'changed_when' condition.
- 3For 'template' tasks, manually compare the rendered template on the control node vs. the file on the target host using 'diff' or 'sha256sum'.
- 4Add a debug task to print the value of the variable that drives the task's 'dest' or 'content' — Jinja2 filters like 'to_nice_yaml' can introduce newlines.
- 5Enable 'ANSIBLE_DIFF_ALWAYS=1' in environment and run with '-v' to see what Ansible thinks changed — sometimes the diff is empty but module still reports change.
The specific files, logs, configs, and dashboards that usually own this bug.
- searchThe specific task definition in the playbook or role (especially 'shell', 'command', 'template', 'copy')
- searchThe Jinja2 template file — check for trailing newlines, extra spaces, or filters that produce non-deterministic output
- searchThe target file on the host (cat, sha256sum) vs. the source (control node copy or template)
- searchAnsible log files under ~/.ansible/log/ or the configured log_path
- searchThe 'vars' dict or host_vars/group_vars for the host — any variable that influences the task content
- searchThe 'check_mode' output — if check says 'ok' but run says 'changed', suspect a check_mode bug in the module
Practical causes, not theory. These are the things you will actually find.
- warningUsing shell/command module without 'creates' or 'changed_when' — these modules always report changed unless you tell Ansible otherwise
- warningJinja2 template rendering differences: trailing newline, carriage return, or whitespace differences between template file and on-disk file
- warningModule bug or limitation: some modules (like 'lineinfile' with backrefs) can have edge cases where they always report change
- warningFile permissions or ownership changes not detected correctly: Ansible compares content but not mode/owner unless specified
- warningUse of 'to_nice_yaml' or 'to_json' filters that add trailing newlines or sort keys differently each run
- warningAnother process (cron, monitoring) modifying the file between Ansible's check and apply phases
Concrete fix directions. Pick the one that matches your root cause.
- buildFor shell/command: add 'creates=<path>' to skip if file exists, or write a custom 'changed_when' condition based on actual outcome (e.g., grep for expected line)
- buildFor template: add 'newline_sequence: unix' and 'trim_blocks: true' in the task to normalize whitespace
- buildExplicitly set 'mode', 'owner', 'group' on file tasks to avoid change from default permissions
- buildUse 'ansible.builtin.copy' with 'checksum' or 'force: no' when content is stable
- buildFor idempotent shell: redirect output to a marker file and use 'creates' or 'changed_when: false' and rely on other tasks to detect drift
- buildUpgrade the Ansible version or module if a known bug exists (check Ansible changelog)
A fix you cannot prove is a guess. Close the loop.
- verifiedRun the playbook twice in a row with '--diff' and confirm the second run shows no 'changed' tasks
- verifiedManually execute the equivalent command on the host and verify the file content is exactly what the template would produce
- verifiedAdd a 'stat' task after the suspect task to compare checksum of the file with a known-good checksum
- verifiedRun in check mode and verify 'ok' for every task; if check says 'ok' but apply says 'changed', that's a module issue
- verifiedWrite a simple test playbook that calls the module in a loop and asserts no 'changed' after the first run
Things that make this bug worse or harder to find.
- warningAdding 'changed_when: false' blindly — it masks the problem and breaks any downstream logic that relies on change detection
- warningAssuming 'template' module is always idempotent — it compares the rendered template with the file, but trailing newline differences can cause false changes
- warningUsing 'lineinfile' with 'regexp' that matches the line you're inserting, causing the module to always think it needs to insert
- warningSetting 'check_mode: no' on the task — it bypasses the idempotency check entirely and can cause other issues
- warningIgnoring the diff output: if the diff is empty but the task reports changed, it's a module bug; upgrade or use a workaround
The Case of the Always-Changed Nginx Config
Timeline
- 09:15Deploy new nginx config via Ansible playbook; all tasks show 'ok' except one: 'template nginx.conf' shows 'changed'.
- 09:17Re-run playbook; same task still 'changed'. Check diff: shows one line added at end of file.
- 09:20SSH to host, cat the file: no extra line. Diff shows a trailing newline in the template that's not on host.
- 09:25Check template file: it ends with a newline. The target file was manually edited earlier and lost the trailing newline.
- 09:30Fix: add 'newline_sequence: unix' and 'trim_blocks: true' to the template task, and re-run.
- 09:32First run after fix: still 'changed' (expected, because it adds the newline). Second run: 'ok'.
- 09:35Document the fix in the runbook and add a pre-check to ensure trailing newline consistency.
We had an Ansible playbook that deployed nginx configs across 50 servers. One specific template task always reported 'changed' on every run, even though the file content on disk looked identical to the template. The diff output showed a single line: a trailing newline. On the host, the file lacked a trailing newline because someone had manually edited it weeks ago and saved without the newline. Ansible's template module compares the rendered template (which ends with a newline) to the file (which doesn't), so it always detects a change.
I initially tried 'changed_when: false' to silence it, but then the monitoring alerts for config drift stopped working. That's when I dug into the diff. The fix was straightforward: add 'newline_sequence: unix' and 'trim_blocks: true' to the task to normalize line endings. After applying the fix, the first run reported 'changed' (as it fixed the newline), but subsequent runs were 'ok'.
The lesson: never assume template idempotency is automatic. Always check the diff, especially for whitespace differences. And never use 'changed_when: false' as a band-aid — it hides the real problem and breaks downstream dependencies.
Root cause
Trailing newline mismatch: the template file ended with a newline, but the target file on the host did not, causing Ansible to detect a difference every time.
The fix
Added 'newline_sequence: unix' and 'trim_blocks: true' to the template task to enforce consistent line endings.
The lesson
Always examine the actual diff output from Ansible's verbose mode. Whitespace differences are a common cause of false idempotency failures.
Ansible modules report 'changed' when they modify the target system. For 'copy' and 'template', the module compares the source with the destination using checksums (by default). If the checksums differ, the module copies the file and reports 'changed'. If they match, it reports 'ok'.
The 'command' and 'shell' modules always report 'changed' unless you provide 'creates', 'removes', or a custom 'changed_when' expression. This is because Ansible cannot know if the command actually changed anything without explicit instructions.
When you must use 'shell' or 'command', the most robust approach is to use 'creates' if the command produces a file. For example: 'creates: /etc/nginx/nginx.conf' tells Ansible to skip the command if that file already exists.
If the command doesn't produce a file, write a wrapper that checks the state before and after. For example, a script that returns exit code 0 if no change, 1 if changed. Then use 'changed_when: result.rc == 1'.
Some modules have bugs where they report 'ok' in check mode but 'changed' in actual run. This happens when the module's check mode implementation doesn't accurately simulate the change detection logic. For example, the 'lineinfile' module with backrefs can have this issue.
Workaround: avoid relying solely on check mode. Always run the playbook without check mode on a test host to verify idempotency.
Use 'ansible-playbook -v' to see the diff for 'template' and 'copy' tasks. If the diff shows no actual content change but the task still reports 'changed', suspect metadata differences like permissions or ownership.
For deeper debugging, add a 'debug' task to print the checksum of the source and destination files. This can be done with the 'stat' module and comparing 'stat.checksum'.
Frequently asked questions
Why does my 'template' task always show 'changed' even though the files are identical?
The most common cause is a trailing newline difference. Ansible's template module compares the rendered template with the file on disk character by character. If one ends with a newline and the other doesn't, it reports a change. Check the diff in verbose mode; if the diff shows a single blank line at the end, that's the culprit. Fix by adding 'newline_sequence: unix' and 'trim_blocks: true' to the task.
How can I make a 'shell' command idempotent?
Use the 'creates' parameter if the command produces a file. For example: 'shell: /opt/install.sh creates=/opt/installed.flag'. If no file is produced, write a script that checks the state and returns a specific exit code, then use 'changed_when: result.rc == 1'. Alternatively, use the 'command' module with 'warn: no' and rely on 'changed_when: false' only if you have another way to detect changes.
What is the difference between 'changed_when: false' and 'check_mode: no'?
'changed_when: false' tells Ansible to always report 'ok' for that task, regardless of what the module does. This can mask real changes and break downstream tasks that depend on change detection. 'check_mode: no' forces the task to run even in check mode, meaning it will actually modify the system when you're only simulating. Both are workarounds, not solutions. The proper fix is to ensure idempotency at the module level.
Can Ansible ignore whitespace in template comparisons?
Not directly. The 'template' module compares the rendered content byte-for-byte. You can control the newline style with 'newline_sequence' (unix, windows, or platform), but there is no built-in option to ignore whitespace differences. You would need to pre-process the file on the host (e.g., strip trailing whitespace) before the template task, or use a custom module that does a fuzzy comparison.
Why does my 'lineinfile' task always report 'changed'?
This often happens when the 'regexp' pattern matches the line you are inserting. For example, if you use 'regexp: "^some line"' and 'line: "some line"', Ansible sees the line already exists but the regex might not match exactly due to whitespace or anchors. Also, if the file you are editing has a trailing newline and your line does not, the module might add it and report a change. Use '--diff' to see what changed.