- Zach Biles
How to debug Ansible
There are a couple of ways to get more information about why a command/playbook is failing. These are listed more or less, in order growing more verbose. This is a pretty long post mostly due to the example output.
Debug module in playbook
The second most common method is using the debug module in your playbook to output content of a variable during a playbook run so you can verify it. This is less desirable as you have to remember to put these in your code, and it can lead to really lengthy playbook output if you have variables with a lot of output. Below is an example of how to use it though.
---
- name: Gather IOS Facts
hosts: testciscocore.test.com
gather_facts: false
connection: network_cli
vars:
ansible_network_os: ios
tasks:
- name: Gather facts
ios_facts:
gather_network_resources: all
register: ios_result
- name: debug
debug:
msg: "{{ ios_result }}"
This will output the contents of ios_result to stdout on the screen during a playbook run.
Verbose Parameter
The first, and most common way to get more information is using the -v parameter on the CLI. Note: you can add more "v"s to get more verbose output.
Example playbook (gather_facts.yml):
---
- name: Gather IOS Facts
hosts: testciscocore.biles.com
gather_facts: false
connection: network_cli
vars:
ansible_network_os: ios
tasks:
- name: Gather facts
ios_facts:
gather_network_resources: all
register: ios_result
Output with standard run of ansible-playbook gather_facts.yml
ansible@blueansible:~$ ansible-playbook gather_facts.yml -u ansible -k
SSH password:
PLAY [Gather IOS Facts] ***********************************************************************
TASK [Gather facts] ***********************************************************************
[WARNING]: default value for `gather_subset` will be changed to `min` from `!config` v2.11 onwards
ok: [testciscocore.test.com]
PLAY RECAP ***********************************************************************
testciscocore.test.com : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
Pretty basic output here, but now, if we run the same playbook with -vv we'll see a lot more info.
ansible@blueansible:~$ ansible-playbook gather_facts.yml -u ansible -k -vv
ansible-playbook 2.9.10
config file = /etc/ansible/ansible.cfg
configured module search path = ['/home/ansible/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /home/ansible/.local/lib/python3.6/site-packages/ansible
executable location = /usr/local/bin/ansible-playbook
python version = 3.6.8 (default, Apr 16 2020, 01:36:27) [GCC 8.3.1 20191121 (Red Hat 8.3.1-5)]
Using /etc/ansible/ansible.cfg as config file
SSH password:
PLAYBOOK: gather_facts.yml *****************************************************************************************************************************************************************************
1 plays in gather_facts.yml
PLAY [Gather IOS Facts] ********************************************************************************************************************************************************************************
META: ran handlers
TASK [Gather facts] ************************************************************************************************************************************************************************************
task path: /home/ansible/gather_facts.yml:10
[WARNING]: default value for `gather_subset` will be changed to `min` from `!config` v2.11 onwards
ok: [testciscocore.test.com] => changed=false
ansible_facts:
ansible_net_all_ipv4_addresses:
- 192.168.3.2
- 192.168.7.2
- 192.168.3.2
- 192.168.252.2
<lines omited for brevity>
META: ran handlers
META: ran handlers
PLAY RECAP *********************************************************************************************************************************************************************************************
testciscocore.test.com : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
With -vv you'll see variable contents output without having to use a debug task in the playbook. You will also see that it prints out what Ansible is using for host, and inventory files, as well as python version. This can be very useful when you run into issues and need to make sure Ansible is using the right host files or inventory files. With -vvv and -vvvv you'll see even more output about what's being done in the background.
ansible@blueansible:~$ ansible-playbook gather_facts.yml -u ansible -k -vvvv
ansible-playbook 2.9.10
config file = /etc/ansible/ansible.cfg
configured module search path = ['/home/ansible/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /home/ansible/.local/lib/python3.6/site-packages/ansible
executable location = /usr/local/bin/ansible-playbook
python version = 3.6.8 (default, Apr 16 2020, 01:36:27) [GCC 8.3.1 20191121 (Red Hat 8.3.1-5)]
Using /etc/ansible/ansible.cfg as config file
SSH password:
setting up inventory plugins
host_list declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
script declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
auto declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
Parsed /etc/ansible/hosts inventory source with ini plugin
Loading callback plugin yaml of type stdout, v2.0 from /home/ansible/.local/lib/python3.6/site-packages/ansible/plugins/callback/yaml.py
PLAYBOOK: gather_facts.yml *****************************************************************************************************************************************************************************
Positional arguments: gather_facts.yml
verbosity: 4
ask_pass: True
private_key_file: /home/ansible/.ssh/ansible_rsa
remote_user: ansible
connection: smart
timeout: 10
become_method: sudo
tags: ('all',)
inventory: ('/etc/ansible/hosts',)
forks: 5
1 plays in gather_facts.yml
PLAY [Gather IOS Facts] ********************************************************************************************************************************************************************************
META: ran handlers
TASK [Gather facts] ************************************************************************************************************************************************************************************
task path: /home/ansible/gather_facts.yml:10
<testciscocore.test.com> attempting to start connection
<testciscocore.test.com> using connection plugin network_cli
<testciscocore.test.com> local domain socket does not exist, starting it
<testciscocore.test.com> control socket path is /home/ansible/.ansible/pc/1890761c07
<testciscocore.test.com> local domain socket listeners started successfully
<testciscocore.test.com> loaded cliconf plugin ios from path /home/ansible/.local/lib/python3.6/site-packages/ansible/plugins/cliconf/ios.py for network_os ios
<testciscocore.test.com>
<testciscocore.test.com> local domain socket path is /home/ansible/.ansible/pc/1890761c07
<testciscocore.test.com> ESTABLISH LOCAL CONNECTION FOR USER: ansible
<testciscocore.test.com> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/ansible/.ansible/tmp/ansible-local-10573803snxlu6a `"&& mkdir /home/ansible/.ansible/tmp/ansible-local-10573803snxlu6a/ansible-tmp-1593572389.8371599-1057499-117182036127322 && echo ansible-tmp-1593572389.8371599-1057499-117182036127322="` echo /home/ansible/.ansible/tmp/ansible-local-10573803snxlu6a/ansible-tmp-1593572389.8371599-1057499-117182036127322 `" ) && sleep 0'
<testciscocore.test.com> Attempting python interpreter discovery
<testciscocore.test.com> EXEC /bin/sh -c 'echo PLATFORM; uname; echo FOUND; command -v '"'"'/usr/bin/python'"'"'; command -v '"'"'python3.7'"'"'; command -v '"'"'python3.6'"'"'; command -v '"'"'python3.5'"'"'; command -v '"'"'python2.7'"'"'; command -v '"'"'python2.6'"'"'; command -v '"'"'/usr/libexec/platform-python'"'"'; command -v '"'"'/usr/bin/python3'"'"'; command -v '"'"'python'"'"'; echo ENDFOUND && sleep 0'
<testciscocore.test.com> EXEC /bin/sh -c '/usr/bin/python3.6 && sleep 0'
Using module file /home/ansible/.local/lib/python3.6/site-packages/ansible/modules/network/ios/ios_facts.py
<testciscocore.test.com> PUT /home/ansible/.ansible/tmp/ansible-local-10573803snxlu6a/tmpetl415e0 TO /home/ansible/.ansible/tmp/ansible-local-10573803snxlu6a/ansible-tmp-1593572389.8371599-1057499-117182036127322/AnsiballZ_ios_facts.py
<testciscocore.test.com> EXEC /bin/sh -c 'chmod u+x /home/ansible/.ansible/tmp/ansible-local-10573803snxlu6a/ansible-tmp-1593572389.8371599-1057499-117182036127322/ /home/ansible/.ansible/tmp/ansible-local-10573803snxlu6a/ansible-tmp-1593572389.8371599-1057499-117182036127322/AnsiballZ_ios_facts.py && sleep 0'
<testciscocore.test.com> EXEC /bin/sh -c '/usr/libexec/platform-python /home/ansible/.ansible/tmp/ansible-local-10573803snxlu6a/ansible-tmp-1593572389.8371599-1057499-117182036127322/AnsiballZ_ios_facts.py && sleep 0'
<testciscocore.test.com> EXEC /bin/sh -c 'rm -f -r /home/ansible/.ansible/tmp/ansible-local-10573803snxlu6a/ansible-tmp-1593572389.8371599-1057499-117182036127322/ > /dev/null 2>&1 && sleep 0'
[WARNING]: default value for `gather_subset` will be changed to `min` from `!config` v2.11 onwards
ok: [testciscocore.test.com] => changed=false
ansible_facts:
ansible_net_all_ipv4_addresses:
- 192.168.3.2
- 192.168.7.2
- 192.168.3.2
- 192.168.252.2
<lines omitted for brevity>
META: ran handlers
META: ran handlers
PLAY RECAP *********************************************************************************************************************************************************************************************
testciscocore.test.com : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
DEBUG_ANSIBLE
This method will give EXTREMELY verbose output to the screen, but is sometimes needed when really low level errors are happening. You can place ANSIBLE_DEBUG=1 in front of your CLI command to get the gritty details of what's going on with your playbook. Output for this is much to verbose to include here, but give it a try on your own with this example. It gives you just about every step happening during the run. You can also combine this with the -vvvv parameter to include both sets of debugging information.
DEBUG_ANSIBLE=1 ansible <host> -i <inventory_file> -m setup