• Zach Biles

AWX Isolated Nodes Broken after AWX Upgrade

tl;dr: Python2 strikes back! Always use Python3.

Ran into this issue after an upgrade of AWX from 14.0.0 to 15.0.1, where jobs will just fail with no output in AWX showing what happened. Just a blank box where stdout normally appears. Well great... Checked /var/log/, journalctl, and any other logging locations I could think of. The only thing I would see was a login event for awx user, then a disconnect by client (tower server). Pretty useless.

AWX_ISOLATED_NODE sshd[20060]: pam_unix(sshd:session): session opened for user awx by (uid=0)
AWX_ISOLATED_NODE sshd[20063]: Received disconnect from port 46462:11: disconnected by user
AWX_ISOLATED_NODE sshd[20063]: Disconnected from port 46462
AWX_ISOLATED_NODE sshd[20060]: pam_unix(sshd:session): session closed for user awx
AWX_ISOLATED_NODE systemd-logind[1057]: Removed session 2269.

Turns out my isolated instance node is installed on a CentOS 7 box, which by default still runs Python2. Unbeknownst to me, this would be key, as most of my other isolated nodes are CentOS 8, which is Python3 default.

Well the ansible-runner version for python2 is 1.0.1, but it seems Ansible 2.9.14/AWX 15.0.1 do not support ansible-runner 1.0.1, and it needs to be at version 1.4.6.

In order to do this, I removed python-ansible-runner through yum, which is version 1.0.1. Then installed via `pip3 install ansible-runner`. Version 1.4.6!

This installation pumped out an warning during installation that /usr/local/bin was not in $PATH.

I added /usr/local/bin to /etc/environment, rebooted and I'm now able to run jobs!

17 views0 comments

Recent Posts

See All

Variables in Ansible

This will be a short post today, feeling pretty dumb. Just learned the hard way that Ansible variable names can not start with numbers. It even says so in the first line of the documentation on variab