Develop Reference

Ansible: Change playbooks location

I have all playbooks in /etc/ansible/playbooks and I want to execute them anywhere on the pc
I tried to configure playbook_dir variable in ansible.cfg
[defaults]
playbook_dir = /etc/ansible/playbooks/
and tried to put ANSIBLE_PLAYBOOK_DIR variable in ~/.bashrc
export ANSIBLE_PLAYBOOK_DIR=/etc/ansible/playbooks/
but I only got the same error in both cases:
nor#nor:~$ ansible-playbook test3.yaml
ERROR! the playbook: test3.yaml could not be found
This is my ansible version:
ansible 2.9.7
config file = /etc/ansible/ansible.cfg
configured module search path = ['/home/nor/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/local/lib/python3.7/dist-packages/ansible
executable location = /usr/local/bin/ansible
python version = 3.7.3 (default, Oct 7 2019, 12:56:13) [GCC 8.3.0]
Does anyone know the problem and how to solve it?

According to https://manpages.debian.org/testing/ansible/ansible-inventory.1.en.html :
--playbook-dir 'BASEDIR'
Since this tool does not use playbooks, use this as a subsitute playbook directory.This sets the relative path for many features including roles/ group_vars/ etc.
This means that ANSIBLE_PLAYBOOK_DIR is not used as a replacement for specifying the the absolute / relative path to your playbook, but it tells the playbook where it should look for roles, host/group vars , etc.
The goal you're trying to achieve is has no solution on the ansible side, you need to achieve this by configuring your shell profile accordingly.
set the following in your .bashrc file:
export playbooks_dir=/path/to/playbooks
when you call the playbook use ansible-playbook $playbooks_dir/test3.yml

As others have said, ANSIBLE_PLAYBOOK_DIR is for setting the relative directory for roles/, files/, etc. IMHO, it's not terribly useful.
If I understand the op, this is how I accomplish a similar result with all versions of ansible ...
PPWD=$PWD cd /my/playbook/dir && ansible-playbook my_playbook.yml; cd $PPWD
Explained,
PPWD=$PWD is to remember the current/present/previous working directory, then
cd /my/playbook/dir and if that succeeds run ansible-playbook my_playbook.yml (everything is relative from there); regardless, always change back to the previous working directory

PLAYBOOK_DIR says:
"A number of non-playbook CLIs have a --playbook-dir argument; this sets the default value for it."
Unfortunately, there is no hint in the doc what "the non-playbook CLIs" might be. ansible-playbook isn't one of them, obviously.
FWIW. If you're looking for a command-line oriented framework try ansible-runner. For example, export the location of private_data_dir
shell> export ansible_private=/path/to/<private-data-dir>
Then run the playbook
shell> ansible-runner -p playbook.yml run $ansible_private

Ansible is using wrong version of Python

I have been dealing with this issue for a couple of days now. I am running ansible on a raspberry pi. I have made Python3.7 the default version of Python. Apparently Ansible wants to use Python 2.7. I have added Version 3.7 in the vars in the playbook but this does not change the module location. It still looks for the modules in Python 2.7.
I am not sure how to tell Ansible to use 3.7 for all the modules and the interpreter.
pi#pi:~ $ python --version
Python 3.7.2
pi#pi:~ $ ansible-playbook vm.yaml -vv
ansible-playbook 2.9.6
config file = /etc/ansible/ansible.cfg
configured module search path = [u'/home/pi/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
ansible python module location = /usr/lib/python2.7/dist-packages/ansible
executable location = /usr/bin/ansible-playbook
python version = 2.7.16 (default, Oct 10 2019, 22:02:15) [GCC 8.3.0]
Using /etc/ansible/ansible.cfg as config file
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'
PLAYBOOK: vm.yaml *************************************************************************************************************************************
1 plays in vm.yaml
PLAY [localhost] **********************************************************************************************************************************************
TASK [Gathering Facts] ****************************************************************************************************************************************
task path: /home/pi/vm.yaml:12
ok: [localhost]
META: ran handlers
TASK [Gather info about the vmware guest vm] ******************************************************************************************************************
task path: /home/pi/vm.yaml:28
An exception occurred during task execution. To see the full traceback, use -vvv. The error was: ModuleNotFoundError: No module named 'pyVim'
fatal: [localhost -> localhost]: FAILED! => {"changed": false, "msg": "Failed to import the required Python library (PyVmomi) on pi's Python /usr/bin/python3. Please read module documentation and install in the appropriate location. If the required library is installed, but Ansible is using the wrong Python interpreter, please consult the documentation on ansible_python_interpreter"}
PLAY RECAP ****************************************************************************************************************************************************
localhost : ok=1 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0
pi#pi:~ $

To let ansible use the python version other than what it is taking by default, you can use ansible.cfg file inside the project root from where you are running ansible-playbook command.
This file is like a configuration file from which ansible picks up details when it executes a playbook.
Once of its configuration option is interpreter_python where you specify the executable path.
Let's do this step by step :
First find the executable path of the python version you would like ansible to use. e.g. if it is python 3.10, you would run which python3.10 from your terminal. It will give you something like /usr/local/bin/python3.10 in the output(which may change if you have it installed in a different directory).
Create a file called ansible.cfg in the project root from where you are running your playbooks and add following :
[defaults]
stdout_callback = debug
interpreter_python= /usr/local/bin/python3.10
Now when you run the playbooks from this directory, ansible will take the interpreter_python configuration and use python3.10
An important note that now if you run ansible-playbook --version it might still show you the older python version it was using. But when you execute the playbook, it will take the ansible.clf file into account.

Q: "How to tell Ansible to use 3.7"
A: It's not possible to select the version of Python the ansible* utilities are running on. This version of Python depends on how the Ansible utilities(package) were built. This is the python version the utilities will display
shell> ansible --version
ansible [core 2.14.1]
config file = /export/scratch/tmp7/test-206/ansible.cfg
configured module search path = ['/home/admin/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /home/admin/.local/lib/python3.9/site-packages/ansible
ansible collection location = /home/admin/.local/lib/python3.9/site-packages
executable location = /home/admin/.local/bin/ansible
python version = 3.9.16 (main, Dec 7 2022, 01:11:51) [GCC 9.4.0] (/usr/bin/python3.9)
jinja version = 3.1.2
libyaml = True
The utilities below will display the same
shell> ansible-playbook --version
shell> ansible-inventory --version
...
There may be more versions of Python installed both on the controller and on the remote hosts. It's a complex process of how Ansible selects which version of Python will be used to execute the modules on a remote host. See:
Interpreter Discovery: "Unless configured otherwise, Ansible will attempt to discover a suitable Python interpreter on each target host."
INTERPRETER_PYTHON: "Configuration options how to select Python on the remote hosts."
Debugging modules to learn details on how the Ansible modules are executed on a remote host.
Setting the Python interpreter in BSD
It's necessary to understand that Ansible can manage both remote hosts and localhost. For example, on the controller (localhost), this ansible utility uses python version = 3.9.16. The version of Python to execute the modules may be different.
shell> ls -1 /usr/bin | egrep '^python[2,3]\.[0-9]$'
python2.7
python3.8
python3.9
By default, Ansible 'discovered' /usr/bin/python3 in Ubuntu which is a link to python3.8
shell> ansible localhost -m setup | grep -i python
"ansible_python": {
"executable": "/usr/bin/python3",
"type": "cpython",
"ansible_python_version": "3.8.5",
"ansible_selinux_python_present": true,
"discovered_interpreter_python": "/usr/bin/python3",
shell> ll /usr/bin/python3
lrwxrwxrwx 1 root root 9 Apr 27 2020 /usr/bin/python3 -> python3.8*
You can configure INTERPRETER_PYTHON in many different ways:
a) In the variable ansible_python_interpreter on the command line
shell> ansible localhost -m setup -e ansible_python_interpreter=/usr/bin/python3.9 | grep -i python
"ansible_python": {
"executable": "/usr/bin/python3.9",
"type": "cpython",
"ansible_python_version": "3.9.16",
"ansible_selinux_python_present": true,
, but you can declare the variable in other precedence and in various scopes. See Variable precedence: Where should I put a variable?. You can even configure different versions for single tasks. For example,
- hosts: localhost
tasks:
- command: "echo {{ ansible_python_interpreter }}"
register: out
vars:
ansible_python_interpreter: /usr/bin/python3.8
- debug:
var: out.stdout
- command: "echo {{ ansible_python_interpreter }}"
register: out
vars:
ansible_python_interpreter: /usr/bin/python3.9
- debug:
var: out.stdout
gives (abridged)
TASK [command] ***********************************************************************************************************************
changed: [localhost]
TASK [debug] *************************************************************************************************************************
ok: [localhost] =>
out.stdout: /usr/bin/python3.8
TASK [command] ***********************************************************************************************************************
changed: [localhost]
TASK [debug] *************************************************************************************************************************
ok: [localhost] =>
out.stdout: /usr/bin/python3.9
b) In the environment variable ANSIBLE_PYTHON_INTERPRETER on the command line
shell> ANSIBLE_PYTHON_INTERPRETER=/usr/bin/python3.9 ansible localhost -m setup | grep -i python
"ansible_python": {
"executable": "/usr/bin/python3.9",
"type": "cpython",
"ansible_python_version": "3.9.16",
"ansible_selinux_python_present": true,
c) In the configuration
shell> grep -B 1 interpreter_python ansible.cfg
[defaults]
interpreter_python = /usr/bin/python3.9
shell> ansible localhost -m setup | grep -i python
"ansible_python": {
"executable": "/usr/bin/python3.9",
"type": "cpython",
"ansible_python_version": "3.9.16",
"ansible_selinux_python_present": true,
d) In the inventory for a single host
shell> cat hosts
localhost ansible_python_interpreter=/usr/bin/python3.9
shell> ansible localhost -m setup | grep -i python
"ansible_python": {
"executable": "/usr/bin/python3.9",
"type": "cpython",
"ansible_python_version": "3.9.16",
"ansible_selinux_python_present": true,
e) In the inventory for a group of hosts
shell> cat hosts
[test]
test_11
test_12
test_13
[test:vars]
ansible_user=admin
ansible_python_interpreter=/usr/local/bin/python3.8
shell> ansible test_11 -m setup | grep -i python
"ansible_python": {
"executable": "/usr/local/bin/python3.8",
"type": "cpython",
"ansible_python_version": "3.8.12",
"status": "Missing selinux Python library"
"ansible_selinux_python_present": false,

You probably need to change this.
ansible-playbook 2.9.6
config file = /etc/ansible/ansible.cfg
configured module search path = [u'/home/pi/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
ansible python module location = /usr/lib/python2.7/dist-packages/ansible
executable location = /usr/bin/ansible-playbook
python version = 2.7.16 (default, Oct 10 2019, 22:02:15) [GCC 8.3.0]
python version is defined as 2.7.16, which you need to change.

Use facts gathered by ansible programmatically

I'd like to write a Python program that uses the facts that Ansible gives me with ansible HOST -m setup.
When I call this, I get a response which makes it only almost pure JSON:
$ ansible localhost -m setup
localhost | success >> {
// actual data
}
Is there some way to get this JSON response directly without parsing the shell output (which might not be too stable)? Could I even use Ansible directly in a Python 3 program?

version stable-2.2, stable-2.3, and 2.4+
The latest ansible releases for 2.2, 2.3, and 2.4 all support ANSIBLE_STDOUT_CALLBACK variable. To use it, you need to add an ansible.cfg file that looks like:
[defaults]
bin_ansible_callbacks = True
callback_plugins = ~/.ansible/callback_plugins
You can place it wherever you're using ansible. Then, you need to create the callback_plugins directory, if you haven't already. Finally, you need to add a custom json parser to the directory. I copied the json parser that is bundled with ansible to the callback_plugins directory, then edited a single line in it to make it work.
I found the json.py file by first executing ansible --version
$ ansible --version
ansible 2.4.0.0
config file = /Users/artburkart/Code/ansible.cfg
configured module search path = [u'/Users/artburkart/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
ansible python module location = /usr/local/lib/python2.7/site-packages/ansible
executable location = /usr/local/bin/ansible
python version = 2.7.13 (default, Jul 18 2017, 09:17:00) [GCC 4.2.1 Compatible Apple LLVM 8.1.0 (clang-802.0.42)]
Then using the "ansible python module location" to find the json.py.
cp /usr/local/lib/python2.7/site-packages/ansible/plugins/callback/json.py ~/.ansible/callback_plugins/
Finally I edited the v2_runner_on_ok function in the json.py file to look like this (courtesy of armab on GitHub):
def v2_runner_on_ok(self, result, **kwargs):
host = result._host
self.results[-1]['tasks'][-1]['hosts'][host.name] = result._result
print(json.dumps({host.name: result._result}, indent=4))
Once that was all set up, the command is very simple:
ANSIBLE_STDOUT_CALLBACK=json ansible all -i localhost, -c local -m setup | jq
If you always want to parse JSON output, you can add the following line to the end of the ansible.cfg file I described above.
stdout_callback = json
That way, you don't need to include the environment variable anymore.
versions <= latest 2.2 stable
When querying against instances, I use the following command:
ansible all --inventory 127.0.0.1, --connection local --module-name setup | sed '1 s/^.*|.*=>.*$/{/g'
If you pipe the output into jq, as leucos suggested, it happily parses the semi-valid JSON. For example:
ansible all -i hosts -m setup | sed '1 s/^.*|.*=>.*$/{/g' | jq -r '.ansible_facts.ansible_distribution'
CentOS
Ubuntu

If Python2 is OK for you, you can use the Ansible API directly. You can find detailled instructions here: http://docs.ansible.com/developing_api.html
It's really easy.
And alternate, shell centric way is to use jq. There is a quick intro here: http://xmodulo.com/how-to-parse-json-string-via-command-line-on-linux.html

Calling Python from Maven in a Cygwin shell

When building with Maven from a Cygwin shell (bash on Windows via Cygwin), the path to /usr/bin is not resolving properly.
Let me explain. Cygwin comes with Python, which is accessible as a symlink from /usr/bin. This symlink should be accessible to Maven, because its location is in the PATH environment variable. Cygwin adds /usr/bin to the PATH environment variable. However, Maven is unable to find Python. E.g.
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2.1:exec (default) on project cloud-devcloud: Command execution failed. Cannot run program "python" (in directory "C:\cygwin\home\myuser\incubator-cloudstack\tools\devcloud"): CreateProcess error=2, The system cannot find the file specified -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
But Python is there:
$ which python
/usr/bin/python
Albeit as a symlink.
$ ls -al /usr/bin/python
lrwxrwxrwx 1 donall Domain Users 13 Sep 19 11:38 /usr/bin/python -> python2.6.exe
Is the issue that Python is a symlink? Or that Maven cannot access files in /usr/bin?

It seems like maven is running as a regular Windows-native application, not a "cygwin-aware application" (if such thing exists :-) )
This is indicated by the path used internally, which is printed in the log: C:\cygwin\home\myuser\incubator-cloudstack\tools\devcloud
For this reason the call from Maven will not recognize the cygwin path when locating the python executable. One possible solution is to add the directory containing python.exe to the system PATH (which means, the Windows PATH variable)

importing cx_Oracle (python) with MacOSX

Importing cx_Oracle in a python script fails.
I have cx_Oracle installed, using "pip install cx_oracle" - that worked fine, reported installed.
Now when i try:
import cx_Oracle
I get the following error
Traceback (most recent call last):
File "reader.py", line 9, in <module>
import cx_Oracle
ImportError: dlopen(/Library/Python/2.7/site-packages/cx_Oracle.so, 2): Symbol not found: _OCIAttrGet
Referenced from: /Library/Python/2.7/site-packages/cx_Oracle.so
Expected in: flat namespace
in /Library/Python/2.7/site-packages/cx_Oracle.so
Other Information:
Python version 2.7 / mac os 10.7.2 (Lion)
$ python
Python 2.7.1 (r271:86832, Jun 16 2011, 16:59:05)
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Oracle 10.2
$ sqlplus -version
SQL*Plus: Release 10.2.0.4.0 - Production
Also, I do not have a /bin directory at all in my ORACLE_HOME folder, I have only the instant client and SDK installed.
ox_Oracle
$ pip freeze
PyRSS2Gen==1.0.0
...
cx-Oracle==5.1.1
(found a lot of questions on getting cx_Oracle installed, but none on this - thanks)

I ran into this problem today and was able to solve it by changing path to the libraries referenced in the InstantClient binaries to the actual locations on the filesystem.
This blog http://blog.caseylucas.com/2013/03/03/oracle-sqlplus-and-instant-client-on-mac-osx-without-dyld_library_path/ provides detailed explanation and the script to adjust all binaries. The only problem is that it uses #executable_path , which does not seem to work anymore with Python 2.7 & El Capitan (I'm not really sure what is responsible for the security exception). Replacing #executable_path with the actual path works just fine.
To summarize, steps to make it work:
install InstantClient to /usr/local/instantclient_11_2
make sure that cx_Oracle.so shared object that you use is at /Library/Python/2.7/site-packages/cx_Oracle.so
copy the following script to /usr/local/instantclient_11_2
#!/bin/sh
# script to change the dynamic lib paths and ids for oracle instant client
# exes and libs
(echo /Library/Python/2.7/site-packages/cx_Oracle.so ; find . -maxdepth 1 -type f \( -perm -1 -o \( -perm -10 -o -perm -100 \) \) -print ) | while read exe
do
echo adjusting executable $exe
baseexe=`basename $exe`
otool -L $exe | awk '/oracle/ {print $1}' | while read lib
do
echo adjusting lib $lib
baselib=`basename $lib`
if [ "$baseexe" = "$baselib" ]
then
echo changing id to $baselib for $exe
install_name_tool -id $baselib $exe
else
echo changing path id for $lib in $exe
install_name_tool -change $lib /usr/local/instantclient_11_2/$baselib $exe
fi
done
done
run the script with root permissions.

Uninstall everything.
Then install oracle instant client:
http://digitalsanctum.com/2007/07/26/installing-oracle-instant-client-on-mac-os-x/
Then use pip to install cx_oracle.
http://www.iceycake.com/2012/02/tutorial-how-to-install-cx_oracle-python-on-mac-os-x-lion/
Then set the path to point to the 32 bit version of oracle.
edit .profile file in your home directory and add the path to your oracle bin home, using this line:
export PATH=$PATH:/usr/local/lib/instantclient/
And it works...