Develop Reference

ansible commands run only with absolute path

On Ubuntu 20.04.2 LTS there is ansible engine installed with pip3 command:
mariusz#g3:~$ pip3 show ansible
Name: ansible
Version: 4.1.0
However running ansible commands ends with below error:
mariusz#g3:~$ ansible
python3: can't open file '/usr/bin/ansible': [Errno 2] No such file or directory
The PATH variable is set correctly:
mariusz#g3:~$ which ansible
/home/mariusz/.local/bin/ansible
And I can run ansible command with absolute path:
mariusz#g3:~$ /home/mariusz/.local/bin/ansible --version
ansible [core 2.11.1]
config file = /home/mariusz/.ansible.cfg
configured module search path = ['/home/mariusz/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /home/mariusz/.local/lib/python3.8/site-packages/ansible
ansible collection location = /home/mariusz/.ansible/collections:/usr/share/ansible/collections
executable location = /home/mariusz/.local/bin/ansible
python version = 3.8.5 (default, May 27 2021, 13:30:53) [GCC 9.3.0]
jinja version = 2.11.3
libyaml = True
Any ideas how to solve it without root privileges i.e. creating /usr/bin/ansible symlink?

The entries in the $PATH variable are tried in order, and thus you'd want to relocate your $HOME/.local/bin to the beginning of the list in order for it to win out over the /usr/bin entry that's there now
You can do this in an interactive shell to confirm or deny the theory, and then put at the end of your ~/.bashrc to make it permanent
PATH=$HOME/.local/bin:$PATH

It seems that ansible package, which was installed before, left bash aliases file that was not removed during package uninstall.
$ cat ~/.bash_aliases
alias ansible='python3 /usr/bin/ansible'
alias ansible-doc='python3 /usr/bin/ansible-doc'
alias ansible-galaxy='python3 /usr/bin/ansible-galaxy'
alias ansible-inventory='python3 /usr/bin/ansible-inventory'
alias ansible-playbook='python3 /usr/bin/ansible-playbook'
alias ansible-vault='python3 /usr/bin/ansible-vault'

Mac OSX: LoadModule wsgi_module not working as mentioned in docs

I upgraded apache from 2.4.25 to 2.4.29 due to security reasons in Mac OSX.
Also installed mod_wsgi via pip.
Now facing difficulty while restarting the apache server!
I ran this command as per the mod_wsgi docs,
> mod_wsgi-express module-config
LoadModule wsgi_module "/Library/Python/2.7/site-packages/mod_wsgi/server/mod_wsgi-py27.so"
WSGIPythonHome "/System/Library/Frameworks/Python.framework/Versions/2.7"
Copied the above two lines in apache.conf file and tried to start the apache server but getting the following error:
> sudo apachectl start
/usr/local/apache/bin/apachectl: line 79: 51863 Illegal instruction: 4 $HTTPD -k $ARGV
Any help would be appreciated. Thanks!
EDIT as per Graham answer:
# I upgraded the mod_wsgi
> sudo pip install --upgrade --no-cache-dir mod_wsgi
Collecting mod_wsgi
Downloading mod_wsgi-4.5.21.tar.gz (2.5MB)
100% |████████████████████████████████| 2.5MB 2.1MB/s
Installing collected packages: mod-wsgi
Found existing installation: mod-wsgi 4.5.20
Uninstalling mod-wsgi-4.5.20:
Successfully uninstalled mod-wsgi-4.5.20
Running setup.py install for mod-wsgi ... done
Successfully installed mod-wsgi-4.5.21
But when i tried:
> mod_wsgi-express start-server
Traceback (most recent call last):
File "/Library/WebServer/Documents/DSFENV/bin/mod_wsgi-express", line 11, in <module>
load_entry_point('mod-wsgi==4.5.21', 'console_scripts', 'mod_wsgi-express')()
File "/Library/WebServer/Documents/DSFENV/lib/python2.7/site-packages/mod_wsgi/server/__init__.py", line 3484, in main
cmd_start_server(args)
File "/Library/WebServer/Documents/DSFENV/lib/python2.7/site-packages/mod_wsgi/server/__init__.py", line 3332, in cmd_start_server
config = _cmd_setup_server('start-server', args, vars(options))
File "/Library/WebServer/Documents/DSFENV/lib/python2.7/site-packages/mod_wsgi/server/__init__.py", line 2999, in _cmd_setup_server
language, encoding = locale.getdefaultlocale()
File "/Library/WebServer/Documents/DSFENV/lib/python2.7/locale.py", line 543, in getdefaultlocale
return _parse_localename(localename)
File "/Library/WebServer/Documents/DSFENV/lib/python2.7/locale.py", line 475, in _parse_localename
raise ValueError, 'unknown locale: %s' % localename
ValueError: unknown locale: UTF-8
getting the above error.
Also I have installed apache 2.4.29 following this blog:
2nd Edit
I have resolved ValueError: unknown locale: UTF-8, now the output of mod_wsgi-express start-server shows:
> mod_wsgi-express start-server
Server URL : http://localhost:8000/
Server Root : /tmp/mod_wsgi-localhost:8000:501
Server Conf : /tmp/mod_wsgi-localhost:8000:501/httpd.conf
Error Log File : /tmp/mod_wsgi-localhost:8000:501/error_log (warn)
Request Capacity : 5 (1 process * 5 threads)
Request Timeout : 60 (seconds)
Startup Timeout : 15 (seconds)
Queue Backlog : 100 (connections)
Queue Timeout : 45 (seconds)
Server Capacity : 20 (event/worker), 20 (prefork)
Server Backlog : 500 (connections)
Locale Setting : en_US.UTF-8
But when I try to do config test, still showing the same error:
> sudo apachectl configtest
/usr/local/apache/bin/apachectl: line 79: 53944 Illegal instruction: 4 $HTTPD -t
xcode version:
> xcodebuild -version
Xcode 8.3.2
Build version 8E2002
3rd EDIT as per comments:
> /usr/local/apache/bin/httpd
Illegal instruction: 4
> /Library/Python/2.7/site-packages/mod_wsgi/server/mod_wsgi-py27.so
-bash: /Library/Python/2.7/site-packages/mod_wsgi/server/mod_wsgi-py27.so: cannot execute binary file
> sys.prefix
'/System/Library/Frameworks/Python.framework/Versions/2.7'
> sys.version
'2.7.10 (default, Feb 7 2017, 00:08:15) \n[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.34)]'
I followed this blog to install Apache 2.4.29:
http://mac-dev-env.patrickbougie.com/apache/
--with-ssl option as we need to https for the website.
4th EDIT:
> file /usr/local/apache/bin/httpd
/usr/local/apache/bin/httpd: Mach-O 64-bit executable x86_64
> file /Library/Python/2.7/site-packages/mod_wsgi/server/mod_wsgi-py27.so
/Library/Python/2.7/site-packages/mod_wsgi/server/mod_wsgi-py27.so: Mach-O universal binary with 2 architectures: [x86_64: Mach-O 64-bit bundle x86_64] [i386: Mach-O bundle i386]
/Library/Python/2.7/site-packages/mod_wsgi/server/mod_wsgi-py27.so (for architecture x86_64): Mach-O 64-bit bundle x86_64
/Library/Python/2.7/site-packages/mod_wsgi/server/mod_wsgi-py27.so (for architecture i386): Mach-O bundle i386
Please let me know how to proceed forward.

If you have updated your Apache installation in some way, usually the mod_wsgi.so binary from an older release should still work as Apache maintains ABI compatibility across patch level updates.
If it isn't working, then you may need to re-build mod_wsgi. If there is no newer version of mod_wsgi, you will need to force a full re-build to ensure cached binaries from pip cache are not used.
pip install --upgrade --no-cache-dir mod_wsgi
Next problem though is if you have upgraded to Xcode 9.1, Apple have further broken things by completely removing apxs, meaning that my prior workaround for Apple brokeness no longer works. In this case you need to ensure you are installing mod_wsgi 4.5.21 or later. That version only just got released with fixes to accomodate for Xcode 9.1
So try that pip command above and make sure mod_wsgi-express still works by running:
mod_wsgi-express start-server
Then validate configuration snippet to use in Apache config by running again:
mod_wsgi-express module-config
If still issues with main Apache, then ensure you do a complete stop of Apache and then a start, and not just restart. Let me know how that all goes.
UPDATE 1
Above presumes you were using Apple supplied Apache in respect of Xcode 9.1 comments. You seem to be using your own Apache, maybe form Homebrew. Still try the same thing.

As suggested by Graham:
pip install --upgrade --no-cache-dir mod_wsgi
Upgrade to mod_wsgi 4.5.22 did the trick.

yum install firefox error - libnssutil3.so

I'm getting this error while installing/listing firefox or python on a Linux server. Any ideas how to fix it.
# yum install firefox
There was a problem importing one of the Python modules
required to run yum. The error leading to this problem was:
/usr/lib64/libnssutil3.so: undefined symbol: PL_ClearArenaPool
Please install a package which provides this module, or
verify that the module is installed correctly.
It's possible that the above module doesn't match the
current version of Python, which is:
2.4.3 (#1, Oct 23 2012, 22:02:41)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-54)]
If you cannot solve this problem yourself, please go to
the yum faq at:
http://wiki.linux.duke.edu/YumFaq
Instead of getting the following output/value for firefox -version command.
$ firefox --version
Mozilla Firefox 17.0.9
I'm getting:
XPCOMGlueLoad error for file /opt/firefox/libxpcom.so:
libxul.so: cannot open shared object file: No such file or directory
Couldn't load XPCOM.
python -h command output looks valid.
python -V shows:
Python 2.4.3
When I'm running Selenium tests (which requires firefox and Xvfb), I get the following error:
org.openqa.selenium.firefox.NotConnectedException: Unable to connect to host 127.0.0.1 on port 7055 after 45000 ms. Firefox console output:
XPCOMGlueLoad error for file /opt/firefox/libxpcom.so:
libxul.so: cannot open shared object file: No such file or directory
Couldn't load XPCOM.
XPCOMGlueLoad error for file /opt/firefox/libxpcom.so:
libxul.so: cannot open shared object file: No such file or directory
Couldn't load XPCOM.

The problem is not python, it’s the LD_LIBRARY_PATH that does not include /lib64 and /usr/lib64 setting values. we fixed this by pre-pending /lib64:/usr/lib64 to the LD_LIBRARY_PATH.
export LD_LIBRARY_PATH=/usr/lib64/:/lib64:$LD_LIBRARY_PATH
After this point, when I did yum help or yum install firefox (it didn't give the above error) BUT:
firefox --version still gave the error:
XPCOMGlueLoad error for file /opt/firefox/libxpcom.so:
libxul.so: cannot open shared object file: No such file or directory
Couldn't load XPCOM.
Now, I finally ran yum install firefox and it installed bunch of components and at the end said:
Installing : alsa-lib 20/23
Installing : xulrunner 21/23
Installing : firefox 22/23
Installing : firefox 23/23
Installed:
firefox.i386 0:17.0.9-1.el5_9 firefox.x86_64 0:17.0.9-1.el5_9
Dependency Installed:
GConf2.i386 0:2.14.0-9.el5 ORBit2.i386 0:2.14.3-5.el5 alsa-lib.i386 0:1.0.17-1.el5 atk.i386 0:1.12.2-1.fc6 avahi.i386 0:0.6.16-10.el5_6 avahi-glib.i386 0:0.6.16-10.el5_6 cairo.i386 0:1.2.4-5.el5
cups-libs.i386 1:1.3.7-30.el5_9.3 gamin.i386 0:0.1.7-10.el5 gnome-vfs2.i386 0:2.16.2-12.el5_9 gnutls.i386 0:1.4.1-10.el5_9.2 gtk2.i386 0:2.10.4-29.el5 libIDL.i386 0:0.8.7-1.fc6 libXcursor.i386 0:1.1.7-1.2
libXfixes.i386 0:4.0.1-2.1 libXinerama.i386 0:1.0.1-2.1 libXrandr.i386 0:1.1.1-3.3 libacl.i386 0:2.2.39-8.el5 libattr.i386 0:2.4.32-1.1 pango.i386 0:1.14.9-8.el5_7.3 xulrunner.i386 0:17.0.9-1.el5_9
Complete!
Now firefox version is showing correctly.
firefox --version
Mozilla Firefox 17.0.9
Everything is working now.

Use facts gathered by ansible programmatically

I'd like to write a Python program that uses the facts that Ansible gives me with ansible HOST -m setup.
When I call this, I get a response which makes it only almost pure JSON:
$ ansible localhost -m setup
localhost | success >> {
// actual data
}
Is there some way to get this JSON response directly without parsing the shell output (which might not be too stable)? Could I even use Ansible directly in a Python 3 program?

version stable-2.2, stable-2.3, and 2.4+
The latest ansible releases for 2.2, 2.3, and 2.4 all support ANSIBLE_STDOUT_CALLBACK variable. To use it, you need to add an ansible.cfg file that looks like:
[defaults]
bin_ansible_callbacks = True
callback_plugins = ~/.ansible/callback_plugins
You can place it wherever you're using ansible. Then, you need to create the callback_plugins directory, if you haven't already. Finally, you need to add a custom json parser to the directory. I copied the json parser that is bundled with ansible to the callback_plugins directory, then edited a single line in it to make it work.
I found the json.py file by first executing ansible --version
$ ansible --version
ansible 2.4.0.0
config file = /Users/artburkart/Code/ansible.cfg
configured module search path = [u'/Users/artburkart/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
ansible python module location = /usr/local/lib/python2.7/site-packages/ansible
executable location = /usr/local/bin/ansible
python version = 2.7.13 (default, Jul 18 2017, 09:17:00) [GCC 4.2.1 Compatible Apple LLVM 8.1.0 (clang-802.0.42)]
Then using the "ansible python module location" to find the json.py.
cp /usr/local/lib/python2.7/site-packages/ansible/plugins/callback/json.py ~/.ansible/callback_plugins/
Finally I edited the v2_runner_on_ok function in the json.py file to look like this (courtesy of armab on GitHub):
def v2_runner_on_ok(self, result, **kwargs):
host = result._host
self.results[-1]['tasks'][-1]['hosts'][host.name] = result._result
print(json.dumps({host.name: result._result}, indent=4))
Once that was all set up, the command is very simple:
ANSIBLE_STDOUT_CALLBACK=json ansible all -i localhost, -c local -m setup | jq
If you always want to parse JSON output, you can add the following line to the end of the ansible.cfg file I described above.
stdout_callback = json
That way, you don't need to include the environment variable anymore.
versions <= latest 2.2 stable
When querying against instances, I use the following command:
ansible all --inventory 127.0.0.1, --connection local --module-name setup | sed '1 s/^.*|.*=>.*$/{/g'
If you pipe the output into jq, as leucos suggested, it happily parses the semi-valid JSON. For example:
ansible all -i hosts -m setup | sed '1 s/^.*|.*=>.*$/{/g' | jq -r '.ansible_facts.ansible_distribution'
CentOS
Ubuntu

If Python2 is OK for you, you can use the Ansible API directly. You can find detailled instructions here: http://docs.ansible.com/developing_api.html
It's really easy.
And alternate, shell centric way is to use jq. There is a quick intro here: http://xmodulo.com/how-to-parse-json-string-via-command-line-on-linux.html

importing cx_Oracle (python) with MacOSX

Importing cx_Oracle in a python script fails.
I have cx_Oracle installed, using "pip install cx_oracle" - that worked fine, reported installed.
Now when i try:
import cx_Oracle
I get the following error
Traceback (most recent call last):
File "reader.py", line 9, in <module>
import cx_Oracle
ImportError: dlopen(/Library/Python/2.7/site-packages/cx_Oracle.so, 2): Symbol not found: _OCIAttrGet
Referenced from: /Library/Python/2.7/site-packages/cx_Oracle.so
Expected in: flat namespace
in /Library/Python/2.7/site-packages/cx_Oracle.so
Other Information:
Python version 2.7 / mac os 10.7.2 (Lion)
$ python
Python 2.7.1 (r271:86832, Jun 16 2011, 16:59:05)
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Oracle 10.2
$ sqlplus -version
SQL*Plus: Release 10.2.0.4.0 - Production
Also, I do not have a /bin directory at all in my ORACLE_HOME folder, I have only the instant client and SDK installed.
ox_Oracle
$ pip freeze
PyRSS2Gen==1.0.0
...
cx-Oracle==5.1.1
(found a lot of questions on getting cx_Oracle installed, but none on this - thanks)

I ran into this problem today and was able to solve it by changing path to the libraries referenced in the InstantClient binaries to the actual locations on the filesystem.
This blog http://blog.caseylucas.com/2013/03/03/oracle-sqlplus-and-instant-client-on-mac-osx-without-dyld_library_path/ provides detailed explanation and the script to adjust all binaries. The only problem is that it uses #executable_path , which does not seem to work anymore with Python 2.7 & El Capitan (I'm not really sure what is responsible for the security exception). Replacing #executable_path with the actual path works just fine.
To summarize, steps to make it work:
install InstantClient to /usr/local/instantclient_11_2
make sure that cx_Oracle.so shared object that you use is at /Library/Python/2.7/site-packages/cx_Oracle.so
copy the following script to /usr/local/instantclient_11_2
#!/bin/sh
# script to change the dynamic lib paths and ids for oracle instant client
# exes and libs
(echo /Library/Python/2.7/site-packages/cx_Oracle.so ; find . -maxdepth 1 -type f \( -perm -1 -o \( -perm -10 -o -perm -100 \) \) -print ) | while read exe
do
echo adjusting executable $exe
baseexe=`basename $exe`
otool -L $exe | awk '/oracle/ {print $1}' | while read lib
do
echo adjusting lib $lib
baselib=`basename $lib`
if [ "$baseexe" = "$baselib" ]
then
echo changing id to $baselib for $exe
install_name_tool -id $baselib $exe
else
echo changing path id for $lib in $exe
install_name_tool -change $lib /usr/local/instantclient_11_2/$baselib $exe
fi
done
done
run the script with root permissions.

Uninstall everything.
Then install oracle instant client:
http://digitalsanctum.com/2007/07/26/installing-oracle-instant-client-on-mac-os-x/
Then use pip to install cx_oracle.
http://www.iceycake.com/2012/02/tutorial-how-to-install-cx_oracle-python-on-mac-os-x-lion/
Then set the path to point to the 32 bit version of oracle.
edit .profile file in your home directory and add the path to your oracle bin home, using this line:
export PATH=$PATH:/usr/local/lib/instantclient/
And it works...