Develop Reference

Running ansible-playbook with Python API

I'm trying to launch an ansible playbook from a python script but I have the following error:
Traceback (most recent call last):
File "py_runner.py", line 1, in <module>
from ansible import context
ModuleNotFoundError: No module named 'ansible'
I'm using one of the code that I found on this link Running ansible-playbook using Python API and here is my ansible location:
ansible 2.8.0
config file = /etc/ansible/ansible.cfg
configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
ansible python module location = /usr/lib/python2.7/site-packages/ansible
executable location = /usr/bin/ansible
python version = 2.7.5 (default, Oct 30 2018, 23:45:53) [GCC 4.8.5 20150623 (Red Hat 4.8.5-36)]
Could you help me figure out why my pythom program doesn't found the ansible module?
Many thanks

Please ignore that.
A figured out that I had two python releases installed on the server: Python2.7 and 3.6.
The Python3.6 directory doesn't have the module installed so I need to launch the program using Python2.7.

six: cannot import name python_2_unicode_compatible

With six 1.10.0 installed under Python and pip 2.6, an old Django 1.0.4 app is not able to import python_2_unicode_compatible even though it finds six 1.10.0 just fine:
>>> import six
>>> six.__version__
'1.10.0'
>>> from six import python_2_unicode_compatible
>>>
I've confirmed with python code within the app that it does indeed have access to six:
['appdirs==1.4.3', 'argparse==1.4.0', 'astkit==0.5.4', 'beautifulsoup==3.2.1',
'coverage==4.3.4', 'django-cms==2.0.0a0', 'django==1.0.4', 'dnspython==1.12.0',
'flup==1.0.2', 'importlib==1.0.4', 'iniparse==0.3.1', 'instrumental==0.5.3',
'mako==1.0.6', 'markupsafe==1.0', 'minimock==1.2.8', 'mysql-python==1.2.5',
'nose==1.3.7', 'packaging==16.8', 'pillow==3.4.2', 'pip==9.0.1', 'pluggy==0.4.0',
'py==1.4.33', 'pyparsing==2.2.0', 'python-dateutil==2.6.0', 'pyzor==1.0.0',
'setuptools==35.0.1', 'six==1.10.0', 'sorl-thumbnail==12.3', 'tox==2.7.0',
'uwsgi==2.0.15','virtualenv==15.1.0', 'wheel==0.29.0']
I am tasked to move a very old site that was running django 1.0.4 (you read that right, 1.0.4) and django_cms 2.0.0 Alpha to a new server. The old server croaked, so all I have is the backup of the main website files and dependencies that were installed long ago.
I am Dockerizing it to help document and deploy this in the future.
Ubuntu 14.04
Python 2.6 (same results with 2.7)
Django 1.0.4 (installed via local zip)
django_cms 2.0.0a0 (installed via local zip)
I have tried Apache mod_wsgi, gunicorn (pip2.6 installed) and currently using uwsgi (preferred, pip2.6 installed) to load the app.
Nginx is running in another Docker container with proxy_pass, and will the frontend proxy and TLS.
uwsgi starts the site up with the custom wsgi.
Upon loading the / index page, I had many import errors. Slowly, I am resolving each and every one of them (mostly related to the Django "MIDDLEWARE_CLASSES", which I have yet to find their definition).
I am currently stuck on the following error:
Traceback (most recent call last):
File "/usr/local/lib/python2.6/dist-packages/django/core/handlers/wsgi.py", line 230, in __call__
self.load_middleware()
File "/usr/local/lib/python2.6/dist-packages/django/core/handlers/base.py", line 41, in load_middleware
raise exceptions.ImproperlyConfigured, 'Error importing middleware %s: "%s"' % (mw_module, e)
django.core.exceptions.ImproperlyConfigured: Error importing middleware cms.middleware.user: "cannot import name python_2_unicode_compatible"
uwsgi starts up with the specified python2.6 just fine:
web_1 | [uWSGI] getting INI configuration from uwsgi.ini
web_1 | *** Starting uWSGI 2.0.15 (64bit) on [Wed Apr 26 16:27:43 2017] ***
web_1 | Python version: 2.6.9 (default, Oct 22 2014, 19:53:49) [GCC 4.8.2]
web_1 | Python main interpreter initialized at 0xef1050
web_1 | python threads support enabled
Also, python2.7 was originally configured and had the exact same error. I thought I read where python_2_unicode_compatible was deprecated in 2.7 or something, so I went back to the original version the site was running under.
Do I need to install virtualenv? I don't usually do it under Docker, and just install everything globally. I can't see how that would make a difference.
If six was not found, wouldn't I get an error that it could not import six, instead of python_2_unicode_compatible?

The python_2_unicode_compatible method was originally in Django, then added to six in 1.9.
One of your installed packages may be trying to import python_2_unicode_compatible from django.utils.encoding, rather than from the six package.

Unable to import _winreg in Python 2.7.9 virtual environment

I'm running an app engine application in a virtual environment on windows 7 64bit, python 2.7.9 x64.
Here's the stacktrace:
p_system = platform.system()
File "C:\Python27\lib\platform.py", line 1310, in system
return uname()[0]
File "C:\Python27\lib\platform.py", line 1206, in uname
release,version,csd,ptype = win32_ver()
File "C:\Python27\lib\platform.py", line 597, in win32_ver
import _winreg
File "C:\Program Files (x86)\Google\google_appengine\google\appengine\tools\devappserver2\python\sandbox.py", line 945, in load_module
raise ImportError('No module named %s' % fullname)
ImportError: No module named _winreg
However, it works just fine from cli (outside venv):
Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation. All rights reserved.
C:\Users\Admin>python
Python 2.7.9 (default, Dec 10 2014, 12:28:03) [MSC v.1500 64 bit (AMD64)] on win
32
Type "help", "copyright", "credits" or "license" for more information.
>>> import _winreg
>>> import platform
>>> platform.system()
'Windows'
>>>
Why does this happen? What can I do to fix this?

Module _winreg, as the docs say, exists to "expose the Windows registry API to Python".
App Engine does not supply a "Windows registry API" (nor any other Windows-specific API). Therefore, its sandbox blocks attempts to import the module -- note, at the end of your stack trace, that the exception is deliberately raised in module sandbox.py of the App Engine SDK.
Python's "virtual env" plays no part here -- it's all about App Engine.
Please clarify what task you're trying to accomplish with _winreg once your GAE app is deployed -- assume it's deployed to Linux servers (although the GAE runtime doesn't supply Linux-specific APIs either:-), so there is no Windows Registry API anywhere in the neighborhood...

The workaround provided by Google, until a fix is implemented, is as follows:
Go to: <sdk_root>\google\appengine\tools\devappserver2\python\sandbox.py
Find the definition of _WHITE_LIST_C_MODULES = [xxx]
Add the following two lines to the list:
'_winreg',
'_ctypes',
If this does not succeed, run python -m pip install google-cloud

I think that the problem is that GAE is not aware that you are in development mode, I suppose because the SERVER_SOFTWARE variable is set to something not starting with "Dev".
If you execute the following code (before calling any GAE library) it should fix the issue:
import os
os.environ['SERVER_SOFTWARE'] = 'Dev'
Note: Make sure this code is removed before going to production.

I had this problem a few days ago.
As said above, the GAE sandbox on Windows blocks some routines or libraries, even built-in one, because it is developed to Unix-like platform.
I opened an issue to Google Team and they passed a workaround:
https://issuetracker.google.com/issues/38290292
That workaround worked well.

Unable to get pyrabbit to run

Im working on a little project that running rabbitmq with python, I need a way to access the management api and pull stats, jobs, etc. I have tried using pyRabbit, but doen't appear to be working unsure why, hoping better programmers might know? Below I was just following the basic tutorial and readme to perform the very basic task. My server is up, I'm able to connect outside of python and pyrabbit fine. I have installed off the dependencies with no luck, at least I think. Also open to other suggestions for just getting queue size, queues, active clients etc outside of pyRabbit.
'Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation. All rights reserved.
C:\Users\user>python
Python 2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
import nose
import httplib2
import mock
from pyrabbit.api import Client
import pyrabbit
cl = Client('my.ip.com:15672', 'guest', 'guest')
cl.is_alive()
No JSON object could be decoded - (Not found.) ()
Traceback (most recent call last):
File "", line 1, in
File "C:\Python27\lib\site-packages\pyrabbit\api.py", line 48, in wrapper if self.has_admin_rights:
File "C:\Python27\lib\site-packages\pyrabbit\api.py", line 175, in has_admin_right whoami = self.get_whoami()
File "C:\Python27\lib\site-packages\pyrabbit\api.py", line 161, in get_whoami whoami = self.http.do_call(path, 'GET')
File "C:\Python27\lib\site-packages\pyrabbit\http.py", line 112, in do_call raise HTTPError(content, resp.status, resp.reason, path, body)
pyrabbit.http.HTTPError: 404 - Object Not Found (None) (whoami) (None)'

I was never able to solve this. But, this forced me to learn what json is, I used simplejson along with httplib2 and it worked like a charm...

This might be a rabbitmq configuration problem: as it is written here: https://www.rabbitmq.com/access-control.html, "guest" user can only connect via localhost:
By default, the guest user is prohibited from connecting to the broker remotely; it can only connect over a loopback interface (i.e. localhost). This applies both to AMQP and to any other protocols enabled via plugins. Any other users you create will not (by default) be restricted in this way.
This is configured via the loopback_users item in the configuration file.
If you wish to allow the guest user to connect from a remote host, you should set the loopback_users configuration item to []. A complete >rabbitmq.config which does this would look like:
[{rabbit, [{loopback_users, []}]}].

GAE: remote_api_shell.py uses wrong Django version

I have a Python 2.7 GAE app that uses Django 1.2. Everything generally works fine.
I'm trying to use remote_api_shell.py, but when I import my models.py, I get this error:
$ PYTHONPATH=.; remote_api_shell.py APP
App Engine remote_api shell
Python 2.7.2 (v2.7.2:8527427914a2, Jun 11 2011, 15:22:34)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]
The db, users, urlfetch, and memcache modules are imported.
s~APP> import models
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "/Users/.../models.py", line 8, in <module>
from django.core.validators import email_re
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/django_0_96/django/core/validators.py", line 13, in <module>
from django.utils.translation import gettext, gettext_lazy, ngettext
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/django_0_96/django/utils/translation/__init__.py", line 3, in <module>
if settings.USE_I18N:
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/django_0_96/django/conf/__init__.py", line 28, in __getattr__
self._import_settings()
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/lib/django_0_96/django/conf/__init__.py", line 53, in _import_settings
raise EnvironmentError, "Environment variable %s is undefined." % ENVIRONMENT_VARIABLE
EnvironmentError: Environment variable DJANGO_SETTINGS_MODULE is undefined.
I don't know why it is using Django 0.96 here. Especially since I think Python 2.7 supports only Django 1.2. Here are the relevant parts of my app.yaml:
application: APP
version: prod
runtime: python27
api_version: 1
threadsafe: true
[...]
libraries:
- name: django
version: "1.2"
If I comment out the from django.core.validators import email_re line then the import succeeds.
Any idea if I am doing something wrong or if this is this a GAE bug?

I had the same issue, and I solved it by running these two lines once I open the remote API session:
from google.appengine.dist import use_library
use_library('django', '1.3')
Though, clearly, it's a hack. Star this issue to encourage the app engine team to fix it:
https://code.google.com/p/googleappengine/issues/detail?id=8716

try typing this:
import django
django.VERSION
To check the django version! I am guessing that the python console that u are running has the old version of django!

FYI this no longer works because
use_library('django', '1.5')
now throws this error:
ValueError: 1.5 is not a supported version for django; supported versions are ['0.96', '1.0', '1.1', '1.2', '1.3']
Hopefully this is a temporary problem, but my solution was to delete (rename):
C:\Program Files (x86)\Google\google_appengine\lib\django-0.96
to something else, then the proper version of django loads w/o any other problems.