On our production server we are running Ubuntu 12.04, to make sure that our application is running in a predefined consistent environment we use Pythonbrew to compile a custom Python. I have created a user that is going to be running our API server, the user has his own Python 2.7.2 environment created with Pythonbrew, with the necessary packages installed using pip.
I've also installed uWSGI on the system, it's going to wrap the API application and make it available to Apache via mod_wsgi. The problem is that I can't get uWSGI to use the correct python binary. The current config looks like this:
[uwsgi]
socket = 127.0.0.1:3031
processes = 4
chdir = /home/app/
pythonpath = /home/app
gid = 1002
uid = 1002
module = api
callable = APP
enable-threads = true
When I try to run this config on the terminal with:
uwsgi --ini app.ini
It fails to import some module. This module is installed only in the Pythonbrew environment that the API user is using. The problem is that uWSGI is using the default python binary in /usr/bin.
One solution is to use a separate uWSGI installed using pip in the API user's environment, but I really want to use the system's uWSGI because it integrates better with the OS.
Is there any way to specify which Python binary uWSGI should use, other then by installing a separate one? If I have to install a separate instance of uWSGI, what is the best way to have it start on system boot?
Edit: just realised that it is probably not using the python binary, but just lining against it's library, so there is no way for me to use the default installation of wsgi, and the non-default python. But the question still remains, what is the best way to integrate a custom built uWSGI into the system.
uWSGI (as well as mod_wsgi) does not use the python binary, but the python shared library, so you have to link the uWSGI python plugin with the specific library of the pythonbrew user.
There are a lot of tricks to accomplish that (expecially if the pythonbrew python version matches the system version), the first one popping into my mind is specifying the pythonbrew user's prefix in its config file with
env = PYTHONHOME=path
By the way, is not better to use virtualenvs instead of full python installations for your users ?
I've run into this issue as well. It seems that uWSGI binds the Python 3 runtime during install. Once you've pip installed a version, it uses the cached wheel (hence cached Python version).
I've found this will fix it by forcing a rebuild of the binary, at least on Ubuntu:
$ pip uninstall uwsgi
$ pip install uwsgi --no-cache
$ COMMAND_TO_RESTART_UWSGI
I had this problem on Ubuntu 18.04 (so a tad bit newer than when the question was asked) and remembered that the [uwsgi] section of the configuration file contains the long command line switches.
It appears that uwsgi in the version that is packaged for Ubuntu 18.04.4 defaults to using the python plugin, which means Python 2.x in this case.
And so I managed to configure my apps by setting them to use the python3 plugin:
[uwsgi]
plugin = python3
This requires the uwsgi-plugin-python3 package to be installed. In case of Ubuntu 18.04 this defaulted to Python 3.6 and hence the above plugin line could also have been:
plugin = python36
You can test if it works with a given plugin by running uwsgi from the command line like so:
uwsgi --plugin python3 -s :0
Related
When running my Flask application on my Ubuntu 16.04.3 x64 web server, I get the following error when trying to import the Flask-OpenID library v1.2.5:
File "/usr/local/lib/python2.7/dist-packages/openid/oidutil.py", line 17, in <module>
from urllib.parse import urlencode
ImportError: No module named parse
I understand that this is happening because this module is getting loaded in using python 2.7.5 (which I have installed on my machine) instead of 3.5.2 (also have it installed).
Checking the version of python verifies that it is installed on the machine:
>> python -V
Python 3.5.2
The virtual environment was initialized with python 3:
python3 -m venv env
The Flask app still seems to be using files from the python 2.7.5.
If this is indeed the issue, is there any way to set a default or to make sure Flask uses the correct version of python? My app runs with no errors in my development environment on my PC.
It turns out that my virtual environment was configured correctly, however the issue was in fact that mod_wsgi running on Apache Web server, was compiled across Python version 2.7.X and loaded up Flask with python 2.7.5 instead of 3.5.2.
To fix this issue I first updated to the version of wsgi that would run python 3.5.1+
sudo apt-get install libapache2-mod-wsgi-py3
From this point on, your flask application may not run since your virtual host, and wsgi file for your flask application needs to be configured properly with python version 3.
Follow this tutorial: http://devmartin.com/blog/2015/02/how-to-deploy-a-python3-wsgi-application-with-apache2-and-debian/
You should also change your VirtualHost configuration to the one shown and explained at this post: https://stackoverflow.com/a/39419825/9367575
I'm trying to create a Django app with WebSockets on a Raspberry Pi 2 using django-websocket-redis pip package. /manage.py runserver doesn't seem to work (got 404 calling /ws/ although it's set by WEBSOCKET_URL = '/ws/' in settings.py), I wanna try a standalone uWSGI server like described here in the official docs.
When I run uwsgi, I got strange errors. I think this is caused by a wrong python version. The output of uwsgi show me that Python 2.7 is used:
Python version: 2.7.13 (default, Jan 19 2017, 14:48:08) [GCC 6.3.0 20170124]
But I need Python 3 (exactly 3.5) for my project. I changed the default python environment, so that the python comamnd points to python 3.5 instead of 2.7. Additionally, I passed the version using --plugin switch like this:
uwsgi --http :9090 --plugin=python35 --wsgi-file wsgi.py
I also used pip3for all pip packages to make sure that no 2.x packages were used. This seems having no effect, cause my script breaks and uwsgi show me that Python 2.7 is used...
install this:
sudo apt-get install uwsgi-plugin-python3
I tried to uninstall a module on heroku with:
heroku run bin/python bin/pip uninstall whatever
Pip shows the module in the /app tree then claims to have uinstalled the module, but running the same command again shows it installed in the same location in the /app tree.
Is there a way to get pip uinstall to succeed?
Heroku run instantiates a new dyno and runs the command specified in that dyno only. Dynos are ephemeral which is why the results of the pip uninstall don't stick.
Updated 2013-09-30: the current way to clear the virtualenv seems to specify a different python runtime version in runtime.txt as stated on Github and in the Heroku's devcenter reference.
Be aware that Heroku currently "only endorses and supports the use of Python 2.7.4 and 3.3.2" so unless your application supports both Python 2.7.4 and 3.3.2, you may want to test it with the runtime you'll want to switch to (currently available at http://envy-versions.s3.amazonaws.com/$PYTHON_VERSION.tar.bz2, though it shouldn't be an issue to switch e.g. between 2.7.4 and 2.7.3 in most cases).
Thanks #Jesse for your up-to-date answer and to the commenters who made me aware of the issue.
Was up-to-date in ~november 2012 (I haven't since updated the linked buildpack, my pull request was closed and the CLEAN_VIRTUALENV feature was dropped at some point by the official buildpack):
As David explained, you cannot pip uninstall one package but you can purge and reinstall the whole virtualenv. Use the user-env-compile lab feature with the CLEAN_VIRTUALENV option to purge the virtualenv:
heroku labs:enable user-env-compile
heroku config:add CLEAN_VIRTUALENV=true
Currently this won't work because there is a bug. You'll need to use my fork of the buildpack until this get fixed upstream (pull request was closed) :
heroku config:add BUILDPACK_URL=https://github.com/blaze33/heroku-buildpack-python.git
Now push your new code and you'll notice that the whole virtualenv gets reinstalled.
Andrey's answer no longer works since March 23 2012. The new style virtualenv commit moved the virtual env from /app to /app/.heroku/venv but the purge branch wasn't updated to catch up so that you end up with a virtualenv not being in PYTHONHOME.
To avoid reinstalling everything after each push, disable the option:
heroku labs:disable user-env-compile
heroku config:remove CLEAN_VIRTUALENV BUILDPACK_URL
There is now a simpler way to clear the pip cache. Just change the runtime environment, for example from 'python-2.7.3' to 'python-2.7.2', or vice versa.
To do this add a file called runtime.txt to the root of your repository that contains just the runtime string (as show above) in it.
For this to work you need to have turned on the Heroku labs user-env-compile feature. See https://devcenter.heroku.com/articles/labs-user-env-compile
By default virtualenv is cached between deploys.
To avoid caching of packages you can run:
heroku config:add BUILDPACK_URL=git#github.com:heroku/heroku-buildpack-python.git#purge
That way everything will be built from scratch after you push some changes. To enable the caching just remove the BUILDPACK_URL config variable.
Now to uninstall specific package(s):
Remove the corresponding record(s) from the requirements.txt;
Commit and push the changes.
Thanks to Lincoln from Heroku Support Team for the clarifications.
I've created some fabfile recipes for Maxime and Jesse answers that allow re-installing the requirements with one fab command: https://gist.github.com/littlepea/5096814 (look at the docstrings for explanation and examples).
For Maxime's answer I've created a task 'heroku_clean' (or 'hc'), it'll look something like that:
fab heroku_clean
Or using an alias and specifying a heroku app:
fab hc:app=myapp
For Jesse's answer I've created a task 'heroku_runtime' (or 'hr'), it sets heroku python runtime and commits runtime.txt (also creates it if it didn't exist):
fab heroku_runtime:2.7.2
If runtime version is not passed it will just toggle it between 2.7.2 and 2.7.3, so the easiest way to change and commit the runtime is:
fab hr
Then you can just deploy (push to heroku origin) you app and the virtualenv will be rebuilt. I've also added a 'heroku_deploy' task ('hr') that I use for heroku push and scale that also can be user together with 'heroku_runtime' task. This is my preferred method of deploying and rebuilding virtualenv - everything happens in one command and I can choose when to rebuild it, I don't like to do it every time like Maxime's answer suggest because it can take a long time:
fab hd:runtime=yes
This is an equivalent of:
fab heroku_runtime
fab heroku_deploy
I have a couple versions of python on my box. My app uses python 2.7 which is found in /usr/local/bin/python. Apache seems to be using an earlier version in /usr/bin/python. How can I configure Apache to use a later version of Python for my app?
To use with Python 3, you need to install the right mod_wsgi for python 3.
On Debian or Ubuntu : sudo apt-get install libapache2-mod-wsgi-py3.
For older versions of Python, when installing mod_wsgi, type : ./configure --with-python=/usr/local/bin/python2.5 for version 2.5 for instance.
You can't. Either rebuild mod_wsgi against the other version of Python, or change the shebang line in your CGI scripts to the other executable.
I could be mistaking, but whatever user apache is started with has a profile which likely has the search path in it. You could change the search path to search /usr/local/bin before /usr/bin, though this is not an ideal approach. Whatever default shell is set for the account used by Apache, just check that you have an rc-file for that shell, and if not, create one.
On Dreamhost, I did the following
installed Python 2.7.1 in my ~/opt directory,
added export PATH=$HOME/opt/bin/:$PATH in my .bash_profile
verified with python --version that 2.7.1 was default
installed setuptools 0.6c11-py2.7
installed Django 1.2.4 with /path/to/python2.7 setup.py install
verified version 1.2.4 with import django and django.VERSION in python shell
wget http://wiki.dreamhost.com/django-setup.py
python2.7 django-setup.py in my site directory ("site.com")
intentionally put a syntax error in ~/site.com/project/urls.py
visited the site in web browser and the error page still says I'm using Python 2.5.2
Why does my Django refuse to use my new version of Python?
According to this, installing-django-with-python-2-5-and-not-with-the-default-version-of-python, the problem might be with mod_wsgi (or I think passenger in my case (but I don't actually know if they do the same thing)).
Should I try to compile mod_wsgi, compile passenger_wsgi, install virtualenv, all of the above, none of the above, A and C, B D and E, or something else altogether?
Maybe you can change the WSGIPythonExecutable directive in your mod_wsgi configuration?
To configure Passenger's Python interpretter, you can follow these steps.