Develop Reference

Upgrading to Python 3.5 but uWSGI using Python 2.7

I am currently trying to upgrade our Python 2.7 project to 3.5. I'm using pipenv to create a virtual environment to do this in. Our server has been upgraded to Ubuntu 16.04. In my current pip environment, when running python -V it correctly returns "Python 3.5.2". However, when attempting to start uWSGI I receive the following error:
Traceback (most recent call last):
File "/usr/local/bin/pipenv", line 7, in <module>
from pipenv import cli
ImportError: No module named 'pipenv'
[uWSGI] getting INI configuration from /opt/site/uwsgi.ini
*** Starting uWSGI 2.0.15 (64bit) on [Tue Jul 21 19:47:23 2020] ***
compiled with version: 4.8.4 on 05 October 2017 19:12:33
os: Linux-4.4.0-185-generic #215-Ubuntu SMP Mon Jun 8 21:53:19 UTC 2020
nodename: ip-172-31-15-50
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 4
current working directory: /opt/site/mysite
detected binary path: /usr/local/bin/uwsgi
chdir() to /opt/site/mysite
your processes number limit is 64022
your memory page size is 4096 bytes
*** WARNING: you have enabled harakiri without post buffering. Slow upload could be rejected on post-unbuffered webservers ***
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to UNIX address /tmp/uwsgi.sock fd 3
Python version: 2.7.16 (default, Mar 26 2019, 10:00:46) [GCC 4.8.4]
Set PythonHome to /home/u_admin/.local/share/virtualenvs/site-qq5I0OuW
ImportError: No module named site
I believe the issue is related to this line:
Python version: 2.7.16 (default, Mar 26 2019, 10:00:46) [GCC 4.8.4]
Although I'm unsure as to why uWSGI is still using Python 2.
Any help would be greatly appreciated. Thanks!
EDIT:
Here is my uwsgi.ini:
[uwsgi]
# Django-related settings
# the base directory (full path)
# main Django wsgi file
chdir = /opt/my_project/portal
module = wsgi.portal:application
# Exit with error code if app can't be imported
need-app = true
# process-related settings
# graceful reload
stopsignal = SIGHUP
# Exit when supervisord sends SIGTERM instead of restarting
die-on-term = true
# master
master = true
# maximum number of worker processes
processes = 5
threads = 3
max-requests = 3000
# Give up if a request takes so long that nginx has already timed out
harakiri = 35
# for NewRelic
enable-threads = true
single-interpreter = true
# the socket (use the full path to be safe)
socket = /tmp/uwsgi.sock
stats = /tmp/uwsgi_stats.sock
chown-socket = myproject:myprojecttech
chmod-socket = 664
# At exit, preserve socket for use on the next launch
vacuum = false
plugins-dir = /usr/lib/uwsgi/plugins
plugins = python35
home = /home/u_admin/.local/share/virtualenvs/agportal-qq5I0OuW
EDIT:
After running uwsgi -H path/to/python I receive the following message:
*** Starting uWSGI 2.0.19.1 (64bit) on [Fri Jul 24 19:14:30 2020] ***
compiled with version: 5.4.0 20160609 on 20 July 2020 17:01:06
os: Linux-4.4.0-185-generic #215-Ubuntu SMP Mon Jun 8 21:53:19 UTC 2020
nodename: ip-172-31-15-50
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 4
current working directory: /opt/spensa/site
detected binary path: /home/u_admin/.local/share/virtualenvs/site-qq5I0OuW/bin/uwsgi
*** WARNING: you are running uWSGI without its master process manager ***
your processes number limit is 64022
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
The -s/--socket option is missing and stdin is not a socket.
This did not change the issue's status.

If you have used the default venv or virtualenv to create the virtual environment you're going to use. Then you can specify the path like the following:
uwsgi -H /fullpath/to/virtualenv
Here -H is the shortcut. Documentation on this: uWSGI Options.

I found a file under our supervisord structure in the "conf.d" folder. (/etc/supervisor/confd).
This file started uwsgi using a file titled vuwsgi.py (theoretically for virtual uwsgi, I'm guessing).
After changing this line:
os.execvp('uwsgi', ['uwsgi', '-H', venv_path] + sys.argv[1:])
to
os.execvp('path/to/my/virtualenv/bin/uwsgi', ['path/to/my/virtualenv/bin/uwsgi', '-H', venv_path] + sys.argv[1:])
uWSGI changed from using Python 2.7 to Python 3.5.

django, pyenv, uwsgi - ModuleNotFoundError: No module named 'django'

I have the following vassal configuration in /etc/uwsgi/vassals/gsd.ini:
[uwsgi]
plugins = python
env = DJANGO_SETTINGS_MODULE=%n.settings
virtualenv = /home/toogy/.pyenv/versions/%n
chdir = /home/webapps/%n
module = %n.wsgi:application
master = true
vacuum = true
pidfile = /tmp/uwsgi-%n.pid
socket = /tmp/uwsgi-%n.sock
daemonize = /var/log/uwsgi/%n.log
chmod-socket = 666
uid = toogy
gid = toogy
Here is the uwsgi log I get
Tue Feb 7 10:49:12 2017 - received message 1 from emperor
...gracefully killing workers...
Gracefully killing worker 1 (pid: 31406)...
worker 1 buried after 1 seconds
binary reloading uWSGI...
chdir() to /etc/uwsgi/vassals
closing all non-uwsgi socket fds > 2 (max_fd = 1024)...
found fd 3 mapped to socket 0 (/tmp/uwsgi-gsd.sock)
running /usr/bin/uwsgi
*** has_emperor mode detected (fd: 7) ***
[uWSGI] getting INI configuration from gsd.ini
*** Starting uWSGI 2.0.14 (64bit) on [Tue Feb 7 10:49:13 2017] ***
compiled with version: 6.3.1 20170109 on 18 January 2017 00:35:47
os: Linux-3.14.32-xxxx-grs-ipv6-64 #7 SMP Wed Jan 27 18:05:09 CET 2016
nodename: renard
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 4
current working directory: /etc/uwsgi/vassals
detected binary path: /usr/bin/uwsgi
chdir() to /home/webapps/gsd
your processes number limit is 15700
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 inherited UNIX address /tmp/uwsgi-gsd.sock fd 3
Python version: 3.6.0 (default, Jan 16 2017, 12:12:55) [GCC 6.3.1 20170109]
PEP 405 virtualenv detected: /home/toogy/.pyenv/versions/gsd
Set PythonHome to /home/toogy/.pyenv/versions/gsd
*** Python threads support is disabled. You can enable it with --enable-threads ***
Python main interpreter initialized at 0x39d21f0
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 145536 bytes (142 KB) for 1 cores
*** Operational MODE: single process ***
added /home/webapps/gsd/ to pythonpath.
Traceback (most recent call last):
File "/home/webapps/gsd/gsd/wsgi.py", line 12, in <module>
from django.core.wsgi import get_wsgi_application
ModuleNotFoundError: No module named 'django'
unable to load app 0 (mountpoint='') (callable not found or import error)
*** no app loaded. going in full dynamic mode ***
*** uWSGI is running in multiple interpreter mode ***
gracefully (RE)spawned uWSGI master process (pid: 27844)
spawned uWSGI worker 1 (pid: 32312, cores: 1)
It cannot find django and I have no idea why because uwsgi seems to detect the python environment (in which django is installed).
Also, it says Python version: 3.6.0 while my virtualenv Python version is 3.5.2. I don't know if this is supposed to happen. The system Python version is 3.6.0.
I am using the last versions of the uwsgi and uwsgi-plugins-python Arch Linux official packages.

The problem was that the system-wide python version linked to uwsgi needs to be the same as the one of the virtualenv, which, I think, is a very stupid thing.
EDIT April 2021: I now recommend using gunicorn, which does not have this problem

In my case it was using the system wide uwsgi, I'm working using a virtualenv so if I execute
$ which uwsgi
I got /usr/local/python3.6/bin/uwsgi
As Valentin Iovene suggests you need to use the uwsgi from your virtual environment
My directories structure is something like this:
~/Env
--/app
--/bin
----/....
----/uwsgi <-- This should be the good one
----/...
--/include
--/lib
(The app directory is where my django app resides)
In my case uwsgi file hasn't execution permissions so I only executed:
$ chmod +x ~/Env/bin/uwsgi
Finally under my app directory I executed the uwsgi command as follows:
../bin/uwsgi --http :8000 --module app.wsgi
Now I can see my app working now :)
I'm following this guide: https://uwsgi-docs.readthedocs.io/en/latest/tutorials/Django_and_nginx.html
Next steps are configuring nginx and https...
I know it's a late response but hope this helps and shared what worked for me.

I also found some possible pitfalls I want to share:
check, if virtualenv (which is the same as venv, pyhome and home) is set to that directory, that contains the bin, include, lib, ... directories
check, if the user (uid) can read the files in your project and the libs in virtual environment (this ends in a ModuleNotFoundError instead of a permission error)
use need-app to exit on failures (this helps for debugging and should be default imho)
use strict to avoid typos in config (this should also be default...)
if your test.py runs, try to import modules of your project and your virtual environment, to test if that works. It also helps to add a
import sys
print(sys.path)
You can also copy the printed sys.path, open a python shell and set sys.path to the same value and try to import the desired wsgi module.

look at this at gsd.ini:
virtualenv = /home/toogy/.pyenv/versions/%n
have you install django under this virtualenv?

When I execute:
uwsgi --socket client_book.sock --module myproject.wsgi --chmod-socket=666
in the directory ~/, and I would get the following error:
ModuleNotFoundError: No module named 'myproject.wsgi'
The solution was to run the command one layer deeper in the directory ~/myproject
This way, uwsgi was able to find myproject.wsgi.
This was what fixed it for me.

'import site' failed while trying to run UWSGI + virtualenv

I have installed Python 2.7 on CentOS and created a virtualenv for my project using:
$ virtualenv -p /usr/local/bin/python2.7 venv
I have installed uwsgi with virtualenv deactivated.
I have also installed uwsgi-plugin-python, as I was facing 'unavailable modifier requested:' issue.
nginx config:
upstream django {
server unix:///tmp/mysite.sock; # for a file socket
}
server {
listen 80;
server_name mysite;
charset utf-8;
client_max_body_size 75M; # adjust to taste
location /media {
alias /projects/mysite/media;
}
location /static {
alias /projects/rebus/rebus/static;
}
location / {
uwsgi_pass django;
include /etc/nginx/uwsgi_params;
}
}
uwsgi ini file:
[uwsgi]
chdir = /projects/mysite
module = mysite.wsgi
virtualenv = /projects/mysite/venv
plugin = python
master = true
processes = 2
socket = /tmp/mysite.sock
chmod-socket = 664
vacuum = true
Now when I launch it and try to access the website:
uwsgi --ini mysite_uwsgi.ini
I get the following log:
*** Starting uWSGI 2.0.11.1 (64bit) on [Wed Aug 19 12:01:22 2015] ***
compiled with version: 4.4.7 20120313 (Red Hat 4.4.7-11) on 21 July 2015 15:58:54
os: Linux-2.6.32-042stab084.12 #1 SMP Tue Nov 26 20:18:08 MSK 2013
nodename: vs23.wovz.net
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 8
current working directory: /var/run
detected binary path: /usr/sbin/uwsgi
chdir() to /projects/mysite
your processes number limit is 1024
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to UNIX address /tmp/mysite.sock fd 3
Python version: 2.6.6 (r266:84292, Jul 23 2015, 15:22:56) [GCC 4.4.7 20120313 (Red Hat 4.4.7-11)]
Set PythonHome to /projects/mysite/venv
'import site' failed; use -v for traceback
*** Python threads support is disabled. You can enable it with --enable-threads ***
Python main interpreter initialized at 0x12e7160
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 218304 bytes (213 KB) for 2 cores
*** Operational MODE: preforking ***
Traceback (most recent call last):
File "./mysite/wsgi.py", line 10, in <module>
import os
ImportError: No module named os
unable to load app 0 (mountpoint='') (callable not found or import error)
*** no app loaded. going in full dynamic mode ***
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 1106)
spawned uWSGI worker 1 (pid: 1107, cores: 1)
spawned uWSGI worker 2 (pid: 1108, cores: 1)
--- no python application found, check your startup logs for errors ---
It looks like my virtualenv with Python 2.7 is not recognized and UWSGI fails to load my Django app. How can I fix that?

This might also have to do with which version of Python uwsgi was compiled against. You'll see the same symptom ('import site' failed) on systems where the default Python is 2.7 if you create a virtual environment using --python=python3 but use a uwsgi obtained via pip install uwsgi instead of pip3 install uwsgi.

maybe you didn't install uwsgi module in your virtualenv
solution:
source /projects/mysite/venv/bin/activate
pip install uwsgi
then rerun your uwsgi server and it maybe work

Uwsgi: "Cannot assign requested address"

So I have a django project all set and ready to go, and then I attempted to start it using uwsgi via this command
# uwsgi --http :5000 --module myProject.wsgi
This then returns this error log
** Starting uWSGI 2.0.3 (32bit) on [Sun Apr 13 14:23:55 2014] ***
compiled with version: 4.6.3 on 13 April 2014 14:06:52
os: Linux-2.6.32-042stab078.26 #1 SMP Wed Jun 19 11:05:34 MSK 2013
nodename: vps
machine: i686
clock source: unix
detected number of CPU cores: 1
current working directory: /root/HomeworkTasker
detected binary path: /usr/local/bin/uwsgi
!!! no internal routing support, rebuild with pcre support !!!
uWSGI running as root, you can use --uid/--gid/--chroot options
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
*** WARNING: you are running uWSGI without its master process manager ***
your processes number limit is 191584
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
bind(): Cannot assign requested address [core/socket.c line 759]
After some googling it seems it has been said that this address is already in use, I have determined this is not the case for multiple reasons. The first of these reasons is that if I go into a python shell, I can bind 0.0.0.0:5000 manually with python. The other reason is that if I go netstat, port 5000 is NOT listed. Anything I can do here or am I SOL?

Solution Found:
Make sure your interfaces are correct by running
ifconfig lo up

switch python version on uwsgi

I am running a django app with nginx and uwsgi. It was was working ok but suddenly I do not know what I changed that the app stop working.
The problem resides in that python version from uWsgi is 2.4.3 which is the default version in Linux CentOs and I have installed django, webpy and other python distributions in version 2.7.3 (will upgrade sooner)
I also tried creating a symbolic link from python 2.7 path to just "python" in usr/bin
and did not work.
How can I make uWsgi use python2.7.3(/usr/local/bin/) instead of 2.4.3?
Django ini:
socket = 127.0.0.1:3031
master = true
processes = 4
env = DJANGO_SETTINGS_MODULE=djtest.settings
module = django.core.handlers.wsgi:WSGIHandler()
When I initialize
./uwsgi --ini django.ini
[uWSGI] getting INI configuration from django.ini
*** Starting uWSGI 1.3 (64bit) on [Sat Nov 24 00:27:55 2012] ***
compiled with version: 4.1.2 20080704 (Red Hat 4.1.2-52) on 23 November 2012 22:37:39
os: Linux-2.6.18-194.17.4.el5xen #1 SMP Mon Oct 25 16:36:31 EDT 2010
nodename: v238545.hostingator.net
machine: x86_64
clock source: unix
detected number of CPU cores: 1
current working directory: /root
detected binary path: /opt/uwsgi/uwsgi
your processes number limit is 32832
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
uWSGI http bound on :8000 fd 4
spawned uWSGI http 1 (pid: 5841)
uwsgi socket 0 bound to TCP address 127.0.0.1:59395 (port auto-assigned) fd 3
Python version: 2.4.3 (#1, Jun 18 2012, 08:55:23) [GCC 4.1.2 20080704 (Red Hat 4.1.2-52)]
*** Python threads support is disabled. You can enable it with --enable-threads ***
Python main interpreter initialized at 0x1cc1130
your server socket listen backlog is limited to 100 connections
mapped 72352 bytes (70 KB) for 1 cores
*** Operational MODE: single process ***
added /usr/local/bin/ to pythonpath.
added /home/indaxone/serv/ to pythonpath.
ImportError: No module named django.core.handlers.wsgi
unable to load app 0 (mountpoint='') (callable not found or import error)
*** no app loaded. going in full dynamic mode ***
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI worker 1 (and the only) (pid: 5840, cores: 1)

Before doing the "make" during the uwsgi installation I did the following:
python2.7 uwsgiconfig.py --build