Develop Reference

Upgrading to Python 3.5 but uWSGI using Python 2.7

I am currently trying to upgrade our Python 2.7 project to 3.5. I'm using pipenv to create a virtual environment to do this in. Our server has been upgraded to Ubuntu 16.04. In my current pip environment, when running python -V it correctly returns "Python 3.5.2". However, when attempting to start uWSGI I receive the following error:
Traceback (most recent call last):
File "/usr/local/bin/pipenv", line 7, in <module>
from pipenv import cli
ImportError: No module named 'pipenv'
[uWSGI] getting INI configuration from /opt/site/uwsgi.ini
*** Starting uWSGI 2.0.15 (64bit) on [Tue Jul 21 19:47:23 2020] ***
compiled with version: 4.8.4 on 05 October 2017 19:12:33
os: Linux-4.4.0-185-generic #215-Ubuntu SMP Mon Jun 8 21:53:19 UTC 2020
nodename: ip-172-31-15-50
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 4
current working directory: /opt/site/mysite
detected binary path: /usr/local/bin/uwsgi
chdir() to /opt/site/mysite
your processes number limit is 64022
your memory page size is 4096 bytes
*** WARNING: you have enabled harakiri without post buffering. Slow upload could be rejected on post-unbuffered webservers ***
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to UNIX address /tmp/uwsgi.sock fd 3
Python version: 2.7.16 (default, Mar 26 2019, 10:00:46) [GCC 4.8.4]
Set PythonHome to /home/u_admin/.local/share/virtualenvs/site-qq5I0OuW
ImportError: No module named site
I believe the issue is related to this line:
Python version: 2.7.16 (default, Mar 26 2019, 10:00:46) [GCC 4.8.4]
Although I'm unsure as to why uWSGI is still using Python 2.
Any help would be greatly appreciated. Thanks!
EDIT:
Here is my uwsgi.ini:
[uwsgi]
# Django-related settings
# the base directory (full path)
# main Django wsgi file
chdir = /opt/my_project/portal
module = wsgi.portal:application
# Exit with error code if app can't be imported
need-app = true
# process-related settings
# graceful reload
stopsignal = SIGHUP
# Exit when supervisord sends SIGTERM instead of restarting
die-on-term = true
# master
master = true
# maximum number of worker processes
processes = 5
threads = 3
max-requests = 3000
# Give up if a request takes so long that nginx has already timed out
harakiri = 35
# for NewRelic
enable-threads = true
single-interpreter = true
# the socket (use the full path to be safe)
socket = /tmp/uwsgi.sock
stats = /tmp/uwsgi_stats.sock
chown-socket = myproject:myprojecttech
chmod-socket = 664
# At exit, preserve socket for use on the next launch
vacuum = false
plugins-dir = /usr/lib/uwsgi/plugins
plugins = python35
home = /home/u_admin/.local/share/virtualenvs/agportal-qq5I0OuW
EDIT:
After running uwsgi -H path/to/python I receive the following message:
*** Starting uWSGI 2.0.19.1 (64bit) on [Fri Jul 24 19:14:30 2020] ***
compiled with version: 5.4.0 20160609 on 20 July 2020 17:01:06
os: Linux-4.4.0-185-generic #215-Ubuntu SMP Mon Jun 8 21:53:19 UTC 2020
nodename: ip-172-31-15-50
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 4
current working directory: /opt/spensa/site
detected binary path: /home/u_admin/.local/share/virtualenvs/site-qq5I0OuW/bin/uwsgi
*** WARNING: you are running uWSGI without its master process manager ***
your processes number limit is 64022
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
The -s/--socket option is missing and stdin is not a socket.
This did not change the issue's status.

If you have used the default venv or virtualenv to create the virtual environment you're going to use. Then you can specify the path like the following:
uwsgi -H /fullpath/to/virtualenv
Here -H is the shortcut. Documentation on this: uWSGI Options.

I found a file under our supervisord structure in the "conf.d" folder. (/etc/supervisor/confd).
This file started uwsgi using a file titled vuwsgi.py (theoretically for virtual uwsgi, I'm guessing).
After changing this line:
os.execvp('uwsgi', ['uwsgi', '-H', venv_path] + sys.argv[1:])
to
os.execvp('path/to/my/virtualenv/bin/uwsgi', ['path/to/my/virtualenv/bin/uwsgi', '-H', venv_path] + sys.argv[1:])
uWSGI changed from using Python 2.7 to Python 3.5.

Attempting to deploy a python pyramid application with uWSGI

I'm attempting to deploy a pyramid application using uWSGI.
The application works fine when served with the included pyramid development server.
Also, I have set this up before, and I swear it worked at one time.
However, putting in the magic phrases right now is resulting in "This webpage is not available"
I'm trying to keep all of the configuration parameters as similar as possible to what I have currently so I don't have to worry about firewall issues.
uWSGI section in development.ini looks like this (from: Setup uWSGI as webserver with pyramid (no NGINX)):
[uwsgi]
socket = localhost:8080
virtualenv = /var/www/finance/finance-env
die-on-term = 1
master = 1
#logto = /var/log/wsgi/uwsgi.log
enable-threads = true
offload-threads = N
py-autoreload = 1
wsgi-file = /var/www/finance/wsgi.py
wsgy.py looks like this:
from pyramid.paster import get_app, setup_logging
ini_path = '/var/www/finance/corefinance/development.ini'
setup_logging(ini_path)
application = get_app(ini_path, 'main')
Here's the output right now. Everything seems to be listening just fine on port 8080.
user1#finance1:~$ sudo /var/www/finance/finance-env/bin/uwsgi --ini-paste-logg /var/www/finance/corefinance/development.ini
[uWSGI] getting INI configuration from /var/www/finance/corefinance/development.ini
*** Starting uWSGI 2.0.11.2 (64bit) on [Fri Jan 15 21:13:31 2016] ***
compiled with version: 4.7.2 on 16 November 2015 20:13:35
os: Linux-4.1.5-x86_64-linode61 #7 SMP Mon Aug 24 13:46:31 EDT 2015
nodename: finance1
machine: x86_64
clock source: unix
detected number of CPU cores: 1
current working directory: /home/user1
detected binary path: /var/www/finance/finance-env/bin/uwsgi
!!! no internal routing support, rebuild with pcre support !!!
uWSGI running as root, you can use --uid/--gid/--chroot options
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
your processes number limit is 3934
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to TCP address localhost:8080 fd 3
Python version: 3.2.3 (default, Feb 20 2013, 14:49:46) [GCC 4.7.2]
Set PythonHome to /var/www/finance/finance-env
Python main interpreter initialized at 0xfd0a10
python threads support enabled
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 145536 bytes (142 KB) for 1 cores
*** Operational MODE: single process ***
WSGI app 0 (mountpoint='') ready in 1 seconds on interpreter 0xfd0a10 pid: 6275 (default app)
mountpoint already configured. skip.
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 6275)
spawned uWSGI worker 1 (pid: 6282, cores: 1)
Python auto-reloader enabled

Unless you are behind a proxy such as nginx, you need to use the internal http routing support in uwsgi, change
socket = localhost:8080
to
http = 0.0.0.0:8080
Here are the uwsgi http support docs

uwsgi Exception: not correct domain name /

I have some application written in python.
I'm trying to run python application under uwsgi.
My app.ini:
[uwsgi]
chdir = /var/www/myapp
wsgi-file = /var/www/myapp/app.wsgi
uid = apache
gid = apache
master = true
processes = 1
socket = :8080
vacuum = true
Start uwsgi:
uwsgi --ini app.ini
[uWSGI] getting INI configuration from app.ini
*** Starting uWSGI 2.0.10 (64bit) on [Sat Mar 21 16:59:09 2015] ***
compiled with version: 4.8.2 20140120 (Red Hat 4.8.2-16) on 21 March 2015 16:55:47
os: Linux-3.10.0-123.el7.x86_64 #1 SMP Mon Jun 30 12:09:22 UTC 2014
nodename: lw_site
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 8
current working directory: /var/www/myapp
detected binary path: /usr/bin/uwsgi
chdir() to /var/www/myapp
your processes number limit is 4096
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to TCP address :8080 fd 3
Python version: 2.7.5 (default, Jun 17 2014, 18:11:42) [GCC 4.8.2 20140120 (Red Hat 4.8.2-16)]
*** Python threads support is disabled. You can enable it with --enable-threads ***
Python main interpreter initialized at 0x2644f70
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 145520 bytes (142 KB) for 1 cores
*** Operational MODE: single process ***
WSGI app 0 (mountpoint='') ready in 0 seconds on interpreter 0x2644f70 pid: 39838 (default app)
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 39838)
spawned uWSGI worker 1 (pid: 39839, cores: 1)
This is
WSGI app 0 (mountpoint='')
confuse me.
I get this error:
Custom Error: Traceback (most recent call last):
File "/var/www/myapp/app/manager.py", line 53, in app
contrller, args = router.mapping(request, city_model)
File "/var/www/myapp/app/router.py", line 478, in mapping
raise Exception('not correct domain name '+str(rurl))
Exception: not correct domain name /
Where is I made a mistake ?

I have found the error. It's problem in code. It get domain name from SERVER_NAME variable. I set in nginx:
uwsgi_param SERVER_NAME $host;

Use virtualenv owned by root in a python web project on production

So I have the following configuration:
Nginx server with uWsgi
Django project in a virtualenv
My virtualenv is owned by an ordinal system user and the whole configuration works fine!
The uwsgi process is configured to run with nginx uid & gid, as well as the nginx process owner in the nginx.conf.
But, if I try to use another virtualenv satisfying the same requirements, but owned by root, my uwsgi can't locate the virtualenv, throwing an import error.
Is this normal and is my aproach correct to use a root owned virtualenv?
Edit:
uwsgi config:
[uwsgi]
virtualenv = /root/Envs/rootenv
thread = 4
uid = nginx
gid = nginx
env = DJANGO_SETTINGS_MODULE=myServer.settings
module = django.core.handlers.wsgi:WSGIHandler()
chdir = /var/www/myServer
socket = /var/run/uwsgi/myserver.sock
logto = /var/log/uwsgi/myserver.log
/root/.bashrc:
### Virtualenv settings ###
export WORKON_HOME=/root/Envs
source /usr/local/bin/virtualenvwrapper.sh
export PIP_VIRTUALENV_BASE=$WORKON_HOME
export PIP_RESPECT_VIRTUALENV=true
uwsgi log traceback:
*** Starting uWSGI 1.9.15 (64bit) on [Fri Sep 6 14:02:41 2013] ***
compiled with version: 4.7.2 on 06 September 2013 10:59:26
os: Linux-3.2.0-4-amd64 #1 SMP Debian 3.2.41-2+deb7u2
nodename: myserver3
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 2
current working directory: /
detected binary path: /usr/local/bin/uwsgi
setgid() to 32
set additional group 4 (adm)
setuid() to 32
*** WARNING: you are running uWSGI without its master process manager ***
your processes number limit is 7929
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
uwsgi socket 0 bound to UNIX address /var/run/uwsgi/myserver.sock fd 3
Python version: 2.7.3 (default, Jan 2 2013, 14:09:21) [GCC 4.7.2]
Set PythonHome to /root/Envs/rootenv
ImportError: No module named site
*** Starting uWSGI 1.9.15 (64bit) on [Fri Sep 6 14:03:13 2013] ***
compiled with version: 4.7.2 on 06 September 2013 10:59:26
os: Linux-3.2.0-4-amd64 #1 SMP Debian 3.2.41-2+deb7u2
nodename: myserver3
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 2
current working directory: /
detected binary path: /usr/local/bin/uwsgi
setgid() to 32
set additional group 4 (adm)
setuid() to 32
*** WARNING: you are running uWSGI without its master process manager ***
your processes number limit is 7929
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
uwsgi socket 0 bound to UNIX address /var/run/uwsgi/myserver.sock fd 3
Python version: 2.7.3 (default, Jan 2 2013, 14:09:21) [GCC 4.7.2]
Set PythonHome to /root/Envs/rootenv
ImportError: No module named site

switch python version on uwsgi

I am running a django app with nginx and uwsgi. It was was working ok but suddenly I do not know what I changed that the app stop working.
The problem resides in that python version from uWsgi is 2.4.3 which is the default version in Linux CentOs and I have installed django, webpy and other python distributions in version 2.7.3 (will upgrade sooner)
I also tried creating a symbolic link from python 2.7 path to just "python" in usr/bin
and did not work.
How can I make uWsgi use python2.7.3(/usr/local/bin/) instead of 2.4.3?
Django ini:
socket = 127.0.0.1:3031
master = true
processes = 4
env = DJANGO_SETTINGS_MODULE=djtest.settings
module = django.core.handlers.wsgi:WSGIHandler()
When I initialize
./uwsgi --ini django.ini
[uWSGI] getting INI configuration from django.ini
*** Starting uWSGI 1.3 (64bit) on [Sat Nov 24 00:27:55 2012] ***
compiled with version: 4.1.2 20080704 (Red Hat 4.1.2-52) on 23 November 2012 22:37:39
os: Linux-2.6.18-194.17.4.el5xen #1 SMP Mon Oct 25 16:36:31 EDT 2010
nodename: v238545.hostingator.net
machine: x86_64
clock source: unix
detected number of CPU cores: 1
current working directory: /root
detected binary path: /opt/uwsgi/uwsgi
your processes number limit is 32832
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
uWSGI http bound on :8000 fd 4
spawned uWSGI http 1 (pid: 5841)
uwsgi socket 0 bound to TCP address 127.0.0.1:59395 (port auto-assigned) fd 3
Python version: 2.4.3 (#1, Jun 18 2012, 08:55:23) [GCC 4.1.2 20080704 (Red Hat 4.1.2-52)]
*** Python threads support is disabled. You can enable it with --enable-threads ***
Python main interpreter initialized at 0x1cc1130
your server socket listen backlog is limited to 100 connections
mapped 72352 bytes (70 KB) for 1 cores
*** Operational MODE: single process ***
added /usr/local/bin/ to pythonpath.
added /home/indaxone/serv/ to pythonpath.
ImportError: No module named django.core.handlers.wsgi
unable to load app 0 (mountpoint='') (callable not found or import error)
*** no app loaded. going in full dynamic mode ***
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI worker 1 (and the only) (pid: 5840, cores: 1)

Before doing the "make" during the uwsgi installation I did the following:
python2.7 uwsgiconfig.py --build