Develop Reference

Attempting to deploy a python pyramid application with uWSGI

I'm attempting to deploy a pyramid application using uWSGI.
The application works fine when served with the included pyramid development server.
Also, I have set this up before, and I swear it worked at one time.
However, putting in the magic phrases right now is resulting in "This webpage is not available"
I'm trying to keep all of the configuration parameters as similar as possible to what I have currently so I don't have to worry about firewall issues.
uWSGI section in development.ini looks like this (from: Setup uWSGI as webserver with pyramid (no NGINX)):
[uwsgi]
socket = localhost:8080
virtualenv = /var/www/finance/finance-env
die-on-term = 1
master = 1
#logto = /var/log/wsgi/uwsgi.log
enable-threads = true
offload-threads = N
py-autoreload = 1
wsgi-file = /var/www/finance/wsgi.py
wsgy.py looks like this:
from pyramid.paster import get_app, setup_logging
ini_path = '/var/www/finance/corefinance/development.ini'
setup_logging(ini_path)
application = get_app(ini_path, 'main')
Here's the output right now. Everything seems to be listening just fine on port 8080.
user1#finance1:~$ sudo /var/www/finance/finance-env/bin/uwsgi --ini-paste-logg /var/www/finance/corefinance/development.ini
[uWSGI] getting INI configuration from /var/www/finance/corefinance/development.ini
*** Starting uWSGI 2.0.11.2 (64bit) on [Fri Jan 15 21:13:31 2016] ***
compiled with version: 4.7.2 on 16 November 2015 20:13:35
os: Linux-4.1.5-x86_64-linode61 #7 SMP Mon Aug 24 13:46:31 EDT 2015
nodename: finance1
machine: x86_64
clock source: unix
detected number of CPU cores: 1
current working directory: /home/user1
detected binary path: /var/www/finance/finance-env/bin/uwsgi
!!! no internal routing support, rebuild with pcre support !!!
uWSGI running as root, you can use --uid/--gid/--chroot options
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
your processes number limit is 3934
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to TCP address localhost:8080 fd 3
Python version: 3.2.3 (default, Feb 20 2013, 14:49:46) [GCC 4.7.2]
Set PythonHome to /var/www/finance/finance-env
Python main interpreter initialized at 0xfd0a10
python threads support enabled
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 145536 bytes (142 KB) for 1 cores
*** Operational MODE: single process ***
WSGI app 0 (mountpoint='') ready in 1 seconds on interpreter 0xfd0a10 pid: 6275 (default app)
mountpoint already configured. skip.
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 6275)
spawned uWSGI worker 1 (pid: 6282, cores: 1)
Python auto-reloader enabled

Unless you are behind a proxy such as nginx, you need to use the internal http routing support in uwsgi, change
socket = localhost:8080
to
http = 0.0.0.0:8080
Here are the uwsgi http support docs

'import site' failed while trying to run UWSGI + virtualenv

I have installed Python 2.7 on CentOS and created a virtualenv for my project using:
$ virtualenv -p /usr/local/bin/python2.7 venv
I have installed uwsgi with virtualenv deactivated.
I have also installed uwsgi-plugin-python, as I was facing 'unavailable modifier requested:' issue.
nginx config:
upstream django {
server unix:///tmp/mysite.sock; # for a file socket
}
server {
listen 80;
server_name mysite;
charset utf-8;
client_max_body_size 75M; # adjust to taste
location /media {
alias /projects/mysite/media;
}
location /static {
alias /projects/rebus/rebus/static;
}
location / {
uwsgi_pass django;
include /etc/nginx/uwsgi_params;
}
}
uwsgi ini file:
[uwsgi]
chdir = /projects/mysite
module = mysite.wsgi
virtualenv = /projects/mysite/venv
plugin = python
master = true
processes = 2
socket = /tmp/mysite.sock
chmod-socket = 664
vacuum = true
Now when I launch it and try to access the website:
uwsgi --ini mysite_uwsgi.ini
I get the following log:
*** Starting uWSGI 2.0.11.1 (64bit) on [Wed Aug 19 12:01:22 2015] ***
compiled with version: 4.4.7 20120313 (Red Hat 4.4.7-11) on 21 July 2015 15:58:54
os: Linux-2.6.32-042stab084.12 #1 SMP Tue Nov 26 20:18:08 MSK 2013
nodename: vs23.wovz.net
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 8
current working directory: /var/run
detected binary path: /usr/sbin/uwsgi
chdir() to /projects/mysite
your processes number limit is 1024
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to UNIX address /tmp/mysite.sock fd 3
Python version: 2.6.6 (r266:84292, Jul 23 2015, 15:22:56) [GCC 4.4.7 20120313 (Red Hat 4.4.7-11)]
Set PythonHome to /projects/mysite/venv
'import site' failed; use -v for traceback
*** Python threads support is disabled. You can enable it with --enable-threads ***
Python main interpreter initialized at 0x12e7160
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 218304 bytes (213 KB) for 2 cores
*** Operational MODE: preforking ***
Traceback (most recent call last):
File "./mysite/wsgi.py", line 10, in <module>
import os
ImportError: No module named os
unable to load app 0 (mountpoint='') (callable not found or import error)
*** no app loaded. going in full dynamic mode ***
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 1106)
spawned uWSGI worker 1 (pid: 1107, cores: 1)
spawned uWSGI worker 2 (pid: 1108, cores: 1)
--- no python application found, check your startup logs for errors ---
It looks like my virtualenv with Python 2.7 is not recognized and UWSGI fails to load my Django app. How can I fix that?

This might also have to do with which version of Python uwsgi was compiled against. You'll see the same symptom ('import site' failed) on systems where the default Python is 2.7 if you create a virtual environment using --python=python3 but use a uwsgi obtained via pip install uwsgi instead of pip3 install uwsgi.

maybe you didn't install uwsgi module in your virtualenv
solution:
source /projects/mysite/venv/bin/activate
pip install uwsgi
then rerun your uwsgi server and it maybe work

uWSGI Emperor configuration

I have been struggling to deploy a flask app to my server using uWSGI in
Emperor mode all day and am at a loss why it is not working. I have read all the other uWSGI Emperor mode questions here & asked on #uwsgi & the mailing list with no reply.
My setup is nginx (1.4.6-1ubuntu3.2), uWSGI==2.0.10 installed via pip
into a virtualenv, Flask==0.10.1, and Python 3.4.0. supervisord is
responsible for launching the Emperor process.
The flask app itself works fine when I start it manually bound to a TCP port, and when I manually run uwsgi like:
uwsgi -s /tmp/oauthsvc.sock -w wsgi --stats /tmp/oauthsvc-stats.sock
I do notice that even though I think I configured the vassal to use a
unix socket, it reports on startup that it is listening to http://127.0.0.1:5000/
I verified with strace that nginx is opening the socket, and permissions
of that socket are 777:
srwxrwxrwx 1 oauthsvc oauthsvc 0 Jun 3 18:34 /tmp/oauthsvc.sock
srwxrwxrwx 1 oauthsvc oauthsvc 0 Jun 3 18:08 /tmp/oauthsvc-stats.sock
But the connection eventually just times out.
My configs & some log output follows.
The nginx config is as simple as can be:
server {
listen 80;
server_name default;
location = /robots.txt { access_log off; log_not_found off; }
location = /favicon.ico { access_log off; log_not_found off; }
location / {
include uwsgi_params;
uwsgi_pass unix:/tmp/oauthsvc.sock;
}
}
supervisor launches the emporer process with this command:
[program:oauthsvc-emperor]
command=/services/oauthsvc/services/oauthsvc/bin/uwsgi --emperor /services/oauthsvc/etc/uwsgi/vassals --die-on-term --uid 1005 --gid 1005 --logto /services/oauthsvc/var/log/uwsgi/emperor.log
user=oauthsvc
autostart=true
autorestart=true
redirect_stderr=true
The vassal's ini file looks like:
[uwsgi]
# do not turn on master mode when spawned by emperor
# http://stackoverflow.com/questions/15055002/uwsgi-master-with-emperor-spawns-two-emperors
#master = true
protocol = uwsgi
socket = /tmp/oauthsvc.sock
stats = /tmp/oauthsvc-stats.sock
daemonize = /services/oauthsvc/var/log/uwsgi/oauthsvc.log
chdir = /services/oauthsvc/services/oauthsvc/oauthsvc-server
wsgi-file = wsgi.py
callable = app
chmod-socket = 777
uid = 1005
gid = 1005
virtualenv = /services/oauthsvc/services/oauthsvc
# PYTHONHOME
home = /services/oauthsvc/services/oauthsvc
#env =
processes = 4
vacuum = true
# If you start uWSGI without threads, the Python GIL will not be enabled, so
# threads generated by your application will never run unless you include:
enable-threads = true
harakiri = 30
And finally, wsgi.py:
#!/usr/bin/env python3
from oauthsvc.flask.server import parse_args, configured_app
args = parse_args()
app = configured_app(args.config, debug=args.debug)
app.run()
The emperor log:
*** Starting uWSGI 2.0.10 (64bit) on [Wed Jun 3 18:34:25 2015] ***
compiled with version: 4.8.2 on 03 June 2015 18:32:12
os: Linux-3.13.0-48-generic #80-Ubuntu SMP Thu Mar 12 11:16:15 UTC 2015
nodename: ip-10-0-1-224
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 1
current working directory: /
detected binary path: /services/oauthsvc/src/bs-oauth-svc.2015-06-03.183113/bin/uwsgi
*** WARNING: you are running uWSGI without its master process manager ***
your processes number limit is 30038
your memory page size is 4096 bytes
detected max file descriptor number: 1024
*** starting uWSGI Emperor ***
*** has_emperor mode detected (fd: 6) ***
[uWSGI] getting INI configuration from oauthsvc.ini
The vassal log:
*** Starting uWSGI 2.0.10 (64bit) on [Wed Jun 3 18:34:25 2015] ***
compiled with version: 4.8.2 on 03 June 2015 18:32:12
os: Linux-3.13.0-48-generic #80-Ubuntu SMP Thu Mar 12 11:16:15 UTC 2015
nodename: ip-10-0-1-224
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 1
current working directory: /services/oauthsvc/etc/uwsgi/vassals
detected binary path: /services/oauthsvc/src/bs-oauth-svc.2015-06-03.183113/bin/uwsgi
chdir() to /services/oauthsvc/services/oauthsvc/oauthsvc-server
your processes number limit is 30038
your memory page size is 4096 bytes
*** WARNING: you have enabled harakiri without post buffering. Slow upload could be rejected on post-unbuffered webservers ***
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to UNIX address /tmp/oauthsvc.sock fd 3
Python version: 3.4.0 (default, Apr 11 2014, 13:08:40) [GCC 4.8.2]
Set PythonHome to /services/oauthsvc/services/oauthsvc
Python main interpreter initialized at 0x135d4d0
python threads support enabled
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 363840 bytes (355 KB) for 4 cores
*** Operational MODE: preforking ***
* Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)

Brian on the uWSGI mailing list solved this for me. The answer is: don't call app.run() in the wsgi.py file! I knew it would be something embarrassing.

How to get uwsgi working with nginx and Django 1.7?

This is what I tried:
I have a django.conf inside /etc/nginx/sites-available/
upstream django {
server unix:///tmp/uwsgi.sock;
}
server {
listen 80;
server_name weasyprint.django.dev;
charset utf-8;
error_log /var/log/nginx/django-weasyprint.log;
location / {
uwsgi_pass django;
include /etc/nginx/uwsgi_params;
}
}
And then I do a sudo ln -s /etc/nginx/sites-available/django.conf inside sites-enabled
And then I restarted nginx.
I created a django folder called weasyprint_site inside /var/virtual/WebApps/virtualenvs/WeasyPrintProject
I used virtualenv on the same folder, so now my structure is like this:
WeasyPrintProject
|---------bin
|---------include
|---------lib
|---------local
|---------share
|---------weasyprint_site
|------------db.sqlite3
|------------manage.py
|------------test.py
|------------uwsgi.ini
|------------weasyprint_site
and then I also put in a uwsgi.ini as you can see above.
Its contents are:
[uwsgi]
socket=/tmp/uwsgi.sock
chmod-socket=666
uid = www-data
gid = www-data
chdir=/var/virtual/WebApps/virtualenvs/WeasyPrintProject
module=weasy_print.wsgi:application
master=true
pidfile=/var/virtual/WebApps/virtualenvs/WeasyPrintProject/weasy_print.pid
vacuum=true
When I am at /var/virtual/WebApps/virtualenvs/WeasyPrintProject/weasyprint_site,
I run
uwsgi --ini uwsgi.ini
I get the following:
[uWSGI] getting INI configuration from uwsgi.ini
*** Starting uWSGI 2.0.9 (64bit) on [Thu Feb 19 11:59:12 2015] ***
compiled with version: 4.8.2 on 16 February 2015 05:39:16
os: Linux-3.13.0-43-generic #72-Ubuntu SMP Mon Dec 8 19:35:06 UTC 2014
nodename: vagrant-ubuntu-trusty-64
machine: x86_64
clock source: unix
detected number of CPU cores: 1
current working directory: /var/virtual/WebApps/virtualenvs/WeasyPrintProject/weasyprint_site
writing pidfile to /var/virtual/WebApps/virtualenvs/WeasyPrintProject/weasy_print.pid
detected binary path: /var/virtual/WebApps/virtualenvs/WeasyPrintProject/bin/uwsgi
!!! no internal routing support, rebuild with pcre support !!!
chdir() to /var/virtual/WebApps/virtualenvs/WeasyPrintProject
your processes number limit is 15934
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to UNIX address /tmp/uwsgi.sock fd 3
Python version: 2.7.6 (default, Mar 22 2014, 23:03:41) [GCC 4.8.2]
*** Python threads support is disabled. You can enable it with --enable-threads ***
Python main interpreter initialized at 0x1915160
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 145536 bytes (142 KB) for 1 cores
*** Operational MODE: single process ***
ImportError: No module named weasy_print.wsgi
unable to load app 0 (mountpoint='') (callable not found or import error)
*** no app loaded. going in full dynamic mode ***
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI master process (pid: 1913)
spawned uWSGI worker 1 (pid: 1914, cores: 1)
I have 2 issues:
I cannot get back to bash unless I do a Ctrl+C
there is an import error saying no module named weasy_print.wsgi
I followed the instructions in here
My virtualenv for this particular project is to use python 2.7.4 and I am attempting to run django 1.7. The environment is Ubuntu 14.04
How do I get the django app to work.

Firstly, you're chdiring into WeasyPrintProject, but your project is in WeasyPrintProject/weasyprint_site. You should chdir into that directory inside your uwsgi.ini file.
Secondly, your project name is weasyprint_site, not weasy_print, so you should call module weasyprint_site.wsgi:application instead.
And last problem is: you should specify path to your virtualenv inside uwsgi.ini, so uwsgi process can now where are additional packages required by your application.
To return to your console without interrupting your uwsgi server, you must put it in background, by forking it, daemonizing it or simply launching from init script. I personally recomment using emperor/vassals system built in into wsgi.
Also having your project inside virtualenv directory is not recommended.

Flask, nginx, and uwsgi

My flask app looks like this...
myapp.py
from flask import Flask
app = Flask(__name__)
#app.route("/")
def hello():
return "Hello World!"
if __name__ == "__main__":
app.run('0.0.0.0')
My nginx setup
server {
root /home/admin.jeremylspencer.com;
server_name admin.jeremylspencer.com;
location / { try_files $uri #yourapplication; }
location #yourapplication {
include uwsgi_params;
uwsgi_pass unix:/tmp/uwsgi.sock;
}
#error_page 404 /404.html;
#error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /usr/share/nginx/www;
}
location ~ /\.ht {
allow all;
}
}
Then finally I restart nginx and run this:
sudo uwsgi -s /tmp/uwsgi.sock --module myapp --callable app
And this is the output
*** Starting uWSGI 1.4.3 (64bit) on [Mon Dec 10 15:41:00 2012] ***
compiled with version: 4.6.3 on 10 December 2012 13:06:15
os: Linux-3.2.0-32-generic #51-Ubuntu SMP Wed Sep 26 21:33:09 UTC 2012
nodename: jeremylspencer.com
machine: x86_64
clock source: unix
detected number of CPU cores: 2
current working directory: /home/admin.jeremylspencer.com
detected binary path: /usr/local/bin/uwsgi
uWSGI running as root, you can use --uid/--gid/--chroot options
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
*** WARNING: you are running uWSGI without its master process manager ***
your processes number limit is 31285
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
uwsgi socket 0 bound to UNIX address /tmp/uwsgi.sock fd 3
Python version: 2.7.3 (default, Aug 1 2012, 05:25:23) [GCC 4.6.3]
*** Python threads support is disabled. You can enable it with --enable-threads ***
Python main interpreter initialized at 0x1dfa790
your server socket listen backlog is limited to 100 connections
mapped 72392 bytes (70 KB) for 1 cores
*** Operational MODE: single process ***
WSGI app 0 (mountpoint='') ready in 0 seconds on interpreter 0x1dfa790 pid: 13645 (default app)
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI worker 1 (and the only) (pid: 13645, cores: 1)
But yet all I get is a 502 error... any how can i fix this?

unix sockets are filesystem objects, so nginx need write permissions to /tmp/uwsgi.sock
You are running uWSGI as root (why ???) so /tmp/uwsgi.sock will be owned by root while nginx generally runs as nobody or www-data.
If you do not want to take in account permissions just use TCP sockets, but obviously do not run your app as root.

You can have a look on this: Python flask with Nginx and uWSGI
as the repo provides the exact case / scenario on how to use Flask in production mode.