Develop Reference

nginx + gunicorn + django not working

I am taking the help of following docs http://tutos.readthedocs.io/en/latest/source/ndg.html and trying to run the server. I have following file in sites-enabled :
upstream test_server {
server unix:/var/www/test/run/gunicorn.sock fail_timeout=10s;
}
# This is not neccessary - it's just commonly used
# it just redirects example.com -> www.example.com
# so it isn't treated as two separate websites
server {
listen 80;
server_name dehatiengineers.in;
return 301 $scheme://www.dehatiengineers.in$request_uri;
}
server {
listen 80;
server_name www.dehatiengineer.in;
client_max_body_size 4G;
access_log /var/www/test/logs/nginx-access.log;
error_log /var/www/test/logs/nginx-error.log warn;
location /static/ {
autoindex on;
alias /var/www/test/ourcase/static/;
}
location /media/ {
autoindex on;
alias /var/www/test/ourcase/media/;
}
location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
if (!-f $request_filename) {
proxy_pass http://test_server;
break;
}
}
#For favicon
location /favicon.ico {
alias /var/www/test/test/static/img/favicon.ico;
}
#For robots.txt
location /robots.txt {
alias /var/www/test/test/static/robots.txt ;
}
# Error pages
error_page 500 502 503 504 /500.html;
location = /500.html {
root /var/www/test/ourcase/static/;
}
}
My server's domain name is dehatiengineers.in.
The gunicorn_start.sh file is:
#!/bin/bash
NAME="ourcase" #Name of the application (*)
DJANGODIR=/var/www/test/ourcase # Django project directory (*)
SOCKFILE=/var/www/test/run/gunicorn.sock # we will communicate using this unix socket (*)
USER=root # the user to run as (*)
GROUP=webdata # the group to run as (*)
NUM_WORKERS=1 # how many worker processes should Gunicorn spawn (*)
DJANGO_SETTINGS_MODULE=ourcase.settings # which settings file should Django use (*)
DJANGO_WSGI_MODULE=ourcase.wsgi # WSGI module name (*)
echo "Starting $NAME as `whoami`"
# Activate the virtual environment
cd $DJANGODIR
source /var/www/test/venv/bin/activate
export DJANGO_SETTINGS_MODULE=$DJANGO_SETTINGS_MODULE
export PYTHONPATH=$DJANGODIR:$PYTHONPATH
# Create the run directory if it doesn't exist
RUNDIR=$(dirname $SOCKFILE)
test -d $RUNDIR || mkdir -p $RUNDIR
# Start your Django Unicorn
# Programs meant to be run under supervisor should not daemonize themselves (do not use --daemon)
exec /var/www/test/venv/bin/gunicorn ${DJANGO_WSGI_MODULE}:application \
--name $NAME \
--workers $NUM_WORKERS \
--user $USER \
--bind=unix:$SOCKFILE
Now, I am running both of my nginx and gunicorn_start.sh file. nginx is running, as you can see at http://www.dehatiengineers.in/ but the django is not connected to nginx. Now on sudo sh gunicorn_start.sh, I am getting following output:
Starting ourcase as root
gunicorn_start.sh: 16: gunicorn_start.sh: source: not found
[2016-05-20 06:21:45 +0000] [10730] [INFO] Starting gunicorn 19.5.0
[2016-05-20 06:21:45 +0000] [10730] [INFO] Listening at: unix:/var/www/test/run/gunicorn.sock (10730)
[2016-05-20 06:21:45 +0000] [10730] [INFO] Using worker: sync
[2016-05-20 06:21:45 +0000] [10735] [INFO] Booting worker with pid: 10735
But, in browser, I am just getting nginx output, not the original one.
Apart from this, I am having issue in running gunicorn_ourcase.service file:
[Unit]
Description=Ourcase gunicorn daemon
[Service]
Type=simple
User=root
ExecStart=/var/www/test/gunicorn_start.sh
[Install]
WantedBy=multi-user.target
running issues:
$ systemctl enable gunicorn_ourcase
systemctl: command not found
Please help me to solve this.

Everything I was doing was fine, except the fact that nginx was using it's default config file. I had to delete/edit the file located in /etc/nginx/sites-enabled/ , with name default. After editing it, everything works fine.

Flask app on uwsgi/nginx - unix socket file is not created on booting

I'm trying to use Flask app on uwsgi/nginx.
Following
http://uwsgi-docs.readthedocs.org/en/latest/tutorials/Django_and_nginx.html
and
http://www.markjberger.com/flask-with-virtualenv-uwsgi-nginx/, I could make wiki.ini file,
[uwsgi]
vhost = true
socket = /tmp/flask_app.sock
venv = /home/ubuntu/webapp/flask/hello/.env
chdir = /home/ubuntu/webapp/flask/hello
module = flaskapp
callable = app
chmod-socket = 666
I checked the wiki.ini file works fine with uwsgi --ini wiki.ini.
Then, I tried to start the Flask app when booting.
From sudo update-rc.d uwsgi enable, I could start the uwsgi service in booting time, and copied the wiki.ini file in /etc/uwsgi/apps-enabled directory.
This is the conf file for nginx.
server {
listen 80;
server_name wiki.example.com;
access_log /var/log/nginx/uwsgi_access.log;
error_log /var/log/nginx/uwsgi_error.log;
location / { try_files $uri #riki; }
location #riki {
include uwsgi_params;
uwsgi_pass unix:/tmp/flask_app.sock;
}
error_page 404 /404.html;
}
However, when I rebooted my ubuntu server, the Flask app isn't working.
I checked the error log to find this error message.
2015/11/07 17:48:17 [crit] 1055#0: *1 connect() to
unix:/tmp/flask_app.sock failed (2: No such file or directory)
while connecting to upstream, client: 68.203.30.28, server: wiki.example.com,
I created the /tmp/flask_app.sock file and run chown -R www-data:www-data /tmp/flask_app.sock to make the application working.
> touch /tmp/flask_app.sock
> sudo chown www-data:www-data /tmp/flask_app.sock
> sudo service uwsgi restart
> sudo service nginx restart
However, I had another connection refuse error.
2015/11/07 17:50:38 [error] 1055#0: *4 connect() to
unix:/tmp/flask_app.sock failed (111: Connection refused) while
connecting to upstream, client: 68.203.30.28,
server: wiki.example.com, request: "GET / HTTP/1.1",
upstream: "uwsgi://unix:/tmp/flask_app.sock:", host: "wiki.example.com"
What might be wrong? How to teach uwsgi to create the unix domain socket? Also, how to make the connection work? I use ubuntu 14.04.
EDIT
Removing the /tmp/flask_app.sock and run uwsgi --ini /etc/uwsgi/apps-enabled/wiki.ini makes the app working fine.

The main issue seems to be from the uwsgi service; it just doesn't work.
I found another way to start uwsgi at startup: upstart and uwsgi --emperor from http://uwsgi-docs.readthedocs.org/en/latest/Upstart.html and http://upstart.ubuntu.com
The process is just make a flask.conf file in /etc/init directory. uwsgi --emperor controls all the ini files in the uwsgi directory.
# simple uWSGI script
# http://uwsgi-docs.readthedocs.org/en/latest/Upstart.html
description "uwsgi tiny instance"
start on runlevel [2345]
stop on runlevel [06]
respawn
exec uwsgi --emperor /etc/uwsgi/apps-enabled
I also had to sudo update-rc.d uwsgi disable so that uwsgi service should be disabled.
I also found this site http://flaviusim.com/blog/Deploying-Flask-with-nginx-uWSGI-and-Supervisor/ for invoking uswgi at startup, but I didn't test it.

Serving django with nginx and uWSGI

I followed this post to serve my django project. The project runs well with manage.py runserver and I want to set it up for production. Here are my setting files:
nginx.conf:
upstream django {
server /tmp/vc.sock;
#server 10.9.1.137:8002;
}
server {
listen 8001;
server_name 10.9.1.137;
charset utf-8;
client_max_body_size 25M;
location /media {
alias /home/deploy/vc/media;
}
location /static {
alias /home/deploy/vc/static;
}
location / {
uwsgi_pass django;
include /etc/nginx/uwsgi_params;
}
}
uwsgi.ini:
[uwsgi]
chdir = /home/deploy/vc
wsgi-file = vc/wsgi.py
master = true
processes = 2
#socket = :8002
socket = /tmp/vc.sock
chmod-socket = 666
vacuum = true
If I use TCP port socket (server 10.9.1.137:8002 and socket = :8002), it's going to be fine. However if I comment them out and use Unix sockets(server /tmp/vc.sock and socket = /tmp/vc.sock), the server will return 502 error. How should I fix it?
EDIT
Here's the nginx error log when I run /etc/init.d/nginx restart
nginx: [emerg] invalid host in upstream "/tmp/vc.sock" in /etc/nginx/conf.d/vc.conf:2
nginx: configuration file /etc/nginx/nginx.conf test failed
And this is the warning when I run uwsgi --ini vc/uwsgi.ini:
*** WARNING: you are running uWSGI as root !!! (use the --uid flag) ***
Can't I run uWSGI as root?

roberto's comment should be an answer!
the syntax in nginx for unix socket path is wrong, you need to prefix
it with unix:

Check your nginx error log, very probably is telling you it does not have permissions to the socket. Unix sockets honour file system permissions, so nginx must have write privilege over the socket file. Short answer: 664 is not enough, you need 666

Nginx is throwing an 403 Forbidden on Static Files

I have a django app, python 2.7 with gunicorn and nginx.
Nginx is throwing a 403 Forbidden Error, if I try to view anything in my static folder #:
/home/ubuntu/virtualenv/myapp/myapp/homelaunch/static
nginx config(/etc/nginx/sites-enabled/myapp) contains:
server {
listen 80;
server_name *.myapp.com;
access_log /home/ubuntu/virtualenv/myapp/error/access.log;
error_log /home/ubuntu/virtualenv/myapp/error/error.log warn;
connection_pool_size 2048;
fastcgi_buffer_size 4K;
fastcgi_buffers 64 4k;
root /home/ubuntu/virtualenv/myapp/myapp/homelaunch/;
location /static/ {
alias /home/ubuntu/virtualenv/myapp/myapp/homelaunch/static/;
}
location / {
proxy_pass http://127.0.0.1:8001;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
add_header P3P 'CP="ALL DSP COR PSAa PSDa OUR NOR ONL UNI COM NAV"';
}
}
error.log contains:
2013/11/24 23:00:16 [error] 18243#0: *277 open() "/home/ubuntu/virtualenv/myapp/myapp/homelaunch/static/img/templated/home/img.png" failed (13: Permission denied), client: xx.xx.xxx.xxx, server: *.myapp.com, request: "GET /static/img/templated/home/img2.png HTTP/1.1", host: "myapp.com", referrer: "http://myapp.com/"
access.log contains
xx.xx.xx.xxx - - [24/Nov/2013:23:02:02 +0000] "GET /static/img/templated/base/animg.png HTTP/1.1" 403 141 "http://myapp.com/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:25.0) Gecko/20100101 Firefox/25.0"
xx.xx.xx.xxx - - [24/Nov/2013:23:02:07 +0000] "-" 400 0 "-" "-"
I tried just viewing say a .css file in /static/ and it throws an error like this in source:
<html>
<head><title>403 Forbidden</title></head>
<body bgcolor="white">
<center><h1>403 Forbidden</h1></center>
<hr><center>nginx/1.1.19</center>
</body>
</html>

MacOs El Capitan: At the top of nginx.conf write user username group_name
My user name is Kamil so i write:
user Kamil staff;
(word 'staff' is very important in macOS). This do the trick. After that you don't need to change any permission in your project folder and files.

It appears the user nginx is running as (nginx?) is missing privileges to read the local file /home/ubuntu/virtualenv/myapp/myapp/homelaunch/static/img/templated/home/img.png. You probably wanna check file permissions as well as permissions on the directories in the hierarchy.

It seems the web server user doesn't have read permissions to the static files.
You can solve this in 2 ways:
(easiest, safer) run the nginx as you app user instead of default nginx user. To do this, add the following in nginx.conf
user your_app_user
Replace your_app_user with appropriate unix username for your app. In this case the your_app_user already has necessary permissions to the static content.
Another way would be to to grant permissions for the web server user to the static dir.

The minimum fix that worked for me is:
sudo chmod -R 664 /home/ubuntu/virtualenv/myapp/myapp/homelaunch/static/
sudo chmod -R a+X /home/ubuntu/virtualenv/myapp/myapp/homelaunch/static/
(BTW, in my case the static folder is called collected_static)

Try specifying a user at the top of your nginx.conf, above the server section.
user www-data;

The best solution in that case would be to add www-data to username group:
gpasswd -a www-data username
For your changes to work, restart nginx
nginx -s reload

I had the same issue no long ago. It might be a combination of factors. I found how to fix 403 access denied by replacing the user in the nginx.conf file.
I deployed my website on an ubuntu server using Digital Ocean.
I created a new user on my new ubuntu server and give admin priviliges
adduser newuser
usermod -aG sudo newuser
I updated my new server and installed few packages
sudo apt update
sudo apt install python3-pip python3-dev libpq-dev postgresql postgresql-contrib nginx curl
I followed all this beautiful instruction on how to deploy your site on Digital Ocean
Since I changed the user and I ssh into my new server using this new user, I need to replace the user on the nginx.conf. By default nginx.conf user is www-data:
user www-data;
worker_processes auto;
pid /run/nginx.pid;
Then I replaced with my sudo user and solved my problem. 😀
user newuser;
worker_processes auto;
pid /run/nginx.pid;
Then I restart nginx, gunicorn and postgresql(even if the last one it is not really necessary)
sudo systemctl restart nginx
sudo systemctl restart gunicorn
sudo systemctl restart postgresql
And tada.. :) no more issue.

Fix 403 error with Django static files on Ubuntu server.
Run this -> gpasswd -a www-data your_proj_username
Reload nginx -> nginx -s reload
Check chmod for your dirs: /home, /home/proj_dir, /home/proj_dir/static
Run this - stat --format '%a' /home . Result must be 755
Run this - stat --format '%a' /home/your_proj_dir/static . Result must be 755
Run this - stat --format '%a' /home/your_proj_dir . Result must be 750
If you have different values you can try to change this:
sudo chmod 755 /home
sudo chmod 755 /home/your_proj_dir/static
sudo chmod 750 /home/your_proj_dir
Reload you project-server. This solve all permission errors

After hours upon hours following so many articles, I ran across :
http://nicholasorr.com/blog/2008/07/22/nginx-engine-x-what-a-pain-in-the-bum/
which had a comment to chmod the whole django app dir, so I did:
sudo chmod -R myapp
This fixed it. Unbelievable!
Thanks to those who offered solutions to fix this.

Problems deploying Pyramid with uWSGI on Nginx

I seem to be having some slight problems deploying a Pyramid web application. The problem seems to lie in my init script that I am using to start my web application on boot. For some reason, uWSGI will not work unless my socket is set to have a permission of "nobody.nobody" OR Nginx is started after my uwsgi init script. I'm changed my init script to reflect these changes, but it does not seem to be working. The init script (or the part that starts uwsgi) looks like so:
#!/sbin/runscript
args="--ini-paste /var/www/pyramid/app1/development.ini"
command="/var/www/pyramid/bin/uwsgi"
pidfile="/var/run/uwsgi.pid"
sock="/var/tmp/proxy/uwsgi.sock"
nobody="nobody.nobody"
start() {
ebegin "Starting app1"
chown $nobody $sock
start-stop-daemon --start --exec $command -- $args \
--pidfile $pidfile
chown $nobody $sock
einfo "app1 started"
eend $?
}
My Nginx configuration looks like so:
location / {
include uwsgi_params;
uwsgi_pass unix:///var/tmp/proxy/uwsgi.sock;
uwsgi_param SCRIPT_NAME "" ;
}
My ini file includes the following:
[uwsgi]
socket = /var/tmp/proxy/uwsgi.sock
pidfile = /var/run/uwsgi.pid
master = true
processes = 1
home = /var/www/pyramid
daemonize = /var/log/uwsgi.log
virtualenv = /var/www/pyramid/
pythonpath = /var/www/pyramid/bin
What happens is that Nginx will start, and then uwsgi will start. Performing a "ls -la" in /var/tmp/proxy reveals that the permissions of uwsgi.sock is set to "root root" instead of "nobody nobody". However, restarting Nginx will fix the problem, regardless of what the socket's permissions are (but Nginx has to be started first).
Thus, the ways I can get this to work is:
start uwsgi
start nginx
restart nginx
or
start nginx
start uwsgi
restart nginx
I'm at a complete loss as to why this isn't working. If anyone has any advice I'd greatly appreciate it!

You can use the following settings to change the permission of its socket in your ini file:
chmod-socket = 777 # socket permission
gid = www-data # socket group
uid = www-data # socket user

Another thing to consider is whether you actually want uWSGI to run as root. If you pass --uid and --gid arguments to uwsgi, uwsgi will masquerade as a different (preferably non-root) user.
For example, nginx usually runs as the www-data user and www-data group. So if you set up your wsgi app to run with "--gid www-data" and then add at least group-write permissions to your socket file with "--chmod-socket 020", then nginx will be able to write to the socket and you'll be in business.
See my blog post on the subject: http://blog.jackdesert.com/common-hurdles-to-deploying-uwsgi-apps-part-1