Develop Reference

Can not reach Flask app served with uWSGI and Nginx

I have a Flask back end that is functional without using uwsgi and nginx.
I'm trying to deploy it on an EC2 instance with its front-end.
No matter what I do, I can't reach the back-end. I opened all the ports for testing purposes but that does not help.
Here's my uwsgi ini file:
[uwsgi]
module = main
callable = app
master = true
processes = 1
socket = 0.0.0.0:5000
vacuum = true
die-on-term = true
Then I use this command to start the app:
uwsgi --ini uwsgi.ini
The message returned is
WSGI app 0 (mountpoint='') ready in 9 seconds.
spawned uWSGI worker 1 (and the only) PID: xxxx
Then here is my Nginx conf file:
server {
server_name my_name.com www.ny_name.com
location / {
root /home/ubuntu/front_end/dist/;
}
location /gan {
proxy_pass https:localhost:5000/gan;
}
## below https conf by certbot
}
If my understanding is correct, whenever a request reaches "my_name.com/gan..." it will be redirected to the localhost on the port 5000 where the back-end is started by uwsgi.
But I can't reach it. I'm trying to simply do a get request on "my_name.com/gan" on my browser (it should return a random image) but I get a 502 by nginx.
Important to note, the front-end works fine and I can access it on browser.

My guess is that url is not in proper form
Try
proxy_pass http://0.0.0.0:5000;

How do I run Flask+Nginx+uWSGI with SELinux in Enforcing mode?

I'm following this tutorial to run Flask on an Nginx server. I've almost got it to work, wherein the page loads when SELinux is set as Permissive but shows a 502 Bad Gateway when SELinux is in the Enforcing mode.
Here are some relevant files:
myproject.ini
[uwsgi]
module = wsgi
master = true
processes = 5
socket = myproject.sock
chmod-socket = 660
vacuum = true
die-on-term = true
myproject.service
[Unit]
Description=uWSGI instance to serve myproject
After=network.target
[Service]
User=thisuser
Group=nginx
WorkingDirectory=/home/thisuser/public_html
Environment="PATH=/home/thisuser/thisuser_env/bin"
ExecStart=/home/thisuser/thisuser_env/bin/uwsgi --ini myproject.ini
[Install]
WantedBy=multi-user.target
thisuser.com.conf (Nginx configuration)
server {
listen 80;
server_name thisuser.com www.thisuser.com;
access_log /home/thisuser/logs/access.log;
error_log /home/thisuser/logs/error.log;
location / {
include uwsgi_params;
uwsgi_pass unix:/home/thisuser/public_html/myproject.sock;
try_files $uri $uri/ =404;
}
}
The location of the Flask files+dirs is /home/thisuser/ and it's contexts are set like so:
[root#dev ~]# ls -ldZ /home/thisuser/
drwx--x--x. thisuser thisuser unconfined_u:object_r:user_home_dir_t:s0 /home/thisuser/
[root#dev ~]# ls -ldZ /home/thisuser/public_html/
drwxrwxr-x. thisuser thisuser unconfined_u:object_r:httpd_sys_content_t:s0 /home/thisuser/public_html/
The errors are as follows:
/var/log/audit/audit.log
type=AVC msg=audit(1498880449.864:156): avc: denied { write } for pid=2667 comm="nginx" name="myproject.sock" dev="dm-2" ino=67165858 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r:httpd_sys_content_t:s0 tclass=sock_file
type=SYSCALL msg=audit(1498880449.864:156): arch=c000003e syscall=42 success=no exit=-13 a0=f a1=7f526e12e548 a2=6e a3=7ffdf52991b0 items=0 ppid=2666 pid=2667 auid=4294967295 uid=997 gid=995 euid=997 suid=997 fsuid=997 egid=995 sgid=995 fsgid=995 tty=(none) ses=4294967295 comm="nginx" exe="/usr/sbin/nginx" subj=system_u:system_r:httpd_t:s0 key=(null)
and
/home/thisuser/logs/error.log
2017/06/30 23:40:49 [crit] 2667#0: *1 connect() to unix:/home/thisuser/public_html/myproject.sock failed (13: Permission denied) while connecting to upstream, client: 192.168.1.15, server: thisuser.com, request: "GET / HTTP/1.1", upstream: "uwsgi://unix:/home/thisuser/public_html/myproject.sock:", host: "thisuser.com"
Steps tried:
tried changing the sock permissions to chmod-socket = 666
used setsebool -P httpd_can_network_connect 1
changed FROM user=thisuser to user=nginx
added thisuser to the nginx group
The only thing that works is changing SELinux to Permissive. Are there some changes/additions I can make, so that SELinux stays Enforcing?
Edit: http(s) has already been allowed in firewalld
[root#dev ~]# firewall-cmd --permanent --zone=public --add-service=https
[root#dev ~]# firewall-cmd --permanent --zone=public --add-service=http
[root#dev ~]# firewall-cmd --reload

Not sure if the below will work but:
The socket needs to be associated with the httpd_sys_content_rw_t type so that processes associated with httpd_t can write it. create "myproject/runtime" and associate type httpd_sys_content_rw_t with "runtime" so that the socket gets created with the httpd_sys_content_rw_t type
Make systemd manually associate the uwsgi app process with the httpd_sys_script_t type so that the webapp is targeted by SELinux (not sure whether systemd is allowed to do this as is in the policy)
The gist is that:
avc: denied { write } for pid=2667 comm="nginx" name="myproject.sock" dev="dm-2" ino=67165858 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r:httpd_sys_content_t:s0 tclass=sock_file
Indicates that nginx process associated with type httpd_t was not allowed to write the myproject.sock sock file because it was associated with the "read only" httpd system content type.
It should have been associated with the "read and write" httpd system content type instead.
ini:
[uwsgi]
module = wsgi
master = true
processes = 5
socket = /home/thisuser/public_html/myproject/runtime/myproject.sock
chmod-socket = 660
vacuum = true
die-on-term = true
unit:
[Unit]
Description=uWSGI instance to serve myproject
After=network.target
[Service]
User=thisuser
Group=nginx
WorkingDirectory=/home/thisuser/public_html/myproject
Environment="PATH=/home/thisuser/thisuser_env/bin"
ExecStart=/home/thisuser/thisuser_env/bin/uwsgi --ini myproject.ini
SELinuxContext=system_u:system_r:httpd_sys_script_t:s0
[Install]
WantedBy=multi-user.target
conf:
server {
listen 80;
server_name thisuser.com www.thisuser.com;
access_log /home/thisuser/logs/access.log;
error_log /home/thisuser/logs/error.log;
location / {
include uwsgi_params;
uwsgi_pass unix:/home/thisuser/public_html/myproject/runtime/myproject.sock;
try_files $uri $uri/ =404;
}
}
Associate labels:
chcon -t httpd_sys_script_exec_t /home/thisuser/thisuser_env/bin/uwsgi
chcon -Rt httpd_sys_content_rw_t /home/thisuser/public_html/myproject/runtime

You need to enable the port 80 on the semanage in order to be able to send traffic through that port.
semanage port -a -t http_port_t -p tcp 80
you may need to enable the port on firewalld too:
firewall-cmd --zone=public --permanent --add-port=80/tcp

Change the uwsgi.ini file as following:
[uwsgi]
plugins = python
project = /home/thisuser/public_html/myproject
chdir = %(project)
module = application # This is the application.py in myproject, you should replace it.
callable = app # This is the call module name of application.py you run
master = true
process = 5
socket = /home/thisuser/public_html/myproject/runtime/myproject.sock
chmod-socket = 666
vacumm = true
die-on-term = true

Try to add this to your myproject.ini file :
plugins = python
project = /home/thisuser/public_html/myproject
chdir = %(project)

Thank you all for your answers.
My goal was to run Python Flask application with NGINX through uWSGI with
SELINUX enabled with enforcing mode. I used Unix socket initially but NGINX cannot access socket file when SELINUX is enforced. There is a workaround to set httpd_t in permissive using the command "semanage permissive -a httpd_t". But I wanted no security loophole so only option is to use HTTP as protocol for uwsgi to talk to my application. This is how I made it happen using the ideas suggested in this thread:
myapp.ini file
;This is uWSGI config file
[uwsgi]
module = wsgi:app
; Allow the Python processes to spawn threads
; uWSGI disables Python threads by default
enable-threads = true
callable = app # This is the call module name of application.py you run
master = true
processes = 1
plugins = python3
project = /usr/local/myapp/bin
chdir = %(project)
socket = 127.0.0.1:3031
chmod-socket = 660
chown-socket=bma:nginx
uid = bma
gid = nginx
socket-timeout = 3600
harakiri = 3600
; remove the socket when the process stops
vacuum = true
; uWSGI will kill the process instead of reloading it
die-on-term = true
My NGINX Conf has:
location /rest/ {
include uwsgi_params;
uwsgi_pass 127.0.0.1:3031;
}
To launch the uWSGI, I used this command (from my .service file)
/usr/local/bin/uwsgi --socket 127.0.0.1:3031 --ini bma.ini
Finally run the following command to allow httpd to act as a relay:
# setsebool -P httpd_can_network_relay 1
Thats all I had to do. SELinux is fully enforcing. No need to set httpd_t in permissive mode.

python: moving from uwsgi to http-socket

I'm using a python app (modoboa) which used to be served with uwsgi to nginx via uwsgi-protocol.
I'm trying out h2o server now which doesn't speak the uwsgi protocol but http.
So I'm trying to migrate uwsgi from using a uwsgi-socket to a http-socket, but uwsgi throws an error "no python application found" at the point I am now.
This was in my nginx.conf:
location /modoboa/ {
root /usr/local/www/modoboa_default/modoboa_default;
include uwsgi_params;
uwsgi_pass unix:/tmp/uwsgi-modoboa.sock;
uwsgi_param UWSGI_SCRIPT modoboa_default.wsgi:application;
uwsgi_param UWSGI_SCHEME https;
uwsgi_param SCRIPT_NAME /modoboa;
uwsgi_modifier1 30;
}
This was my uwsgi.ini:
[uwsgi]
chdir = /usr/local/www/modoboa_default
module = modoboa_default.wsgi:application
master = true
harakiri = 60
processes = 4
vhost = true
no-default-app = true
This is my uwsgi.ini now:
[uwsgi]
chdir = /usr/local/www/modoboa_default
module = modoboa_default.wsgi:application
master = true
harakiri = 60
processes = 4
vhost = true
no-default-app = true
http-socket=/tmp/uwsgi-http.sock
enable-threads=true
buffer-size=60000
http-keepalive=3000
I guess I'm missing something in my uwsgi.ini
update
this is what I have so far
h2o.conf
"/modoboa/":
proxy.reverse.url: "http://[unix:/tmp/uwsgi-http.sock]/"
proxy.timeout.keepalive: 1000
proxy.preserve-host: ON
"/modoboa/sitestatic/":
file.dir: /usr/local/www/modoboa_default/sitestatic/
"/modoboa/media/":
file.dir: /usr/local/www/modoboa_default/media/
uwsgi.ini
[uwsgi]
chdir = /usr/local/www/modoboa_default
harakiri = 60
processes = 4
http-socket = /tmp/uwsgi-http.sock
enable-threads = true
mount = /modoboa=modoboa_default.wsgi:application
manage-script-name = true
This works for the start page.
However, when I login I will get redirected to example.com/accounts/login/ instead of example.com/modoboa/accounts/login/

You should remove vhost and no-default-app options if you're explicitly setting module in uWSGI configuration. With that options set, uWSGI is expecting to get information about that from HTTP server, but H2O is not setting anything.
Also, it is insecure to user that settings with servers that can send proper headers unless you're exactly know what they're for, so remove them also from your existing configuration for nginx. You have module set in uWSGI config.

I think you must have the following in the [uwsgi] section (i.e. the same socket name in uwsgi.ini and nginx.conf)
socket = /tmp/uwsgi-modoboa.sock
chmod-socket = 644
If 644 doesn't work, try 666 or even 777 (be aware of the security issues with wide open permission on a shared server)

running uwsgi as master won't handle requests

I have the following set up
src
|
|--flask_app
|--|--controllers.py
|--|--provider.py
|--|--__init__.py
|--config.py
|--wsgi.py
|--myproject.ini
|--myproject.sock
init.py creates the flask application
from flask import Flask, g
from flask_app.controllers import api_module
from flask_app.provider import CassandraDbProvider
# Define WSGI application object
app = Flask(__name__)
# Configure it
app.config.from_object('config')
cassandra = CassandraDbProvider(**app.config['DATABASE_CONNECT_OPTIONS'])
#app.before_request
def before_request():
g.db = cassandra
app.register_blueprint(api_module)
controler.py has the views that run on endpoints and also creates the blueprint
Finally wsgi.py has the following code
from cassandra_flask_app import app as application
if __name__ == "__main__":
application.run(host="0.0.0.0", port=5000)
myproject.ini
[uwsgi]
module = wsgi
logto = /var/log/uwsgi/uwsgi.log
threads = 10
socket = myproject.sock
chmod-socket = 664
vacum = true
die-on-term = true
Upstart script
description "uWSGI server instance configured to serve myproject"
start on runlevel [2345]
stop on runlevel [!2345]
setuid myuser
setgid www-data
env PATH=/home/myuser/myproject/myprojectenv/bin
chdir /home/myuser/myproject/src
exec uwsgi --ini myproject.ini
and nginx
server {
listen 80;
server_name myIpHere;
location / {
include uwsgi_params;
uwsgi_pass unix:/home/myuser/myproject/src/myproject.sock;
}
}
controller.py
api_module = Blueprint('my_api', __name__, prefix='/api') # this might be wrong now because I don't have code infront of me
#myvmip/api/ works fine when uwsgi is master
#api_module.route('/', methods=['GET']):
def test_url():
return 'c'
#myvmip/api/normal_view/?query1=some_value" doesn't work when in master with no error. Only connection timeout error in nginx error.log.
#api_module.route('/nomral_view/', methods=['GET'])
def normal_view():
get_parameter = request.args.get('query1')
#uses g.db to connect to database and fetch some results
#database is cassandra db
return jsonify(**json)
It runs good on my developement machine. On my vm I load my flask application using nginx and uwsgi. I have set up uwsgi as described on many tutorial on the internet. The problem is that If I run uwsgi as master then It won't access some of my urls. Disabling it works properly. What do you think it could be? Does it matter uwsgi isn't loaded as master?

You need to add the following to your myproject.ini file
chdir = /path/to/project
callable = application
http://uwsgi-docs.readthedocs.org/en/latest/WSGIquickstart.html#deploying-flask
Also if you have a virtualenv you should add
venv=/path/to/venv

Why are Django debugging errors suppressed by uWSGI?

I am running a nginx and uWSGI setup with Django but the errors are no longer shown in Django even though debugging is enabled: DEBUG = True. All errors that occur are saved in the uWSGI log file instead. How can i enable Django to show them again?
nginx.conf:
server {
access_log /var/www/servers/myserver/development/logs/nginx_access.log;
error_log /var/www/servers/myserver/development/logs/nginx_error.log warn;
server_name localhost
listen [::]:80;
charset utf-8;
client_max_body_size 75M;
location / {
uwsgi_pass unix:/var/www/servers/myserver/development/sockets/myserver-dev.sock;
include /var/www/servers/myserver/development/configs/uwsgi_params;
deny all;
}
location /static {
autoindex on;
alias /var/www/servers/myserver/development/static;
}
location /media {
autoindex on;
alias /var/www/servers/myserver/development/media;
}
}
uwsgi.conf:
[uwsgi]
;enable master process manager
master = true
;spawn 2 uWSGI worker processes
workers = 2
;unix socket (referenced in nginx configuration)
socket = /var/www/servers/myserver/development/sockets/myserver-dev.sock
# set mode of created UNIX socket
chmod-socket = 666
# place timestamps into log
log-date = true
# user identifier of uWSGI processes
uid = www-data
# group identifier of uWSGI processes
gid = www-data
; number of worker processes
;processes = 2
;vacuum = true
; project-level logging to the logs/ folder
;logto = /var/www/servers/myserver/development/logs/uwsgi.log
; django >= 1.4 project
chdir = /var/www/servers/myserver/development/webapp
wsgi-file = /var/www/servers/myserver/development/webapp/webapp/wsgi.py
;enable-threads = true
virtualenv = /var/www/servers/myserver/development/env
vacuum = true
env = DJANGO_SETTINGS_MODULE=webapp.settings
pidfile = /var/www/servers/myserver/development/logs/myserver-dev.pid
;harakiri = 20 # respawn processes taking more than 20 seconds
;max-requests = 5000 # respawn processes after serving 5000 requests

Try double checking that DEBUG == True is actually correct. I suspect it's not. You could do this in one of your views with the following code.
## inside a view function
from django.conf import settings
raise Exception('Value of DEBUG is %s' % (settings.DEBUG,))
Restart uWSGI and try visiting that view. You should see if your supposition is correct immediately.

Have you configured log handlers in your application? I'm not really familiar with Django anymore but in Flask having debug=True with uwsgi will actually strip all your logging handlers. Instead you need to set up handlers and have debug=False.
I would have suspected that general error handling won't work within uwsgi in the same way as with runserver - http://librelist.com/browser/flask/2011/10/19/debug-when-deploy-in-uwsgi/#7be089baf631971dfb73a5a7b79e2248