My flask web app is based on socket.io, flask, nginx and gunicorn. One of frontend features sends 1.3MB message to the backend using socket.io. I know that from 3.x socket.io maxHttpBufferSize was decreased from 100MB to 1MB so I have max_http_buffer_size limit set to 2MB on the server side. Everything works good, backend receiving message but after few seconds of executing code on backend side it stops without any information in nginx error.log and gunicorn.log. Web browser(Opera, Chrome) console does not return any error too. I think that the problem is on the nginx side because when I execute my app using only gunicorn it works in the right way.
Have you any idea what should I change in my nginx config? Or the problem is elsewhere?
I'm using
Python 3.7.9,
python-engineio 4.0.0,
python-socketio 5.0.2,
Flask-SocketIO 5.0.0,
Javascript SocketIO 3.0.0
gunicorn 20.0.4,
nginx 1.14.0
My .service config file
[Unit]
Description=Gunicorn instance to serve <my_app>
After=network.target
[Service]
User=<my_user>
Group=www-data
WorkingDirectory=/path/to/my/app
Environment="PATH=/path/to/my/env/bin"
ExecStart=/path/to/my/env/bin/gunicorn --certfile=/path/to/my/app/cert.crt --keyfile=/path/to/my/app/key.key --access-logfile /path/to/my/app/gunicorn_logs.log --worker-class eventlet -w 1 --timeout 300 -b 0.0.0.0:5000 <my_app>:app
[Install]
WantedBy=multi-user.target
My nginx proxy requests config
server{
listen 80;
listen [::]:80;
server_name <my_domain> <my_server_ip>;
return 301 https://$host$request_uri;
}
server{
listen 443 default_server ssl;
listen [::]:443 default_server ssl;
access_log /path/to/my/app/access.log;
error_log /path/to/my/app/error.log;
ssl_certificate /etc/letsencrypt/live/<my_domain>/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/<my_domain>/privkey.pem;
server_name <my_domain> <my_server_ip>;
proxy_read_timeout 300;
# reverse proxy for HTTP connection
location / {
include proxy_params;
# proxy_set_header Host $host;
# proxy_set_header X-Real-IP $remote_addr;
# proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass https://0.0.0.0:5000/;
proxy_redirect off;
}
# reverse proxy for Socket.IO connections
location /socket.io {
proxy_pass https://0.0.0.0:5000/socket.io;
proxy_redirect off;
proxy_buffering off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
}
# let's encrypt host validation
location ~ /.well-known{
root /path/to/my/app/static/;
}
}
Related
I think this erros happens, because of proxy_pass is incorrect, if someone has any idea please tell me thanks
NGINX FILE
proxy_cache_path /var/nginx/cache/project
keys_zone=djangoproject_cache:60m;
gzip on;
gzip_proxied any;
gzip_types text/plain text/xml text/css application/javascript application/x-javascript;
gzip_vary on;
gzip_disable msie6;
upstream appserver {
server unix:/home/dir/projects/djangoproject/gunicorn.sock fail_timeout=0;
}
server {
listen 80;
listen [::]:80 default_server;
return 301 https://$host$request_uri;
}
server {
listen 443;
server_name www.subdominion.dominion www.subdominion.dominion;
add_header X-Frame-Options "SAMEORIGIN";
ssl_certificate certificate.crt;
ssl_certificate_key certificate.key;
ssl on;
server_tokens off;
root /var/www/html/;
index index.html index.htm;
location / {
try_files $uri $uri/ =404;
}
location /djangoproject{
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# Fix the “It appears that your reverse proxy set up is broken" error.
rewrite ^/djangoproject(.*) /$1 break;
proxy_pass http://unix:/home/sftpserver/projects/podaexpress/gunicorn.sock; -- I think the error is here
try_files $uri #proxy_to_appserver;
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
sendfile_max_chunk 1024k;
root /home/dir/projects/djangoproject/static;
proxy_read_timeout 90;
}
location /flaskproject{
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# Fix the “It appears that your reverse proxy set up is broken" error.
rewrite ^/flaskproject(.*) /$1 break;
proxy_pass http://unix:/home/dir/projects/flaskproject/gunicorn.sock;
proxy_read_timeout 90;
}
location #proxy_to_appserver {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header REMOTE_ADDR $remote_addr;
proxy_redirect off;
proxy_pass http://appserver;
}
}
using this nginx file I got one page manager index.html, in this page has two buttons with href link for my dominion/djangoproject or dominion/flaskproject , everything is correct until here, but when I click in button whose link directs for another project has one error http 502, but gunicorn file is correct
gunicorn file from djangoproject
#!/bin/bash
NAME="djangoproject" # Name of the application (*)
DJANGODIR=/home/dir/projects/djangoproject/ # Django project directory (*)
SOCKFILE=/home/dir/projects/djangoproject/gunicorn.sock # we will communicate using this unix socket (*)
USER=root # the user to run as (*)
GROUP=root # the group to run as (*)
NUM_WORKERS=15 # how many worker processes should Gunicorn spawn (*)
DJANGO_SETTINGS_MODULE=project.qasettings # which settings file should Django use (*)
DJANGO_WSGI_MODULE=project.wsgi # WSGI module name (*)
echo "------------- INICIANDO $NAME gunicorn -------------"
# Activate the virtual environment
cd $DJANGODIR
source /home/dir/projects/djangoproject/bin/activate
export DJANGO_SETTINGS_MODULE=$DJANGO_SETTINGS_MODULE
export PYTHONPATH=$DJANGODIR:$PYTHONPATH
# Create the run directory if it doesn't exist
RUNDIR=$(dirname $SOCKFILE)
test -d $RUNDIR || mkdir -p $RUNDIR
# Start your Django Unicorn Programs meant to be run under supervisor should not daemonize themselves (do not use --daemon)
exec /home/dir/projects/djangoproject/bin/gunicorn ${DJANGO_WSGI_MODULE}:application \
--name $NAME \
--workers $NUM_WORKERS \
--user $USER \
--bind=unix:$SOCKFILE \
--daemon
in both links I get the correct url example https://subdominion.dominion.com.br/djangoprojects or https://subdominion.dominion.com.br/flaskproject
After some ubuntu 16.04 upgrades and app code modifications, my Flask on gunicorn and nginx website which previously was working is now giving a 502 bad gateway error.
myapp.conf:
upstream app_server_wsgiapp {
server localhost:8000 fail_timeout=0;
}
server {
listen 80;
server_name www.myserver.com;
access_log /var/log/nginx/www.myapp.access.log;
error_log /var/log/nginx/www.myapp.error.log info;
keepalive_timeout 5;
location /static {
autoindex on;
alias /myapp/static;
}
location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
if (!-f $request_filename) {
proxy_pass http://app_server_wsgiapp;
break;
}
client_max_body_size 2097152;
#to get around upstream sent too big header while reading response header from upstream error
proxy_buffer_size 128k;
proxy_buffers 4 256k;
proxy_busy_buffers_size 256k;
}
# this section allows Nginx to reverse proxy for websockets
location /socket.io {
proxy_pass http://app_server_wsgiapp/socket.io;
proxy_redirect off;
proxy_buffering off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
}
}
/var/log/nginx/myapp.error.log
2017/06/11 06:42:52 [error] 31054#31054: *1 connect() failed (111: Connection refused) while connecting to upstream, client: clientip, server: www.myserver.com, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "www.myapp.com"
From my apps log file I note that Flask gets part of the way through starting up, then restarts in a continuous fail/restart loop.
Any ideas how I can go about debugging what could be causing this issue?
Looks like the problem is caused by flask app running error. Could you please try to run the app in interactive way? With same params and environment settings in daemon way and check whether the service can be started correctly?
I want to create a socket in Flask-SocketIO through Nginx and Unicorn. The following configuration works on localhost without Nginx.
When I access the web application through Nginx in Chrome, i get the error:
Failed to load resource: Could not connect to the server.
http://52.34.18.48:6419/socket.io/?EIO=3&transport=polling&t=1454455363683-6
When I access the web application through Nginx in Firefox, I get the error:
Cross-Origin Request Blocked: The Same Origin Policy disallows reading
the remote resource at
http://52.34.18.48:6419/socket.io/?EIO=3&transport=polling&t=1454464333740-25. (Reason: CORS request failed).
This is how I initialize the socket connection in JavaScript:
import io from "socket.io-client"
const socketUrl = 'http://' + document.domain + ':6419' + '/FlaskApp'
const socket = io(socketUrl);
export default socket;
My Nginx config file looks like this:
server {
listen 80;
server_name 52.34.18.48;
error_log /var/www/Flaskapp/nginx_errorlog.log;
access_log /var/www/Flaskapp/nginx_accesslog.log;
root /var/www/Flaskapp;
location /socket.io {
proxy_pass http://127.0.0.1:6419/socket.io;
proxy_redirect off;
proxy_buffering off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
location / {
proxy_pass http://127.0.0.1:6419;
proxy_buffering off;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header Host $host;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
The Gunicorn command I used is:
gunicorn --worker-class eventlet -w 1 -b 127.0.0.1:6419 flask_app:application
I wrapped the socket in the Flask application like this.
from flask_app import application
from flask_socketio import SocketIO
socketio = SocketIO(application)
Why do I get this error via Nginx but not when connecting via localhost? How can I access the socket.io via Nginx correctly?
I'm trying to serve both a Django and a Flask site using gunicorn and nginx. The Django site is currently available at example.com, and I'd like for the Flask site to be availabe at myapp.example.com .
My nginx configuration file is as follows for the Django site:
cat /etc/nginx/sites-available/example.com
server {
listen 80;
server_name example.com;
location /static {
alias /home/me/sites/example.com/static;
}
location / {
proxy_set_header Host $host;
proxy_pass http://unix:/tmp/example.com.socket;
}
}
I also have a second nginx config file for the Flask site:
cat /etc/nginx/sites-available/myapp.example.com
server {
listen 80;
server_name myapp.example.com;
access_log /var/log/nginx/access.log;
location / {
proxy_pass http://127.0.0.1:8001;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
I'm pretty sure that I want to change my proxy_pass variable, but am not sure what I should change it to.
My gunicorn file currently is as follows:
description "Gunicorn server for example.com"
start on net-device-up
stop on shutdown
respawn
setuid me
chdir /home/rowan/sites/example.com/source
exec ../virtualenv/bin/gunicorn \
--bind unix:/tmp/example.com.socket \
example.com.wsgi:application
Do I need a second gunicorn config file for the Flask site? Which I would then put in /etc/init/ and start through the sudo start gunicorn-myapp.example.com command?
So I have a Python application running with an NginX reverse proxy providing an SSL connection to the running app. I have configured the reverse proxy with the appropriate proxy_set_header directives to forward the $remote_addr variable to the X-Real-IP header. What I'm struggling with is that the Python application is still pulling the localhost address as the IP that is accessing the app.
This exact configuration is setup on two other servers, run by other people without problems. I'm just not sure if there is a configuration option in the main NginX conf file that I need to modify, although neither of the other admins mentioned having this issue with their configurations.
Attached is my site configuration. Any recommendations would be appreciated.
server {
listen 80;
listen [::]:80;
return 301 https://$host$request_uri;
}
server {
listen 443;
server_name redacted;
ssl_certificate /etc/nginx/ssl/redacted.crt;
ssl_certificate_key /etc/nginx/ssl/redacted.key;
ssl on;
ssl_session_cache builtin:1000 shared:SSL:10m;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers HIGH:!aNULL:!eNULL:!EXPORT:!CAMELLIA:!DES:!MD5:!PSK:!RC4;
ssl_prefer_server_ciphers on;
access_log /var/log/nginx/redacted.access.log;
location / {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://127.0.0.1:8765;
proxy_read_timeout 90;
proxy_redirect http://127.0.0.1:8765 https://redacted;
}
}