I want to create a socket in Flask-SocketIO through Nginx and Unicorn. The following configuration works on localhost without Nginx.
When I access the web application through Nginx in Chrome, i get the error:
Failed to load resource: Could not connect to the server.
http://52.34.18.48:6419/socket.io/?EIO=3&transport=polling&t=1454455363683-6
When I access the web application through Nginx in Firefox, I get the error:
Cross-Origin Request Blocked: The Same Origin Policy disallows reading
the remote resource at
http://52.34.18.48:6419/socket.io/?EIO=3&transport=polling&t=1454464333740-25. (Reason: CORS request failed).
This is how I initialize the socket connection in JavaScript:
import io from "socket.io-client"
const socketUrl = 'http://' + document.domain + ':6419' + '/FlaskApp'
const socket = io(socketUrl);
export default socket;
My Nginx config file looks like this:
server {
listen 80;
server_name 52.34.18.48;
error_log /var/www/Flaskapp/nginx_errorlog.log;
access_log /var/www/Flaskapp/nginx_accesslog.log;
root /var/www/Flaskapp;
location /socket.io {
proxy_pass http://127.0.0.1:6419/socket.io;
proxy_redirect off;
proxy_buffering off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
location / {
proxy_pass http://127.0.0.1:6419;
proxy_buffering off;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header Host $host;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
The Gunicorn command I used is:
gunicorn --worker-class eventlet -w 1 -b 127.0.0.1:6419 flask_app:application
I wrapped the socket in the Flask application like this.
from flask_app import application
from flask_socketio import SocketIO
socketio = SocketIO(application)
Why do I get this error via Nginx but not when connecting via localhost? How can I access the socket.io via Nginx correctly?
Related
My flask web app is based on socket.io, flask, nginx and gunicorn. One of frontend features sends 1.3MB message to the backend using socket.io. I know that from 3.x socket.io maxHttpBufferSize was decreased from 100MB to 1MB so I have max_http_buffer_size limit set to 2MB on the server side. Everything works good, backend receiving message but after few seconds of executing code on backend side it stops without any information in nginx error.log and gunicorn.log. Web browser(Opera, Chrome) console does not return any error too. I think that the problem is on the nginx side because when I execute my app using only gunicorn it works in the right way.
Have you any idea what should I change in my nginx config? Or the problem is elsewhere?
I'm using
Python 3.7.9,
python-engineio 4.0.0,
python-socketio 5.0.2,
Flask-SocketIO 5.0.0,
Javascript SocketIO 3.0.0
gunicorn 20.0.4,
nginx 1.14.0
My .service config file
[Unit]
Description=Gunicorn instance to serve <my_app>
After=network.target
[Service]
User=<my_user>
Group=www-data
WorkingDirectory=/path/to/my/app
Environment="PATH=/path/to/my/env/bin"
ExecStart=/path/to/my/env/bin/gunicorn --certfile=/path/to/my/app/cert.crt --keyfile=/path/to/my/app/key.key --access-logfile /path/to/my/app/gunicorn_logs.log --worker-class eventlet -w 1 --timeout 300 -b 0.0.0.0:5000 <my_app>:app
[Install]
WantedBy=multi-user.target
My nginx proxy requests config
server{
listen 80;
listen [::]:80;
server_name <my_domain> <my_server_ip>;
return 301 https://$host$request_uri;
}
server{
listen 443 default_server ssl;
listen [::]:443 default_server ssl;
access_log /path/to/my/app/access.log;
error_log /path/to/my/app/error.log;
ssl_certificate /etc/letsencrypt/live/<my_domain>/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/<my_domain>/privkey.pem;
server_name <my_domain> <my_server_ip>;
proxy_read_timeout 300;
# reverse proxy for HTTP connection
location / {
include proxy_params;
# proxy_set_header Host $host;
# proxy_set_header X-Real-IP $remote_addr;
# proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass https://0.0.0.0:5000/;
proxy_redirect off;
}
# reverse proxy for Socket.IO connections
location /socket.io {
proxy_pass https://0.0.0.0:5000/socket.io;
proxy_redirect off;
proxy_buffering off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
}
# let's encrypt host validation
location ~ /.well-known{
root /path/to/my/app/static/;
}
}
After some ubuntu 16.04 upgrades and app code modifications, my Flask on gunicorn and nginx website which previously was working is now giving a 502 bad gateway error.
myapp.conf:
upstream app_server_wsgiapp {
server localhost:8000 fail_timeout=0;
}
server {
listen 80;
server_name www.myserver.com;
access_log /var/log/nginx/www.myapp.access.log;
error_log /var/log/nginx/www.myapp.error.log info;
keepalive_timeout 5;
location /static {
autoindex on;
alias /myapp/static;
}
location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
if (!-f $request_filename) {
proxy_pass http://app_server_wsgiapp;
break;
}
client_max_body_size 2097152;
#to get around upstream sent too big header while reading response header from upstream error
proxy_buffer_size 128k;
proxy_buffers 4 256k;
proxy_busy_buffers_size 256k;
}
# this section allows Nginx to reverse proxy for websockets
location /socket.io {
proxy_pass http://app_server_wsgiapp/socket.io;
proxy_redirect off;
proxy_buffering off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
}
}
/var/log/nginx/myapp.error.log
2017/06/11 06:42:52 [error] 31054#31054: *1 connect() failed (111: Connection refused) while connecting to upstream, client: clientip, server: www.myserver.com, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "www.myapp.com"
From my apps log file I note that Flask gets part of the way through starting up, then restarts in a continuous fail/restart loop.
Any ideas how I can go about debugging what could be causing this issue?
Looks like the problem is caused by flask app running error. Could you please try to run the app in interactive way? With same params and environment settings in daemon way and check whether the service can be started correctly?
I have a AWS Load balancer with SSL which is pointing to a ec2(windows) on 80, i.e on LB,
https 443 http 80
However the app i'm running is on another port 9100 which is configured in ngnix, now i'm unable to access the server via https
Please help me out on this
here is my nginx config,
server {
listen 80;
server_name mydomain.com;
location / {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto https;
proxy_pass http://127.0.0.1:9100;
proxy_connect_timeout 150;
proxy_send_timeout 100;
proxy_read_timeout 100;
proxy_buffers 4 32k;
client_max_body_size 8m;
client_body_buffer_size 128k;
}
}
There are many examples of Nginx with https around. Merging the Nginx HTTPS documentation with your sample configuration gives something like:
server {
listen 80;
listen 443 ssl;
server_name mydomain.com;
ssl_certificate www.example.com.crt;
ssl_certificate_key www.example.com.key;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
location / {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto https;
proxy_pass http://127.0.0.1:9100;
proxy_connect_timeout 150;
proxy_send_timeout 100;
proxy_read_timeout 100;
proxy_buffers 4 32k;
client_max_body_size 8m;
client_body_buffer_size 128k;
}
}
This will get Nginx to serve traffic as HTTPS on port 443. You would need to update the ELB to send HTTPS traffic to port 443 if it's coming via that device.
You would also need to update the Nginx ssl_certificate & ssl_certificate_key options to point to somewhere on the EC2 instance that contains those files.
I'm trying to set up a Bokeh server and reverse proxy it with Nginx.
My p-website.conf now looks like:
server {
listen 80;
server_name website.com;
client_max_body_size 25M;
access_log /var/www/logs/p-website.access.nginx.log;
error_log /var/www/logs/p-website.error.nginx.log error;
root /var/www/pydocs/website/production/src;
include global/restrictions.conf;
location /plot/ {
proxy_pass http://website.com:5100;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_http_version 1.1;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host:$server_port;
proxy_buffering off;
}
location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
# enable this if and only if you use HTTPS, this helps Rack
# set the proper protocol for doing redirects:
# proxy_set_header X-Forwarded-Proto https;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_connect_timeout 10;
proxy_read_timeout 60s;
proxy_pass http://production_website_gunicorn;
}
}
On the server I run the Bokeh server with:
bokeh serve bokehserver.py --port 5100 --host website.com:80
But when I visit website.com/plot/ I get a 404 from Bokeh and the servers terminal gives me: WARNING:tornado.access:404 GET / ("here was ip address") 3.04ms
I don't understand why it always gives a 404, or has it something to do with Nginx?
Thanks!
Update 30/06
Ok, I think I'm a step further, and hopefully in the good direction.
My p-website.conf now looks like:
server {
listen 80 default_server;
server_name website.com;
client_max_body_size 25M;
access_log /var/www/logs/p-website.access.nginx.log;
error_log /var/www/logs/p-website.error.nginx.log error;
root /var/www/pydocs/magnify/production/src;
include global/restrictions.conf;
location / {
allow 127.0.0.1:5100
proxy_pass http://127.0.0.1:5100;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_http_version 1.1;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host:$server_port;
proxy_buffering off;
}
}
I now run the server with:
bokeh serve bokehserver.py --port 5100 --host 127.0.0.1:80
Now I don't get the 404 any more, but a "403: Forbidden" when I go to website.com:5100/bokehserver/. And this message on in the bokehserver terminal:
INFO:bokeh.server.tornado:Rejected connection from host 'website.com:5100' because it is not in the --host whitelist
WARNING:tornado.access:403 GET /bokehserver/ (213.152.161.35) 0.78ms
I tried to fix the whitelist problem by adding allow 127.0.0.1:5100 in the p-website.conf, no luck.
I figured it out, apparently you need to add --prefix= with same location to the Bokeh command:
bokeh serve bokehserver.py --port 5100 --prefix=/plot/ --host website.com:80
And the location block in p-website.conf will look like:
location /plot/ {
proxy_pass http://127.0.0.1:5100;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_http_version 1.1;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host:$server_port;
proxy_buffering off;
}
So I have a Python application running with an NginX reverse proxy providing an SSL connection to the running app. I have configured the reverse proxy with the appropriate proxy_set_header directives to forward the $remote_addr variable to the X-Real-IP header. What I'm struggling with is that the Python application is still pulling the localhost address as the IP that is accessing the app.
This exact configuration is setup on two other servers, run by other people without problems. I'm just not sure if there is a configuration option in the main NginX conf file that I need to modify, although neither of the other admins mentioned having this issue with their configurations.
Attached is my site configuration. Any recommendations would be appreciated.
server {
listen 80;
listen [::]:80;
return 301 https://$host$request_uri;
}
server {
listen 443;
server_name redacted;
ssl_certificate /etc/nginx/ssl/redacted.crt;
ssl_certificate_key /etc/nginx/ssl/redacted.key;
ssl on;
ssl_session_cache builtin:1000 shared:SSL:10m;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers HIGH:!aNULL:!eNULL:!EXPORT:!CAMELLIA:!DES:!MD5:!PSK:!RC4;
ssl_prefer_server_ciphers on;
access_log /var/log/nginx/redacted.access.log;
location / {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://127.0.0.1:8765;
proxy_read_timeout 90;
proxy_redirect http://127.0.0.1:8765 https://redacted;
}
}