I am using Flask and when developing locally, all this works fine. However, when deployed using nginx with a proxy pass through gunicorn, the url_for seems to be adding a trailing / to the url which is causing errors when pulling the json data off the url. Here's the relevant code:
#app.route('/save', methods=['POST'])
def save():
name = request.form['name'];
email = request.form['email']
data = json.dumps({"name": name, "email": email})
return redirect(url_for('.record', data=data))
#app.route("/record")
def record():
// error here because of trailing /
data = json.loads(request.args['data'])
//... more code
I'm pretty new to using Flask and super new to gunicorn so if I'm missing any required info to get help, just let me know. Here's the proxy_pass config in nginx:
location / {
proxy_pass http://localhost:8000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
}
And then I'm just running gunicorn as a service with:
gunicorn --bind 0.0.0.0:8000 app:app
Example breaking URL
https://<ipaddress>/record?data=%7B%22name%22%3A+%22fdaf%22%2C+%22email%22%3A+%22fsd%40sfd.com%22%7D/
Related
I need to get the client IP address of a websocket connection for some extra functionality I would like to implement. I have an existing deployed Django server running an Nginx-Gunicorn-Uvicorn Worker-Redis configuration. As one might expect, during development, whilst running a local server, everything works as expected. However, when deployed, I receive the error NoneType object is not subscriptable when attempting to access the client IP address of the websocket via self.scope["client"][0].
Here are the configurations and code:
NGINX Config:
upstream uvicorn {
server unix:/run/gunicorn.sock;
}
server {
listen 80;
server_name <ip address> <hostname>;
location = /favicon.ico { access_log off; log_not_found off; }
location / {
include proxy_params;
proxy_set_header Connection "";
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_pass http://uvicorn;
proxy_headers_hash_max_size 512;
proxy_headers_hash_bucket_size 128;
}
location /ws/ {
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_redirect off;
proxy_pass http://uvicorn;
}
location /static/ {
root /var/www/serverfiles/;
autoindex off;
}
location /media {
alias /mnt/apps;
}
}
Gunicorn Config:
NOTE: ExecStart has been formatted for readability, it is one line in the actual config
[Unit]
Description=gunicorn daemon
Requires=gunicorn.socket
After=network.target
[Service]
User=django
Group=www-data
WorkingDirectory=/srv/server
Environment=DJANGO_SECRET_KEY=
Environment=GITEA_SECRET_KEY=
Environment=MSSQL_DATABASE_PASSWORD=
ExecStart=/bin/bash -c "
source venv/bin/activate;
exec /srv/server/venv/bin/gunicorn
--workers 3
--bind unix:/run/gunicorn.sock
--timeout 300
--error-logfile /var/log/gunicorn/error.log
--access-logfile /var/log/gunicorn/access.log
--log-level debug
--capture-output
-k uvicorn.workers.UvicornWorker
src.server.asgi:application
"
[Install]
WantedBy=multi-user.target
Code throwing the error:
#database_sync_to_async
def _set_online_if_model(self, set_online: bool) -> None:
model: MyModel
for model in MyModel.objects.all():
if self.scope["client"][0] == model.ip:
model.online = set_online
model.save()
This server has been running phenomenally in its current configuration before my need to access connect client IP addresses. It handles other websocket connections just fine without any issues.
I've already looked into trying to configure my own custom UvicornWorker according to the docs. I'm not at all an expert in this, so I might have misunderstood what I was supposed to do: https://www.uvicorn.org/deployment/#running-behind-nginx
from uvicorn.workers import UvicornWorker
class ServerUvicornWorker(UvicornWorker):
def __init__(self, *args, **kwargs) -> None:
self.CONFIG_KWARGS.update({"proxy_headers": True, "forwarded_allow_ips": "*"})
super().__init__(*args, **kwargs)
I also looked at https://github.com/django/channels/issues/546 which mentioned a --proxy-headers config for Daphne, however, I am not running Daphne. https://github.com/django/channels/issues/385 mentioned that HTTP headers are passed to the connect method of a consumer, however, that post is quite old and no longer relavent as far as I can tell. I do not get any additional **kwargs to my connect method.
Client IP has nothing to do with channels
self.scope["client"][0] is undefined because when you receive data from the front end at the backend there is no data with the name client. so try to send it from the frontend. you can send a manual, static value at first to verify and then find techniques to read the IP address and then send it.
I have a website that has the following setup:
the client is an angular 14 project
the server is a python app with strawberry for graphql
using nginx as web server
using asgi python script to run the app using daphne
I'm having cors related errors when I try to access graphql from the angular app
Access to XMLHttpRequest at 'https://URL/graphql' from origin 'https://URL' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: Redirect is not allowed for a preflight request.
in the nginx server i have listen 443 ssl http2; set with certificates from lets encrypt
I created an upstream setup for the python project:
upstream myproj-server {
server 127.0.0.1:8001;
}
created a location backend:
location #backend {
proxy_pass http://myproj-server; # <--- THIS DOES NOT HAVE A TRAILING '/'
#proxy_set_header 'Access-Control-Allow-Origin' "*";
#proxy_set_header 'Access-Control-Allow-Credentials' 'true';
#proxy_set_header 'Access-Control-Allow-Methods' 'GET, POST, PUT, DELETE, OPTIONS';
#proxy_set_header 'Access-Control-Allow-Headers' 'Accept,Authorization,Cache-Control,Content-Type,DNT,If-Modified-Since,Keep-Alive,Origin,User-Agent,X-Requested-With';
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_http_version 1.1;
proxy_set_header X-NginX-Proxy true;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_cache_bypass $http_upgrade;
proxy_redirect off;
proxy_set_header X-Forwarded-Proto $scheme;
}
and setup graphql location:
location /graphql {
# add_header 'Access-Control-Allow-Origin' "*" always;
# add_header 'Access-Control-Allow-Credentials' 'true' always;
# add_header 'Access-Control-Allow-Methods' 'GET, POST, PUT, DELETE, OPTIONS' always;
# add_header 'Access-Control-Allow-Headers' 'Accept,Authorization,Cache-Control,Content-Type,DNT,If-Modified-Since,Keep-Alive,Origin,User-Agent,X-Requested-With' always;
try_files $uri $uri/ #backend;
}
i commented out the CORS lines with # but i did try to enable them, to add cors at the /graphql location or at the proxy configured in the backend, both configurations did not change anything.
next, i have a server.py with the asgi application with the strawberry plugin for the graphql:
from strawberry.asgi import GraphQL
from app import schema
app = GraphQL(schema, graphiql=False)
and i start it with daphne -b 0.0.0.0 -p 8001 server:app
and here I tried to modify server.py to use the Starlette CORS middleware
from strawberry.asgi import GraphQL
from starlette.middleware.cors import CORSMiddleware
from starlette.middleware import Middleware
from starlette.applications import Starlette
from app import schema
middleware = [
Middleware(CORSMiddleware, allow_origins=['*'],
allow_methods=["*"],
allow_credentials=True,
allow_headers=["*"],)
]
graphql_app=GraphQL(schema, graphiql=True)
app = Starlette(middleware=middleware)
app.add_route("/graphql", graphql_app)
app.add_websocket_route("/graphql", graphql_app)
and also here the results are the same
I'm sure that the reverse proxy works properly because if i set graphiql to True and i browse mydomain/graphql it does open the graphql-playground.
so I tried anything i can think of and i'm pretty lots, so any ideas or any information regarding this issue would be greatly appreciated.
I checked the network tab of my browser and I noticed that what the OPTIONS request (for the CORS preflight request) fails with error 301, which is redirect. and then I noticed that the graphql url is mydomain.com/graphql and not www.mydomain.com/graphql and I do redirect to www in my nginx configuration.
i disabled the headers in the nginx i prefer to control it outside of the nginx configuration.
the server.py configuration with starlette did the trick. of course now i'll make it more secured.
I work on a Django project + django-rest-framework. On localhost, PATCH requests work perfectly fine. However, on server, PATCH requests do not work. I get a 400 Bad request error. I use nginx to configure the web server.
Here is my configuration:
server {
listen 80;
server_name x.y.z.com;
root /var/www/path-to/project;
location / {
error_page 404 = 404;
proxy_pass http://127.0.0.1:5555/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_intercept_errors on;
}
}
I get this error when I try PATCH requests on server :
How can I make so that django accept PATCH requests? The log does not show anything. As if it does not even receives the request. I run the django server like this:
python manage.py runserver 5555
Friend, I faced this problem, I made all the possible settings in nginx, but the problem was in my js fetch that tried with the patch method in lowercase, 'patch', changing to 'PATCH' worked normally.
I've got a flask app daemonized via supervisor. I want to proxy_pass a subfolder on the localhost to the flask app. The flask app runs correctly when run directly, however it gives 404 errors when called through the proxy. Here is the config file for nginx:
upstream apiserver {
server 127.0.0.1:5000;
}
location /api {
rewrite /api/(.*) /$1 break;
proxy_pass_header Server;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Scheme $scheme;
proxy_pass http://apiserver;
proxy_next_upstream error timeout http_502;
proxy_buffering off;
}
For instance, when I go to http://127.0.0.1:5000/me, I get a valid response from the app. However when I go to http://127.0.0.1/api/me I get a 404 from the flask app (not nginx). Also, the flask SERVER_NAME variable is set to 127.0.0.1:5000, if that's important.
I'd really appreciate any suggestions; I'm pretty stumped! If there's anything else I need to add, let me know!
I suggest not setting SERVER_NAME.
If SERVER_NAME is set, it will 404 any requests that don't match the value.
Since Flask is handling the request, you could just add a little bit of information to the 404 error to help you understand what's passing through to the application and give you some real feedback about what effect your nginx configuration changes cause.
from flask import request
#app.errorhandler(404)
def page_not_found(error):
return 'This route does not exist {}'.format(request.url), 404
So when you get a 404 page, it will helpfully tell you exactly what Flask was handling, which should help you to very quickly narrow down your problem.
I ran into the same issue. Flask should really provide more verbose errors here since the naked 404 isn't very helpful.
In my case, SERVER_NAME was set to my domain name (e.g. example.com).
nginx was forwarding requests without the server name, and as #Zoidberg notes, this caused Flask to trigger a 404.
The solution was to configure nginx to use the same server name as Flask.
In your nginx configuration file (e.g. sites_available or nginx.conf, depending on where you're defining your server):
server {
listen 80;
server_name example.com; # this should match Flask SERVER_NAME
...etc...
I'm making a website where users can log on and download files, using the Flask micro-framework (based on Werkzeug) which uses Python (2.6 in my case).
I need to get the IP address of users when they log on (for logging purposes).
Does anyone know how to do this? Surely there is a way to do it with Python?
See the documentation on how to access the Request object and then get from this same Request object, the attribute remote_addr.
Code example
from flask import request
from flask import jsonify
#app.route("/get_my_ip", methods=["GET"])
def get_my_ip():
return jsonify({'ip': request.remote_addr}), 200
For more information see the Werkzeug documentation.
Proxies can make this a little tricky, make sure to check out ProxyFix (Flask docs) if you are using one. Take a look at request.environ in your particular environment. With nginx I will sometimes do something like this:
from flask import request
request.environ.get('HTTP_X_REAL_IP', request.remote_addr)
When proxies, such as nginx, forward addresses, they typically include the original IP somewhere in the request headers.
Update See the flask-security implementation. Again, review the documentation about ProxyFix before implementing. Your solution may vary based on your particular environment.
Actually, what you will find is that when simply getting the following will get you the server's address:
request.remote_addr
If you want the clients IP address, then use the following:
request.environ['REMOTE_ADDR']
The below code always gives the public IP of the client (and not a private IP behind a proxy).
from flask import request
if request.environ.get('HTTP_X_FORWARDED_FOR') is None:
print(request.environ['REMOTE_ADDR'])
else:
print(request.environ['HTTP_X_FORWARDED_FOR']) # if behind a proxy
I have Nginx and With below Nginx Config:
server {
listen 80;
server_name xxxxxx;
location / {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://x.x.x.x:8000;
}
}
#tirtha-r solution worked for me
#!flask/bin/python
from flask import Flask, jsonify, request
app = Flask(__name__)
#app.route('/', methods=['GET'])
def get_tasks():
if request.environ.get('HTTP_X_FORWARDED_FOR') is None:
return jsonify({'ip': request.environ['REMOTE_ADDR']}), 200
else:
return jsonify({'ip': request.environ['HTTP_X_FORWARDED_FOR']}), 200
if __name__ == '__main__':
app.run(debug=True,host='0.0.0.0', port=8000)
My Request and Response:
curl -X GET http://test.api
{
"ip": "Client Ip......"
}
The user's IP address can be retrieved using the following snippet:
from flask import request
print(request.remote_addr)
httpbin.org uses this method:
return jsonify(origin=request.headers.get('X-Forwarded-For', request.remote_addr))
If you use Nginx behind other balancer, for instance AWS Application Balancer, HTTP_X_FORWARDED_FOR returns list of addresses. It can be fixed like that:
if 'X-Forwarded-For' in request.headers:
proxy_data = request.headers['X-Forwarded-For']
ip_list = proxy_data.split(',')
user_ip = ip_list[0] # first address in list is User IP
else:
user_ip = request.remote_addr # For local development
Here is the simplest solution, and how to use in production.
from flask import Flask, request
from werkzeug.middleware.proxy_fix import ProxyFix
app = Flask(__name__)
# Set environment from any X-Forwarded-For headers if proxy is configured properly
app.wsgi_app = ProxyFix(app.wsgi_app, x_host=1)
#app.before_request
def before_process():
ip_address = request.remote_addr
Add include proxy_params to /etc/nginx/sites-available/$project.
location / {
proxy_pass http://unix:$project_dir/gunicorn.sock;
include proxy_params;
}
include proxy_params forwards the following headers which are parsed by ProxyFix.
$ sudo cat /etc/nginx/proxy_params
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
If You are using Gunicorn and Nginx environment then the following code template works for you.
addr_ip4 = request.remote_addr
This should do the job.
It provides the client IP address (remote host).
Note that this code is running on the server side.
from mod_python import apache
req.get_remote_host(apache.REMOTE_NOLOOKUP)
I did not get any of the above work with Google Cloud App Engine. This worked, however
ip = request.headers['X-Appengine-User-Ip']
The proposed request.remote_addr did only return local host ip every time.