Since my app has background tasks, I use the Flask context. For the context to work, the Flask setting SERVER_NAME should be set.
When the SERVER_NAME is set the incoming requests are checked to match this value or the route isn't found. When placing an nginx (or other webserver in front), the SERVER_NAME should also include the port and the reverse proxy should handle the rewrite stuff, hiding the port number from the outside world (which it does).
For session cookies to work in modern browsers, the URL name passed by the proxy should be the same as the SERVER_NAME, otherwise the browser refuses to send the cookies. This can be solved by adding the official hostname in the /etc/hosts and setting it to 127.0.0.1.
There is one thing that I haven't figured out yet and it is the URL in the background tasks. url_for() is used with the _external option to generate URLs in the mail it sends out. But that URL includes the port, which is different from the 443 port used by my nginx instance.
Removing the port from the SERVER_NAME makes the stuff described in the first paragraph fail.
So what are my best options for handling the url_for in the mail. Create a separate config setting? Create my own url_for?
You should use url_for(location, _external=True)
or include proxy_params if you use nginx.
Related
This question seems to be asked often, but I have not found a good resolution to the problem I am having.
I have a flask application that is behind nginx. The app and nginx communicate via uwsgi unix socket. The application has a publicly exposed endpoint that is exposed via Route53. It is also exposed via AWS API Gateway. The reason for this dual exposure is that the application is replacing an existing Lambda solution. With the API Gateway, I can support legacy requests until they can transition to the new publicly exposed endpoint. An additional fact about my application, it is running in a Kubernetes pod, behind a load balancer.
I need to get access to the IP address of the client that made the request so I can use geoIP lookups and exclude collection of data for users outside of US (GDPR) among other things. With two paths into the application, I have two different ways to get to the IP address.
Hitting the endpoint from API Gateway
When I come in through the legacy path, I get an X-Forwarded-For, but I am only seeing IP addresses that are registered to Amazon. I was using the first one in the list, but I only see one or two different IP address. This is a test environment, and that may be correct, but I don't think so because when I hit it from my local browser, I do not find my IP.
Directly hitting the endpoint:
In this case, there is no data in the X-Forwarded-For list, and the only ip address I can find is request.remote_addr. This, unfortunately only has the IP address of either the pod, or maybe the load balancer. I'm not sure which as it is in the same class, but matches neither. Regardless, it is definitely not the client IP address. I found documentation in nginx that describes available variables including $realip_remote_addr. However, when I logged that value, it was the same as remote_addr.
The following is the code that I am using to get the remote_addr:
def remote_addr(self, request):
x_forwarded_for = request.headers.get("X-Forwarded-For")
if x_forwarded_for:
ip_list = x_forwarded_for.split(",")
return ip_list[0]
else:
return request.remote_addr
If it is helpful, this is my nginx server config:
server {
listen 8443 ssl;
ssl_certificate /etc/certs/cert;
ssl_certificate_key /etc/certs/key;
ssl_dhparam /etc/certs/dhparam.pem;
ssl_protocols TLSv1.2;
ssl_prefer_server_ciphers on;
ssl_ciphers "EECDH+ECDSA+AESGCM EECDH+aRSA+AESGCM EECDH+ECDSA+SHA384 EECDH+ECDSA+SHA256 EECDH+aRSA+SHA384 EECDH+aRSA+SHA256 EECDH+aRSA+RC4 EECDH EDH+aRSA HIGH !RC4 !aNULL !eNULL !LOW !3DES !MD5 !EXP !PSK !SRP !DSS";
server_tokens off;
location = /log {
limit_except POST {
deny all;
}
include uwsgi_params;
uwsgi_pass unix:///tmp/uwsgi.sock;
}
location = /ping {
limit_except GET {
deny all;
}
include uwsgi_params;
uwsgi_pass unix:///tmp/uwsgi.sock;
}
location = /health-check {
return 200 '{"response": "Healthy"}';
}
location /nginx_status {
stub_status;
}
}
I have spent over a day trying to sort this out. I am sure that the solution is trivial and is likely caused by lack of knowledge/experience using nginx.
Kubernetes, by default do not preserve the client address in the target container (https://kubernetes.io/docs/tasks/access-application-cluster/create-external-load-balancer/).
Therefore, I solved that by editing my Nginx ingress configuration, where I changed the externalTrafficPolicy property to local.
...
ports:
- port: 8765
targetPort: 9376
externalTrafficPolicy: Local
type: LoadBalancer
...
However, be aware that if you change that you may have a risk of unbalanced traffic among the pods:
Cluster obscures the client source IP and may cause a second hop to another node, but should have good overall load-spreading. Local preserves the client source IP and avoids a second hop for LoadBalancer and NodePort type Services, but risks potentially imbalanced traffic spreading.
Heroku proxies requests from a client to server, so you have to parse the X-Forwarded-For to find the originating IP address.
The general format of the X-Forwarded-For is:
X-Forwarded-For: client1, proxy1, proxy2
Using werkzeug on flask, I'm trying to come up with a solution in order to access the originating IP of the client.
Does anyone know a good way to do this?
Thank you!
Werkzeug (and Flask) store headers in an instance of werkzeug.datastructures.Headers. You should be able to do something like this:
provided_ips = request.headers.getlist("X-Forwarded-For")
# The first entry in the list should be the client's IP.
Alternately, you could use request.access_route (thanks #Bastian for pointing that out!):
provided_ips = request.access_route
# First entry in the list is the client's IP
This is what I use in Django. See this https://docs.djangoproject.com/en/dev/ref/request-response/#django.http.HttpRequest.get_host
Note: At least on Heroku HTTP_X_FORWARDED_FOR will be an array of IP addresses. The first one is the client IP the rest are proxy server IPs.
in settings.py:
USE_X_FORWARDED_HOST = True
in your views.py:
if 'HTTP_X_FORWARDED_FOR' in request.META:
ip_adds = request.META['HTTP_X_FORWARDED_FOR'].split(",")
ip = ip_adds[0]
else:
ip = request.META['REMOTE_ADDR']
I need configure flask application to handle requests with any host in HTTP header
If some fqdn is specified in SERVER_NAME I have 404 error if request goes with any other domain.
How should be defined SERVER_NAME in configuration?
How can be requested/routed/blueprint-ed HTTP hostname?
Use app.run(host='0.0.0.0') if you want flask to accept any host name.
To allow any domain name just remove 'SERVER_NAME' from application config
Currently I am using nginx and uWSGI to host my website. I need to append www. to my urls, but I'm not sure what is the best route to take.
Should I be doing this at the nginx level?
Yes, nginx is the most efficient way to prepend (or append) www, though Django provides a settings PREPEND_WWW that does the exact same thing when set to True.
E.g. in your nginx config:
server {
listen 80;
server_name example.com;
return 301 http://www.example.com$request_uri;
}
How can I specify the port used for the Flask url_for method? Or, can I configure Flask to use whatever port it is running on for url_for? My issue is that I'm running a server on port 8080 but url_for does not add this port to any URLs generated, so any generated URLs use port 80 and do not resolve.
It seems the only way to specify a port in url_for is to use the _external=True argument like so:
url_for('handle_contact_form', _external=True)
This generates a URL like http://localhost:5000/contact-us. Unfortunately a :5000/contact-us isn't a valid relative URL. So without using a full, external URL, the port cannot be specified.