I am using nginx as a proxy server for a Django app using gunicorn. The Django app is binded to http://127.0.0.1:8000. And here's my nginx setup from etc/nginx/sites-enabled/parkitbackend:
server {
server_name AA.BB.CC.DD;
access_log off;
location /static/ {
autoindex on;
alias /home/zihe/parkitbackend/parkitbackend/common-static/;
}
location / {
proxy_pass http://127.0.0.1:8000;
}
}
I am using python requests module:
requests.post("http://AA.BB.CC.DD/dashboard/checkin/", data=unicode(json.dumps(payload), "utf8"))
to post JSON objects to my django app called dashboard, where I have a function in dashboard/views.py called checkin to process the JSON object.
I did not receive any errors from running JSON posting script. However, Nginx does not seem to be able to pass the request to gunicorn binded at 127.0.0.1:8000. What should I do so I can use Nginx to pass the JSON to my django app? Thank you!
Additional notes:
I am very sure JSON posting code and my django app work properly since I tested it by binding Django app to http://AA.BB.CC.DD:8000 and ran this code in python:
requests.post("http://AA.BB.CC.DD:8000/dashboard/checkin/", data=unicode(json.dumps(payload), "utf8"))
and my django app received the JSON as expected.
I checked the error.log located at /var/log/nginx/. It turns out that the JSON I was sending was too large and was giving this error:
[error] 3450#0: *9 client intended to send too large body: 1243811 bytes, client: 127.0.0.1, server: _, request: "POST /dashboard/checkin/ HTTP/1.1", host: "127.0.0.1"
After reading up on this link: http://gunicorn-docs.readthedocs.org/en/19.3/deploy.html#nginx-configuration
I reduced the size of the JSON and modified etc/nginx/sites-enabled/parkitbackend to be like this:
upstream app_server {
server 127.0.0.1:8000;
}
server {
listen AA.BB.CC.DD:80;
server_name = _;
location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://app_server;
}
location /static/ {
autoindex on;
alias /home/username/parkitbackend/parkitbackend/common-static/;
}
}
and replaced this line in /etc/nginx/nginx.conf:
include /etc/nginx/sites-enabled/*;
with this:
include /etc/nginx/sites-enabled/parkitbackend;
And the problem is resolved.
Related
I work on a Django project + django-rest-framework. On localhost, PATCH requests work perfectly fine. However, on server, PATCH requests do not work. I get a 400 Bad request error. I use nginx to configure the web server.
Here is my configuration:
server {
listen 80;
server_name x.y.z.com;
root /var/www/path-to/project;
location / {
error_page 404 = 404;
proxy_pass http://127.0.0.1:5555/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_intercept_errors on;
}
}
I get this error when I try PATCH requests on server :
How can I make so that django accept PATCH requests? The log does not show anything. As if it does not even receives the request. I run the django server like this:
python manage.py runserver 5555
Friend, I faced this problem, I made all the possible settings in nginx, but the problem was in my js fetch that tried with the patch method in lowercase, 'patch', changing to 'PATCH' worked normally.
I have set up AWS Cloudfront Distribution for streaming objects from one of my S3 bucket. After generating urls, I am able to stream. Now since I have a server running in EC2 and my web app is backed by Nginx with already configured proxy_pass for the backend server. Now how do I use that generated cloudfront url for files to start playing them in my web app.
I am totally new to nginx following things I have tried
here is my nginx server config
server {
listen 8888;
server_name localhost;
}
location /app{
alias /opt/mw_web_app;
index index.html index.htm;
}
#Create proxy_pass for DataService.
location /service/{
proxy_pass http://server-ip:9003/;
proxy_set_header USER-IP $proxy_add_x_forwarded_for;
}
First fix your confguration and move the second curly brace to the end . then allow your server to be default server to assign all requests to this server
server {
listen 8888;
server_name localhost default_server;
location /app{
alias /opt/mw_web_app;
index index.html index.htm;
}
#Create proxy_pass for DataService.
location /service/{
proxy_pass http://server-ip:9003/;
proxy_set_header USER-IP $proxy_add_x_forwarded_for;
}
}
second, you sould use cloud front url direclty or assign CNAME using your own domain. if you use CF through your nginx CF will be useless with extra fees.
After a try and going to through Nginx Docs : Following worked for me and I a able to stream using cloudfront :
server {
listen 8888;
server_name localhost;
}
location /app{
alias /opt/mw_web_app;
index index.html index.htm;
}
#Create proxy_pass for DataService.
location /service/{
proxy_pass http://server-ip:9003/;
proxy_set_header USER-IP $proxy_add_x_forwarded_for;
}
#Create proxy_pass for DataService.
location /cloudfront/{
proxy_pass https://ddddddddd.cloudfront.net/;
}
}
PS: thanks #Ahmed Abdelazim for your time and yeah there was a syntax error also which u pointed out
Background
A docker container running a supervisord process with 2 processes started - nginx and uwsgi (yes I understand this may be doing docker 'wrong'. That's not the question)
The uwsgi hits a python flask app. This has a logger connected, and prints the headers dictionary to the info log.
I have a postman request that tests from my local box, hits the docker container, routes via nginx and hits the python app, with the info log appended.
Custom headers sent by postman are being logged (thanks to ignore_invalid_headers off;)
The Problem
I'd like to use nginx to decorate incoming requests with some further headers. No matter what I try I can't get it to work. None of the headers I add in the nginx conf seems to make it through to the flask app.
I've tried proxy_set_header or uwgi_param. No variant seems to work.
Please note - I want a request header. I believe add_header is for response headers.
nginx.conf:
user nginx;
worker_processes auto;
pid /run/nginx.pid;
events {
worker_connections 768;
}
http {
include /etc/nginx/mime.types;
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
include /etc/nginx/conf.d/*.conf;
include /etc/nginx/sites-enabled/*;
underscores_in_headers off;
ignore_invalid_headers off;
upstream myapp {
server unix:/run/myapp.sock;
}
server {
listen 80;
location / {
include uwsgi_params;
uwsgi_pass myapp;
proxy_set_header x-proxy-set-header x-proxy-set-header-value;
proxy_set_header sampl-header ONE;
uwsgi_param X-add-uwsgi-param x-added-uwsfi-param-value;
}
}
}
daemon off;
Any help would be hugely appreciated!!
So. Solved. As Richard Smith also found- proxy_pass doesn't work as I'm using uwsgi_pass for the custom protocol.
SO. this works:
location / {
include uwsgi_params;
uwsgi_pass myapp;
uwsgi_pass_request_headers on;
uwsgi_param HTTP_X_TESTING 'bar';
}
And we're cooking on gas...
Air Punch
I am running Django, uwsgi, ngix server.
My server works fine for GET, POST requests of smaller size. But when POSTing requests of large size, nginx returns 502:
nginx error.log is:
2016/03/01 13:52:19 [error] 29837#0: *1482 sendfile() failed (32: Broken pipe) while sending request to upstream, client: 175.110.112.36, server: server-ip, request: "POST /me/amptemp/ HTTP/1.1", upstream: "uwsgi://unix:///tmp/uwsgi.sock:", host: "servername"
So, in order to find where the real problem is, I ran uwsgi on a different port and checked if any error occurs with the same request. But the request was successful. So, the problem is with nginx or unix socket configuration.
Ngin-x configuration:
# the upstream component nginx needs to connect to
upstream django {
server unix:///tmp/uwsgi.sock; # for a file socket
# server 127.0.0.1:8001; # for a web port socket (we'll use this first)
}
# configuration of the server
server {
# the port your site will be served on
listen 80;
# the domain name it will serve for
server_name 52.25.29.179; # substitute your machine's IP address or FQDN
charset utf-8;
# max upload size
client_max_body_size 75M; # adjust to taste
# Django media
location /media {
alias /home/usman/Projects/trequant/trequant-python/trequant/media; # your Django project's media files - amend as required
}
location /static {
alias /home/usman/Projects/trequant/trequant-python/trequant/static; # your Django project's static files - amend as required
}
# Finally, send all non-media requests to the Django server.
location / {
######## proxy_pass http://127.0.0.1:8000;
######## proxy_set_header Host $host;
######## proxy_set_header X-Real-IP $remote_addr;
######## proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
uwsgi_pass django;
uwsgi_read_timeout 600s;
uwsgi_send_timeout 600s;
include /etc/nginx/uwsgi_params; # the uwsgi_params file you installed
}
}
So, any idea what I am doing wrong? Thank you in advance.
Supposedly setting post-buffering = 8192 in your uwsgi.ini file will fix this. I got this answer from a 2.5-yr-old answer here and it implies this fix is not the root cause. Hope it helps!
Another fix is to use a TCP socket instead of a unix socket in your conf files:
In uwsgi.ini, use something like socket = 127.0.0.1:8000 in the [uwsgi] section instead of:
socket = /tmp/uwsgi.sock
chown-socket = nginx:nginx
chmod-socket = 664
In your nginx.conf file (btw in Ubuntu, I'm referring to /etc/nginx/conf.d/nginx.conf, NOT the one simply in /etc/nginx/) use uwsgi_pass 127.0.0.1:8000; instead of include uwsgi_params;
I've posted this as a separate answer because either answer may work, and I'm interested to see which answer helps others the most.
In my case this seemed to be for requests that would have given a 308 redirect. I think my Node backend was sending response before postdata was fully received. Updating the client to hit new endpoint (no redirect) may permanently fix my case. Seems promising.
Set higher body buffer size client_body_buffer_size 1M; This will fix.
References:
http://nginx.org/en/docs/http/ngx_http_core_module.html#client_body_buffer_size
https://www.nginx.com/resources/wiki/start/topics/examples/full/
I'm using nginx as reverse proxy with gunicorn for my Django app, and am new to webserver configuration. My app has a postgres backend, and the machine hosting it has Ubuntu 14.04 lts installed.
I have reason to suspect that my nginx configuration is not forwarding proxy set header to the Django app correctly. Is there a way I can see (e.g. print?) the host, http_user_agent, remote_addr etc. forwarded, on the linux command line to test my gut feel?
Secondly, how do I check whether my Django app received the correct forwarded IP? E.g. can I somehow print?
/etc/nginx/sites-available/myproject:
server {
listen 80;
server_name example.cloudapp.net;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root /home/mhb11/folder/myproject;
}
location / {
proxy_set_header Host $host;
proxy_set_header User-Agent $http_user_agent;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://unix:/home/mhb11/folder/myproject/myproject.sock;
}
error_page 500 502 503 504 /500.html;
location = /500.html {
root /home/mhb11/folder/myproject/templates/;
}
}
All you have to do is print out request.META at the Django project level to see what all of those values are being set to. This automatically happens for you if you get an error and debug is set to True (just scroll down, you'll see a big table with all request.META values populated therein).
Or you can print it yourself from your views.py, or if that doesn't work, then from any middleware you have. You can even write custom middleware for this. Let me know if you need further clarification on this, I can give you basic code snippets too.
This question was posted a long time ago but since I landed here when needed help, below are some code which I ended up using to attain the same goal for the help of other people.
def getClientIP(request):
x_forwarded_for = request.META.get('HTTP_X_FORWARDED_FOR')
if x_forwarded_for:
ip = x_forwarded_for.split(',')[-1].strip()
else:
ip = request.META.get('REMOTE_ADDR')
return ip