I have what I believe to be a slightly convoluted Django/Gunicorn/NGINX stack that is giving me trouble as I try to migrate it from the django development server to a production setup with Gunicorn and NGINX. Specifically, I'm having trouble serving static files.
SYSTEM ARCHITECTURE: I have one physical server with a public IP address. This physical server hosts 3 VM's on a private virtual network (NAT). Each VM run's it's own django project on port 8001. I forward port 8001 on each VM to unique available ports on the physical machine. So, in summary, the architecture looks like the following:
PRIVATE VIRTUAL NETWORK VM's:
VM1 runs "site 1" on VM1 port 8001
VM2 runs "site 2" on VM2 port 8001
VM3 runs "site 3" on VM3 port 8001
HOST SERVER:
Publishes "site 1" on host port 8001 (VM port 8001 fwd.to Host port 8001)
Publishes "site 2" on host port 8002 (VM port 8001 fwd.to Host port 8002)
Publishes "site 3" on host port 8003 (VM port 8001 fwd.to Host port 8003)
This works really well for development with the django development server. I just need to include a port # in the URL. As in:
myserver.edu:8001 for site 1
myserver.edu:8002 for site 2
myserver.edu:8003 for site 3
For production I would like to setup NGINX to serve the sites as:
myserver.edu/site1 for site 1
myserver.edu/site2 for site 2
myserver.edu/site3 for site 3
I installed NGINX on the host machine and configured it to use TLS. The NGINX configuration on the host machine defines 3 locations below with following config. You can see that I use the proxy_pass to route traffic for each site to the Virtual Network on the host machine.
location /site1/ {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X_Forwarded-Proto $scheme;
proxy_pass http://192.168.122.243:8001/;
}
location /site2/ {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X_Forwarded-Proto $scheme;
proxy_pass http://192.168.122.244:8001/;
}
location /site3/ {
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X_Forwarded-Proto $scheme;
proxy_pass http://192.168.122.245:8001/;
}
So any request for:
List item
myserver.edu/site1 goes to port 8001 on VM1
List item
myserver.edu/site2 goes to port 8001 on VM2
List item
myserver.edu/site3 goes to port 8001 on VM3
On VM1 I have a django site + gunicorn + NGINX with the following config (same setup on all VMs):
server {
listen 8001;
server_name 0.0.0.0;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root /home/rcrv/bubbles/s2uds_user/gui;
}
location / {
include proxy_params;
proxy_pass http://unix:/home/rcrv/bubbles/s2uds_user/bubs.sock;
}
}
My Site 1 root URL from urls.py is this: url(r'^$', gv.home),
BEHAVIOR UNDER THE ABOVE NGINX PRODUCTION CONFIG:
If I browse site 1 on VM1 from within the NAT with a URL such as:
192.168.122.243:8001/
This will work perfectly - Static and media file are served
If I browse from public IP space with a URL such as:
myerver.edu/site1/
This will render all dynamic content from django but will fail to serve static content (with a 404). My browser developer console shows that the browser is looking for the static content at https://myserver.com/static
Note that I expected it to look for static content at myserver/site1/static
If I modify that URL directly in the browser to be myserver.com/site1/static I can access the static content that was missing
ATTEMPTS TO MITIGATE/FIX INCLUDE:
I changed the location block on the NGINX config of the VM to be:
location = /site1/static/
No luck.
Debugging shows that the browser is still trying to find the static content at: myserver.edu/static
QUESTION:
How the heck do I modify or fix the configuration such that django includes that "/site1/" part in the static URL? I'm inclined to think that this problem is not an NGINX config issue but instead a django problem. Rather, I believe I need to tell django to prepend /site1/ to it's static URL.
Ideas? I've read numerous responses to similar django static file issues but yet on that fixes this.
Thank you.
UPDATE: I've started to figure this out. Here is what worked for me.
I created a /static/ directory at /var/www/static on each VM and set the STATIC_ROOT in the settings.py file to be this location, then ran collect static to copy all static content to this directory.
I then modified static file location block within the NGINX conf on the VM to be:
location /static/ {
root /var/www/;
}
Finally I modified the STATIC_URL in settings.py from: STATIC_URL = /static/ to STATIC_URL = /site1/static/
The result is the static content is now served in production under NGINX. The cause was my incomplete understanding of static content methods in general. Typing out the question here sort of clarified the problem and led me to the solution. Hopes this helps someone with a similar architecture. Now to fix the media files - likely the same approach.
Related
I have a Domain Name buy from Gandi, the relative DNS is set down,too. The domain name can access my website (use XAMPP).This shows that the current domain name is valid.
~ About my API project ~
The project path is "/home/API_AIcustomerservice"
The API url is "XXX.XXX.XXX.XXX:5062"
The python file of API named "start_test.py"
I'm very sure Gunicorn and Flask is worked. (I can access the API url by IP address successfully)
I run Gunicorn by command "gunicorn -c /home/API_AIcustomerservice/gconfig.py start_test:app" with gconfig.py
[gconfig.py]
from gevent import monkey
monkey.patch_all()
import multiprocessing
debug = True
loglevel = 'debug'
bind = 'XXX.XXX.XXX.XXX:5062'
pidfile = 'log/gunicorn.pid'
logfile = 'log/debug.log'
workers = multiprocessing.cpu_count() * 2 + 1
worker_class = 'gevent'
About Nginx I edited the file: vim /etc/nginx/sites-available/default
server {
listen 81;
root /home/API_AIcustomerservice;
server_name domainName www.domainName;
location / {
proxy_set_header x-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_pass http://XXX.XXX.XXX.XXX:5062/;
}
location ~ ^\/static\/.*$ {
root /home/API_AIcustomerservice;
}
}
After edited, I run sudo service nginx restart
~ Conclusion ~
XXX.XXX.XXX.XXX:5062 can be accessed (of course)
XXX.XXX.XXX.XXX:81 can be accessed like Above (seems Nginx worked)
But DomainName:81 or www.DomainName:81 can't be accessed
Are there another setting I missed?
Thank you for taking the time. I thank you from the bottom of my heart.
Check your Domain DNS server and resolve ip
nginx config true
you can test resolve domain localy and test it again
definds dns in os hosts file, like:
xxx.xxx.xxx.xxx DomainName
in windows:
c:\Windows\System32\Drivers\etc\hosts
in linux:
/etc/hosts
I'm using nginx as reverse proxy with gunicorn for my Django app, and am new to webserver configuration. My app has a postgres backend, and the machine hosting it has Ubuntu 14.04 lts installed.
I have reason to suspect that my nginx configuration is not forwarding proxy set header to the Django app correctly. Is there a way I can see (e.g. print?) the host, http_user_agent, remote_addr etc. forwarded, on the linux command line to test my gut feel?
Secondly, how do I check whether my Django app received the correct forwarded IP? E.g. can I somehow print?
/etc/nginx/sites-available/myproject:
server {
listen 80;
server_name example.cloudapp.net;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root /home/mhb11/folder/myproject;
}
location / {
proxy_set_header Host $host;
proxy_set_header User-Agent $http_user_agent;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://unix:/home/mhb11/folder/myproject/myproject.sock;
}
error_page 500 502 503 504 /500.html;
location = /500.html {
root /home/mhb11/folder/myproject/templates/;
}
}
All you have to do is print out request.META at the Django project level to see what all of those values are being set to. This automatically happens for you if you get an error and debug is set to True (just scroll down, you'll see a big table with all request.META values populated therein).
Or you can print it yourself from your views.py, or if that doesn't work, then from any middleware you have. You can even write custom middleware for this. Let me know if you need further clarification on this, I can give you basic code snippets too.
This question was posted a long time ago but since I landed here when needed help, below are some code which I ended up using to attain the same goal for the help of other people.
def getClientIP(request):
x_forwarded_for = request.META.get('HTTP_X_FORWARDED_FOR')
if x_forwarded_for:
ip = x_forwarded_for.split(',')[-1].strip()
else:
ip = request.META.get('REMOTE_ADDR')
return ip
I have an active site that's hosted on Ubuntu, uses nginx, and the site is written in Python (CherryPy is the server, Bottle is the framework).
I have a shell script that copies python files that I upload over the existing live site ones which then of course results in CherryPy restarting the server so it's running the latest code (how I want it). The problem is, in between the time it's stopping and started a default static page is displayed to any unlucky person who tries to view a page on the site at that time (hope they aren't submitting a form). I've seen this page a bunch of times while updating.
My current setup is two copies of the site running on two ports reverse proxied with nginx. So I figured if I update one, wait a few seconds, then update the other then the site will be up 100% of the time, but this doesn't appear to be the case?
Lets say I have reverse proxy on ports 8095 and 8096, both show the same site but two identical copies of it on the hard drive. I update the python files for port 8095 which causes that port to go down while CherryPy restarts it. Shouldn't everyone then be hitting 8096? It doesn't seem to be working like this. I have an 8 second delay in my file copy script and according to CherryPy logs the 2nd stopped to restart 6 seconds after the 1st was already finished restarting, yet I saw the default static offline page that's displayed when the server is down. I'm confused. According to logs there was always one port up.
Here's part of my nginx.conf:
upstream app_servers {
server 127.0.0.1:8095;
server 127.0.0.1:8096;
}
server {
server_name www.mydomain.com;
listen 80;
error_page 502 503 /offline/offline.html;
location /offline {
alias /usr/share/nginx/html/mysite/1/views/;
}
location / {
proxy_pass http://app_servers;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $server_name;
proxy_set_header X-Scheme $scheme;
proxy_connect_timeout 10;
proxy_read_timeout 10;
}
}
Try this: From the manual
upstream app_servers {
server 127.0.0.1:8095 max_fails=1 fail_timeout=1;
server 127.0.0.1:8096 max_fails=1 fail_timeout=1;;
}
I am using this tutorial - part 1, but i am not sure how to test if the app is running with nginx serving static files or not.
I have exactly the same code.
/etc/nginx/sites-available/flask_project
server {
location / {
proxy_pass http://localhost:8000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
}
location /static {
alias /home/www/flask_project/static/;
}
}
And then:
gunicorn app:app -b localhost:8000
All routes are working fine. However if I do http://localhost:8000/static i will see
Not Found
The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.
And apparently I should see the page with <h1>Test!</h1> from static folder.
What i am doing wrong?
Basically i want to know how configure nginx to serve static files and then confirm.
-app.py
-static
-index.html
First, requests to port 8000 completely bypass nginx, so nothing strange here. You should go to localhost without port number.
Second, you have to symlink this config to /etc/nginx/sites-enabled and reload nginx.
Third, your static location is wrong. You have location without trailing slash and alias with one. They should always be with or without trailing slash simultaneously. And in this case it's even better to have root directive.
server {
root /home/www/flask_project;
index index.html;
location / {
proxy_pass http://localhost:8000/;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
}
location /static/ {
# empty. Will serve static files from ROOT/static.
}
}
You should make request directly http://localhost:8000/static/index.html then you will see response.
But if you want to see on index.html by default, you should have something like in conf:
location /static {
alias /home/www/flask_project/static/;
try_files $uri $uri/index.html index.html;
}
I'm new to anything related to servers and am trying to deploy a django application. Today I bought a domain name for the app and am having trouble configuring it so that the base URL does not need the port number at the end of it. I have to type www.trackthecharts.com:8001 to see the website when I only want to use www.trackethecharts.com. I think the problem is somewhere in my nginx, gunicorn or supervisor configuration.
gunicorn_config.py
command = '/opt/myenv/bin/gunicorn'
pythonpath = '/opt/myenv/top-chart-app/'
bind = '162.243.76.202:8001'
workers = 3
root#django-app:~#
nginx config
server {
server_name 162.243.76.202;
access_log off;
location /static/ {
alias /opt/myenv/static/;
}
location / {
proxy_pass http://127.0.0.1:8001;
proxy_set_header X-Forwarded-Host $server_name;
proxy_set_header X-Real-IP $remote_addr;
add_header P3P 'CP="ALL DSP COR PSAa PSDa OUR NOR ONL UNI COM NAV"';
}
}
supervisor config
[program:top_chart_gunicorn]
command=/opt/myenv/bin/gunicorn -c /opt/myenv/gunicorn_config.py djangoTopChartApp.wsgi
autostart=true
autorestart=true
stderr_logfile=/var/log/supervisor_gunicorn.err.log
stdout_logfile=/var/log/supervisor_gunicorn.out.log
Thanks for taking a look.
You should bind to port 80, the default http port. Then make sure in /etc/nginx/sites-enabled/, your are listening on port 80.
By binding to port 80, you will not need to explicitly specify one in your url.