Develop Reference

How to serve staticfiles from docker django app to hosted nginx

My goal is to serve the staticfiles to nginx on ubuntu from a docker django app container without using docker nginx
I setup the reverse proxy to django all works fine in dev mode but when i turn Debug to False nginx doesn't recognize the staticfiles path
here's a screenshot
Here's my dockerfile for django app
FROM python:3.9
RUN mkdir /app
WORKDIR /app
COPY . /app
RUN pip install pipenv
RUN pipenv install --system --deploy --ignore-pipfile
EXPOSE 8000
ENTRYPOINT ["python", "manage.py"]
CMD ["runserver", "0.0.0.0:8000"]
And here's my nginx config
upstream django {
server 127.0.0.1:8001;
}
server {
server_name django.com;
listen 80;
listen 8000;
client_max_body_size 100M;
location / {
proxy_pass http://django;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forward-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forward-Proto http;
proxy_set_header X-Nginx-Proxy true;
proxy_redirect off;
proxy_read_timeout 1000;
proxy_connect_timeout 1000;
proxy_send_timeout 1000;
}
}

To have NGINX serve your staticfiles, your config needs to be edited. How do you expect nginx to find your files otherwise?
Luckily, it's a simple edit. :)
upstream django {
server 127.0.0.1:8001;
}
server {
server_name django.com;
listen 80;
listen 8000;
client_max_body_size 100M;
location /static {
alias /PATH/TO/STATIC_FILES;
}
location / {
proxy_pass http://django;
...
}
}
Don't forget to configure your static root, and run:
py manage.py collectstatic
This will make sure Django's default CSS/JS is included in your staticfiles.
When DEBUG=True, your staticfiles will be served by Django. This is not good, and can lead to some serious perfomance and security problems.
Edit:
Alternatively, have a look at Django-Whitenoise. It's great for serving staticfiles.
http://whitenoise.evans.io/en/stable/

Nginx, fastapi and streamlit in docker - reverse proxy does not work for streamlit

i want to use docker to containerize nginx, fastapi and 2 streamlit apps. All 3 apps (fastapi, 2streamlit apps) do not interact with each other. Nginx should work as reverse-proxy for the 3 apps.
For fastapi it is working. I can send rest-api requests to http://ip:80/twxservices .
"twxservices" as endpoint is added in the app.py file and not in the nginxconfig.
The streamlit apps are not reachable through this http://ip:80/stream1 and http://ip:80/stream2 I get the error: 404: Not found
Thanks in advance for your help. Find below the filestructures and the config files.
This is the filestructure and how the reverse-proxy should work:
docker-compose file:
services:
web:
build:
context: ./nginxfolder
dockerfile: Dockerfile
container_name: web
ports:
- 80:80
networks:
- my-network
depends_on:
- app
app:
build:
context: ./twxservicesfolder
dockerfile: Dockerfile
container_name: app
networks:
- my-network
spectrum:
build:
context: ./spectrumfolder
dockerfile: Dockerfile
container_name: spectrum
networks:
- my-network
dgraph:
build:
context: ./dgraphfolder
dockerfile: Dockerfile
container_name: dgraph
networks:
- my-network
networks:
my-network:
driver: bridge
nginx config (default.conf):
upstream fastapi-backend {
server app:8000;
}
server {
listen 80;
location / {
proxy_pass http://fastapi-backend;
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_redirect off;
proxy_buffering off;
client_body_buffer_size 100M;
client_max_body_size 100M;
}
location /stream1 {
proxy_pass http://spectrum:8501/stream1;
}
location ^~ /stream1/static {
proxy_pass http://spectrum:8501/stream1/static/;
}
location ^~ /stream1/healthz {
proxy_pass http://spectrum:8501/stream1/healthz;
}
location ^~ /stream1/vendor {
proxy_pass http://spectrum:8501/stream1/vendor;
}
location /stream1/spectrum {
proxy_pass http://spectrum:8501/stream1/spectrum;
proxy_http_version 1.1;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_read_timeout 86400;
}
location /stream2 {
proxy_pass http://dgraph:8503/stream2;
}
location ^~ /stream2/static {
proxy_pass http://dgraph:8503/stream2/static/;
}
location ^~ /stream2/healthz {
proxy_pass http://dgraph:8503/stream2/healthz;
}
location ^~ /stream2/vendor {
proxy_pass http://dgraph:8503/stream2/vendor;
}
location /stream2/dgraph {
proxy_pass http://dgraph:8503/stream2/dgraph;
proxy_http_version 1.1;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_read_timeout 86400;
}
}
Dockerfile nginx container:
FROM nginx:latest
RUN rm /etc/nginx/conf.d/default.conf
COPY default.conf /etc/nginx/conf.d
Dockerfile streamlit app dgraph container:
FROM python:3.8
COPY requirements.txt requirements.txt
RUN pip3 install -r requirements.txt
EXPOSE 8503
COPY app2.py /var/dashboard/app2.py
CMD streamlit run /var/dashboard/app2.py --server.address="0.0.0.0" --server.port="8503"
Dockerfile streamlit app spectrum container:
FROM python:3.8
COPY requirements.txt requirements.txt
RUN pip3 install -r requirements.txt
EXPOSE 8501
COPY app1.py /var/dashboard/app1.py
CMD streamlit run /var/dashboard/app1.py --server.address="0.0.0.0" --server.port="8501"
Dockerfile fastapi app twxservices container:
FROM python:3.8-slim-buster
COPY requirements.txt requirements.txt
RUN pip3 install -r requirements.txt
COPY . .
ENV PYTHONUNBUFFERED 1
EXPOSE 8000
CMD ["uvicorn","app:app","--proxy-headers","--host","0.0.0.0","--forwarded-allow-ips","*"]

I made it work with these nginx config
location /spectrum {
proxy_pass http://spectrum1:8501/spectrum;
proxy_http_version 1.1;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_read_timeout 86400;
}
and this Dockerfile config for the streamlit app:
FROM python:3.8
COPY requirements.txt requirements.txt
RUN pip3 install -r requirements.txt
EXPOSE 8501
COPY app1.py /var/dashboard/app1.py
CMD streamlit run /var/dashboard/app1.py --server.address="0.0.0.0" --server.port="8501" --server.baseUrlPath="spectrum" --server.enableCORS=false --server.enableXsrfProtection=false

Nginx not serving my Flask website

I am following this example and this answer on stackoverflow and I am stuck.
I am running this example on a digitalocean VPS. My file structure is as follows:
project structure
docker-compose.yml
mainweb/
nginx/
README
docker-compose.yml
version: '2'
services:
app:
restart: always
build: ./mainweb
command: gunicorn -w 2 -b :5000 wsgi:app
networks:
- mainnet
expose:
- "5000"
ports:
- "5000:5000"
nginx:
restart: always
build: ./nginx
networks:
- mainnet
links:
- app
volumes:
- /www/static
expose:
- 8080
ports:
- "8880:8080"
networks:
mainnet:
mainweb/
app.py
Dockerfile
requirements.txt
templates/
wsgi.py
mainweb/app.py
from flask import Flask, render_template
app=Flask(__name__)
#app.route('/')
def home()():
return render_template('templates/home.html')
if __name__=="__main__":
app.run(host="0.0.0.0", port=5000)
mainweb/Dockerfile
FROM python:3.5
MAINTAINER castellanprime
RUN mkdir /mainweb
COPY . /mainweb
WORKDIR /mainweb
RUN pip install -r requirements.txt
mainweb/templates/
home.html
mainweb/templates/home.html
<!doctype html>
<html>
<head>
<title> My website </title>
</head>
<body>
<h1> I am here </h1>
</body>
</html>
mainweb/wsgi.py
from app import app
if __name__=="__main__":
app.run()
nginx
Dockerfile
sites-enabled.conf
static/
nginx/Dockerfile
FROM nginx:1.13.1-alpine
MAINTAINER castellanprime
ADD sites-enabled.conf /etc/nginx/conf.d/sites-enabled.conf
ADD static/ /www/static/
nginx/sites-enabled.conf
server{
listen 8080;
server_name app; # Should I put my actual www.XXXXXX.XXXXX address here
charset utf-8;
location /static{
alias /www/static/;
}
location / {
proxy_pass http://app:5000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X_Forwared-For $proxy_add_x_forwarded_for;
}
}
nginx/static
css/
js/
After I run the command docker-compose up -d, I check the www.XXXXXX.com:8880, or www.XXXXXX.com:8080 from another web client on another system.
I get the standard nginx web page.
How do I redirect it to the home.html?

Take a step back and run the Flask app alone.
You have some syntax errors.
from flask import Flask, render_template
app=Flask(__name__)
#app.route('/')
def home(): # Remove double brackets
return render_template('home.html') # The templates folder is already picked up
if __name__=="__main__":
app.run(host="0.0.0.0", port=5000)
Then in a Docker container, and without gunicorn
FROM python:3.5
RUN mkdir /mainweb
COPY . /mainweb
WORKDIR /mainweb
RUN pip install -r requirements.txt
EXPOSE 5000
CMD ["python3","/mainweb/app.py"]
And run it, and see if it works.
cd mainapp
docker build -t flask:test .
docker run --rm -p 5000:5000 flask:test
Open http://server:5000
Then start on docker-compose with just that container and define nginx if you want.
nginx/Dockerfile
FROM nginx:1.13.1-alpine
ADD flask.conf /etc/nginx/conf.d/
EXPOSE 8080
nginx/flask.conf (I changed this based on a file that I have in a project)
server {
listen 8080; # This is the port to EXPOSE in nginx container
server_name app; # You can change this, but not necessary
charset utf-8;
location ^~ /static/ {
alias /usr/share/nginx/html/;
}
location / {
try_files $uri $uri/ #flask;
}
location #flask {
proxy_pass http://app:5000; # This is the port Flask container EXPOSE'd
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X_Forwared-For $proxy_add_x_forwarded_for;
}
}
And finally, the compose. You don't want to have your site exposing both 5000 and 80 (you don't want people to bypass nginx), so just don't expose 5000
version: '2'
services:
app:
restart: always
build: ./mainweb
networks:
- mainnet
nginx:
restart: always
build: ./nginx
networks:
- mainnet
links:
- app
volumes:
- ./mainweb/static:/usr/share/nginx/html
ports:
- "80:8080"
networks:
mainnet:

504 error with nginx and flask docker containers

Two days of work and I'm still stuck. I'm running separate nginx and application containers. The application container has a flask app that runs a gunicorn process on port 8000.
Everytime I nav to localhost:8080 which is the nginx port 80 is mapped to on localhost, I get a loading screen and a nginx 504 error.
This is what I see on the terminal:
docker-compose.yml
version: '2'
services:
web:
restart: always
build: ./web_app
expose:
- "8000"
ports:
- "8000:8000"
volumes:
- ./web_app:/data/web
command: /usr/local/bin/gunicorn web_interface:app -w 4 -t 90 --log-level=info -b :8000 --reload
depends_on:
- postgres
nginx:
restart: always
build: ./nginx
ports:
- "8080:80"
volumes_from:
- web
depends_on:
- web
postgres:
restart: always
image: postgres:latest
volumes_from:
- data
volumes:
- ./postgres/docker-entrypoint-initdb.d:/docker-entrypoint-initdb.d
- ./backups/postgresql:/backup
expose:
- "5432"
data:
restart: always
image: alpine
volumes:
- /var/lib/postgresql
tty: true
nginx.conf
server {
listen 80;
server_name localhost;
charset utf-8;
location /static/ {
alias /data/web/crm/web_interface;
}
location = /favicon.ico {
alias /data/web/crm/web_interface/static/favicon.ico;
}
location / {
proxy_pass http://web:8000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
nginx Dockerfile
FROM nginx:latest
RUN rm /etc/nginx/conf.d/default.conf
COPY ./nginx.conf /etc/nginx/conf.d/nginx.conf
I'll provide more info if needed to get some help on this issue that Im struggling on.

NGINX response 504 indicate gateway timeout, because of NGINX can not connection the backend server. So, you can locate the issue at proxy_pass directory.
I guess NGINX can not resolve web domain name, There is two solution:
instead of IP
location / {
proxy_pass http://<real_ip>:8000;
}
use upstream
upstream web {
server <real_ip>;
}
location / {
proxy_pass http://web:8000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}

Ok, so after three days of bashing my head, I re-started from the ground up. Rebuilt the app container and ran gunicorn.
From there I was able to determine that the gunicorn process was timing out because the database host name was incorrect. Instead of the an error being returned through my application, the failure went silent.
I fixed this by linking the postgres container and the web container. In my code I was able to use "postgres" (the name of the container) as the postgres host name.
Check the addresses to your external hosts.

Serving multiple sites with gunicorn and nginx?

I'm trying to serve both a Django and a Flask site using gunicorn and nginx. The Django site is currently available at example.com, and I'd like for the Flask site to be availabe at myapp.example.com .
My nginx configuration file is as follows for the Django site:
cat /etc/nginx/sites-available/example.com
server {
listen 80;
server_name example.com;
location /static {
alias /home/me/sites/example.com/static;
}
location / {
proxy_set_header Host $host;
proxy_pass http://unix:/tmp/example.com.socket;
}
}
I also have a second nginx config file for the Flask site:
cat /etc/nginx/sites-available/myapp.example.com
server {
listen 80;
server_name myapp.example.com;
access_log /var/log/nginx/access.log;
location / {
proxy_pass http://127.0.0.1:8001;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
I'm pretty sure that I want to change my proxy_pass variable, but am not sure what I should change it to.
My gunicorn file currently is as follows:
description "Gunicorn server for example.com"
start on net-device-up
stop on shutdown
respawn
setuid me
chdir /home/rowan/sites/example.com/source
exec ../virtualenv/bin/gunicorn \
--bind unix:/tmp/example.com.socket \
example.com.wsgi:application
Do I need a second gunicorn config file for the Flask site? Which I would then put in /etc/init/ and start through the sudo start gunicorn-myapp.example.com command?