Develop Reference

AWS EC2, LINUX, NGINX, DJANGO, GUNICORN static files not found

I am currently trying to launch my Django website using Nginx and Gunicorn but my static files are not being found. I am on a Linux AMI not Ubuntu which makes things a bit harder and different because I got Nginx with
sudo amazon-linux-extras install nginx1.12
I was looking at these tutorials
http://www.threebms.com/index.php/2020/07/27/set-up-a-django-app-in-aws-with-gunicorn-nginx/
https://linuxtut.com/en/ce98f7afda7738c8cc1b/
but whenever I launch my website with
gunicorn --bind 0.0.0.0:8000 myappname.wsgi
it always says my static files are not found....
I have already done
python manage.py collectstatic
STATIC_URL = '/static/'
STATIC_ROOT = os.path.join(BASE_DIR, "static/")
This is my config file found at. sudo vi /etc/nginx/nginx.conf
I don't really know if I should keep the first server part that was there as default but the only part that is not default is the second server but the tutorials say just add a new one to the end
# For more information on configuration, see:
# * Official English Documentation: http://nginx.org/en/docs/
# * Official Russian Documentation: http://nginx.org/ru/docs/
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;
# Load dynamic modules. See /usr/share/doc/nginx/README.dynamic.
include /usr/share/nginx/modules/*.conf;
events {
worker_connections 1024;
}
http {
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 4096;
include /etc/nginx/mime.types;
default_type application/octet-stream;
# Load modular configuration files from the /etc/nginx/conf.d directory.
# See http://nginx.org/en/docs/ngx_core_module.html#include
# for more information.
include /etc/nginx/conf.d/*.conf;
server {
listen 80;
listen [::]:80;
server_name _;
root /usr/share/nginx/html;
# Load configuration files for the default server block.
include /etc/nginx/default.d/*.conf;
error_page 404 /404.html;
location = /404.html {
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
}
}
server {
listen 8000;
#not real address but same format
server_name 12.18.123.613;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/
{
autoindex on;
alias /home/ec2-user/pydjangoenv/myprojname/static/;
}
location / {
proxy_pass http://12.18.123.613;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
# Settings for a TLS enabled server.
#
# server {
# listen 443 ssl http2;
# listen [::]:443 ssl http2;
# server_name _;
# root /usr/share/nginx/html;
#
# ssl_certificate "/etc/pki/nginx/server.crt";
# ssl_certificate_key "/etc/pki/nginx/private/server.key";
# ssl_session_cache shared:SSL:1m;
# ssl_session_timeout 10m;
# ssl_ciphers PROFILE=SYSTEM;
# ssl_prefer_server_ciphers on;
#
# # Load configuration files for the default server block.
# include /etc/nginx/default.d/*.conf;
and the tree goes like this
Django-cloud9 - /home/ec2/user
pydjangoenv
myprojname
blog - this is an app
myprojname
static
users - this is an app
manage.py
requirements.txt
env
I have really been stuck on this for three days, any help is appreciated :-)
EDIT
After adding
urlpatterns =+ static(settings.STATIC_URL, document_root=settings.STATIC_ROOT)
to my urls.py the Django serves the static files, but not when debug is false
the static files don't work, I really have tried everything, please help.

nginx - upstream sent too big header while reading response header from upstream

I have a e-commerce project written in python and flask framework, I keep shopping cart information on session, when I try add product to session, nginx gives this error:
upstream sent too big header while reading response header from upstream, client: xx.xxx.xx.xxx, server: mysite.com, request: "POST /add_to_cart HTTP/1.1", upstream: "uwsgi://unix:/path/uwsgi.sock:", host: "mysite.com"
This occurs when I have a lot of information in session,
I tried adding fastcgi and proxy_buffer parameters, but still not working, here is my nginx conf file:
server {
listen 443 ssl;
server_name mysite.com;
ssl_certificate /path/nginx.pem;
ssl_certificate_key /path/nginx.key;
include /etc/letsencrypt/options-ssl-nginx.conf;
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;
access_log /path/access.log main;
fastcgi_buffers 16 16k;
fastcgi_buffer_size 32k;
proxy_buffering on;
proxy_buffer_size 128k;
proxy_buffers 4 256k;
proxy_busy_buffers_size 256k;
location /static/ {
alias /path/web/static/;
access_log off;
index index.html index.htm;
}
location / {
try_files $uri #uwsgi;
root /path/www/;
index index.html index.htm;
}
location #uwsgi {
include uwsgi_params;
uwsgi_pass unix:/path/web/uwsgi.sock;
}
}

If you're capable of reconstructing exact POST request via curl or otherwise measure the actual header size, you can specify the proper size for uwsgi_buffer_size (the directive that is related in your case).
Here's my post that has some insight into a similiar directive, proxy_buffer_size. There are many *_buffer_size directives, each "proxy"-like NGINX module has one (fastcgi, proxy, uwsgi), but how you approach their tuning (and how they essentially work) is the same.
You can try, without measurement, by placing directly in server block:
uwsgi_buffer_size 16k;
uwsgi_busy_buffers_size 24k;

Tried everything but still cannot serve static files of Django project via nginx+gunicorn

I have to deploy Django project via nginx+gunicorn(static files by nginx and other files by gunicorn.)
dev.py file inside settings folder of the project have these settings -
STATIC_ROOT = os.path.join(BASE_DIR,'static')
STATIC_URL = '/static/'
nginx.conf looks like this -
server {
listen 8000;
server_name localhost;
access_log /home/user/access.log;
error_log /home/user/error.log;
location / {
proxy_pass http://localhost:8001;
}
location /static/ {
autoindex on;
alias /home/user/project_revamped/project/static;
}
}
I am running my server as -
gunicorn project.wsgi:application --bind=127.0.0.1:8001
On accessing server at URL at localhost:8001, page shows up without any static files.
The error.log of nginx does not get populated. The error is that browser is trying to request static files from url
localhost:8001/static/...
however it should request files from URL -
localhost:8000/static/...
as nginx is configured to serve through that url.

server {
listen 8000;
server_name localhost;
root /home/user/project_dir/project;
access_log /home/user/access.log;
error_log /home/user/error.log;
location /static/ {
alias /home/user/project_dir/project/static/;
}
location / {
proxy_set_header Host $http_host;
proxy_pass http://localhost:8001;
proxy_redirect default;
}
i think this should be correct configuration for your case.
ideally you should listen all the http request on port 80 ,so listen
8000 shall be changed to 80 in production.
if you write location / before and static things after code would
never reach the location /static/
there must be "proxy_set_header Host $http_host;" so that in debug=
False (in production) allowed host gets some value

django production deployment does not load static files

I've been trying for about a whole day to deploy Django's static files in production, but till now I had no luck, so I absolutely need community's help!
My nginx config is:
worker_processes 1;
user nobody nogroup;
pid /tmp/nginx.pid;
error_log /tmp/nginx.error.log;
events {
worker_connections 1024;
accept_mutex off;
}
http {
include mime.types;
default_type application/octet-stream;
access_log /tmp/nginx.access.log combined;
sendfile on;
upstream app_server {
server unix:/tmp/gunicorn.sock fail_timeout=0;
# For a TCP configuration:
# server 192.168.0.7:8000 fail_timeout=0;
}
server {
listen 80 default;
client_max_body_size 4G;
server_name _;
keepalive_timeout 5;
# path for static files
root /home/ubuntu/src/static;
location / {
# checks for static file, if not found proxy to app
try_files $uri #proxy_to_app;
}
location #proxy_to_app {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://app_server;
}
error_page 500 502 503 504 /500.html;
location = /500.html {
root /home/ubuntu/src/static;
}
}
}
and in my django settings.py file I've set the following:
STATIC_URL = '/static/'
STATICFILES_DIRS = ()
STATICFILES_FINDERS = (
'django.contrib.staticfiles.finders.FileSystemFinder',
'django.contrib.staticfiles.finders.AppDirectoriesFinder',
)
STATIC_ROOT = os.path.join(BASE_DIR, 'static')
and of course I've run the collectstatic django command and the folder in path /home/ubuntu/src/static exists containing all appropriate files.
Still my deployment does not server any static file :/
Does anybody have any idea what might be wrong with my setup?
Thank you in advance

I used a great tutorial here for my setup. Hopefully this can help. One obvious change I can see is using location /static/ for static files and just forwarding location / straight to gunicorn (or uwsgi in my case)

Add static alias to server conf
location /static {
alias /home/ubuntu/src/static; # your Django project's static files - amend as required
}
Full configuration should be like
server {
listen 80 default;
client_max_body_size 4G;
server_name _;
keepalive_timeout 5;
# path for static files
# root /home/ubuntu/src/static;
location /static {
alias /home/ubuntu/src/static; # your Django project's static files - amend as required
}
location / {
# checks for static file, if not found proxy to app
try_files $uri #proxy_to_app;
}
location #proxy_to_app {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://app_server;
}
error_page 500 502 503 504 /500.html;
location = /500.html {
root /home/ubuntu/src/static;
}
}
}

NGINX/Django: 403 error when serving static files

I've spent many, many hours debugging this, and have not quite come to a solution yet. I've tried applying the solutions from the dozen or so relevant threads, but none solved the problem (that or I implemented the solution wrong).
I'm trying to serve media and static files using nginx and django, and am using a media file to test with (following this tutorial). The logs show that it's trying to grab the correct file, but it just doesn't have the permissions to do so (failed (13: Permission denied)). Everything is set to 755. My nginx.conf and mysite_nginx.conf are as follows. Any ideas?
nginx.conf
worker_processes 1;
error_log /usr/local/etc/nginx/logs/error.log debug;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /usr/local/etc/nginx/logs/access.log main;
sendfile on;
keepalive_timeout 65;
index index.html index.php;
include /usr/local/etc/nginx/sites-enabled/mysite_nginx.conf;
}
mysite_nginx.conf
upstream django {
server 127.0.0.1:8001; # for a web port socket (we'll use this first)
}
server {
listen 8000;
server_name localhost;
root /var/www/;
access_log /usr/local/etc/nginx/logs/default.access.log main;
location / { } #go to default index.html
location /media/ {
alias /Users/meep_me/Desktop/ram_stuff/combining/box/mysite/media/; # your Django project's media files
}
location /static/ {
alias /Users/meep_me/Desktop/ram_stuff/combining/box/mysite/static/; # your Django project's static files
}
location /other {
uwsgi_pass django;
include /Users/meep_me/Desktop/ram_stuff/combining/box/mysite/uwsgi_params; # the uwsgi_params file you installed
}
}

/Users/meep_me/Desktop/ram_stuff/combining/box/mysite/media/: Do each of the folders have the +x bit set for permissions (usually required to list/access a folder/subfolder)?

The fix was as mentioned by warath-coder in the comments. I forgot to explicitly make sure every single folder actually had the +x bit set. Going through and finding the one that didn't solved the issue.