I've spent many, many hours debugging this, and have not quite come to a solution yet. I've tried applying the solutions from the dozen or so relevant threads, but none solved the problem (that or I implemented the solution wrong).
I'm trying to serve media and static files using nginx and django, and am using a media file to test with (following this tutorial). The logs show that it's trying to grab the correct file, but it just doesn't have the permissions to do so (failed (13: Permission denied)). Everything is set to 755. My nginx.conf and mysite_nginx.conf are as follows. Any ideas?
nginx.conf
worker_processes 1;
error_log /usr/local/etc/nginx/logs/error.log debug;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /usr/local/etc/nginx/logs/access.log main;
sendfile on;
keepalive_timeout 65;
index index.html index.php;
include /usr/local/etc/nginx/sites-enabled/mysite_nginx.conf;
}
mysite_nginx.conf
upstream django {
server 127.0.0.1:8001; # for a web port socket (we'll use this first)
}
server {
listen 8000;
server_name localhost;
root /var/www/;
access_log /usr/local/etc/nginx/logs/default.access.log main;
location / { } #go to default index.html
location /media/ {
alias /Users/meep_me/Desktop/ram_stuff/combining/box/mysite/media/; # your Django project's media files
}
location /static/ {
alias /Users/meep_me/Desktop/ram_stuff/combining/box/mysite/static/; # your Django project's static files
}
location /other {
uwsgi_pass django;
include /Users/meep_me/Desktop/ram_stuff/combining/box/mysite/uwsgi_params; # the uwsgi_params file you installed
}
}
/Users/meep_me/Desktop/ram_stuff/combining/box/mysite/media/: Do each of the folders have the +x bit set for permissions (usually required to list/access a folder/subfolder)?
The fix was as mentioned by warath-coder in the comments. I forgot to explicitly make sure every single folder actually had the +x bit set. Going through and finding the one that didn't solved the issue.
Related
I am currently trying to launch my Django website using Nginx and Gunicorn but my static files are not being found. I am on a Linux AMI not Ubuntu which makes things a bit harder and different because I got Nginx with
sudo amazon-linux-extras install nginx1.12
I was looking at these tutorials
http://www.threebms.com/index.php/2020/07/27/set-up-a-django-app-in-aws-with-gunicorn-nginx/
https://linuxtut.com/en/ce98f7afda7738c8cc1b/
but whenever I launch my website with
gunicorn --bind 0.0.0.0:8000 myappname.wsgi
it always says my static files are not found....
I have already done
python manage.py collectstatic
STATIC_URL = '/static/'
STATIC_ROOT = os.path.join(BASE_DIR, "static/")
This is my config file found at. sudo vi /etc/nginx/nginx.conf
I don't really know if I should keep the first server part that was there as default but the only part that is not default is the second server but the tutorials say just add a new one to the end
# For more information on configuration, see:
# * Official English Documentation: http://nginx.org/en/docs/
# * Official Russian Documentation: http://nginx.org/ru/docs/
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;
# Load dynamic modules. See /usr/share/doc/nginx/README.dynamic.
include /usr/share/nginx/modules/*.conf;
events {
worker_connections 1024;
}
http {
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 4096;
include /etc/nginx/mime.types;
default_type application/octet-stream;
# Load modular configuration files from the /etc/nginx/conf.d directory.
# See http://nginx.org/en/docs/ngx_core_module.html#include
# for more information.
include /etc/nginx/conf.d/*.conf;
server {
listen 80;
listen [::]:80;
server_name _;
root /usr/share/nginx/html;
# Load configuration files for the default server block.
include /etc/nginx/default.d/*.conf;
error_page 404 /404.html;
location = /404.html {
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
}
}
server {
listen 8000;
#not real address but same format
server_name 12.18.123.613;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/
{
autoindex on;
alias /home/ec2-user/pydjangoenv/myprojname/static/;
}
location / {
proxy_pass http://12.18.123.613;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
# Settings for a TLS enabled server.
#
# server {
# listen 443 ssl http2;
# listen [::]:443 ssl http2;
# server_name _;
# root /usr/share/nginx/html;
#
# ssl_certificate "/etc/pki/nginx/server.crt";
# ssl_certificate_key "/etc/pki/nginx/private/server.key";
# ssl_session_cache shared:SSL:1m;
# ssl_session_timeout 10m;
# ssl_ciphers PROFILE=SYSTEM;
# ssl_prefer_server_ciphers on;
#
# # Load configuration files for the default server block.
# include /etc/nginx/default.d/*.conf;
and the tree goes like this
Django-cloud9 - /home/ec2/user
pydjangoenv
myprojname
blog - this is an app
myprojname
static
users - this is an app
manage.py
requirements.txt
env
I have really been stuck on this for three days, any help is appreciated :-)
EDIT
After adding
urlpatterns =+ static(settings.STATIC_URL, document_root=settings.STATIC_ROOT)
to my urls.py the Django serves the static files, but not when debug is false
the static files don't work, I really have tried everything, please help.
When I tried to upload a big csv file of size about 600MB in my project which is hosted in the digital ocean, it tries to upload but shows 502 Bad Gateway Error (Nginx).
The application is a data conversion application.
This works fine while working locally.
sudo tail -30 /var/log/nginx/error.log
shows
[error] 132235#132235: *239 upstream prematurely closed connection while reading response header from upstream, client: client's ip , server: ip, request: "POST /submit/ HTTP/1.1", upstream: "http://unix:/run/gunicorn.sock:/submit/", host: "ip", referrer: "http://ip/"
sudo nano /etc/nginx/sites-available/myproject
shows
server {
listen 80;
server_name ip;
client_max_body_size 999M;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
alias /root/static/;
}
location / {
include proxy_params;
proxy_pass http://unix:/run/gunicorn.sock;
}
}
nginx.conf
user root;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;
events {
worker_connections 768;
# multi_accept on;
}
http {
##
# Basic Settings
##
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
# server_tokens off;
# server_names_hash_bucket_size 64;
# server_name_in_redirect off;
include /etc/nginx/mime.types;
default_type application/octet-stream;
I have also the javascript loader running while the conversion process takes place.
How can I fix this?
If you are using django 3.1 or higher you can make your file processing asynchronous this way and return a response to the user while the file conversion takes place.
Your view should look something like this...
import asyncio
from django.http import JsonResponse
from asgiref.sync import sync_to_async
#sync_to_async
def crunching_stuff(my_file):
# do your conversion here
async def upload(request):
json_payload = {
"message": "Your file is being converted"
}
my_file = request.POST.get('file')
asyncio.create_task(crunching_stuff(my_file))
return JsonResponse(json_payload)
On the front end if you use Dropzone.js your user can see the file upload progress and will get a response quicker. this is a better user experience.
https://www.dropzonejs.com/
This error can indicate multiple problems. The fact it works for you locally strengthen the probability the issue relies on the nginx side.
You can try to solve it by increasing the timeout thresholds (as suggested here), and the buffers size. Add this to your server's nginx.conf:
proxy_read_timeout 300s;
proxy_connect_timeout 300s;
proxy_buffer_size 128k;
proxy_buffers 4 256k;
proxy_busy_buffers_size 256k;
502 error can be of anything.
Check your Nginx error log as following
tail /var/log/nginx/error.log -f
In my case it is because the header is too big. So one had to increase the buffer size in /etc/nginx/sites-enabled/default as Chen.A had described.
my application is working fine at
location / {
include uwsgi_params;
uwsgi_pass unix:/tmp/socket_file.sock;
}
but when i include static assets location location ~* .\(css|js|jpg|jpeg|gif|png|ico|zip|tgz|gz|rtf)it gives 404 not found error for css,js files
server {
listen 8080;
server_name your_domain www.your_domain;
location ~* \.(css|js|jpg|jpeg|gif|png|ico|zip|tgz|gz|rtf)$ {
expires max;
log_not_found off;
access_log off;
}
location / {
include uwsgi_params;
uwsgi_pass unix:/tmp/socket_file.sock;
}
}
nginx error log show `
"/usr/share/nginx/html/static/js/jspdf.js" failed (2: No such file or directory)
'
While my static assets are in /ProjectDirectory/assets/
locations are mutually exclusive, i.e. only one may match at a time. When the first location (regex) matches, directives in the second location / are ignored, and Nginx will proceed with whatever defined outside of this location block (if not defined). In your case, it's apparent that root /usr/share/nginx/html is defined somewhere at an outer level.
A straightforward solution would be to add the same include and uwsgi_pass to the first location block, but since you're serving static files, why not let Nginx handle it by adding root /path/to/your/static/files?
In practice, instead of matching file extension by regex, it's better to put all your files in one location, and use prefix-based matching:
location /static/ {
alias /ProjectDirectory/assets/;
}
You might want to read this answer to learn about the differences between root and alias.
I have a e-commerce project written in python and flask framework, I keep shopping cart information on session, when I try add product to session, nginx gives this error:
upstream sent too big header while reading response header from upstream, client: xx.xxx.xx.xxx, server: mysite.com, request: "POST /add_to_cart HTTP/1.1", upstream: "uwsgi://unix:/path/uwsgi.sock:", host: "mysite.com"
This occurs when I have a lot of information in session,
I tried adding fastcgi and proxy_buffer parameters, but still not working, here is my nginx conf file:
server {
listen 443 ssl;
server_name mysite.com;
ssl_certificate /path/nginx.pem;
ssl_certificate_key /path/nginx.key;
include /etc/letsencrypt/options-ssl-nginx.conf;
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;
access_log /path/access.log main;
fastcgi_buffers 16 16k;
fastcgi_buffer_size 32k;
proxy_buffering on;
proxy_buffer_size 128k;
proxy_buffers 4 256k;
proxy_busy_buffers_size 256k;
location /static/ {
alias /path/web/static/;
access_log off;
index index.html index.htm;
}
location / {
try_files $uri #uwsgi;
root /path/www/;
index index.html index.htm;
}
location #uwsgi {
include uwsgi_params;
uwsgi_pass unix:/path/web/uwsgi.sock;
}
}
If you're capable of reconstructing exact POST request via curl or otherwise measure the actual header size, you can specify the proper size for uwsgi_buffer_size (the directive that is related in your case).
Here's my post that has some insight into a similiar directive, proxy_buffer_size. There are many *_buffer_size directives, each "proxy"-like NGINX module has one (fastcgi, proxy, uwsgi), but how you approach their tuning (and how they essentially work) is the same.
You can try, without measurement, by placing directly in server block:
uwsgi_buffer_size 16k;
uwsgi_busy_buffers_size 24k;
Sorry for my poor english, I'm french... no one is perfect.
I'm trying to install ReviewBoard under Nginx (LEMP with Debian Wheezy, Nginx 1.4.5, MySQL 14.14, PHP 5.4.4). My python installation use Python 2.7.3, easy_install 0.6.24dev-r0, pip 1.1, virtualenv 1.11.4 and gunicorn 18.0.
I succeeded to install all that stuff and create a reviewboard site but I fail to access the reviewboard site from a browser.
I know that reviewboard recommends Apache but also that is possible to install it on Nginx. I can't find any step-by-step tutorial except many links to http: / /rramsden.ca/blog/2011/09/26/nginx-reviewboard/ but this link is dead. Does anyone know another link ?
My Nginx conf is:
server {
listen 80;
server_name review.unskontrollables.org;
root /var/www/review.unskontrollables.org/htdocs;
# Error handlers
error_page 500 502 503 504 /errordocs/500.html;
# serve directly - analogous for static/staticfiles
# Alias static media requests to filesystem
location /media {
alias /var/www/review.unskontrollables.org/htdocs/media;
# if asset versioning is used
if ($query_string) {
expires max;
}
}
location /static {
alias /var/www/review.unskontrollables.org/htdocs/static;
}
location /errordocs {
alias /var/www/review.unskontrollables.org/htdocs/errordocs;
}
location /favicon.ico {
alias /var/www/review.unskontrollables.org/htdocs/static/rb/images/favicon.png;
}
#~ location /admin/media/ {
#~ # this changes depending on your python version
#~ root /usr/local/lib/python2.6/site-packages/django/contrib;
#~ }
location / {
#alias /var/www/review.unskontrollables.org/htdocs/reviewboard.wsgi;
#root /var/www/review.unskontrollables.org/htdocs/reviewboard.wsgi/;
#proxy_pass_request_headers on;
proxy_pass_header Server;
proxy_set_header Host $proxy_host;
proxy_redirect off;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Scheme $scheme;
proxy_connect_timeout 10;
proxy_read_timeout 10;
proxy_pass http://127.0.0.1:8000/;
}
# Prevent the server from processing or allowing the rendering of
# certain file types.
location /media/uploaded {
types {
text/plain .html .htm .shtml .php .php3 .php4 .php5 .phps .asp .pl .py .fcgi .cgi .phtml .phtm .pht .jsp .sh .rb;
}
}
}
To launch the WSGI server, I use:
root#jotunn:~# source /opt/reviewboard/rbenv/bin/activate
(rbenv)root#jotunn:~# rb-site manage /var/www/review.unskontrollables.org/ runserver
Validating models...
0 errors found
Django version 1.4.10, using settings 'reviewboard.settings'
Development server is running at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
If I try to access
http://review.unskontrollables.org/
I'm redirected to
http://127.0.0.1:8000/r/
and the console output is:
/opt/reviewboard/rbenv/local/lib/python2.7/site-packages/Django-1.4.10-py2.7.egg/django/views/generic/list_detail.py:10: DeprecationWarning: Function-based generic views have been deprecated; use class-based views instead.
DeprecationWarning
/opt/reviewboard/rbenv/local/lib/python2.7/site-packages/Django-1.4.10-py2.7.egg/django/conf/__init__.py:75: DeprecationWarning: The ADMIN_MEDIA_PREFIX setting has been removed; use STATIC_URL instead.
"use STATIC_URL instead.", DeprecationWarning)
/opt/reviewboard/rbenv/local/lib/python2.7/site-packages/Django-1.4.10-py2.7.egg/django/views/generic/simple.py:8: DeprecationWarning: Function-based generic views have been deprecated; use class-based views instead.
DeprecationWarning
[14/Mar/2014 16:59:50] "GET / HTTP/1.0" 302 0
Does any one has an idea ?
Thanks.
EDIT: I found a solution based on the following tutorial
https://github.com/shuge/man/blob/master/sa/review/reviewboard/reviewboard-quick-install-guide.md
The missing link to the config file rb.m.com.conf point this file: https://github.com/shuge/man/blob/master/sa/review/reviewboard/rb.m.com.conf
With that data, I've just add flup (fastcgi python module) and changed my conf to the following:
server {
listen 80;
server_name review.unskontrollables.org;
root /var/www/review.unskontrollables.org/htdocs;
# Error handlers
error_page 500 502 503 504 /errordocs/500.html;
# serve directly - analogous for static/staticfiles
# Alias static media requests to filesystem
location /media {
alias /var/www/review.unskontrollables.org/htdocs/media;
# if asset versioning is used
if ($query_string) {
expires max;
}
}
location /static {
alias /var/www/review.unskontrollables.org/htdocs/static;
}
location /errordocs {
alias /var/www/review.unskontrollables.org/htdocs/errordocs;
}
location /favicon.ico {
alias /var/www/review.unskontrollables.org/htdocs/static/rb/images/favicon.png;
}
#~ location /admin/media/ {
#~ # this changes depending on your python version
#~ root /usr/local/lib/python2.6/site-packages/django/contrib;
#~ }
location / {
fastcgi_pass 127.0.0.1:9090;
fastcgi_param PATH_INFO $fastcgi_script_name;
fastcgi_param REQUEST_METHOD $request_method;
fastcgi_param QUERY_STRING $query_string;
fastcgi_param CONTENT_TYPE $content_type;
fastcgi_param CONTENT_LENGTH $content_length;
fastcgi_pass_header Authorization;
fastcgi_intercept_errors off;
}
# Prevent the server from processing or allowing the rendering of
# certain file types.
location /media/uploaded {
types {
text/plain .html .htm .shtml .php .php3 .php4 .php5 .phps .asp .pl .py .fcgi .cgi .phtml .phtm .pht .jsp .sh .rb;
}
}
}
After that nginx is restarted, everything works fine.