Develop Reference

Static images disappeared after using AWS for Heroku site

I made a site for Heroku using Django and I got it to the point where it kept all the static images and files on Heroku just fine but the images the user uploads got deleted on dyno reset; that's why I wanted to use AWS to host the files the user uploads.
This is the code I am using in my settings:
AWS_ACCESS_KEY_ID = os.environ.get('my key')
AWS_SECRET_ACCESS_KEY = os.environ.get('my secret key')
AWS_STORAGE_BUCKET_NAME = 'my bucket name'
STATICFILES_STORAGE = 'storages.backends.s3boto3.S3StaticStorage'
DEFAULT_FILE_STORAGE = 'storages.backends.s3boto3.S3StaticStorage'
STATIC_URL = 'http://' + AWS_STORAGE_BUCKET_NAME + '.s3.eu-west-1.amazonaws.com/'
ADMIN_MEDIA_PREFIX = STATIC_URL + 'admin/'
AWS_QUERYSTRING_AUTH = False
I added the "eu-west-1" part in the static URL because it was in the bucket's URL but not in my site's src for the image.
The problem is that now most of my JavaScript and CSS have disappeared as have all my static files that were previously just on Heroku and worked fine, furthermore the files that the user uploads also don't show up and the src doesn't containt the "eu-west-1" that I added (and it doesn't work without that part either).
Can somebody help me make it so that my static files are on Heroku as before while user uploaded files are on AWS?

How to pull or read image in Django from AWS s3 bucket for OpenCV to process?

When user uploads images from (react as frontend) and django receives the images from static folder and begins to process those images and later user can download.
Below is my django python code
imagePath = os.path.commonprefix(['images']) #<------coming from static files when user
# upload via react axios.post. instead
# of "imagePath" i want django to read
# from s3 bucket 'images' folder.
#other process
for img in imagePath:
image = cv2.imread(img)
##other process
but now i'm using amzazon s3 instead of local static files.
AWS_ACCESS_KEY_ID = '********'
AWS_SECRET_ACCESS_KEY = '***************'
AWS_STORAGE_BUCKET_NAME = '********'
AWS_S3_FILE_OVERWRITE = False
AWS_DEFAULT_ACL = None
DEFAULT_FILE_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
STATICFILES_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
my question is :
How to make django to read images directly from s3 bucket? so that instead of "imagePath" i can use s3 files?
Any related answer will be really appreciated thank you :)
I hope i get response ! i'am even ready to shift to any server if my code plan works there ! so give me any idea how to work my plan.
Hope i get some response !

#You are missing the url settings for static and media. Here is what to you can do, based on my days of struggle on the matter of S3 - django - static files.
1. Make sure you have the right S3 Permissions and Policy in place. Access 'Policy' via permissions tab in AWS S3 console for your specific bucket.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Statement1",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::<your_bucket_name>/*"
},
{
"Sid": "Statement2",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::<your_bucket_name>/images/"
}
]
}
2. Install WhiteNoiseMiddleware & django-storages in your project environment.
pip install whitenoise
pip install django-storages
3. Add the following to MIDDLEWARE= in settings.py
'whitenoise.middleware.WhiteNoiseMiddleware',
4. Following additions in settings.py are required to handle URLs from S3 correctly. The handling is done by django middleware & django-storages automatically
STATICFILES_LOCATION = 'static'
MEDIAFILES_LOCATION = 'media'
AWS_S3_CUSTOM_DOMAIN = '%s.s3.amazonaws.com' % os.environ['BUCKET_NAME']
AWS_ACCESS_KEY_ID = os.environ['AWS_KEY']
AWS_SECRET_ACCESS_KEY = os.environ['AWS_ACC_KEY']
AWS_STORAGE_BUCKET_NAME = os.environ['BUCKET_NAME']
AWS_S3_FILE_OVERWRITE = False
AWS_DEFAULT_ACL = None
DEFAULT_FILE_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'
STATICFILES_STORAGE = 'storages.backends.s3boto3.S3StaticStorage'
MEDIA_ROOT = os.path.join (BASE_DIR, 'static/images/')
STATIC_ROOT = os.path.join (BASE_DIR, 'static')
STATIC_URL = "https://%s/%s/" % (AWS_S3_CUSTOM_DOMAIN, STATICFILES_LOCATION)
MEDIA_URL = "https://%s/%s/" % (AWS_S3_CUSTOM_DOMAIN, MEDIAFILES_LOCATION)
5. For aiming uploads into a precise S3 Bucket folder. (Additional)
In setting.py set media root:
MEDIA_ROOT = os.path.join (BASE_DIR, 'static/images/')
In models.py use ImageFiled and add upload_to= takes in a folder name and creates it with the first upload!:
image_variable = models.ImageField(null=True, default="{default_filename)", upload_to='uploads/')
Reference: django-storages , whiteNoiseMiddelware, S3 Access Troubleshooting

Here Is the Best Solution. Link Image read by presignedurl and in Opencv.

Django with S3 staticfiles optimization

I am trying to optimize my staticfiles using django with S3. I am using django compressor to compress and cache js and css files.
Here are my settings :
AWS_ACCESS_KEY_ID = access_key
AWS_SECRET_ACCESS_KEY = secret_key
AWS_STORAGE_BUCKET_NAME='mybucketname'
AWS_QUERYSTRING_AUTH = False
S3_URL = 'https://%s.s3.amazonaws.com/' %AWS_STORAGE_BUCKET_NAME
MEDIA_URL = S3_URL + "media/"
STATIC_URL = S3_URL + "static/"
ADMIN_MEDIA_PREFIX = STATIC_URL + "admin/"
STATICFILES_DIRS = (
os.path.join(BASE_DIR, "static","static_dirs"),
#'/var/www/static/',
)
AWS_HEADERS = {
'Cache-Control': 'public,max-age=86400',
}
STATIC_ROOT = os.path.join(BASE_DIR, "static","static_root")
STATICFILES_STORAGE = 'lafabrique.settings.s3utils.CachedS3BotoStorage'
DEFAULT_FILE_STORAGE = 'lafabrique.settings.s3utils.MediaRootS3BotoStorage'
COMPRESS_STORAGE = 'lafabrique.settings.s3utils.CachedS3BotoStorage'
COMPRESS_URL = S3_URL
and in another file :
class CachedS3BotoStorage(S3BotoStorage):
def __init__(self, *args, **kwargs):
super(CachedS3BotoStorage, self).__init__(*args, **kwargs)
self.local_storage = get_storage_class(
"compressor.storage.GzipCompressorFileStorage")()
def save(self, name, content):
name = super(CachedS3BotoStorage, self).save(name, content)
self.local_storage._save(name, content)
return name
What I don't understand is that when I test my page on https://developers.google.com/speed/pagespeed/insights/, google still tells me that I should use gzip and cache on my static files...
Also in my amazon http response I get : Cache-Control:max-age=0 ... ( actual website is lafabrique.io, just in case)
Does somebody know what I did wrong ?
Thanks a lot

Are you using Django-storages? Try adding this to your settings:
AWS_IS_GZIPPED = True
GZIP_CONTENT_TYPES = (
'text/css',
'application/javascript',
'application/x-javascript',
'text/javascript'
)
It looks like you're using gzipped storage on your local machine, but not for the file that you upload to S3.
For the caching issue, try the solution here: Trouble setting cache-cotrol header for Amazon S3 key using boto

Error importing static files from S3 Bucket Django

I tried uploading my static and media files onto AWS S3 bucket but, the css is not being rendered.
settings.py
AWS_HEADERS = { # see http://developer.yahoo.com/performance /rules.html#expires
'Expires': 'Thu, 31 Dec 2099 20:00:00 GMT',
'Cache-Control': 'max-age=94608000',
}
AWS_STORAGE_BUCKET_NAME = '*'
AWS_ACCESS_KEY_ID = '*'
AWS_SECRET_ACCESS_KEY = '*'
AWS_S3_CUSTOM_DOMAIN = '%s.s3.amazonaws.com' % AWS_STORAGE_BUCKET_NAME
STATIC_URL = "https://%s/" % AWS_S3_CUSTOM_DOMAIN
STATICFILES_STORAGE = 'storages.backends.s3boto.S3BotoStorage'
STATICFILES_LOCATION = 'static'
STATICFILES_STORAGE = 'custom_storages.StaticStorage'
STATIC_URL = "https://%s/%s/" % (AWS_S3_CUSTOM_DOMAIN, STATICFILES_LOCATION)
MEDIAFILES_LOCATION = 'media'
MEDIA_URL = "https://%s/%s/" % (AWS_S3_CUSTOM_DOMAIN, MEDIAFILES_LOCATION)
DEFAULT_FILE_STORAGE = 'custom_storages.MediaStorage'
And when I try to access the css file through its source I get the following message -
<Error><Code>AccessDenied</Code><Message>Access Denied</Message>
<RequestId>BD606AFA0AF488B7</RequestId>
<HostId>2aZOfc5qdORtn2VhfDQsJ2gZHPoYBeBV9ciAdOnk+wGuK7azaWngXwfv+rEy1XLMFOrOgs+qDpI= </HostId>
</Error>

Sorry was making a silly mistake. The root of the static folders was not specified in STATICFILES_DIRS so collectstatic command couldn't collect those files.

Django-Storages + Easy_Thumbnails: [Errno 30] Read-only file system

I have installed easy_thumbnails and am trying to deploy my solution on to S3. I'm using https://github.com/jamstooks/django-s3-folder-storage to separate my /media/ and /static/ folders, with media containing uploaded content.
My settings file works like this:
# static file config
DEFAULT_FILE_STORAGE = 's3_folder_storage.s3.DefaultStorage'
DEFAULT_S3_PATH = "media"
STATICFILES_STORAGE = 's3_folder_storage.s3.StaticStorage'
STATIC_S3_PATH = "static"
AWS_ACCESS_KEY_ID = os.environ.get('AWS_ACCESS_KEY_ID')
AWS_SECRET_ACCESS_KEY = os.environ.get('AWS_SECRET_ACCESS_KEY')
MEDIA_ROOT = '/media/'
MEDIA_URL = 'https://%s.s3.amazonaws.com/media/' % AWS_STORAGE_BUCKET_NAME
STATIC_ROOT = "/%s/" % STATIC_S3_PATH
STATIC_URL = '//%s.s3.amazonaws.com/static/' % AWS_STORAGE_BUCKET_NAME
ADMIN_MEDIA_PREFIX = STATIC_URL + 'admin/'
AWS_PRELOAD_METADATA = True
CKEDITOR_UPLOAD_PATH = 'uploads'
AWS_DEFAULT_ACL = 'public-read'
ADMIN_MEDIA_PREFIX = STATIC_URL + 'admin/'
But I'm encountering this error:
TemplateSyntaxError at /
Couldn't get the thumbnail teams/alumni/images/thumbs/alumni.png: [Errno 30] Read-only file system: '/media'

After doing a lot of research I discovered http://gibuloto.com/blog/easy-thumbnails-with-amazon-s3/. This should resolve any issues one may be having in implementing easy_thumbnails using S3.