Develop Reference

For Django file upload can I set the models.FileField directly without using any Form or web input

Every example of uploading I can find for Django uses a web input to get the file. But I am getting the files from an array of REST api end points and I receive the files as byte strings. I am using a Model class with boto3 backend storage to upload to S3. I can upload files using the django admin but fail to do it manually. The actual code is too long but here is some pseudo code. My question is what is the data type I need for x in this line:
zipf.zipfile = x
Here is the pseudo code:
class UploadedZip(models.Model):
user = models.ForeignKey(CustomUser, on_delete=models.CASCADE, related_name='zipfiles')
name = models.CharField(max_length=102)
zipfile = models.FileField(upload_to="zipfile")
def getfiles(request):
files = apicall() # Returns include a list of links
for f in files:
# the return can be a byte string of data or I could save it to a tmp file if I need to
x = callapi_inline(request, f.apiurl)
if x and len(x) > 0:
zipf = UploadedZip()
zipf.zipfile = x
zipf.name = f[0]['name']
zipf.user = request.user
zipf.save()

Unable to save webp file to model field

I'm not sure if this is entirely django related, but if someone could help me, that would be so much appreciated! I'm having trouble generating a webp file from the following code
from io import BytesIO
from PIL import Image
import requests
I've got the following model
class UserImage(models.Model):
user_provided_image = VersatileImageField(upload_to=folder10, null=True, blank=True)
nextgen_image = models.FileField(upload_to=folder10,null=True, blank=True) #for WebP images
I'm creating a webp file. This code works, but it saved it to the file to the root directory of my project and I'm not sure how to save it to the FileField (i.e. nextgen_image ) on my model
def create_webp_image(sender, instance, *args, **kwargs):
image_url = instance.image.thumbnail['1920x1080'].url
try:
response = requests.get(image_url, stream=True)
path = image_url
except: #local env
path = "http://localhost:8000" + image_url
response = requests.get(path, stream=True)
img = Image.open(BytesIO(response.content))
#build file path
position = path.rfind("/") + 1
newpath = path[0:position]
#build file name
image_name = path[position:]
name_of_file = image_name.split('.')[0] + ".webp"
#this creates the webp file
img.save(name_of_file,"webp")
#save image to model
#instance.nextgen_image = ?
post_save.connect(create_webp_image, sender=UserImage)
Thanks!

You can use something like that:
from django.core.files.base import ContentFile
...
img_content = ContentFile(BytesIO(response.content))
instance.nextgen_image.save(name_of_file, img_content, save=True)
...

If you want to use packages to get the job done then use this package django-resized.
Based on the code provided above this should do the trick. Hoping that this will solve your issue. Cheers
nextgen_image = ResizedImageField(force_format="WEBP",
upload_to=folder10,null=True, blank=True)

Django-admin: images in zip file and inserting each image info in a database

I want to build an image gallery using Django. Each image is a post, of course. Now, I don't want to be uploading each image independently. I want to zip them all and upload them in the Django admin page and maybe create some kind of a trigger to:
decompressed the zip
read all the images info
store the info in a database, each image in a row
Is this possible with Django? what would be your best way to accomplish this? I'll appreciate any kind of help, I'm very new to Django (like 5 hours new)

Yes, it's possible. Here is a broad outline inspired entirely by how Mezzanine implements this.
First you define a field for accepting the zip file:
class BaseGallery(models.Model):
zip_import = models.FileField(blank=True, upload_to=upload_to("galleries")
Then you have a separate model that is Foreign keyed to your parent model. In the example here the parent model is BaseGallery and the image model is GalleryImage:
class GalleryImage(Orderable):
gallery = models.ForeignKey(Gallery, related_name="images")
file = models.ImageField(upload_to="galleries")
Then in your model's save method, you can extract this zip file and save the individual images:
from django.core.files import ContentFile
from django.conf import settings
from zipfile import ZipFile
def save(self, delete_zip_import=True, *args, **kwargs):
"""
If a zip file is uploaded, extract any images from it and add
them to the gallery, before removing the zip file.
"""
super(BaseGallery, self).save(*args, **kwargs)
if self.zip_import:
zip_file = ZipFile(self.zip_import)
for name in zip_file.namelist():
data = zip_file.read(name)
try:
from PIL import Image
image = Image.open(BytesIO(data))
image.load()
image = Image.open(BytesIO(data))
image.verify()
except ImportError:
pass
except:
continue
name = os.path.split(name)[1]
# You now have an image which you can save
path = os.path.join(settings.MEDIA_ROOT, "galleries",
native(str(name, errors="ignore")))
saved_path = default_storage.save(path, ContentFile(data))
self.images.create(file=saved_path)
if delete_zip_import:
zip_file.close()
self.zip_import.delete(save=True)
Note, the bit where the image is actually saved has been simplified and if you look at the source I have linked to there is a bit more jiggery-pokery required to handle unicode file names etc.
Also note that Mezzanine uses its own FileField which isn't the same as Django's FileField. I have tried to refactor this in the example above.

Here's code to extract files from an uploaded ZIP file:
source
import zipfile
localFile = 'beer.zip'
with zipfile.ZipFile(localFile, 'r') as myzip:
myzip.extractall()
print 'got:', [info.filename for info in myzip.infolist()]
reference
Python zipfile module

changing file name in django

I have a this model...
class MyModel(models.Model):
...
file = models.FileField(upload_to='files/',null=True, blank=True)
...
when i upload a file, example file name is docfile.doc. when i change the file or i rewrite it and upload again docfile.doc the file will become docfile_1.doc and the old docfile.doc is still exist.
i am doing the uploading and saving data in django-admin
my question is, how can i remove the old docfile.doc if i upload the new docfile.doc and the file name is still docfile.doc?
can anyone help me in my case? thanks in advance
i try this one :
def content_file_name(instance, filename):
print instance
print filename
file = os.path.exists(filename)
print file
if file:
os.remove(filename)
return "file/"+str(filename)
class MyModel(models.Model):
...
file = models.FileField(upload_to=content_file_name,null=True, blank=True)
...
but nothing happend, when i upload docfile.doc again, it will become docfile_1.doc and the old docfile.doc still exist.

i got it... i use this
def content_file_name(instance, filename):
print instance
print filename
file = os.path.exists("media/file/"+str(filename))
print file
if file:
os.remove("media/file/"+str(filename))
return "file/"+str(filename)

I don't know exactly how to do it, but i think these links can help you:
Here you can find the two options that a FileField accept. The one that i think will interest you the most is FileField.storage. You can pass a storage object in that parameter.
It says:
FileField.storage: Optional. A storage object, which handles the storage and retrieval of your files.
Then, if you read this you would see that you can write your own storage object. Here is some explanation on how to do it. I think that you could just override the _save method in order to accomplish what you want to do (i.e: if the file already exists, remove it before saving the new copy.)
But be careful! I don't know which is the source of the files you are going to store. Maybe, your app is going to recieve lots of files with the same name, although they are all different. In this case, you would want to use a callable as the FileField.upload_to parameter, so that determine a unique filename for each file your site recieve.
I hope this helps you!

You could also have a look here: ImageField overwrite image file with same name
Define your own storage and overwrite its get available_name method.

The next code solves your problem. You override pre_save method where image is actually saved to storage. Please, rename functions for your project. Use newly created image field ImageFieldWithPermantName with your upload_to function (content_file_name).
If the code is too complicated you could simplify it. I use the code to do more complex operations for uploading images: I create thumbnails on-the-fly in custom _save_image function. So, you can simplify it.
from PIL import Image
import StringIO
from django.db.models import ImageField
from django.db.models.fields.files import FileField
from dargent.settings import MEDIA_ROOT
import os
class ImageFieldWithPermanentName( ImageField ):
def pre_save( self, model_instance, add ):
file = super( FileField, self ).pre_save(model_instance, add)
if file and not file._committed:
if callable( self.upload_to ):
path = self.upload_to( model_instance, "" )
else:
path = self.upload_to
file.name = path # here we set the same name to a file
path = os.path.join( MEDIA_ROOT, path )
chunks = _get_chunks( file.chunks() )
_save_image( chunks, path )
return file
def _get_chunks( chunks ):
chunks_ = ""
for chunk in chunks:
chunks_ += chunk
return chunks_
def _get_image( chunks ):
chunks_ = ""
for chunk in chunks:
chunks_ += chunk
virt_file = StringIO.StringIO( chunks_ )
image = Image.open( virt_file )
return image
def _save_image( chunks, out_file_path ):
image = _get_image( chunks )
image.save( out_file_path, "JPEG", quality = 100 )

ZipExtFile to Django File

I am wondering whether there is a way to upload a zip file to django web server and put the zip's files into django database WITHOUT accessing the actual file system in the process (e.g. extracting the files in the zip into a tmp dir and then load them)
Django provides a function to convert python File to Django File, so if there is a way to convert ZipExtFile to python File, it should be fine.
thanks for help!
Django model:
from django.db import models
class Foo:
file = models.FileField(upload_to='somewhere')
Usage:
from zipfile import ZipFile
from django.core.exceptions import ValidationError
from django.core.files import File
from io import BytesIO
z = ZipFile('zipFile')
istream = z.open('subfile')
ostream = BytesIO(istream.read())
tmp = Foo(file=File(ostream))
try:
tmp.full_clean()
except Validation, e:
print e
Output:
{'file': [u'This field cannot be blank.']}
[SOLUTION] Solution using an ugly hack:
As correctly pointed out by Don Quest, file-like classes such as StringIO or BytesIO should represent the data as a virtual file. However, Django File's constructor only accepts the build-in file type and nothing else, although the file-like classes would have done the job as well. The hack is to set the variables in Django::File manually:
buf = bytesarray(OPENED_ZIP_OBJECT.read(FILE_NAME))
tmp_file = BytesIO(buf)
dummy_file = File(tmp_file) # this line actually fails
dummy_file.name = SOME_RANDOM_NAME
dummy_file.size = len(buf)
dummy_file.file = tmp_file
# dummy file is now valid
Please keep commenting if you have a better solution (except for custom storage)

There's an easier way to do this:
from django.core.files.base import ContentFile
uploaded_zip = zipfile.ZipFile(uploaded_file, 'r') # ZipFile
for filename in uploaded_zip.namelist():
with uploaded_zip.open(filename) as f: # ZipExtFile
my_django_file = ContentFile(f.read())
Using this, you can convert a file that was uploaded to memory directly to a django file. For a more complete example, let's say you wanted to upload a series of image files inside of a zip to the file system:
# some_app/models.py
class Photo(models.Model):
image = models.ImageField(upload_to='some/upload/path')
...
# Upload code
from some_app.models import Photo
for filename in uploaded_zip.namelist():
with uploaded_zip.open(filename) as f: # ZipExtFile
new_photo = Photo()
new_photo.image.save(filename, ContentFile(f.read(), save=True)

Without knowing to much about Django, i can tell you to take a look at the "io" package.
You could do something like:
from zipfile import ZipFile
from io import StringIO
zname,zipextfile = 'zipcontainer.zip', 'file_in_archive'
istream = ZipFile(zname).open(zipextfile)
ostream = StringIO(istream.read())
And then do whatever you would like to do with your "virtual" ostream Stream/File.

I've used the following django file class to avoid the need to read ZipExtFile into a another datastructure (StingIO or BytesIO) while properly impelementing what Django needs in order to save the file directly.
from django.core.files.base import File
class DjangoZipExtFile(File):
def __init__(self, zipextfile, zipinfo):
self.file = zipextfile
self.zipinfo = zipinfo
self.mode = 'r'
self.name = zipinfo.filename
self._size = zipinfo.file_size
def seek(self, position):
if position != 0:
#this will raise an unsupported operation
return self.file.seek(position)
#TODO if we have already done a read, reopen file
zipextfile = archive.open(path, 'r')
zipinfo = archive.getinfo(path)
djangofile = DjangoZipExtFile(zipextfile, zipinfo)
storage = DefaultStorage()
result = storage.save(djangofile.name, djangofile)