In my project I have a model where I use Imagekit to process an image. When I save an image I have following requirements:
rename image and thumbnail to a unique name
when a new image is loaded, the old one should be removed (and the thumbnail in the cache should refresh to the new image).
To accomplish this, I use following code:
The model:
def generate_cache_filename(instance, path, specname, extension):
extension = '.jpg'
return 'cache/images_upload/%s_%s%s' % (instance.pk, specname, extension)
def generate_image_filename_1(instance, filename):
filename = '1'
extension = '.jpg'
return 'images_upload/%s_%s%s' % (instance.pk, filename, extension)
class Model(models.Model):
name = models.CharField(max_length=40)
image_1 = ProcessedImageField([Adjust(contrast=1.2, sharpness=1.1), ResizeToFill(500, 370)], upload_to=generate_image_filename_1, format='JPEG', options={'quality': 90})
thumbnail_1 = ImageSpec([Adjust(contrast=1.2, sharpness=1.1), ResizeToFill(83, 78)], image_field='image_1', cache_to=generate_cache_filename, format='JPEG', options={'quality': 90})
The form (to delete the image when it is replaced by a new one):
if form.is_valid():
form_image = form.cleaned_data['image_1']
try:
details = Model.objects.get(pk=pk)
if details.image_1 != form_image:
details.image_1.delete(save=False)
except Model.DoesNotExist:
pass
form.save()
The part of renaming the images and replacing image_1 (= loading new and deleting old) works just fine. But for some reason the thumbnail_1 in the cache does not refresh (= is still the thumbnail of the old image).
I think it has something to do with the deletion code in the form, but I can't figure out why and how to solve it. Someone with suggestions?
UPDATE 1: it has also something to do with the 'renaming'. I did some extra tests: when I don't rename the image_1 file, then everything works fine (also refreshing of the thumbnail). But when I load another image with the same name, then I have the same problem: image_1 is updated, but thumbnail_1 is still the thumbnail of the old image.
UPDATE 2: did some more tests and when uploading a new image with the same filename, I definitely enter the if statement in try. So the old image is deleted. According to the documentation of Imagekit, the thumbnail should also be deleted. But this is not the case.
Many thanks!
In the mean time I found a working solution.
The main reason why the above code didn't work was because of the thumbnail in the cache that was not deleted after deleting the original image, especially in cases where the new image had the same filename as the previous one => then the thumbnail was never deleted.
=> I still don't know why..., because I expected that the cached image is always deleted when the original is deleted.
Using the following code everything works as expected:
Basically I made sure the new uploaded image has always another filename:
Model.py
def generate_cache_filename(instance, path, specname, extension):
extension = '.jpg'
return 'cache/images_upload/%s_%s%s' % (instance.pk, specname, extension)
# generate random string of 10 characters
def id_generator(size=10, chars=string.ascii_uppercase + string.digits):
return ''.join(random.choice(chars) for x in range(size))
def generate_random_filename(instance, filename):
filename = id_generator()
extension = '.jpg'
return 'images_upload/%s_%s%s' % (instance.pk, filename, extension)
class Model(models.Model):
name = models.CharField(max_length=20)
image_1 = ProcessedImageField([Adjust(contrast=1.2, sharpness=1.1), ResizeToFill(500, 370)], upload_to=generate_random_filename, format='JPEG', options={'quality': 90})
thumbnail_1 = ImageSpec([Adjust(contrast=1.2, sharpness=1.1), ResizeToFill(83, 78)], image_field='image_1', cache_to=generate_cache_filename, format='JPEG', options={'quality': 90})
View.py:
# thanks to jdi for the help in the for-loop
if form.is_valid():
# A for loop is used here, because my database contains 5 images and 5 thumbnails
image_list = ['image_%d' % i for i in xrange(1,6)]
for image_name in image_list:
form_image = form.cleaned_data[image_name]
try:
details = Model.objects.get(pk=pk)
if getattr(details, image_name, None) != form_image:
getattr(details, image_name, None).delete(save=False)
except Model.DoesNotExist:
pass
Hopefully this can help out others as well.
Kind Regards
Related
i'm on a struggle. The problem is with the unit testing ("test.py"), and i figured out how to upload images with tempfile and PIL, but those temporary images never get deleted. I think about making a temporary dir and then, with os.remove, delete that temp_dir, but the images upload on different media directorys dependings on the model, so i really don't know how to post temp_images and then delete them.
This is my models.py
class Noticia(models.Model):
...
img = models.ImageField(upload_to="noticias", storage=OverwriteStorage(), default="noticias/tanque_arma3.jpg")
...
test.py
def temporary_image():
import tempfile
from PIL import Image
image = Image.new('RGB', (100, 100))
tmp_file = tempfile.NamedTemporaryFile(suffix='.jpg', prefix="test_img_")
image.save(tmp_file, 'jpeg')
tmp_file.seek(0)
return tmp_file
class NoticiaTest(APITestCase):
def setUp(self):
...
url = reverse('api:noticia-create')
data = {'usuario': usuario.pk, "titulo":"test", "subtitulo":"test", "descripcion":"test", "img": temporary_image()}
response = client.post(url, data,format="multipart")
...
So, to summarize, the question is, ¿How can i delete a temporary file from different directories, taking into account that those files strictly have to be upload on those directorys?
For testing you can use the package dj-inmemorystorage and Django will not save to disk. Serializers and models will still work as expected, and you can read the data back out if needed.
In your settings, when you are in test mode, overwrite the default file storage. You can also put any other "test mode" settings in here, just make sure it runs last, after your other settings.
if 'test' in sys.argv :
# store files in memory, no cleanup after tests are finished
DEFAULT_FILE_STORAGE = 'inmemorystorage.InMemoryStorage'
# much faster password hashing, default one is super slow (on purpose)
PASSWORD_HASHERS = ['django.contrib.auth.hashers.MD5PasswordHasher']
When you are uploading a file you can use SimpleUploadFile, which is purely in-memory. This takes care of the "client" side, while the dj-inmemorystorage package takes care of Django's storage.
def temporary_image():
bts = BytesIO()
img = Image.new("RGB", (100, 100))
img.save(bts, 'jpeg')
return SimpleUploadedFile("test.jpg", bts.getvalue())
def tearDown(self) -> None:
self.Noticia.img.delete()
i'm making a site for uploading and processing of files (mostly images but also other stuff). sometimes, some files need some editing before saving.
i want edit the uploaded file before saving it to media dir (mostly to avoid IO interaction and only keep original file in memory and discard after saving edit version).
edit: so i thought i have to run the edit function in POST, and change information in request.FILES. but i failed, and don't know what to do anymore.
NOTE: i'm mostly looking for a way to do it via class views.
here's some code for reference:
The model:
class FilePatient(models.Model):
file_imag = models.FileField(upload_to='')
The View:
class FileAddView(LoginRequiredMixin, UserPassesTestMixin, SuccessMessageMixin, CreateView):
model = FilePatient
fields = ['file_imag']
def post(self, request, *args, **kwargs):
if request.FILES['file_imag'].name.endswith('.png'):
newFile=editFile(request.FILES['image_imag'])
# what to do, what not to do
return super().post(request, *args, **kwargs)
I hope the following will help you... this code "edits" the file by resizing the original file if it is too big. Based on working code, I substituted the vars you used in your question, not tested as below.
from pathlib import Path
import PIL
from six import BytesIO
from django.core.files.uploadedfile import InMemoryUploadedFile
img = request.FILES['fileInput']
if img.name.endswith('.png'):
fp = FilePatient.objects.get(...) # get the object for uploading file
img2 = PIL.Image.open(img)
exif = img2.info['exif'] if 'exif' in img2.info else None
max_hw = 800 # px width / height maximum
width_percent = (max_hw/float(img2.size[0]))
heigth_percent = (max_hw/float(img2.size[1]))
min_wh_percent = float(min(width_percent, heigth_percent))
if min_wh_percent <= 1:
width_size = int((float(img2.size[0])*min_wh_percent))
height_size = int((float(img2.size[1])*min_wh_percent))
img2 = img2.resize((width_size, height_size), PIL.Image.ANTIALIAS)
fp.file_imag = img2
buffer = BytesIO()
if exif:
img2.save(buffer, format='JPEG', exif=exif, quality=90)
else:
img2.save(buffer, format='JPEG', quality=90)
buffer.seek(0)
fp.file_imag.name = Path(img.name).stem
fp.file_imag = InMemoryUploadedFile(buffer,
'ImageField',
f"{fp.file_imag.name}.jpg",
'image/jpeg',
img2.size,
"utf-8")
fp.save()
I want to build an image gallery using Django. Each image is a post, of course. Now, I don't want to be uploading each image independently. I want to zip them all and upload them in the Django admin page and maybe create some kind of a trigger to:
decompressed the zip
read all the images info
store the info in a database, each image in a row
Is this possible with Django? what would be your best way to accomplish this? I'll appreciate any kind of help, I'm very new to Django (like 5 hours new)
Yes, it's possible. Here is a broad outline inspired entirely by how Mezzanine implements this.
First you define a field for accepting the zip file:
class BaseGallery(models.Model):
zip_import = models.FileField(blank=True, upload_to=upload_to("galleries")
Then you have a separate model that is Foreign keyed to your parent model. In the example here the parent model is BaseGallery and the image model is GalleryImage:
class GalleryImage(Orderable):
gallery = models.ForeignKey(Gallery, related_name="images")
file = models.ImageField(upload_to="galleries")
Then in your model's save method, you can extract this zip file and save the individual images:
from django.core.files import ContentFile
from django.conf import settings
from zipfile import ZipFile
def save(self, delete_zip_import=True, *args, **kwargs):
"""
If a zip file is uploaded, extract any images from it and add
them to the gallery, before removing the zip file.
"""
super(BaseGallery, self).save(*args, **kwargs)
if self.zip_import:
zip_file = ZipFile(self.zip_import)
for name in zip_file.namelist():
data = zip_file.read(name)
try:
from PIL import Image
image = Image.open(BytesIO(data))
image.load()
image = Image.open(BytesIO(data))
image.verify()
except ImportError:
pass
except:
continue
name = os.path.split(name)[1]
# You now have an image which you can save
path = os.path.join(settings.MEDIA_ROOT, "galleries",
native(str(name, errors="ignore")))
saved_path = default_storage.save(path, ContentFile(data))
self.images.create(file=saved_path)
if delete_zip_import:
zip_file.close()
self.zip_import.delete(save=True)
Note, the bit where the image is actually saved has been simplified and if you look at the source I have linked to there is a bit more jiggery-pokery required to handle unicode file names etc.
Also note that Mezzanine uses its own FileField which isn't the same as Django's FileField. I have tried to refactor this in the example above.
Here's code to extract files from an uploaded ZIP file:
source
import zipfile
localFile = 'beer.zip'
with zipfile.ZipFile(localFile, 'r') as myzip:
myzip.extractall()
print 'got:', [info.filename for info in myzip.infolist()]
reference
Python zipfile module
When working with manual file uploads, do I need to place the file in the final location before saving it to the model? Or, does the model move the file at some point? If I do need to place it myself, why do I need the upload_to param in the model field? That seems like I would have to keep parity with the upload_to param and the logic I'm using to copy it.
I think I'm just confused. Can someone help me do this right?
My form gets in image url from the web:
class ProductForm(ModelForm):
main_image_url = forms.URLField()
# etc...
My view retrieves it, checks it, and makes a thumbnail:
main_img_temp = NamedTemporaryFile(delete=True)
main_img_temp.write(urllib2.urlopen(main_image_url).read())
main_img_temp.flush()
img_type = imghdr.what(main_img_temp.name)
if not img_type:
errors = form._errors.setdefault("main_image_url", ErrorList())
errors.append(u"Url does not point to a valid image")
return render_to_response('add_image.html', {'form':form}, context_instance=RequestContext(request))
# build a temporary path name
filename = str(uuid.uuid4())
dirname = os.path.dirname(main_img_temp.name)
full_size_tmp = os.path.join(dirname, filename+'_full.jpg')
thumb_size_tmp = os.path.join(dirname, filename+'_thumb.jpg')
shutil.copy2(main_img_temp.name, full_size_tmp)
shutil.copy2(main_img_temp.name, thumb_size_tmp)
# build full size and thumbnail
im = Image.open(full_size_tmp)
im.thumbnail(full_image_size, Image.ANTIALIAS)
im.save(full_size_tmp, "JPEG")
im = Image.open(thumb_size_tmp)
im.thumbnail(thumb_image_size, Image.ANTIALIAS)
im.save(thumb_size_tmp, "JPEG")
# close to delete the original temp file
main_img_tmp.close()
### HERE'S WHERE I'M STUCK. This doesn't move the file... ####
main_image = UploadedImage(image=full_size_tmp, thumbnail=thumb_size_tmp)
main_image.save()
In my models, I've got an UploadedImage model that has the basic fields:
class UploadedImage(models.Model):
image = models.ImageField(upload_to='uploads/images/%Y/%m/%d/full')
thumbnail = models.ImageField(upload_to='uploads/images/%Y/%m/%d/thumb/')
Normally, when you save the model, it writes the files to the location that points upload_to. It handles that by itself so you won't need to do it manually.
Here you are writing the file to a temp, moving it after and a lot of things that can be done automatically. Check the answer to this question where he also uses urllib to get an image and save it into the db.
Note that you can pass a buffer in memory to create a FileField or ImageField which be suitable to do the logic for the thumbnail. Also you may consider using django-thumbnails for this purposes. It's a good library.
Hope this helps!
Answered My Own Question...
When I passed the path to the ImageField in the model, I was just handing it a path. I see now that to invoke all the storage handling built in to Django, you have to hand it a File object. This was enough to make it copy to the upload_to path:
from django.core.files import File
main_image = UploadedImage(image=File(open(full_size_tmp)), thumbnail=File(open(thumb_size_tmp)), creator=request.user)
main_image.save()
I have a this model...
class MyModel(models.Model):
...
file = models.FileField(upload_to='files/',null=True, blank=True)
...
when i upload a file, example file name is docfile.doc. when i change the file or i rewrite it and upload again docfile.doc the file will become docfile_1.doc and the old docfile.doc is still exist.
i am doing the uploading and saving data in django-admin
my question is, how can i remove the old docfile.doc if i upload the new docfile.doc and the file name is still docfile.doc?
can anyone help me in my case? thanks in advance
i try this one :
def content_file_name(instance, filename):
print instance
print filename
file = os.path.exists(filename)
print file
if file:
os.remove(filename)
return "file/"+str(filename)
class MyModel(models.Model):
...
file = models.FileField(upload_to=content_file_name,null=True, blank=True)
...
but nothing happend, when i upload docfile.doc again, it will become docfile_1.doc and the old docfile.doc still exist.
i got it... i use this
def content_file_name(instance, filename):
print instance
print filename
file = os.path.exists("media/file/"+str(filename))
print file
if file:
os.remove("media/file/"+str(filename))
return "file/"+str(filename)
I don't know exactly how to do it, but i think these links can help you:
Here you can find the two options that a FileField accept. The one that i think will interest you the most is FileField.storage. You can pass a storage object in that parameter.
It says:
FileField.storage: Optional. A storage object, which handles the storage and retrieval of your files.
Then, if you read this you would see that you can write your own storage object. Here is some explanation on how to do it. I think that you could just override the _save method in order to accomplish what you want to do (i.e: if the file already exists, remove it before saving the new copy.)
But be careful! I don't know which is the source of the files you are going to store. Maybe, your app is going to recieve lots of files with the same name, although they are all different. In this case, you would want to use a callable as the FileField.upload_to parameter, so that determine a unique filename for each file your site recieve.
I hope this helps you!
You could also have a look here: ImageField overwrite image file with same name
Define your own storage and overwrite its get available_name method.
The next code solves your problem. You override pre_save method where image is actually saved to storage. Please, rename functions for your project. Use newly created image field ImageFieldWithPermantName with your upload_to function (content_file_name).
If the code is too complicated you could simplify it. I use the code to do more complex operations for uploading images: I create thumbnails on-the-fly in custom _save_image function. So, you can simplify it.
from PIL import Image
import StringIO
from django.db.models import ImageField
from django.db.models.fields.files import FileField
from dargent.settings import MEDIA_ROOT
import os
class ImageFieldWithPermanentName( ImageField ):
def pre_save( self, model_instance, add ):
file = super( FileField, self ).pre_save(model_instance, add)
if file and not file._committed:
if callable( self.upload_to ):
path = self.upload_to( model_instance, "" )
else:
path = self.upload_to
file.name = path # here we set the same name to a file
path = os.path.join( MEDIA_ROOT, path )
chunks = _get_chunks( file.chunks() )
_save_image( chunks, path )
return file
def _get_chunks( chunks ):
chunks_ = ""
for chunk in chunks:
chunks_ += chunk
return chunks_
def _get_image( chunks ):
chunks_ = ""
for chunk in chunks:
chunks_ += chunk
virt_file = StringIO.StringIO( chunks_ )
image = Image.open( virt_file )
return image
def _save_image( chunks, out_file_path ):
image = _get_image( chunks )
image.save( out_file_path, "JPEG", quality = 100 )