How do you register an image file in a Django ImageField without using a form, and not copying any files?
I have several thousand JPGs located at /images, and I want to register them in an Image model similar to:
class Image(models.Model):
image = models.ImageField(upload_to='images', max_length=1000)
hash = models.CharField(max_length=1000, unique=True)
However, all the docs I can find on "loading" images into a Django project assume I'm doing so via a form, which also implies the image will be copied to MEDIA_ROOT. I'm not using a form, and I don't want to re-copy the several thousand JPGs, since they're already where they're supposed to be. I just want to create Image records that will store the filename of all the images I currently have. I've written a simple Python script to loop over each image, but I can't find how to properly create the Image record.
I also want to store a hash of the image content, to prevent duplicate records. e.g.
import hashlib
content = open(image_filename).read()
h = hashlib.sha512()
h.update(content)
imgobj.hash = h.hexdigest()
imgobj.save()
Would I override the default model.Model.save() method to do this?
If you have the script to loop over the images in your directory, you're nearly to a solution. Django will only store the path to the image in your Image.image field so basically all you need to do in your loop is:
#pseudo-code
for image_file in image_files:
image, created = Image.objects.get_or_create(hash=the_hash, \
defaults={'image' : 'path/to/image', 'hash' : the_hash)
That's a pretty easy way to build up only the unique records in your database without having to move the files, or use a form. You're either going to harmlessly return the image by the hash if it exists, or you're going to create a new record.
Hope that helps!
After digging through the code, and piecing together a few snippets I found, the following seems to work for me:
models.py
import os, hashlib
from django.db import models
class Image(models.Model):
image = models.ImageField(upload_to=IMAGE_UPLOAD_TO, max_length=1000)
hash = models.CharField(max_length=1000, unique=True)
def save(self, *args, **kwargs):
# Update image hash to ensure uniqueness.
h = hashlib.sha512()
h.update(self.image.read())
self.hash = h.hexdigest()
return models.Model.save(self, *args, **kwargs)
import_images.py
import os
from django.conf import settings
from django.core.files import File
from myapp import models
fn = os.path.join(settings.MEDIA_ROOT, 'images', 'mytestimage.jpg')
img = models.Image()
img.image.save(fn, File(open(fn, 'r')))
img.save()
In Django, if you have a ImageFile in a model, deleting will remove the associated file from disk as well as removing the record from the database.
Shouldn't replacing an image also remove the unneeded file from disk? Instead, I see that it keeps the original and adds the replacement.
Now deleting the object won't delete the original file only the replacement.
Are there any good strategies to doing this? I don't want to have a bunch of orphan files if my users replace their images frequently.
The best strategy I've found is to make a custom save method in the model:
class Photo(models.Model):
image = ImageField(...) # works with FileField also
def save(self, *args, **kwargs):
# delete old file when replacing by updating the file
try:
this = Photo.objects.get(id=self.id)
if this.image != self.image:
this.image.delete(save=False)
except: pass # when new photo then we do nothing, normal case
super(Photo, self).save(*args, **kwargs)
And beware, as with the updating which doesn't delete the back end file, deleting an instance model (here Photo) will not delete the back-end file, not in Django 1.3 anyway, you'll have to add more custom code to do that (or regularly do some dirty cron job).
Finally test all your update/delete cases with your ForeignKey, ManytoMany and others relations to check if the back-end files are correctly deleted. Believe only what you test.
Shouldn't replacing an image also remove the unneeded file from disk?
In the olden days, FileField was eager to clean up orphaned files. But that changed in Django 1.2:
In earlier Django versions, when a model instance containing a FileField was deleted, FileField took it upon itself to also delete the file from the backend storage. This opened the door to several potentially serious data-loss scenarios, including rolled-back transactions and fields on different models referencing the same file. In Django 1.2.5, FileField will never delete files from the backend storage.
The code in the following working example will, upon uploading an image in an ImageField, detect if a file with the same name exists, and in that case, delete that file before storing the new one.
It could easily be modified so that it deletes the old file regardless of the filename. But that's not what I wanted in my project.
Add the following class:
from django.core.files.storage import FileSystemStorage
class OverwriteStorage(FileSystemStorage):
def _save(self, name, content):
if self.exists(name):
self.delete(name)
return super(OverwriteStorage, self)._save(name, content)
def get_available_name(self, name):
return name
And use it with ImageField like so:
class MyModel(models.Model):
myfield = models.ImageField(
'description of purpose',
upload_to='folder_name',
storage=OverwriteStorage(), ### using OverwriteStorage here
max_length=500,
null=True,
blank=True,
height_field='height',
width_field='width'
)
height = models.IntegerField(blank=True, null=True)
width = models.IntegerField(blank=True, null=True)
If you don't use transactions or you don't afraid of loosing files on transaction rollback, you can use django-cleanup
There have been a number of tickets regarding this issue though it is likely this will not make it into the core. The most comprehensive is http://code.djangoproject.com/ticket/11663. The patches and ticket comments can give you some direction if you are looking for a solution.
You can also consider using a different StorageBackend such as the Overwrite File Storage System given by Django snippet 976. http://djangosnippets.org/snippets/976/. You can change your default storage to this backend or you can override it on each FileField/ImageField declaration.
Here is a code that can work with or without upload_to=... or blank=True, and when the submitted file has the same name as the old one.
(py3 syntax, tested on Django 1.7)
class Attachment(models.Model):
document = models.FileField(...) # or ImageField
def delete(self, *args, **kwargs):
self.document.delete(save=False)
super().delete(*args, **kwargs)
def save(self, *args, **kwargs):
if self.pk:
old = self.__class__._default_manager.get(pk=self.pk)
if old.document.name and (not self.document._committed or not self.document.name):
old.document.delete(save=False)
super().save(*args, **kwargs)
Remember that this kind of solution is only applicable if you are in a non transactional context (no rollback, because the file is definitively lost)
I used a simple method with popen, so when i save my Info model i delete the former file before linking to the new:
import os
try:
os.popen("rm %s" % str(info.photo.path))
except:
#deal with error
pass
info.photo = nd['photo']
I save the original file and if it has changed - delete it.
class Document(models.Model):
document = FileField()
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self._document = self.document
def save(self, *args, **kwargs):
if self.document != self._document:
self._document.delete()
super().save(*args, **kwargs)
I'm using Django to create a stock photo site, I have an ImageField in my model, the problem is that when the user updates the image field, the original image file isn't deleted from the hard disk.
How can I delete the old images after an update?
Use django-cleanup
pip install django-cleanup
settings.py
INSTALLED_APPS = (
...
'django_cleanup.apps.CleanupConfig', # should be placed after your apps
)
You'll have to delete the old image manually.
The absolute path to the image is stored in your_image_field.path. So you'd do something like:
os.remove(your_image_field.path)
But, as a convenience, you can use the associated FieldFile object, which gives easy access to the underlying file, as well as providing a few convenience methods. See http://docs.djangoproject.com/en/dev/ref/models/fields/#filefield-and-fieldfile
Use this custom save method in your model:
def save(self, *args, **kwargs):
try:
this = MyModelName.objects.get(id=self.id)
if this.MyImageFieldName != self.MyImageFieldName:
this.MyImageFieldName.delete()
except: pass
super(MyModelName, self).save(*args, **kwargs)
It works for me on my site. This problem was bothering me as well and I didn't want to make a cleanup script instead over good bookkeeping in the first place. Let me know if there are any problems with it.
Before updating the model instance, you can use the delete method of FileField object. For example, if the FileField or ImageField is named as photo and your model instance is profile, then the following will remove the file from disk
profile.photo.delete(False)
For more clarification, here is the django doc
https://docs.djangoproject.com/en/1.11/ref/models/fields/#django.db.models.fields.files.FieldFile.delete
You can define pre_save reciever in models:
#receiver(models.signals.pre_save, sender=UserAccount)
def delete_file_on_change_extension(sender, instance, **kwargs):
if instance.pk:
try:
old_avatar = UserAccount.objects.get(pk=instance.pk).avatar
except UserAccount.DoesNotExist:
return
else:
new_avatar = instance.avatar
if old_avatar and old_avatar.url != new_avatar.url:
old_avatar.delete(save=False)
My avatrs has unique url for each person like "avatars/ceb47779-8833-4719-8711-6f4e5cabb2b2.png". If user upload new image with different extension like jpg, delete_file_on_change_extension reciever remove old image, before save new with url "avatars/ceb47779-8833-4719-8711-6f4e5cabb2b2.jpg" (in this case). If user uploads new image with same extension django overwrite old image on storage (disk), because images paths are the same.
This works fine with AWS S3 django-storage.
Here is an app that deletes orphan files by default: django-smartfields.
It will remove files whenever:
field value was replaced with a new one (either uploaded or set manually)
field is cleared through the form (in case that field is not required, of course)
the model instance itself containing the field is deleted.
It is possible to turn that cleanup feature off using an argument: ImageField(keep_orphans=True) on per field basis, or globally in settings SMARTFIELDS_KEEP_ORPHANS = True.
from django.db import models
from smartfields import fields
class MyModel(models.Model):
image = fields.ImageField()
document = fields.FileField()
try this, it will work even if old file is deleted
def logo_file(instance, filename):
try:
this = business.objects.get(id=instance.id)
if this.logo is not None:
path = "%s" % (this.logo)
os.remove(path)
finally:
pass..
code will work even without "try .. finally" but it will generate problem if file was accidently deleted.
changed: move model matching inside "try" so it will not throw any error at user signup
Let me know if there are any problems.
Completing Chris Lawlor's answer, tried this and works.
from YOURAPP.settings import BASE_DIR
try:
os.remove(BASE_DIR + user.userprofile.avatarURL)
except Exception as e:
pass
The URL has a pattern of /media/mypicture.jpg
What I did is saving the path to the old image and if form is valid I would delete the old one.
if request.method == 'POST':
old_image = ""
if request.user.profile.profile_picture:
old_image = request.user.profile.profile_picture.path
form = UpdateProfileForm(request.POST,request.FILES,instance = profile)
if form.is_valid():
if os.path.exists(old_image):
os.remove(old_image)
form.save()
It is a little messy , but you do not install third parties or anythin
Ok, I've tried about near everything and I cannot get this to work.
I have a Django model with an ImageField on it
I have code that downloads an image via HTTP (tested and works)
The image is saved directly into the 'upload_to' folder (the upload_to being the one that is set on the ImageField)
All I need to do is associate the already existing image file path with the ImageField
I've written this code about 6 different ways.
The problem I'm running into is all of the code that I'm writing results in the following behavior:
(1) Django will make a 2nd file, (2) rename the new file, adding an _ to the end of the file name, then (3) not transfer any of the data over leaving it basically an empty re-named file. What's left in the 'upload_to' path is 2 files, one that is the actual image, and one that is the name of the image,but is empty, and of course the ImageField path is set to the empty file that Django try to create.
In case that was unclear, I'll try to illustrate:
## Image generation code runs....
/Upload
generated_image.jpg 4kb
## Attempt to set the ImageField path...
/Upload
generated_image.jpg 4kb
generated_image_.jpg 0kb
ImageField.Path = /Upload/generated_image_.jpg
How can I do this without having Django try to re-store the file? What I'd really like is something to this effect...
model.ImageField.path = generated_image_path
...but of course that doesn't work.
And yes I've gone through the other questions here like this one as well as the django doc on File
UPDATE
After further testing, it only does this behavior when running under Apache on Windows Server. While running under the 'runserver' on XP it does not execute this behavior.
I am stumped.
Here is the code which runs successfully on XP...
f = open(thumb_path, 'r')
model.thumbnail = File(f)
model.save()
I have some code that fetches an image off the web and stores it in a model. The important bits are:
from django.core.files import File # you need this somewhere
import urllib
# The following actually resides in a method of my model
result = urllib.urlretrieve(image_url) # image_url is a URL to an image
# self.photo is the ImageField
self.photo.save(
os.path.basename(self.url),
File(open(result[0], 'rb'))
)
self.save()
That's a bit confusing because it's pulled out of my model and a bit out of context, but the important parts are:
The image pulled from the web is not stored in the upload_to folder, it is instead stored as a tempfile by urllib.urlretrieve() and later discarded.
The ImageField.save() method takes a filename (the os.path.basename bit) and a django.core.files.File object.
Let me know if you have questions or need clarification.
Edit: for the sake of clarity, here is the model (minus any required import statements):
class CachedImage(models.Model):
url = models.CharField(max_length=255, unique=True)
photo = models.ImageField(upload_to=photo_path, blank=True)
def cache(self):
"""Store image locally if we have a URL"""
if self.url and not self.photo:
result = urllib.urlretrieve(self.url)
self.photo.save(
os.path.basename(self.url),
File(open(result[0], 'rb'))
)
self.save()
Super easy if model hasn't been created yet:
First, copy your image file to the upload path (assumed = 'path/' in following snippet).
Second, use something like:
class Layout(models.Model):
image = models.ImageField('img', upload_to='path/')
layout = Layout()
layout.image = "path/image.png"
layout.save()
tested and working in django 1.4, it might work also for an existing model.
Just a little remark. tvon answer works but, if you're working on windows, you probably want to open() the file with 'rb'. Like this:
class CachedImage(models.Model):
url = models.CharField(max_length=255, unique=True)
photo = models.ImageField(upload_to=photo_path, blank=True)
def cache(self):
"""Store image locally if we have a URL"""
if self.url and not self.photo:
result = urllib.urlretrieve(self.url)
self.photo.save(
os.path.basename(self.url),
File(open(result[0], 'rb'))
)
self.save()
or you'll get your file truncated at the first 0x1A byte.
Ok, If all you need to do is associate the already existing image file path with the ImageField, then this solution may be helpfull:
from django.core.files.base import ContentFile
with open('/path/to/already/existing/file') as f:
data = f.read()
# obj.image is the ImageField
obj.image.save('imgfilename.jpg', ContentFile(data))
Well, if be earnest, the already existing image file will not be associated with the ImageField, but the copy of this file will be created in upload_to dir as 'imgfilename.jpg' and will be associated with the ImageField.
Here is a method that works well and allows you to convert the file to a certain format as well (to avoid "cannot write mode P as JPEG" error):
import urllib2
from django.core.files.base import ContentFile
from PIL import Image
from StringIO import StringIO
def download_image(name, image, url):
input_file = StringIO(urllib2.urlopen(url).read())
output_file = StringIO()
img = Image.open(input_file)
if img.mode != "RGB":
img = img.convert("RGB")
img.save(output_file, "JPEG")
image.save(name+".jpg", ContentFile(output_file.getvalue()), save=False)
where image is the django ImageField or your_model_instance.image
here is a usage example:
p = ProfilePhoto(user=user)
download_image(str(user.id), p.image, image_url)
p.save()
Hope this helps
What I did was to create my own storage that will just not save the file to the disk:
from django.core.files.storage import FileSystemStorage
class CustomStorage(FileSystemStorage):
def _open(self, name, mode='rb'):
return File(open(self.path(name), mode))
def _save(self, name, content):
# here, you should implement how the file is to be saved
# like on other machines or something, and return the name of the file.
# In our case, we just return the name, and disable any kind of save
return name
def get_available_name(self, name):
return name
Then, in my models, for my ImageField, I've used the new custom storage:
from custom_storage import CustomStorage
custom_store = CustomStorage()
class Image(models.Model):
thumb = models.ImageField(storage=custom_store, upload_to='/some/path')
A lot of these answers were outdated, and I spent many hours in frustration (I'm fairly new to Django & web dev in general). However, I found this excellent gist by #iambibhas: https://gist.github.com/iambibhas/5051911
import requests
from django.core.files import File
from django.core.files.temp import NamedTemporaryFile
def save_image_from_url(model, url):
r = requests.get(url)
img_temp = NamedTemporaryFile(delete=True)
img_temp.write(r.content)
img_temp.flush()
model.image.save("image.jpg", File(img_temp), save=True)
Another possible way to do that:
from django.core.files import File
with open('path_to_file', 'r') as f: # use 'rb' mode for python3
data = File(f)
model.image.save('filename', data, True)
If you want to just "set" the actual filename, without incurring the overhead of loading and re-saving the file (!!), or resorting to using a charfield (!!!), you might want to try something like this --
model_instance.myfile = model_instance.myfile.field.attr_class(model_instance, model_instance.myfile.field, 'my-filename.jpg')
This will light up your model_instance.myfile.url and all the rest of them just as if you'd actually uploaded the file.
Like #t-stone says, what we really want, is to be able to set instance.myfile.path = 'my-filename.jpg', but Django doesn't currently support that.
This is might not be the answer you are looking for. but you can use charfield to store the path of the file instead of ImageFile. In that way you can programmatically associate uploaded image to field without recreating the file.
With Django 3,
with a model such as this one:
class Item(models.Model):
name = models.CharField(max_length=255, unique=True)
photo= models.ImageField(upload_to='image_folder/', blank=True)
if the image has already been uploaded, we can directly do :
Item.objects.filter(...).update(photo='image_folder/sample_photo.png')
or
my_item = Item.objects.get(id=5)
my_item.photo='image_folder/sample_photo.png'
my_item.save()
You can try:
model.ImageField.path = os.path.join('/Upload', generated_image_path)
class tweet_photos(models.Model):
upload_path='absolute path'
image=models.ImageField(upload_to=upload_path)
image_url = models.URLField(null=True, blank=True)
def save(self, *args, **kwargs):
if self.image_url:
import urllib, os
from urlparse import urlparse
file_save_dir = self.upload_path
filename = urlparse(self.image_url).path.split('/')[-1]
urllib.urlretrieve(self.image_url, os.path.join(file_save_dir, filename))
self.image = os.path.join(file_save_dir, filename)
self.image_url = ''
super(tweet_photos, self).save()
class Pin(models.Model):
"""Pin Class"""
image_link = models.CharField(max_length=255, null=True, blank=True)
image = models.ImageField(upload_to='images/', blank=True)
title = models.CharField(max_length=255, null=True, blank=True)
source_name = models.CharField(max_length=255, null=True, blank=True)
source_link = models.CharField(max_length=255, null=True, blank=True)
description = models.TextField(null=True, blank=True)
tags = models.ForeignKey(Tag, blank=True, null=True)
def __unicode__(self):
"""Unicode class."""
return unicode(self.image_link)
def save(self, *args, **kwargs):
"""Store image locally if we have a URL"""
if self.image_link and not self.image:
result = urllib.urlretrieve(self.image_link)
self.image.save(os.path.basename(self.image_link), File(open(result[0], 'r')))
self.save()
super(Pin, self).save()
Working!
You can save image by using FileSystemStorage.
check the example below
def upload_pic(request):
if request.method == 'POST' and request.FILES['photo']:
photo = request.FILES['photo']
name = request.FILES['photo'].name
fs = FileSystemStorage()
##### you can update file saving location too by adding line below #####
fs.base_location = fs.base_location+'/company_coverphotos'
##################
filename = fs.save(name, photo)
uploaded_file_url = fs.url(filename)+'/company_coverphotos'
Profile.objects.filter(user=request.user).update(photo=photo)
class DemoImage(models.Model):
title = models.TextField(max_length=255, blank=False)
image = models.ImageField(blank=False, upload_to="images/DemoImages/")
import requests
import urllib.request
from django.core.files import File
url = "https://path/to/logo.jpg"
# Below 3 lines is to fake as browser agent
# as many sites block urllib class suspecting to be bots
opener = urllib.request.build_opener()
opener.addheaders = [("User-agent", "Mozilla/5.0")]
urllib.request.install_opener(opener)
# Issue command to actually download and create temp img file in memory
result = urllib.request.urlretrieve(url)
# DemoImage.objects.create(title="title", image=File(open(result[0], "rb")))
# ^^ This erroneously results in creating the file like
# images/DemoImages/path/to/temp/dir/logo_image_file
# as opposed to
# images/DemoImages/logo_image_file
# Solution to get the file in images/DemoImages/
reopen = open(result[0], "rb") # Returns a BufferedReader object of the temp image
django_file = File(reopen) # Create the file from the BufferedReader object
demoimg = DemoImage()
demoimg.title = "title"
demoimg.image.save("logo.png", django_file, save=True)
This approach also triggers file upload to cloudinary/S3 if so configured
So, if you have a model with an imagefield with an upload_to attribute set, such as:
class Avatar(models.Model):
image_file = models.ImageField(upload_to=user_directory_path_avatar)
then it is reasonably easy to change the image, at least in django 3.15.
In the view, when you process the image, you can obtain the image from:
self.request.FILES['avatar']
which is an instance of type InMemoryUploadedFile, as long as your html form has the enctype set and a field for avatar...
<form method="post" class="avatarform" id="avatarform" action="{% url avatar_update_view' %}" enctype="multipart/form-data">
{% csrf_token %}
<input id="avatarUpload" class="d-none" type="file" name="avatar">
</form>
Then, setting the new image in the view is as easy as the following (where profile is the profile model for the self.request.user)
profile.avatar.image_file.save(self.request.FILES['avatar'].name, self.request.FILES['avatar'])
There is no need to save the profile.avatar, the image_field already saves, and into the correct location because of the 'upload_to' callback function.
Your can use Django REST framework and python Requests library to Programmatically saving image to Django ImageField
Here is a Example:
import requests
def upload_image():
# PATH TO DJANGO REST API
url = "http://127.0.0.1:8080/api/gallery/"
# MODEL FIELDS DATA
data = {'first_name': "Rajiv", 'last_name': "Sharma"}
# UPLOAD FILES THROUGH REST API
photo = open('/path/to/photo', 'rb')
resume = open('/path/to/resume', 'rb')
files = {'photo': photo, 'resume': resume}
request = requests.post(url, data=data, files=files)
print(request.status_code, request.reason)
I save the image with uuid in django 2 python 3 because thats how django do it:
import uuid
from django.core.files import File
import urllib
httpUrl = "https://miimgeurl/image.jpg"
result = urllib.request.urlretrieve(httpUrl)
mymodel.imagefield.save(os.path.basename(str(uuid.uuid4())+".jpg"),File(open(result[0], 'rb')))
mymodel.save()
if you use admin.py you can solve the problem override (doc on django):
def save_model(self, request, obj, form, change):
obj.image_data = bytes(obj.image_name.read())
super().save_model(request, obj, form, change)
with models.py:
image_name = models.ImageField()
image_data = models.BinaryField()