Develop Reference

Getting the Django-generated unique name of a file uploaded multiple times

I am using DRF backend to upload files. In my specific case I will want to get the name of the file, after it has been uploaded. The reason is that if a user uploads a file with same name, I am still able to process it independently.
views.py:
class ImageUploaderView(viewsets.ModelViewSet):
renderer_classes = [renderers.JSONRenderer]
queryset = ImageUploader.objects.all()
serializer_class = ImageUploaderSerializer
parser_classes = (MultiPartParser,)
serializer.py:
class ImageUploaderSerializer(serializers.ModelSerializer):
class Meta:
model = ImageUploader
models.py:
class ImageUploader(models.Model):
# name=models.ImageField(upload_to='media')
name=models.FileField(upload_to='media')
I tried to put signals and hooks after the model definitions but I am not being able to get this filename. Can someone shed some light pls?
UPDATE: Let me elaborate what I want to achieve essentially:
User1 hits endpoint "/api/calculate_interest_rate" which is rendered
by a frontend React component. "calculate_interest_rate" is served by
DRF, and lets the user upload a CSV file. This will be stored as
"user1.csv", the file is processed and then tabulated (rendered by
React).
At the same time and in parallel to User1, User2 hits the same endpoint "/api/calculate_interest_rate" and
by mistake he saves his file as "user1.csv", and uploads it to the systemn.
So I want to be able to detect both names of the file in order to process it. By using always the same default filename (ex. using the OverwriteStorage() technique), I will probably cause chaos when two or more users are using the same filename. Therefore I am looking into a technique that allows me to get the filename as is, and process it immediately.

How about using storage option?
class OverwriteStorage(FileSystemStorage):
def get_available_name(self, name, max_length=None):
print("filename", name)
#parts = name.split('.') you can separate name and extension.
return super().get_available_name(name)
upload_image = models.ImageField(
upload_to=[yourpath],
default=[defaultname],
storage=OverwriteStorage()
)

I suggest you to following this configuration:
1. Change your MEDIA_ROOT and MEDIA_URL inside file of settings.py
MEDIA_URL = '/media/'
MEDIA_ROOT = '/path/to/env/projectname/media'
2. Then, I suggest you to change your upload_to='media to upload_to='images/%Y/%m/%d, also rename your field of name with image.
class ImageUploader(models.Model):
image = models.FileField(upload_to='images/%Y/%m/%d')
# OR
# image = models.ImageField(upload_to='images/%Y/%m/%d')
Explanation; If you following this configuration, you could have uploaded images are following, eg: /media/images/2017/01/29/yourimage.jpg. This is one way to handle the problem of duplicated files.
3. But if you want to upload file with multiple times without duplicate files, you can using deconstructible;
import os, time, uuid
from django.db import models
from django.utils.deconstruct import deconstructible
class ImageUploader(models.Model):
#deconstructible
class PathAndRename(object):
def __init__(self, sub_path):
self.path = sub_path
def __call__(self, instance, filename):
# eg: filename = 'my uploaded file.jpg'
ext = filename.split('.')[-1] #eg: '.jpg'
uid = uuid.uuid4().hex[:10] #eg: '567ae32f97'
# eg: 'my-uploaded-file'
new_name = '-'.join(filename.replace('.%s' % ext, '').split())
# eg: 'my-uploaded-file_64c942aa64.jpg'
renamed_filename = '%(new_name)s_%(uid)s.%(ext)s' % {'new_name': new_name, 'uid': uid, 'ext': ext}
# eg: 'images/2017/01/29/my-uploaded-file_64c942aa64.jpg'
return os.path.join(self.path, renamed_filename)
image_path = time.strftime('images/%Y/%m/%d')
image = models.ImageField(upload_to=PathAndRename(self.image_path))

django submit image fom url file name stored wrong

I'm trying to submit image from url based on the answers of this question but as a result I have image file stored successfully , but with filename .jpg instead of image-name.jpg
code
import urllib2
from django.core.files import File
from django.core.files.temp import NamedTemporaryFile
from myapp.models import Mymodel
# Mymodel contain ImageField named image
extract_url = 'http://example.com/image-name.jpg'
my_temp = Mymodel()
image_name = extract_url.split('/')[-1]
image_temp = NamedTemporaryFile(delete=True)
image_temp.write(urllib2.urlopen(extract_url).read())
image_temp.flush()
my_temp.image.save(image_name, File(image_temp), save=True)
I did not overwrite Mymodel save method and image_name value is image-name.jpg
>> print image_name
u'image-name.jpg'
EDIT : Include Mymodel code
def _image_container(instance, filename):
return os.path.join(settings.IMAGE_FILES_ROOT, instance.slug + '.' + filename.split('.')[-1].lower())
class Mymodel(models.Model):
...
slug = AutoSlugField(_('Slug'), unique=True, populate_from='title', editable=True)
image = thumbnail.ImageField(_('Image'), upload_to=_image_container)
I want to keep _image_container functionality because it works nice on manual upload

Your upload_to _image_container function renames the image based on the Mymodel instance's slug, but the model instance you use for saving your image has no slug, so obviously you have exactly what you asked for... To make a long story short you have to explicitely set the slug attribute of your temporary Mymodel instance to whatever name you want for the image.
import os
extract_url = 'http://example.com/image-name.jpg'
image_name = extract_url.split('/')[-1]
slug = os.path.splitext(image_name)[0]
my_temp = Mymodel(slug=slug)
# etc

Simple Django Admin App - How to track file versions

If I have a Django model with a FileField e.g.,
class Probe(models.Model):
name = models.CharField(max_length=200, unique=True)
nanoz_file = models.FileField(upload_to='nanoz_file', blank=True)
is there a way to prevent an uploaded file from being overwritten if a user uploads a new file in the Admin interface?
Also if I do keep the old files around is there a way I can relate the previous files back to the model instance?
I.e., I'd like to be able to list all files uploaded to the nanoz_file field for a given model instance.

Django never overwrites an uploaded file. If you upload 'foo.png' twice, the second will be 'foo_1.png' - I just tested this but don't take my word for it: try it too !
All you have to do (or let django-reversion do) is keep track of the previous file names.

You can use this structure:
class File(models.Model):
name = models.CharField()
file = models.FileField(upload_to='files_storage/')
belongs = models.ForeignKey('self')
creation = models.DateTimeField(auto_now_add=True)
Then in the view you can use something like:
def edit_file(request, ...):
# Get the file model instance
file_model = ... # Code to get the instance
# Create a new instance of the model with the old file path
old_file = File(name='file1-v2', file=file_model.file, belongs=file_model)
old_file.save()
# Update the file_model with the new file data
Hope this helps!

Just as jpic said, you could try django-reversion, or
track names of past files in sorts of DB, for example in seperated table row, in customized field or in gfk field.
glob files online, as long as filename is managed.
For the second way, actually for dealing w/ all user uploads, its better to name the file by the pattern you designed instead of using the raw name (you could also store the raw name for later usage) . For your case, since the name field is unique, the field is suitable as the base for generating filename of the uploaded files, if it rarely changes:
import os.path
from django.hash_compat import sha_constructor
def upload_to(self, filename):
return 'nanoz_file/%s%s' % (
sha_constructor(self.name).hexdigest(), os.path.splitext(filename)[-1])
class Probe(models.Model):
name = models.CharField(max_length=200, unique=True)
nanoz_file = models.FileField(upload_to=upload_to, blank=True)
Then in your view, you could fetch the list of names of all files of Probe instance probe by
import glob
# be careful to operate directory securely
glob.glob(os.path.join(
os.path.dirname(probe.nanoz_file.path),
'%s*' % sha_constructor(probe.name).hexdigest()))

Cannot change model instance after a ValidationError is thrown in django admin area

Imagine a model like this:
class CFile(models.Model):
filepath = models.FileField(upload_to=...)
collection = models.ForeignKey("FileCollection",null=True)
... # other attributes that are not relevant
def clean(self):
bname = os.path.basename
if self.collection:
cfiles = self.baseline.attachment_set.all()
with_same_basename = filter(lambda e: bname(e.filepath.path) == bname(self.filepath.path),cfiles)
if len(with_same_basename) > 0:
raise ValidationError("There already exists a file with the same name in this collection")
class FileCollection(models.Model):
name = models.CharField(max_length=255)
files= models.ManyToManyField("CFile")
I want to disallow the upload of a CFile if there already exists a CFile with the same basename, that's why I added the clean. The problem is:
I upload a CFile, with the name file1.png -> gets uploaded because no other files with this name exist
I upload another CFile, with the name file1.png -> I get the expected error that I already have a file with this name. So, I try to change the file, and upload a file with a different name ( file2.png ). The problem is, I stopped via pdb in the clean, and the model instance is still file1.png. I imagine this happens because of my ValidationError and django allows me to "correct my mistake". The problem is I cannot correct it if I cannot upload another file. How can I handle this?
EDIT: This happens in the admin area, sorry for forgetting to mention this before. I don't have anything custom ( besides inlines = [ FileInline ] ).

I think the clearest way is to declare another field in your model for filename and make it unique for every collection. Like this:
class CFile(models.Model):
filepath = models.FileField(upload_to=...)
collection = models.ForeignKey("FileCollection",null=True, related_name='files')
filename = models.CharField(max_length=255)
... # other attributes that are not relevant
class Meta:
unique_together = (('filename', 'collection'),)
def save(self, *args, **kwargs):
self.filename = bname(self.filepath.path)
super(CFile, self).save(args, kwargs)

How to Register Images using Django's ORM

How do you register an image file in a Django ImageField without using a form, and not copying any files?
I have several thousand JPGs located at /images, and I want to register them in an Image model similar to:
class Image(models.Model):
image = models.ImageField(upload_to='images', max_length=1000)
hash = models.CharField(max_length=1000, unique=True)
However, all the docs I can find on "loading" images into a Django project assume I'm doing so via a form, which also implies the image will be copied to MEDIA_ROOT. I'm not using a form, and I don't want to re-copy the several thousand JPGs, since they're already where they're supposed to be. I just want to create Image records that will store the filename of all the images I currently have. I've written a simple Python script to loop over each image, but I can't find how to properly create the Image record.
I also want to store a hash of the image content, to prevent duplicate records. e.g.
import hashlib
content = open(image_filename).read()
h = hashlib.sha512()
h.update(content)
imgobj.hash = h.hexdigest()
imgobj.save()
Would I override the default model.Model.save() method to do this?

If you have the script to loop over the images in your directory, you're nearly to a solution. Django will only store the path to the image in your Image.image field so basically all you need to do in your loop is:
#pseudo-code
for image_file in image_files:
image, created = Image.objects.get_or_create(hash=the_hash, \
defaults={'image' : 'path/to/image', 'hash' : the_hash)
That's a pretty easy way to build up only the unique records in your database without having to move the files, or use a form. You're either going to harmlessly return the image by the hash if it exists, or you're going to create a new record.
Hope that helps!

After digging through the code, and piecing together a few snippets I found, the following seems to work for me:
models.py
import os, hashlib
from django.db import models
class Image(models.Model):
image = models.ImageField(upload_to=IMAGE_UPLOAD_TO, max_length=1000)
hash = models.CharField(max_length=1000, unique=True)
def save(self, *args, **kwargs):
# Update image hash to ensure uniqueness.
h = hashlib.sha512()
h.update(self.image.read())
self.hash = h.hexdigest()
return models.Model.save(self, *args, **kwargs)
import_images.py
import os
from django.conf import settings
from django.core.files import File
from myapp import models
fn = os.path.join(settings.MEDIA_ROOT, 'images', 'mytestimage.jpg')
img = models.Image()
img.image.save(fn, File(open(fn, 'r')))
img.save()