If I have a Django model with a FileField e.g.,
class Probe(models.Model):
name = models.CharField(max_length=200, unique=True)
nanoz_file = models.FileField(upload_to='nanoz_file', blank=True)
is there a way to prevent an uploaded file from being overwritten if a user uploads a new file in the Admin interface?
Also if I do keep the old files around is there a way I can relate the previous files back to the model instance?
I.e., I'd like to be able to list all files uploaded to the nanoz_file field for a given model instance.
Django never overwrites an uploaded file. If you upload 'foo.png' twice, the second will be 'foo_1.png' - I just tested this but don't take my word for it: try it too !
All you have to do (or let django-reversion do) is keep track of the previous file names.
You can use this structure:
class File(models.Model):
name = models.CharField()
file = models.FileField(upload_to='files_storage/')
belongs = models.ForeignKey('self')
creation = models.DateTimeField(auto_now_add=True)
Then in the view you can use something like:
def edit_file(request, ...):
# Get the file model instance
file_model = ... # Code to get the instance
# Create a new instance of the model with the old file path
old_file = File(name='file1-v2', file=file_model.file, belongs=file_model)
old_file.save()
# Update the file_model with the new file data
Hope this helps!
Just as jpic said, you could try django-reversion, or
track names of past files in sorts of DB, for example in seperated table row, in customized field or in gfk field.
glob files online, as long as filename is managed.
For the second way, actually for dealing w/ all user uploads, its better to name the file by the pattern you designed instead of using the raw name (you could also store the raw name for later usage) . For your case, since the name field is unique, the field is suitable as the base for generating filename of the uploaded files, if it rarely changes:
import os.path
from django.hash_compat import sha_constructor
def upload_to(self, filename):
return 'nanoz_file/%s%s' % (
sha_constructor(self.name).hexdigest(), os.path.splitext(filename)[-1])
class Probe(models.Model):
name = models.CharField(max_length=200, unique=True)
nanoz_file = models.FileField(upload_to=upload_to, blank=True)
Then in your view, you could fetch the list of names of all files of Probe instance probe by
import glob
# be careful to operate directory securely
glob.glob(os.path.join(
os.path.dirname(probe.nanoz_file.path),
'%s*' % sha_constructor(probe.name).hexdigest()))
Related
I am using Django admin to upload files to my websites DB daily. How do I keep track of the names of the files that I have uploaded to the database? because in Django admin they just show up as objects1 object2 etc. I want to return my the_file title of the file that I upload. I am trying to show the first 50 characters of the name of the files that I have uploaded. However, it is not working for me. I think I am having trouble because it is of type FileField
class UploadedFile(models.Model):
the_file = models.FileField()
def __str__(self):
return self.the_file[:50]
The object a FileField [Django-doc] wraps is a FieldFile [Django-doc] (note that the two words are swapped). You can obtain the name of the file with the .name attribute [Django-doc]:
The name of the file including the relative path from the root of the Storage of the associated FileField.
So we can use this in the __str__ method:
class UploadedFile(models.Model):
the_file = models.FileField()
def __str__(self):
return self.the_file.name[:50]
I am using DRF backend to upload files. In my specific case I will want to get the name of the file, after it has been uploaded. The reason is that if a user uploads a file with same name, I am still able to process it independently.
views.py:
class ImageUploaderView(viewsets.ModelViewSet):
renderer_classes = [renderers.JSONRenderer]
queryset = ImageUploader.objects.all()
serializer_class = ImageUploaderSerializer
parser_classes = (MultiPartParser,)
serializer.py:
class ImageUploaderSerializer(serializers.ModelSerializer):
class Meta:
model = ImageUploader
models.py:
class ImageUploader(models.Model):
# name=models.ImageField(upload_to='media')
name=models.FileField(upload_to='media')
I tried to put signals and hooks after the model definitions but I am not being able to get this filename. Can someone shed some light pls?
UPDATE: Let me elaborate what I want to achieve essentially:
User1 hits endpoint "/api/calculate_interest_rate" which is rendered
by a frontend React component. "calculate_interest_rate" is served by
DRF, and lets the user upload a CSV file. This will be stored as
"user1.csv", the file is processed and then tabulated (rendered by
React).
At the same time and in parallel to User1, User2 hits the same endpoint "/api/calculate_interest_rate" and
by mistake he saves his file as "user1.csv", and uploads it to the systemn.
So I want to be able to detect both names of the file in order to process it. By using always the same default filename (ex. using the OverwriteStorage() technique), I will probably cause chaos when two or more users are using the same filename. Therefore I am looking into a technique that allows me to get the filename as is, and process it immediately.
How about using storage option?
class OverwriteStorage(FileSystemStorage):
def get_available_name(self, name, max_length=None):
print("filename", name)
#parts = name.split('.') you can separate name and extension.
return super().get_available_name(name)
upload_image = models.ImageField(
upload_to=[yourpath],
default=[defaultname],
storage=OverwriteStorage()
)
I suggest you to following this configuration:
1. Change your MEDIA_ROOT and MEDIA_URL inside file of settings.py
MEDIA_URL = '/media/'
MEDIA_ROOT = '/path/to/env/projectname/media'
2. Then, I suggest you to change your upload_to='media to upload_to='images/%Y/%m/%d, also rename your field of name with image.
class ImageUploader(models.Model):
image = models.FileField(upload_to='images/%Y/%m/%d')
# OR
# image = models.ImageField(upload_to='images/%Y/%m/%d')
Explanation; If you following this configuration, you could have uploaded images are following, eg: /media/images/2017/01/29/yourimage.jpg. This is one way to handle the problem of duplicated files.
3. But if you want to upload file with multiple times without duplicate files, you can using deconstructible;
import os, time, uuid
from django.db import models
from django.utils.deconstruct import deconstructible
class ImageUploader(models.Model):
#deconstructible
class PathAndRename(object):
def __init__(self, sub_path):
self.path = sub_path
def __call__(self, instance, filename):
# eg: filename = 'my uploaded file.jpg'
ext = filename.split('.')[-1] #eg: '.jpg'
uid = uuid.uuid4().hex[:10] #eg: '567ae32f97'
# eg: 'my-uploaded-file'
new_name = '-'.join(filename.replace('.%s' % ext, '').split())
# eg: 'my-uploaded-file_64c942aa64.jpg'
renamed_filename = '%(new_name)s_%(uid)s.%(ext)s' % {'new_name': new_name, 'uid': uid, 'ext': ext}
# eg: 'images/2017/01/29/my-uploaded-file_64c942aa64.jpg'
return os.path.join(self.path, renamed_filename)
image_path = time.strftime('images/%Y/%m/%d')
image = models.ImageField(upload_to=PathAndRename(self.image_path))
I need to change default file upload behavior in django and the documentation on the django site is rather confusing.
I have a model with a field as follows:
class document (models.Model):
name = models.CharField(max_length=200)
file = models.FileField(null=True, upload_to='uploads/')
I need to create a .json file that will contain meta data when a file is uploaded. For example if I upload a file mydocument.docx I need to create mydocument.json file within the uploads/ folder and add meta information about the document.
From what I can decipher from the documentation I need to create a file upload handler as a subclass of django.core.files.uploadhandler.FileUploadHandler. It also goes on to say I can define this anywhere I want.
My questions: Where is the best place to define my subclass? Also from the documentation found here https://docs.djangoproject.com/en/1.8/ref/files/uploads/#writing-custom-upload-handlers looks like the subclass would look like the following:
class FileUploadHandler(object):
def handle_raw_input(self, input_data, META, content_length, boundary, encoding=None):
# do the acctual writing to disk
def file_complete(self, file_size):
# some logic to create json file
Does anyone have a working example of a upload handler class that works for django version 1.8?
One option could be to do the .json file generation on the (model) form used to initially upload the file. Override the save() method of the ModelForm to generate the file immediately after the model has been saved.
class DocumentForm(forms.ModelForm):
class Meta(object):
model = Document
fields = 'name', 'file'
def save(self, commit=True):
saved_document = super().save(commit)
with open(saved_document.file.path + '.json', mode='w') as fh:
fh.write(json.dumps({
"size": saved_document.file.size,
"uploaded": timezone.now().isoformat()
}))
return saved_document
I've tested this locally but YMMV if you are using custom storages for working with things like S3.
Imagine a model like this:
class CFile(models.Model):
filepath = models.FileField(upload_to=...)
collection = models.ForeignKey("FileCollection",null=True)
... # other attributes that are not relevant
def clean(self):
bname = os.path.basename
if self.collection:
cfiles = self.baseline.attachment_set.all()
with_same_basename = filter(lambda e: bname(e.filepath.path) == bname(self.filepath.path),cfiles)
if len(with_same_basename) > 0:
raise ValidationError("There already exists a file with the same name in this collection")
class FileCollection(models.Model):
name = models.CharField(max_length=255)
files= models.ManyToManyField("CFile")
I want to disallow the upload of a CFile if there already exists a CFile with the same basename, that's why I added the clean. The problem is:
I upload a CFile, with the name file1.png -> gets uploaded because no other files with this name exist
I upload another CFile, with the name file1.png -> I get the expected error that I already have a file with this name. So, I try to change the file, and upload a file with a different name ( file2.png ). The problem is, I stopped via pdb in the clean, and the model instance is still file1.png. I imagine this happens because of my ValidationError and django allows me to "correct my mistake". The problem is I cannot correct it if I cannot upload another file. How can I handle this?
EDIT: This happens in the admin area, sorry for forgetting to mention this before. I don't have anything custom ( besides inlines = [ FileInline ] ).
I think the clearest way is to declare another field in your model for filename and make it unique for every collection. Like this:
class CFile(models.Model):
filepath = models.FileField(upload_to=...)
collection = models.ForeignKey("FileCollection",null=True, related_name='files')
filename = models.CharField(max_length=255)
... # other attributes that are not relevant
class Meta:
unique_together = (('filename', 'collection'),)
def save(self, *args, **kwargs):
self.filename = bname(self.filepath.path)
super(CFile, self).save(args, kwargs)
I have an existing file on disk (say /folder/file.txt) and a FileField model field in Django.
When I do
instance.field = File(file('/folder/file.txt'))
instance.save()
it re-saves the file as file_1.txt (the next time it's _2, etc.).
I understand why, but I don't want this behavior - I know the file I want the field to be associated with is really there waiting for me, and I just want Django to point to it.
How?
just set instance.field.name to the path of your file
e.g.
class Document(models.Model):
file = FileField(upload_to=get_document_path)
description = CharField(max_length=100)
doc = Document()
doc.file.name = 'path/to/file' # must be relative to MEDIA_ROOT
doc.file
<FieldFile: path/to/file>
If you want to do this permanently, you need to create your own FileStorage class
import os
from django.conf import settings
from django.core.files.storage import FileSystemStorage
class MyFileStorage(FileSystemStorage):
# This method is actually defined in Storage
def get_available_name(self, name):
if self.exists(name):
os.remove(os.path.join(settings.MEDIA_ROOT, name))
return name # simply returns the name passed
Now in your model, you use your modified MyFileStorage
from mystuff.customs import MyFileStorage
mfs = MyFileStorage()
class SomeModel(model.Model):
my_file = model.FileField(storage=mfs)
try this (doc):
instance.field.name = <PATH RELATIVE TO MEDIA_ROOT>
instance.save()
It's right to write own storage class. However get_available_name is not the right method to override.
get_available_name is called when Django sees a file with same name and tries to get a new available file name. It's not the method that causes the rename. the method caused that is _save. Comments in _save is pretty good and you can easily find it opens file for writing with flag os.O_EXCL which will throw an OSError if same file name already exists. Django catches this Error then calls get_available_name to get a new name.
So I think the correct way is to override _save and call os.open() without flag os.O_EXCL. The modification is quite simple however the method is a little be long so I don't paste it here. Tell me if you need more help :)
I had exactly the same problem! then I realize that my Models were causing that. example I hade my models like this:
class Tile(models.Model):
image = models.ImageField()
Then, I wanted to have more the one tile referencing the same file in the disk! The way that I found to solve that was change my Model structure to this:
class Tile(models.Model):
image = models.ForeignKey(TileImage)
class TileImage(models.Model):
image = models.ImageField()
Which after I realize that make more sense, because if I want the same file being saved more then one in my DB I have to create another table for it!
I guess you can solve your problem like that too, just hoping that you can change the models!
EDIT
Also I guess you can use a different storage, like this for instance: SymlinkOrCopyStorage
http://code.welldev.org/django-storages/src/11bef0c2a410/storages/backends/symlinkorcopy.py
You should define your own storage, inherit it from FileSystemStorage, and override OS_OPEN_FLAGS class attribute and get_available_name() method:
Django Version: 3.1
Project/core/files/storages/backends/local.py
import os
from django.core.files.storage import FileSystemStorage
class OverwriteStorage(FileSystemStorage):
"""
FileSystemStorage subclass that allows overwrite the already existing
files.
Be careful using this class, as user-uploaded files will overwrite
already existing files.
"""
# The combination that don't makes os.open() raise OSError if the
# file already exists before it's opened.
OS_OPEN_FLAGS = os.O_WRONLY | os.O_TRUNC | os.O_CREAT | getattr(os, 'O_BINARY', 0)
def get_available_name(self, name, max_length=None):
"""
This method will be called before starting the save process.
"""
return name
In your model, use your custom OverwriteStorage
myapp/models.py
from django.db import models
from core.files.storages.backends.local import OverwriteStorage
class MyModel(models.Model):
my_file = models.FileField(storage=OverwriteStorage())
The answers work fine if you are using the app's filesystem to store your files. But, If your are using boto3 and uploading to sth like AWS S3 and maybe you want to set a file already existing in an S3 bucket to your model's FileField then, this is what you need.
We have a simple model class with a filefield:
class Image(models.Model):
img = models.FileField()
owner = models.ForeignKey(get_user_model(), on_delete=models.CASCADE, related_name='images')
date_added = models.DateTimeField(editable=False)
date_modified = models.DateTimeField(editable=True)
from botocore.exceptions import ClientError
import boto3
s3 = boto3.client(
's3',
aws_access_key_id=os.getenv("AWS_ACCESS_KEY_ID"),
aws_secret_access_key=os.getenv("AWS_SECRET_ACCESS_KEY")
)
s3_key = S3_DIR + '/' + filename
bucket_name = os.getenv("AWS_STORAGE_BUCKET_NAME")
try:
s3.upload_file(local_file_path, bucket_name, s3_key)
# we want to store it to our db model called **Image** after s3 upload is complete so,
image_data = Image()
image_data.img.name = s3_key # this does it !!
image_data.owner = get_user_model().objects.get(id=owner_id)
image_data.save()
except ClientError as e:
print(f"failed uploading to s3 {e}")
Setting the S3 KEY into the name field of the FileField does the trick. As much i have tested everything related works as expected e.g previewing the image file in django admin. fetching the images from db appends the root s3 bucket prefix (or, the cloudfront cdn prefix) to the s3 keys of the files too. Ofcourse, its given that, i already had a working setup of the django settings.py for boto and s3.