I am building a Django app that saves an .stl file passed using a formulary and my goal is to open the file, extract some information with a script that is already tested, and save this information in the same register that the file.
I am doing this:
from stl import mesh # numpy-stl library
def informationGeneration(stl_route, *args, **kwargs):
# scripts that generates the information
myMesh = mesh.Mesh.from_file(stl_route) # here the error appears
return myMesh.areas.shape[0]
class Piece(models.Model):
"""Piece model."""
# ...
file = models.FileField(upload_to='pieces/files', default='NA')
# ...
information = models.IntegerField(default=0)
def save(self, *args, **kwargs):
"""Overriding the save method."""
self.information = informationGeneration(self.file)
super().save(*args, **kwargs)
def __str__(self):
# ...
The problem is that when I try to save a new instance, numpy-stl detects an error, self.file is not the .stl file, is an alement of the formulary.
Then, I use a form:
class PieceForm(forms.ModelForm):
"""Pieces model form."""
class Meta:
"""Form settings."""
model = Piece
fields = ('file')
How can I pass the file and not the route?
Piece.file is not a path, it's a models.FileField. To get the path, you have to use self.file.path.
Just beware that if there's actually no file for this field, self.file.path will raise an exception (ValueError, "the file attribute has no file associated with it"), so it's better to test before. models.FileField have a false value in a boolean context, so you want:
if self.file:
self.information = informationGeneration(self.file.path)
A couple notes:
1/ a function is an action, so it's name should be a verb (ie "extract_informations")
2/ you probably don't want to re-parse the file's content each and every time your object is saved, only when the file has changed. You can use a md5sum (stored in the model) to check this.
3/ I have not double-checked but I really dont think you should use a default for this field - if you want to make it optional, use blank=True and null=True.
Related
I want to create a Django model that contains a FileField to store image and video files, but I want to validate the files before saving the instance. I've thought about adding three fields:
file: A FileField field. This will only be used to have a file column in the database, but serializers won't use it (instead they will use the two next fields).
file_image: An ImageField to perform image file validation. Before the model instance is saved, the file will be assigned to the file field. I don't want this field to have a dabatase representation.
file_video: A VideoField (custom field) to perform video file validation. Before the model instance is saved, the file will be assigned to the file field. I don't want this field to have a dabatase representation.
Of course, file_image and file_video won't be set at the same time.
The problem is preventing makemigrations from including file_image and file_video in the migrations. I could edit the migration file by hand, but I wonder if there is any way to automatically ignore these fields.
class MyModel(models.Model):
file = models.ImageField()
file_image = models.ImageField() # Not an actual column
file_video = models.VideoField() # Not an actual column
def save(self, *args, **kwargs):
if self.file_image.file is not None:
self.file.file = self.file_image.file
elif self.file_video.file is not None:
self.file.file = self.file_video.file
else:
raise ValidationError()
super().save(*args, **kwargs)
Your model is a representation of what's in the database. I would advise you not to fight against the ORM in this manner. Instead, I would perform the validation within the form class that's used when creating/updating the instance. With the form you can define the fields, file_video and file_image, then whichever is used, use that field to write to file.
Ok, so I abandoned my original idea of adding extra fields to the model and I only left file field, as suggested by #schillingt. Now I'm using a custom validation in the clean() method to validate the file type.
from django.db import models
from django.db.models.fields.files import ImageFieldFile
from django import forms
from custom_fields import forms as custom_forms
from custom_fields import fields as custom_fields
class MyModel(models.Model):
file = models.ImageField()
def clean(self):
# Get the MIME type to have a hint of the file type
import magic
mime_type = magic.from_buffer(self.file.file.read(1024), mime=True)
if mime_type.startswith('image'):
image_field = ImageFieldFile(self, models.ImageField(),
self.file.name)
image_field.file = self.file.file
if image_field.width:
self.width = image_field.width
self.height = image_field.height
self.media_type = mime_type
else:
raise ValidationError({'file': forms.ImageField.
default_error_messages['invalid_image']})
elif mime_type.startswith('video'):
video_field = custom_fields.VideoFieldFile(self,
custom_fields.VideoField(),
self.file.name)
video_field.file = self.file.file
if video_field.width:
self.width = video_field.width
self.height = video_field.height
self.duration = video_field.duration
self.media_type = mime_type
else:
raise ValidationError({'src': custom_forms.VideoField.
default_error_messages['invalid_video']})
else:
raise ValidationError({'src': 'The file is neither an image nor a video.'})
def save(self, *args, **kwargs):
self.full_clean()
super().save(*args, **kwargs)
I know it's an ugly solution, but could this work? I mean, I tested it and it seems to work, but surely there will be a better and more elegant way.
I need to change default file upload behavior in django and the documentation on the django site is rather confusing.
I have a model with a field as follows:
class document (models.Model):
name = models.CharField(max_length=200)
file = models.FileField(null=True, upload_to='uploads/')
I need to create a .json file that will contain meta data when a file is uploaded. For example if I upload a file mydocument.docx I need to create mydocument.json file within the uploads/ folder and add meta information about the document.
From what I can decipher from the documentation I need to create a file upload handler as a subclass of django.core.files.uploadhandler.FileUploadHandler. It also goes on to say I can define this anywhere I want.
My questions: Where is the best place to define my subclass? Also from the documentation found here https://docs.djangoproject.com/en/1.8/ref/files/uploads/#writing-custom-upload-handlers looks like the subclass would look like the following:
class FileUploadHandler(object):
def handle_raw_input(self, input_data, META, content_length, boundary, encoding=None):
# do the acctual writing to disk
def file_complete(self, file_size):
# some logic to create json file
Does anyone have a working example of a upload handler class that works for django version 1.8?
One option could be to do the .json file generation on the (model) form used to initially upload the file. Override the save() method of the ModelForm to generate the file immediately after the model has been saved.
class DocumentForm(forms.ModelForm):
class Meta(object):
model = Document
fields = 'name', 'file'
def save(self, commit=True):
saved_document = super().save(commit)
with open(saved_document.file.path + '.json', mode='w') as fh:
fh.write(json.dumps({
"size": saved_document.file.size,
"uploaded": timezone.now().isoformat()
}))
return saved_document
I've tested this locally but YMMV if you are using custom storages for working with things like S3.
This is my models.py:
class College(models.Model):
name = models.CharField(unique=True, max_length=50,
help_text='Name of the college.'
)
slug = models.SlugField(unique=True)
description = models.TextField(blank = True)
image = models.ImageField(upload_to='site-media/media/college_images/',
default = 'site-media/media/college_images/default.jpeg'
)
user = models.ForeignKey(User)
def get_absolute_url(self):
return "/%s/" % self.slug
def create_thumbnail(self):
if not self.image:
return
THUMBNAIL_SIZE = (250,193)
image = Image.open(StringIO(self.image.read()))
thumb = ImageOps.fit(image, THUMBNAIL_SIZE, Image.ANTIALIAS)
temp_handle = StringIO()
thumb.convert('RGB').save(temp_handle, 'jpeg')
temp_handle.seek(0)
suf = SimpleUploadedFile(os.path.split(self.image.name)[-1],
temp_handle.read(), content_type='image/jpeg')
self.image.save('%s_college_.%s'%(os.path.splitext(suf.name)[0],'jpeg'), suf, save=False)
def save(self, *args, **kwargs):
self.slug = slugify(self.name)
self.create_thumbnail()
super(College, self).save(*args, **kwargs)
I have presented the user with a form to edit just the description. When the description 'POST' is made the 'save()' method above is called. The problem with this is that the thumbnail is created over and over again with a bigger name every time. And, also the previous thumbnail is not deleted from the hard disk. Is it possible, that this 'thumbnail' method doesn't get called over and over again with each edit of the 'description'.
You can check whether you are sending image file in you request post or not. For this You need to call your save in view with one argument request like : college.save(request)
def save(self, request=False, *args, **kwargs):
self.slug = slugify(self.name)
if request and request.FILES.get('image',False):
self.create_thumbnail()
super(College, self).save(*args, **kwargs)
OR
you can differentiate your save and edit using
if self.pk is not None
But it can create problem if you edit your image.
So its your choice How you want to go with it.
There are two reasonable paths I see to handle this. Neither are ideal, so I'll be interested to see if anyone has a better option to offer.
One is to save the filename of the most recent image for which you created a thumbnail as a model field. Then in your save method you can check the filename of your image field against it and create a new thumbmail if it has changed. This has the disadvantage of requiring a new model field, but is more universal in its application.
The other is to override the save method of the form class. You can then check the old image filename by looking at self.instance.image and comparing that against self.cleaned_data['image']. This has the disadvantage of only affecting views that use that form class, but doesn't require changing your data model. You can pass a custom form to the admin, if you're using it, or override the admin class's save_model method.
The simple solution is to NOT try to create the thumbnail at this stage, but only when it's needed, ie (pseudocode example):
class Whatever(models.Model):
image = models.ImageField(...)
#property
def thumbnail(self):
thumb = do_i_have_a_thumbnail_yet(self.image)
if not thumb:
thumb = ok_then_make_me_a_thumbnail_and_store_it_please(self.image)
return thumb
Implementing do_i_have_a_thumbnail_yet() and ok_then_make_me_a_thumbnail_and_store_it_please() is left as an exercise to the reader, but there are librairies or django apps providing such services.
Another - and even better imho - solution is to delegate thumbnails handling to a templatetag or template filter. Why ? because
thumbnails are mostly presentation stuff and do not belong to the model
most of the time you'll need different thumbnails sizes from a same image
the front-end developer may want to be free to change the thumbnail's size without having to ask you to change your backend code and write a migration script to recompute all existing thumbnails.
Imagine a model like this:
class CFile(models.Model):
filepath = models.FileField(upload_to=...)
collection = models.ForeignKey("FileCollection",null=True)
... # other attributes that are not relevant
def clean(self):
bname = os.path.basename
if self.collection:
cfiles = self.baseline.attachment_set.all()
with_same_basename = filter(lambda e: bname(e.filepath.path) == bname(self.filepath.path),cfiles)
if len(with_same_basename) > 0:
raise ValidationError("There already exists a file with the same name in this collection")
class FileCollection(models.Model):
name = models.CharField(max_length=255)
files= models.ManyToManyField("CFile")
I want to disallow the upload of a CFile if there already exists a CFile with the same basename, that's why I added the clean. The problem is:
I upload a CFile, with the name file1.png -> gets uploaded because no other files with this name exist
I upload another CFile, with the name file1.png -> I get the expected error that I already have a file with this name. So, I try to change the file, and upload a file with a different name ( file2.png ). The problem is, I stopped via pdb in the clean, and the model instance is still file1.png. I imagine this happens because of my ValidationError and django allows me to "correct my mistake". The problem is I cannot correct it if I cannot upload another file. How can I handle this?
EDIT: This happens in the admin area, sorry for forgetting to mention this before. I don't have anything custom ( besides inlines = [ FileInline ] ).
I think the clearest way is to declare another field in your model for filename and make it unique for every collection. Like this:
class CFile(models.Model):
filepath = models.FileField(upload_to=...)
collection = models.ForeignKey("FileCollection",null=True, related_name='files')
filename = models.CharField(max_length=255)
... # other attributes that are not relevant
class Meta:
unique_together = (('filename', 'collection'),)
def save(self, *args, **kwargs):
self.filename = bname(self.filepath.path)
super(CFile, self).save(args, kwargs)
In Django, if you have a ImageFile in a model, deleting will remove the associated file from disk as well as removing the record from the database.
Shouldn't replacing an image also remove the unneeded file from disk? Instead, I see that it keeps the original and adds the replacement.
Now deleting the object won't delete the original file only the replacement.
Are there any good strategies to doing this? I don't want to have a bunch of orphan files if my users replace their images frequently.
The best strategy I've found is to make a custom save method in the model:
class Photo(models.Model):
image = ImageField(...) # works with FileField also
def save(self, *args, **kwargs):
# delete old file when replacing by updating the file
try:
this = Photo.objects.get(id=self.id)
if this.image != self.image:
this.image.delete(save=False)
except: pass # when new photo then we do nothing, normal case
super(Photo, self).save(*args, **kwargs)
And beware, as with the updating which doesn't delete the back end file, deleting an instance model (here Photo) will not delete the back-end file, not in Django 1.3 anyway, you'll have to add more custom code to do that (or regularly do some dirty cron job).
Finally test all your update/delete cases with your ForeignKey, ManytoMany and others relations to check if the back-end files are correctly deleted. Believe only what you test.
Shouldn't replacing an image also remove the unneeded file from disk?
In the olden days, FileField was eager to clean up orphaned files. But that changed in Django 1.2:
In earlier Django versions, when a model instance containing a FileField was deleted, FileField took it upon itself to also delete the file from the backend storage. This opened the door to several potentially serious data-loss scenarios, including rolled-back transactions and fields on different models referencing the same file. In Django 1.2.5, FileField will never delete files from the backend storage.
The code in the following working example will, upon uploading an image in an ImageField, detect if a file with the same name exists, and in that case, delete that file before storing the new one.
It could easily be modified so that it deletes the old file regardless of the filename. But that's not what I wanted in my project.
Add the following class:
from django.core.files.storage import FileSystemStorage
class OverwriteStorage(FileSystemStorage):
def _save(self, name, content):
if self.exists(name):
self.delete(name)
return super(OverwriteStorage, self)._save(name, content)
def get_available_name(self, name):
return name
And use it with ImageField like so:
class MyModel(models.Model):
myfield = models.ImageField(
'description of purpose',
upload_to='folder_name',
storage=OverwriteStorage(), ### using OverwriteStorage here
max_length=500,
null=True,
blank=True,
height_field='height',
width_field='width'
)
height = models.IntegerField(blank=True, null=True)
width = models.IntegerField(blank=True, null=True)
If you don't use transactions or you don't afraid of loosing files on transaction rollback, you can use django-cleanup
There have been a number of tickets regarding this issue though it is likely this will not make it into the core. The most comprehensive is http://code.djangoproject.com/ticket/11663. The patches and ticket comments can give you some direction if you are looking for a solution.
You can also consider using a different StorageBackend such as the Overwrite File Storage System given by Django snippet 976. http://djangosnippets.org/snippets/976/. You can change your default storage to this backend or you can override it on each FileField/ImageField declaration.
Here is a code that can work with or without upload_to=... or blank=True, and when the submitted file has the same name as the old one.
(py3 syntax, tested on Django 1.7)
class Attachment(models.Model):
document = models.FileField(...) # or ImageField
def delete(self, *args, **kwargs):
self.document.delete(save=False)
super().delete(*args, **kwargs)
def save(self, *args, **kwargs):
if self.pk:
old = self.__class__._default_manager.get(pk=self.pk)
if old.document.name and (not self.document._committed or not self.document.name):
old.document.delete(save=False)
super().save(*args, **kwargs)
Remember that this kind of solution is only applicable if you are in a non transactional context (no rollback, because the file is definitively lost)
I used a simple method with popen, so when i save my Info model i delete the former file before linking to the new:
import os
try:
os.popen("rm %s" % str(info.photo.path))
except:
#deal with error
pass
info.photo = nd['photo']
I save the original file and if it has changed - delete it.
class Document(models.Model):
document = FileField()
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self._document = self.document
def save(self, *args, **kwargs):
if self.document != self._document:
self._document.delete()
super().save(*args, **kwargs)