Develop Reference

File manipulation in Django models.py

I am building a Django app that saves an .stl file passed using a formulary and my goal is to open the file, extract some information with a script that is already tested, and save this information in the same register that the file.
I am doing this:
from stl import mesh # numpy-stl library
def informationGeneration(stl_route, *args, **kwargs):
# scripts that generates the information
myMesh = mesh.Mesh.from_file(stl_route) # here the error appears
return myMesh.areas.shape[0]
class Piece(models.Model):
"""Piece model."""
# ...
file = models.FileField(upload_to='pieces/files', default='NA')
# ...
information = models.IntegerField(default=0)
def save(self, *args, **kwargs):
"""Overriding the save method."""
self.information = informationGeneration(self.file)
super().save(*args, **kwargs)
def __str__(self):
# ...
The problem is that when I try to save a new instance, numpy-stl detects an error, self.file is not the .stl file, is an alement of the formulary.
Then, I use a form:
class PieceForm(forms.ModelForm):
"""Pieces model form."""
class Meta:
"""Form settings."""
model = Piece
fields = ('file')
How can I pass the file and not the route?

Piece.file is not a path, it's a models.FileField. To get the path, you have to use self.file.path.
Just beware that if there's actually no file for this field, self.file.path will raise an exception (ValueError, "the file attribute has no file associated with it"), so it's better to test before. models.FileField have a false value in a boolean context, so you want:
if self.file:
self.information = informationGeneration(self.file.path)
A couple notes:
1/ a function is an action, so it's name should be a verb (ie "extract_informations")
2/ you probably don't want to re-parse the file's content each and every time your object is saved, only when the file has changed. You can use a md5sum (stored in the model) to check this.
3/ I have not double-checked but I really dont think you should use a default for this field - if you want to make it optional, use blank=True and null=True.

Overiding save for just one field in Django

This is my models.py:
class College(models.Model):
name = models.CharField(unique=True, max_length=50,
help_text='Name of the college.'
)
slug = models.SlugField(unique=True)
description = models.TextField(blank = True)
image = models.ImageField(upload_to='site-media/media/college_images/',
default = 'site-media/media/college_images/default.jpeg'
)
user = models.ForeignKey(User)
def get_absolute_url(self):
return "/%s/" % self.slug
def create_thumbnail(self):
if not self.image:
return
THUMBNAIL_SIZE = (250,193)
image = Image.open(StringIO(self.image.read()))
thumb = ImageOps.fit(image, THUMBNAIL_SIZE, Image.ANTIALIAS)
temp_handle = StringIO()
thumb.convert('RGB').save(temp_handle, 'jpeg')
temp_handle.seek(0)
suf = SimpleUploadedFile(os.path.split(self.image.name)[-1],
temp_handle.read(), content_type='image/jpeg')
self.image.save('%s_college_.%s'%(os.path.splitext(suf.name)[0],'jpeg'), suf, save=False)
def save(self, *args, **kwargs):
self.slug = slugify(self.name)
self.create_thumbnail()
super(College, self).save(*args, **kwargs)
I have presented the user with a form to edit just the description. When the description 'POST' is made the 'save()' method above is called. The problem with this is that the thumbnail is created over and over again with a bigger name every time. And, also the previous thumbnail is not deleted from the hard disk. Is it possible, that this 'thumbnail' method doesn't get called over and over again with each edit of the 'description'.

You can check whether you are sending image file in you request post or not. For this You need to call your save in view with one argument request like : college.save(request)
def save(self, request=False, *args, **kwargs):
self.slug = slugify(self.name)
if request and request.FILES.get('image',False):
self.create_thumbnail()
super(College, self).save(*args, **kwargs)
OR
you can differentiate your save and edit using
if self.pk is not None
But it can create problem if you edit your image.
So its your choice How you want to go with it.

There are two reasonable paths I see to handle this. Neither are ideal, so I'll be interested to see if anyone has a better option to offer.
One is to save the filename of the most recent image for which you created a thumbnail as a model field. Then in your save method you can check the filename of your image field against it and create a new thumbmail if it has changed. This has the disadvantage of requiring a new model field, but is more universal in its application.
The other is to override the save method of the form class. You can then check the old image filename by looking at self.instance.image and comparing that against self.cleaned_data['image']. This has the disadvantage of only affecting views that use that form class, but doesn't require changing your data model. You can pass a custom form to the admin, if you're using it, or override the admin class's save_model method.

The simple solution is to NOT try to create the thumbnail at this stage, but only when it's needed, ie (pseudocode example):
class Whatever(models.Model):
image = models.ImageField(...)
#property
def thumbnail(self):
thumb = do_i_have_a_thumbnail_yet(self.image)
if not thumb:
thumb = ok_then_make_me_a_thumbnail_and_store_it_please(self.image)
return thumb
Implementing do_i_have_a_thumbnail_yet() and ok_then_make_me_a_thumbnail_and_store_it_please() is left as an exercise to the reader, but there are librairies or django apps providing such services.
Another - and even better imho - solution is to delegate thumbnails handling to a templatetag or template filter. Why ? because
thumbnails are mostly presentation stuff and do not belong to the model
most of the time you'll need different thumbnails sizes from a same image
the front-end developer may want to be free to change the thumbnail's size without having to ask you to change your backend code and write a migration script to recompute all existing thumbnails.

Django admin error in many-to-many relationship

For example.
class One(models.Model):
text=models.CharField(max_length=100)
class Two(models.Model):
test = models.Integer()
many = models.ManyToManyField(One, blank=True)
When I try save my object in admin panel, I take error such as:
"'Two' instance needs to have a primary key value before a many-to-many relationship can be used."
I use django 1.3. I tried add AutoField to Two class, but it's not work too.
This is my code.
from django.http import HttpResponse, HttpResponseRedirect
from django.shortcuts import render_to_response, redirect
from django.template import RequestContext
from django.core.urlresolvers import reverse
from project.foo.forms import FooForm
from project.foo.models import Foo
from project.fooTwo.views import fooTwoView
def foo(request, template_name="foo_form.html"):
if request.method == 'POST':
form = FooForm(data=request.POST)
if form.is_valid():
foo = Foo()
foo.name = request.POST.get("name")
foo.count_people = request.POST.get("count_people")
foo.date_time = request.POST.get("date_time")
foo.save()
return fooTwoView(request)
else:
form = FooForm()
return render_to_response(template_name, RequestContext(request, {
"form": form,
}))
P.S. I find my fail. It is in model. I used many-to-many in save method. I add checking before using, but it's not help.
class Foo(models.Model):
name = models.CharField(max_length=100, null=False, blank=False)
count_people = models.PositiveSmallIntegerField()
menu = models.ManyToManyField(Product, blank=True, null=True)
count_people = models.Integer()
full_cost = models.IntegerField(blank=True)
def save(self, *args, **kwargs):
if(hasattr(self,'menu')):
self.full_cost = self.calculate_full_cost()
super(Foo, self).save(*args, **kwargs)
def calculate_full_cost(self):
cost_from_products = sum([product.price for product in self.menu.all()])
percent = cost_from_products * 0.1
return cost_from_products + percent
I try hack in save method such as
if(hasattr(self,Two)):
self.full_cost = self.calculate_full_cost()
This is help me, but i dont think that is the django way. What is interesting, that is without this checking admin panel show error, but create object. Now, if i select item from Two and save, my object does not have full_cost, but when i view my object, admin panel remember my choice and show me my Two item, what i select... I dont know why.
How do i save this?

There are quite a few problems with your code. The most obvious one are
1/ in your view, using a form for user inputs validation/sanitization/conversion then ignoring the santized/converted data and getting unsanitized inputs directly from the request. Use form.cleaned_data instead of request.POST to get your data, or even better use a ModelForm which will take care of creating a fully populated Foo instance for you.
2/ there's NO implicit "this" (or "self" or whatever) pointer in Python methods, you have to explicitely use "self" to get at the instance attributes. Here's what your model's "save" method really do:
def save(self, *args, **kwargs):
# test the truth value of the builtin "id" function
if(id):
# create a local variable "full_cost"
full_cost = self.calculate_full_cost()
# call on super with a wrong base class
super(Banquet, self).save(*args, **kwargs)
# and exit, discarding the value of "full_cost"
Now with regard to your question: Foo.save is obviously not the right place to compute someting based on m2m related objects. Either write a distinct method that run the computation AND update Foo AND save it and call it after the m2m are saved (hint : a ModelForm will take care of saveing the m2m related objects for you), or just use the m2m_changed signal.
This being said, I strongly suggest you spend a few hours learning Python and Django - it will save you a lot of time.

Why not use "OneToOneField" instead of Many-to-Many

Django file upload: filename not sticking

I'm uploading files and storing metadata in a db. Part of the metadata is the file name itself. However, somewhere down the line, the filename seems to not be getting saved! I will paste only what I think are relevant parts of the code to keep this short.
class UploadFile(models.Model):
...
theFile = models.FileField(upload_to = "Genius/Uploads/", null = True)
filename = models.CharField(max_length = 50, blank = True, null = False)
class UploadFileForm(ModelForm):
class Meta:
model = UploadFile
fields = ('title', 'theFile', 'date_uploaded',) # Don't prompt for filename
def files_upload(request):
if request.method == 'POST':
form = UploadFileForm(request.POST, request.FILES)
if form.is_valid():
form.filename = request.FILES['theFile'].name # TODO: sanitize!
# form.filename = 'foo'
form.save()
return HttpResponseRedirect('/files/upload/successful/')
else:
form = UploadFileForm()
return render_to_response('files/upload_file.html', { 'form': form })
I have checked the value of request.FILES['theFile'].name before & after saving the form. For whatever reason it is intact but never seems to make it into the DB.

That's because form.filename is the form field, not the value it will be saving.
You are looking for something like this:
class UploadFileForm(ModelForm):
def save(self, commit=True):
instance = ModelForm.save(self, commit=False)
instance.filename = self.files['theFile'].name
if commit:
instance.save()
return instance
class Meta:
model = UploadFile
fields = ('title', 'theFile', 'date_uploaded',) # Don't prompt for filename
Alternative solution:
upload_file = form.save(commit=False)
upload_file.filename = request.FILES['theFile'].name
upload_file.save()

Form field values aren't accessed via attributes on the form. So setting 'form.filename' doesn't set the value to be saved in the filename field. Instead, set the value on the instance returned by form.save().
upload_file = form.save(commit=False)
upload_file.filename = filename
upload_file.save()

I just wanted to add that, in the future, you might try to avoid putting such business logic on the model form. While WoLpH's answer is correct and a great example of how to handle additional model instance processing through ModelForm, particular cases of having fields dependent on other fields data is handled in the Model, Form and ModelForm API through their respected clean() methods and is mentioned in several places in the official reference docs (here's one on forms, though the same holds true for the Model and ModelForm APIs).
In your case this would mean:
import os
class UploadFile(models.Model):
# ...
def clean(self):
# Field data has already been populated by this point.
# Note that `FieldFile` inherits from `File` and that
# `File.name` is actually the full path to the file
# so we need to get the base path component sans the extension
path, extension = os.path.splitext(self.thefile.file.name)
self.filename = os.path.basename(path)
And that's about it! If you properly set the editable attribute on your model fields, you'll find that you can rely on Django to automatically generate the ModelForm for the UploadFile model. That means you don't have to define a ModelForm for the generic create, update views or your model's ModelAdmin and that's less lines of code to manage!
The general rule of thumb is that you think twice about whether overriding default behavior is ever justified or self-contained, especially when your working at the far end of the business logic chain, otherwise you may feel the wrath of unexpected flops.

Replacing a Django image doesn't delete original

In Django, if you have a ImageFile in a model, deleting will remove the associated file from disk as well as removing the record from the database.
Shouldn't replacing an image also remove the unneeded file from disk? Instead, I see that it keeps the original and adds the replacement.
Now deleting the object won't delete the original file only the replacement.
Are there any good strategies to doing this? I don't want to have a bunch of orphan files if my users replace their images frequently.

The best strategy I've found is to make a custom save method in the model:
class Photo(models.Model):
image = ImageField(...) # works with FileField also
def save(self, *args, **kwargs):
# delete old file when replacing by updating the file
try:
this = Photo.objects.get(id=self.id)
if this.image != self.image:
this.image.delete(save=False)
except: pass # when new photo then we do nothing, normal case
super(Photo, self).save(*args, **kwargs)
And beware, as with the updating which doesn't delete the back end file, deleting an instance model (here Photo) will not delete the back-end file, not in Django 1.3 anyway, you'll have to add more custom code to do that (or regularly do some dirty cron job).
Finally test all your update/delete cases with your ForeignKey, ManytoMany and others relations to check if the back-end files are correctly deleted. Believe only what you test.

Shouldn't replacing an image also remove the unneeded file from disk?
In the olden days, FileField was eager to clean up orphaned files. But that changed in Django 1.2:
In earlier Django versions, when a model instance containing a FileField was deleted, FileField took it upon itself to also delete the file from the backend storage. This opened the door to several potentially serious data-loss scenarios, including rolled-back transactions and fields on different models referencing the same file. In Django 1.2.5, FileField will never delete files from the backend storage.

The code in the following working example will, upon uploading an image in an ImageField, detect if a file with the same name exists, and in that case, delete that file before storing the new one.
It could easily be modified so that it deletes the old file regardless of the filename. But that's not what I wanted in my project.
Add the following class:
from django.core.files.storage import FileSystemStorage
class OverwriteStorage(FileSystemStorage):
def _save(self, name, content):
if self.exists(name):
self.delete(name)
return super(OverwriteStorage, self)._save(name, content)
def get_available_name(self, name):
return name
And use it with ImageField like so:
class MyModel(models.Model):
myfield = models.ImageField(
'description of purpose',
upload_to='folder_name',
storage=OverwriteStorage(), ### using OverwriteStorage here
max_length=500,
null=True,
blank=True,
height_field='height',
width_field='width'
)
height = models.IntegerField(blank=True, null=True)
width = models.IntegerField(blank=True, null=True)

If you don't use transactions or you don't afraid of loosing files on transaction rollback, you can use django-cleanup

There have been a number of tickets regarding this issue though it is likely this will not make it into the core. The most comprehensive is http://code.djangoproject.com/ticket/11663. The patches and ticket comments can give you some direction if you are looking for a solution.
You can also consider using a different StorageBackend such as the Overwrite File Storage System given by Django snippet 976. http://djangosnippets.org/snippets/976/. You can change your default storage to this backend or you can override it on each FileField/ImageField declaration.

Here is a code that can work with or without upload_to=... or blank=True, and when the submitted file has the same name as the old one.
(py3 syntax, tested on Django 1.7)
class Attachment(models.Model):
document = models.FileField(...) # or ImageField
def delete(self, *args, **kwargs):
self.document.delete(save=False)
super().delete(*args, **kwargs)
def save(self, *args, **kwargs):
if self.pk:
old = self.__class__._default_manager.get(pk=self.pk)
if old.document.name and (not self.document._committed or not self.document.name):
old.document.delete(save=False)
super().save(*args, **kwargs)
Remember that this kind of solution is only applicable if you are in a non transactional context (no rollback, because the file is definitively lost)

I used a simple method with popen, so when i save my Info model i delete the former file before linking to the new:
import os
try:
os.popen("rm %s" % str(info.photo.path))
except:
#deal with error
pass
info.photo = nd['photo']

I save the original file and if it has changed - delete it.
class Document(models.Model):
document = FileField()
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self._document = self.document
def save(self, *args, **kwargs):
if self.document != self._document:
self._document.delete()
super().save(*args, **kwargs)