Develop Reference

Django REST framework - parse uploaded csv file

I have setup Django REST framework endpoint that allows me to upload a csv file.
The serializers.py looks like this:
from rest_framework import serializers
class UploadSerializer(serializers.Serializer):
file_uploaded = serializers.FileField()
class Meta:
fields = ['file_uploaded']
In my views.py file, I'm trying to read data from uploaded csv like this:
class UploadViewSet(viewsets.ViewSet):
serializer_class = UploadSerializer
def create(self, request):
file_uploaded = request.FILES.get('file_uploaded')
with open(file_uploaded, mode ='r')as file:
csvFile = csv.reader(file)
for lines in csvFile:
print(lines)
I'm getting the following error:
... line 37, in create
with open(file_uploaded, mode ='r') as file:
TypeError: expected str, bytes or os.PathLike object, not InMemoryUploadedFile
I have checked type() of file_uploaded and It is <class 'django.core.files.uploadedfile.InMemoryUploadedFile'>
How can I read this file into dictionary or dataframe so I can extract the data I need from it?

When you do request.FILES.get('file_uploaded') it returns back an InMemoryUploadedFile which is a wrapper around a file object. You can access the file object using the file attribute.
file_uploaded # <InMemoryUploadedFile: xxx (xxx/xxx)>
file_object = file_uploaded.file
This file_object can then be opened.

an InMemoryFileObject can be used pretty much in the same way as an open file., so you don't need to open it.
def create(self, request):
file_upload = request.FILES.get("file_uploaded")
csvFile = csv.reader(file_upload)
for line in csvFile:
print(line)
This has some good information on file handling in django.
https://docs.djangoproject.com/en/4.1/topics/http/file-uploads/#handling-uploaded-files-with-a-model

Returning title of DB entries of type "FileField" in django

I am using Django admin to upload files to my websites DB daily. How do I keep track of the names of the files that I have uploaded to the database? because in Django admin they just show up as objects1 object2 etc. I want to return my the_file title of the file that I upload. I am trying to show the first 50 characters of the name of the files that I have uploaded. However, it is not working for me. I think I am having trouble because it is of type FileField
class UploadedFile(models.Model):
the_file = models.FileField()
def __str__(self):
return self.the_file[:50]

The object a FileField [Django-doc] wraps is a FieldFile [Django-doc] (note that the two words are swapped). You can obtain the name of the file with the .name attribute [Django-doc]:
The name of the file including the relative path from the root of the Storage of the associated FileField.
So we can use this in the __str__ method:
class UploadedFile(models.Model):
the_file = models.FileField()
def __str__(self):
return self.the_file.name[:50]

How to programmatically upload local file as Django model field?

I'm having troubles trying to upload files to FileField from local path.
I have correctly configurred CDN backend in S3 bucket and use it as PrivateMediaStorage for one of my model fields:
class MyModel(models.Model):
some_file = models.FileField(storage=PrivateMediaStorage())
...
With this very simple configuration whenever I'm creating/updating model through django-admin it is saved and file attached as some_file is correctly uploaded to S3 bucket.
Yet if I try to create/update model instance programmatically, say through custom manage.py command, model instance itself is created but attachment is never uploaded to CDN. Here's simplified version of code I'm using to upload files:
class Command(BaseCommand):
help = 'Creates dummy instance for quicker configuration'
def handle(self, *args, **options):
some_file = os.path.join(os.path.dirname(__file__), '../../../temporary/some_image.png')
if not os.path.exists(some_file):
raise CommandError(f'File {some_file} does not exist')
else:
instance, created = MyModel.objects.get_or_create(defaults={'some_file': some_file}, ...)
What is missing in my implementation and what needs to be adjusted to allow file uploads from local storage?

You're passing a string (the result of os.path.join()) to your some_file field, but you need to pass it an actual File object.
The easiest way to save a file on a model directly is to use the FieldFile's save() method.
As a working solution for case provided in question a valid way of creating a record would be:
instance = MyModel.objects.create(some_file=File(file=open(some_file, 'rb'), name='some_name.png'))
Or even better to use pathlib to obtain name dynamically:
from pathlib import Path
instance = MyModel.objects.create(some_file=File(file=open(some_file, 'rb'), name=Path(some_file).name))
Note that fetching a row based on the file is unlikely to work, AFAIK each time you open a file, doing a get_or_create() with the File instance as argument will probably create a new row each time. Better put file fields into defaults:
with open(some_file, 'rb') as file:
instance, created = MyModel.objects.get_or_create(
some_other_field=...,
defaults={'some_file': File(
file=file,
name=pathlib.Path(some_file).name
)}
)

you can also do something like this.
some_file = os.path.join(os.path.dirname(__file__), '../../../temporary/some_image.png')
instance.some_file.name = some_file
instance.save()

updated file name in django

This is an admin form handler, for example I upload a test.txt file in the django admin panel:
def save_model(self, request, obj, form, change):
if 'file' in form.changed_data:
print("file has changed: ")
print(obj.file)
else:
print("file has not changed")
super(FileAdmin, self).save_model(request, obj, form, change)
here I get the original file name from the upload form, but by fact the file is saved with another name if there is already a file with this name, but in the above code i get only the original name in all cases, how to can I get the changed/updated file name that was saved?..

The "_somehash" part is added by your project's filestorage when he sees there's already a file by the same name in the destination directory. This happens when the model instance is saved, so if all you need is to read the "final" name, you can get it from your (saved) model field's .name attribute (I assume you use a FileField of course).

"Upload" file from disk in Django

I generate a file in python, and want to "upload" that file to the django database. This way it is automatically put inside the media folder, and organized neatly with all other files of my application.
Now here is what I tried: (type hinting used, since it's python 3.6)
# forms.py
class UploadForm(forms.ModelForm):
class Meta:
model = UploadedFile
fields = ('document',)
# models.py
class UploadedFile(models.Model):
document = models.FileField(upload_to=get_upload_path)
# mimetype is generated by filename on save
mimetype = models.CharField(max_length=255)
# ... additional fields like temporary
def get_upload_path(instance: UploadedFile, filename):
if instance.temporary:
return "uploaded_files/temp/" + filename
return "uploaded_files/" + filename
# views.py, file_out has been generated
with open(file_out, 'rb') as local_file:
from django.core.files import File
form = UploadForm(dict(), {'document': File(local_file)})
print(form.errors)
if form.is_valid():
file = form.save(commit=False)
# ... set additional fields
file.save()
form.save_m2m()
return file
Now this is not the only thing I've tried. First I've gone with setting the FileField directly, but that resulted in the save() to fail, while the mimetype field is set. Because the original file sits outside the media folder, and thus a suspicious file action is triggered.
Also, the form gives some feedback about the "upload", through the form.errors.
Depending on my approach, either the save() fails as mentioned above -- meaning the "uploading" does not actually copy the file in the media folder -- or the form returns the error that no file was transmitted, and tells to check the form protocol.
Now my theory is, that I would have to go and initialize my own instance of InMemoryUploadedFile, but I could not figure out how to do that myself, and no documentation was available on the internet.
It feels like I'm taking the wrong approach from the get go. How would one do this properly?

Do you have get_upload_path defined? If not, that would explain the errors you're getting.
From what I can see you're on the right track. If you don't need a dynamic path for your uploads, if you just want them in media/uploads, you can pass in a string value for upload_to (from the Django docs):
# file will be uploaded to MEDIA_ROOT/uploads
document = models.FileField(upload_to='uploads/')

First of all, thanks to Franey for pointing me at storage documentation which lead me to contentfile documentation.
The ContentFile actually solves the problem, because it basically is the self-instantiated version of InMemoryUploadedFile that I was looking for. It's a django File that is not stored on disk.
Here's the full solution:
# views.py, file_out has been generated
with open(file_out, 'rb') as local_file:
from django.core.files.base import ContentFile
# we need to provide a name. Otherwise the Storage.save
# method reveives a None-parameter and breaks.
form = UploadForm(dict(), {'document': ContentFile(local_file.read(), name=name)})
if form.is_valid():
file = form.save(commit=False)
# ... set additional fields
file.save()
form.save_m2m()
return file