I have setup Django REST framework endpoint that allows me to upload a csv file.
The serializers.py looks like this:
from rest_framework import serializers
class UploadSerializer(serializers.Serializer):
file_uploaded = serializers.FileField()
class Meta:
fields = ['file_uploaded']
In my views.py file, I'm trying to read data from uploaded csv like this:
class UploadViewSet(viewsets.ViewSet):
serializer_class = UploadSerializer
def create(self, request):
file_uploaded = request.FILES.get('file_uploaded')
with open(file_uploaded, mode ='r')as file:
csvFile = csv.reader(file)
for lines in csvFile:
print(lines)
I'm getting the following error:
... line 37, in create
with open(file_uploaded, mode ='r') as file:
TypeError: expected str, bytes or os.PathLike object, not InMemoryUploadedFile
I have checked type() of file_uploaded and It is <class 'django.core.files.uploadedfile.InMemoryUploadedFile'>
How can I read this file into dictionary or dataframe so I can extract the data I need from it?
When you do request.FILES.get('file_uploaded') it returns back an InMemoryUploadedFile which is a wrapper around a file object. You can access the file object using the file attribute.
file_uploaded # <InMemoryUploadedFile: xxx (xxx/xxx)>
file_object = file_uploaded.file
This file_object can then be opened.
an InMemoryFileObject can be used pretty much in the same way as an open file., so you don't need to open it.
def create(self, request):
file_upload = request.FILES.get("file_uploaded")
csvFile = csv.reader(file_upload)
for line in csvFile:
print(line)
This has some good information on file handling in django.
https://docs.djangoproject.com/en/4.1/topics/http/file-uploads/#handling-uploaded-files-with-a-model
Related
Trying to make endpoint that can read uploaded .sav (SPSS) file and create model with data from it. For getting data from it I'm using pyreadstat library. But now when I'm trying to run it I have an error expected str, bytes or os.PathLike object, not InMemoryUploadedFile
How I can change this code so pyreadstat can correctly read the given file?
from rest_framework import generics, status
import pandas as pd
import pyreadstat
from rest_framework.response import Response
from .models import Research, Data
from .serializers import FileUploadSerializer, SaveFileSerializer
class UploadFileView(generics.CreateAPIView):
serializer_class = FileUploadSerializer
def post(self, request, *args, **kwargs):
serializer = self.get_serializer(data=request.data)
serializer.is_valid(raise_exception=True)
file = serializer.validated_data['file']
df, meta = pyreadstat.read_sav(file, user_missing=True)
json_df = df.to_json(orient='table')
rsch = Research.objects.get(pk=1)
Data.objects.create(research=rsch, data={})
Change this:
pyreadstat.read_sav(file, user_missing=True)
To this:
pyreadstat.read_sav(file.getvalue(), user_missing=True)
Hope it works.
Django uploaded files are mostly file like, and they only proxy certain methods. See the FileProxyMixin source for more details on specific methods, if you are curious.
To get the uploaded contents, use the read() method:
file = serializer.validated_data['file']
contents = file.read()
df, meta = pyreadstat.read_sav(contents, user_missing=True)
In this case you are getting an InMemoryUploadFile but this could change if Django decides to write to disk due to the upload being too large.
Documentation on django uploaded file objects:
https://docs.djangoproject.com/en/4.0/ref/files/uploads/#.
This is my view.. In the blank space after open I want to read a file but I dont to what to write to make it dynamic..
def home(request):
image_data = open('', 'rb').read()
return HttpResponse(image_data, mimetype='application/pdf')
This my model here file is uploaded .. Its is either pdf or .doc .. I want to read the file uploaded
from django.db import models
class CV(models.Model):
file = models.FileField(upload_to='media')
try this
def home(request):
file = request.FILES['yourHtmlFileFieldName']
image_date = open(file,'rb').read()
I'm sending a excel file from Angular to Django. I want to read the file using Pandas and perform some operations in the file, but I'm not sure how to do it.
class fileupload(APIView) :
def post(self, request):
f = request.FILES
print(f)
When I print, it shows below,
<MultiValueDict: {'excelfile': [<InMemoryUploadedFile: New_Excel.xlsx (application/vnd.openxmlformats-officedocument.spreadsheetml.sheet)>]}>
Here, I want to save this file to some location and use pandas to perform operations or if possible, directly would need to read the file using pandas. I'm new to Django and Pandas so if anything is wrong, please help.. Thanks in advance
from django.core.files.storage import default_storage
from django.core.files.base import ContentFile
file_objs = request.data.getlist('files')
for file_obj in file_objs:
path = default_storage.save(settings.MEDIA_ROOT, ContentFile(file_obj.read()))
print("images path are",path)
class fileupload(APIView) :
def post(self, request):
f = request.data.getlist("excelfile")
print(f) # list of elements
Now loop the f then store one by one
I'm trying to export data from queryset to different file formats and I'm getting an issue with json format.
This is my little piece of code :
from django.core import serializers
def export_categories_json(request):
with open("categories.json", "w") as out:
data = serializers.serialize("json", Category.objects.all().values_list('id', 'name'))
out.write(data)
Then, I set in my template a button which call this function and should download the json file.
But I have this issue :
Traceback:
File "/home/val/.pyenv/versions/Publication3.6.2/lib/python3.6/site-packages/django/core/handlers/exception.py" in inner
41. response = get_response(request)
File "/home/val/.pyenv/versions/Publication3.6.2/lib/python3.6/site-packages/django/core/handlers/base.py" in _get_response
187. response = self.process_exception_by_middleware(e, request)
File "/home/val/.pyenv/versions/Publication3.6.2/lib/python3.6/site-packages/django/core/handlers/base.py" in _get_response
185. response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/home/val/Bureau/Projets/Publication/publication/src/web/views/exports.py" in export_categories_json
276. data = serializers.serialize("json", Category.objects.all().values_list('id', 'name'))
File "/home/val/.pyenv/versions/Publication3.6.2/lib/python3.6/site-packages/django/core/serializers/__init__.py" in serialize
129. s.serialize(queryset, **options)
File "/home/val/.pyenv/versions/Publication3.6.2/lib/python3.6/site-packages/django/core/serializers/base.py" in serialize
84. concrete_model = obj._meta.concrete_model
Exception Type: AttributeError at /Category/json
Exception Value: 'tuple' object has no attribute '_meta'
Something is wrong in my code ?
I can't send a queryset to json file ?
EDIT :
I wrote this and it works, but json is shown in my template. How I can export this one to a json file ?
def export_categories_json(request):
from django.http import JsonResponse
data = list(Category.objects.values())
return JsonResponse(data, safe=False)
Please do not do serialization yourself: Django has some builtin serializiation functionality, and you can subclass a serializer to change its behavior.
Your view also does not return a HTTP response, but this is a contract it should satisfy (well it should return a HTTP response, or it should raise some error).
Instead you write content to a file, but writing to files is typically not a good idea (unless you expect the filesize to be huge, in which case you can use a temporary file). By using files, you create race conditions, a hacker might also aim to "inject" a different filename and thus overwrting certain files to run arbitrary code, or changing credentials, and finally it is possible that the server has certain permissions making it impossible to write to a file (the permissions of the directory).
Django allows you to see a HTTP response a s a stream object, to which content can be written, like:
from django.http import HttpResponse
from django.core import serializers
def export_categories_json(request):
response = new HttpResponse(content_type='application/json')
response['Content-Disposition'] = 'attachment;filename=categories.json'
serializers.serialize(
'json',
Category.objects.all(),
fields=['name'],
stream=response
)
return response
Django's serialization is for models, but you are using .values_list() which returns plain Python lists.
In your specific case, you can simply use the built-in json module:
import json
def export_categories_json(request):
with open("categories.json", "w") as out:
values = list(Category.objects.all().values_list('id', 'name'))
json.dump(values, out)
django serialize except the querset object ,
Category.objects.all().values_list('id', 'name')
this will return the tuple, you can replace with below
data = list(Category.objects.all().values('id', 'name'))
with open("file.json", "w+") as file:
file.write(data)
If you want to write the json representation of something to a file, use json.dump:
import json
from django.shortcuts import redirect
def export_to_json(request):
with open('export.json', 'w') as f:
json.dump(list(Category.objects.all().values_list('id', 'name')), f)
return redirect('/')
I have this serializer which I am trying to test:
class AttachmentSerializer(CustomModelSerializer):
order = serializers.PrimaryKeyRelatedField()
file = FileField()
class Meta:
model = Attachment
fields = (
'id',
'order',
'name',
'file_type',
'file',
'created_at',
)
My test simply checks whether it is valid or not:
def test_serializer_create(self):
self.data = {
'order': self.order.pk,
'name': 'sample_name',
'file_type': 'image',
'created_at': datetime.now(),
'file': open(self.image_path, 'rb').read()
}
serializer = AttachmentSerializer(data=self.data)
self.assertTrue(serializer.is_valid())
And I am constantly getting this error:
{'file': ['No file was submitted. Check the encoding type on the form.']}
I tried to create a file in a number of different ways, such as with StringIO/BytesIO, File and etc. to no avail.
What might be wrong?
from django.core.files.uploadedfile import SimpleUploadedFile
content = SimpleUploadedFile("file.txt", "filecontentstring")
data = {'content': content}
try smth like that , because if you check FileField serializer's code - it expects UploadedFile that should have name and size:
def to_internal_value(self, data):
try:
# `UploadedFile` objects should have name and size attributes.
file_name = data.name
file_size = data.size
except AttributeError:
self.fail('invalid')
and StringIO or opened file objects doesn't have size attribute.
I ran into a similar problem. It turned out that Django REST Framework FileField cannot be used with a JSON API parser. DRF documentation states that "Most parsers, such as e.g. JSON don't support file uploads."
Your question does not show which parser you configured, but given how common JSON is, it may be the culprit. You can set a different parser either across the board, or for a specific API view, as described here.
Once the parser problem was fixed, I made the test work with a Django File, but perhaps other approaches could work too:
from django.core.files import File
def test_create(self):
...
data = {
'file': File(open(path_to_test_file, 'rb')),
}
...
The thing is that you pass an opened file to the APIClient / APIRequestFactory, not to the view itself. The Django request will wraps the file to an UploadedFile which is what you should use.