Develop Reference

Django REST framework - parse uploaded csv file

I have setup Django REST framework endpoint that allows me to upload a csv file.
The serializers.py looks like this:
from rest_framework import serializers
class UploadSerializer(serializers.Serializer):
file_uploaded = serializers.FileField()
class Meta:
fields = ['file_uploaded']
In my views.py file, I'm trying to read data from uploaded csv like this:
class UploadViewSet(viewsets.ViewSet):
serializer_class = UploadSerializer
def create(self, request):
file_uploaded = request.FILES.get('file_uploaded')
with open(file_uploaded, mode ='r')as file:
csvFile = csv.reader(file)
for lines in csvFile:
print(lines)
I'm getting the following error:
... line 37, in create
with open(file_uploaded, mode ='r') as file:
TypeError: expected str, bytes or os.PathLike object, not InMemoryUploadedFile
I have checked type() of file_uploaded and It is <class 'django.core.files.uploadedfile.InMemoryUploadedFile'>
How can I read this file into dictionary or dataframe so I can extract the data I need from it?

When you do request.FILES.get('file_uploaded') it returns back an InMemoryUploadedFile which is a wrapper around a file object. You can access the file object using the file attribute.
file_uploaded # <InMemoryUploadedFile: xxx (xxx/xxx)>
file_object = file_uploaded.file
This file_object can then be opened.

an InMemoryFileObject can be used pretty much in the same way as an open file., so you don't need to open it.
def create(self, request):
file_upload = request.FILES.get("file_uploaded")
csvFile = csv.reader(file_upload)
for line in csvFile:
print(line)
This has some good information on file handling in django.
https://docs.djangoproject.com/en/4.1/topics/http/file-uploads/#handling-uploaded-files-with-a-model

Json data not saving in django database Admin

I am building a BlogApp and I am trying to access user location through geoplugin BUT it was sending through json so i made a view to send json data to database BUT json data is not saving in Database (Admin).
When i click on link then it is showing :-
'NoneType' object has no attribute 'json'
BUT then i use json.loads then it is showing :-
the JSON object must be str, bytes or bytearray, not NoneType
models.py
class Location(models.Model):
user = models.ForeignKey(User,on_delete=models.CASCADE)
city = models.JSONField()
views.py
def jsonLocation(request):
r = request.POST.get('http://www.geoplugin.net/javascript.gp')
# data = json.loads(r)
data = r.json()
for x in data:
title = x["geoplugin_city"]
user = request.user
addin = Location.objects.create(city=title,user=request.user)
addin.save()
return HttpResponse("Successfully submitted!")
I have seen many answers but did't find any solution. And it is still not saving in database.
Any help would be Appreciated.

According to your comments on the post, this should be the error:
You are trying to make a POST-Request to the URL you provided us, doing the following:
def jsonLocation(request):
r = request.POST.get('http://www.geoplugin.net/javascript.gp')
This will not work, request, is an object that describes the INCOMING request of the user loading YOUR webpage. This has a POST-property, from where you can get, e.g. form data.
To make a POST-request and save the data, you will need a Python package for it, I always use Requests, I think it is also the most popular.
Then you would do something along the lines of this:
import requests
r = requests.get('http://www.geoplugin.net/javascript.gp')
data = r.json()
# Continue your script
So, basically, your attempt was good, I think you just mixed up request and requests

How to read or save a file from MultiValueDict in Django

I'm sending a excel file from Angular to Django. I want to read the file using Pandas and perform some operations in the file, but I'm not sure how to do it.
class fileupload(APIView) :
def post(self, request):
f = request.FILES
print(f)
When I print, it shows below,
<MultiValueDict: {'excelfile': [<InMemoryUploadedFile: New_Excel.xlsx (application/vnd.openxmlformats-officedocument.spreadsheetml.sheet)>]}>
Here, I want to save this file to some location and use pandas to perform operations or if possible, directly would need to read the file using pandas. I'm new to Django and Pandas so if anything is wrong, please help.. Thanks in advance

from django.core.files.storage import default_storage
from django.core.files.base import ContentFile
file_objs = request.data.getlist('files')
for file_obj in file_objs:
path = default_storage.save(settings.MEDIA_ROOT, ContentFile(file_obj.read()))
print("images path are",path)

class fileupload(APIView) :
def post(self, request):
f = request.data.getlist("excelfile")
print(f) # list of elements
Now loop the f then store one by one

Django issue with json : 'tuple' object has not attribute '_meta'

I'm trying to export data from queryset to different file formats and I'm getting an issue with json format.
This is my little piece of code :
from django.core import serializers
def export_categories_json(request):
with open("categories.json", "w") as out:
data = serializers.serialize("json", Category.objects.all().values_list('id', 'name'))
out.write(data)
Then, I set in my template a button which call this function and should download the json file.
But I have this issue :
Traceback:
File "/home/val/.pyenv/versions/Publication3.6.2/lib/python3.6/site-packages/django/core/handlers/exception.py" in inner
41. response = get_response(request)
File "/home/val/.pyenv/versions/Publication3.6.2/lib/python3.6/site-packages/django/core/handlers/base.py" in _get_response
187. response = self.process_exception_by_middleware(e, request)
File "/home/val/.pyenv/versions/Publication3.6.2/lib/python3.6/site-packages/django/core/handlers/base.py" in _get_response
185. response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/home/val/Bureau/Projets/Publication/publication/src/web/views/exports.py" in export_categories_json
276. data = serializers.serialize("json", Category.objects.all().values_list('id', 'name'))
File "/home/val/.pyenv/versions/Publication3.6.2/lib/python3.6/site-packages/django/core/serializers/__init__.py" in serialize
129. s.serialize(queryset, **options)
File "/home/val/.pyenv/versions/Publication3.6.2/lib/python3.6/site-packages/django/core/serializers/base.py" in serialize
84. concrete_model = obj._meta.concrete_model
Exception Type: AttributeError at /Category/json
Exception Value: 'tuple' object has no attribute '_meta'
Something is wrong in my code ?
I can't send a queryset to json file ?
EDIT :
I wrote this and it works, but json is shown in my template. How I can export this one to a json file ?
def export_categories_json(request):
from django.http import JsonResponse
data = list(Category.objects.values())
return JsonResponse(data, safe=False)

Please do not do serialization yourself: Django has some builtin serializiation functionality, and you can subclass a serializer to change its behavior.
Your view also does not return a HTTP response, but this is a contract it should satisfy (well it should return a HTTP response, or it should raise some error).
Instead you write content to a file, but writing to files is typically not a good idea (unless you expect the filesize to be huge, in which case you can use a temporary file). By using files, you create race conditions, a hacker might also aim to "inject" a different filename and thus overwrting certain files to run arbitrary code, or changing credentials, and finally it is possible that the server has certain permissions making it impossible to write to a file (the permissions of the directory).
Django allows you to see a HTTP response a s a stream object, to which content can be written, like:
from django.http import HttpResponse
from django.core import serializers
def export_categories_json(request):
response = new HttpResponse(content_type='application/json')
response['Content-Disposition'] = 'attachment;filename=categories.json'
serializers.serialize(
'json',
Category.objects.all(),
fields=['name'],
stream=response
)
return response

Django's serialization is for models, but you are using .values_list() which returns plain Python lists.
In your specific case, you can simply use the built-in json module:
import json
def export_categories_json(request):
with open("categories.json", "w") as out:
values = list(Category.objects.all().values_list('id', 'name'))
json.dump(values, out)

django serialize except the querset object ,
Category.objects.all().values_list('id', 'name')
this will return the tuple, you can replace with below
data = list(Category.objects.all().values('id', 'name'))
with open("file.json", "w+") as file:
file.write(data)

If you want to write the json representation of something to a file, use json.dump:
import json
from django.shortcuts import redirect
def export_to_json(request):
with open('export.json', 'w') as f:
json.dump(list(Category.objects.all().values_list('id', 'name')), f)
return redirect('/')

How to test a FileField in Django REST Framework

I have this serializer which I am trying to test:
class AttachmentSerializer(CustomModelSerializer):
order = serializers.PrimaryKeyRelatedField()
file = FileField()
class Meta:
model = Attachment
fields = (
'id',
'order',
'name',
'file_type',
'file',
'created_at',
)
My test simply checks whether it is valid or not:
def test_serializer_create(self):
self.data = {
'order': self.order.pk,
'name': 'sample_name',
'file_type': 'image',
'created_at': datetime.now(),
'file': open(self.image_path, 'rb').read()
}
serializer = AttachmentSerializer(data=self.data)
self.assertTrue(serializer.is_valid())
And I am constantly getting this error:
{'file': ['No file was submitted. Check the encoding type on the form.']}
I tried to create a file in a number of different ways, such as with StringIO/BytesIO, File and etc. to no avail.
What might be wrong?

from django.core.files.uploadedfile import SimpleUploadedFile
content = SimpleUploadedFile("file.txt", "filecontentstring")
data = {'content': content}
try smth like that , because if you check FileField serializer's code - it expects UploadedFile that should have name and size:
def to_internal_value(self, data):
try:
# `UploadedFile` objects should have name and size attributes.
file_name = data.name
file_size = data.size
except AttributeError:
self.fail('invalid')
and StringIO or opened file objects doesn't have size attribute.

I ran into a similar problem. It turned out that Django REST Framework FileField cannot be used with a JSON API parser. DRF documentation states that "Most parsers, such as e.g. JSON don't support file uploads."
Your question does not show which parser you configured, but given how common JSON is, it may be the culprit. You can set a different parser either across the board, or for a specific API view, as described here.
Once the parser problem was fixed, I made the test work with a Django File, but perhaps other approaches could work too:
from django.core.files import File
def test_create(self):
...
data = {
'file': File(open(path_to_test_file, 'rb')),
}
...

The thing is that you pass an opened file to the APIClient / APIRequestFactory, not to the view itself. The Django request will wraps the file to an UploadedFile which is what you should use.