Develop Reference

Django: Use JSON and requests in my django app

I am using JSON, re and requests to get the number of followers from an instagram page:
import json
import re
import requests
PROFILE = 'espn'
response = requests.get('https://www.instagram.com/' + PROFILE)
json_match = re.search(r'window\._sharedData = (.*);</script>', response.text)
profile_json = json.loads(json_match.group(1))['entry_data']['ProfilePage'][0]['graphql']['user']
print(profile_json['edge_followed_by']['count'])
I am wondering how I can then add this to my django project to use that number to display on my website, updating each time someone vists the page to reflect the dynamic number. I had a look around but I dont really understand where you would add this code. I would image it would be added in the views section to then be rendered in the html page but not sure what the format would be.
The view for the page I would like it to be used for example is
View.py:
def index(request):
post = Post.objects.all()[0]
recent_posts = Post.objects.all()[1:4]
latest_posts = Post.objects.all()[:5]
data = {"post":post, "recent_posts":recent_posts, "latest_posts":latest_posts}
return render(request, 'index.html', data)
Also as an extra question - how would the process differ if I was using a Generic Class-Based view vs a function view.
Thank you very much.

Just get the data as you are - but instead of printing the number, put it in a variable and include it in your template context:
def index(request):
post = Post.objects.all()[0]
recent_posts = Post.objects.all()[1:4]
latest_posts = Post.objects.all()[:5]
PROFILE = 'espn'
response = requests.get('https://www.instagram.com/' + PROFILE)
json_match = re.search(r'window\._sharedData = (.*);</script>', response.text)
profile_json = json.loads(json_match.group(1))['entry_data']['ProfilePage'][0]['graphql']['user']
page_visitors = profile_json['edge_followed_by']['count']
data = {"post":post, "recent_posts":recent_posts, "latest_posts":latest posts, "page_visitors":page_visitors}
return render(request, 'index.html', data)
In a class based view you would add this code to the get_context_data method.
Note though that this approach really isn't great - you appear to be scraping content from the Instagram webpage, which means you are heavily reliant on how that works, and your code could unexpectedly break at basically any time. I do not know if Instagram has an API that allows access to this information - but they likely do, and I would advise to make your requests to that instead.

You can give the JSON profile_json as context of your function view and just access each element of the JSON in the Django Template.
import json
import re
import requests
def index(request):
PROFILE = 'espn'
response = requests.get('https://www.instagram.com/' + PROFILE)
json_match = re.search(r'window\._sharedData = (.*);</script>', response.text)
profile_json = json.loads(json_match.group(1))['entry_data']['ProfilePage'][0]['graphql']['user']
context = {'profile': profile_json}
return render(request, 'index.html', context)
Now, assuming that profile_json has a structure of {'followers': 2000}, you can access it in the Django template as follows: {{ profile.followers }} since the JSON was given under the name profile to the context of the view.
If you want to use Generic Class-Based Views, I suggest to use a TemplateView.
from django.views.generic import TemplateView
class ProfileTemplateView(TemplateView):
template_name = 'index.html'
def get_context_data(self, **kwargs):
context = super().get_context_data(**kwargs)
PROFILE = 'espn'
response = requests.get('https://www.instagram.com/' + PROFILE)
json_match = re.search(r'window\._sharedData = (.*);</script>', response.text)
profile_json = json.loads(json_match.group(1))['entry_data']['ProfilePage'][0]['graphql']['user']
context['profile'] = profile_json
return context
Next, you can access the profile context the same way as before in your Django template.

Django sorl: not enough values to unpack (expected 2, got 1)

I am trying to have a form to generate thumbnails from images that will be uploaded
I will be using sorl for the thumb generation and I am following the following documentation:
Django multiple file upload: https://docs.djangoproject.com/en/3.0/topics/http/file-uploads/
Sorl Low level API: https://sorl-thumbnail.readthedocs.io/en/latest/examples.html
When I try to generate the thumbnail I get the error of
not enough values to unpack (expected 2, got 1)
I dont understand what I am doing wrong, in summary I upload the image and this get saved in my root directory, then im trying to create the thumb
Also is there a way to void saving this original image in the root? I am planning to send both Image and thumb to google cloud storage
My forms.py:
from django import forms
class FileFieldForm(forms.Form):
file_field = forms.FileField(widget=forms.ClearableFileInput(attrs={'multiple': True}))
My html file: upload.html
<html>
<head></head>
<body>
<h3>Read File Content</h3>
<form enctype="multipart/form-data" action="" method="post">
{% csrf_token %}
{{ form }}
<input type="submit" value="Save">
</form>
</body>
</html>
My views.py looks like:
from sorl.thumbnail import ImageField, get_thumbnail
from .forms import FileFieldForm
class FileFieldView(FormView):
form_class = FileFieldForm
template_name = 'app_workflow/upload.html' # Replace with your template.
success_url = '/photo' # Replace with your URL or reverse().
def post(self, request, *args, **kwargs):
form_class = self.get_form_class()
form = self.get_form(form_class)
files = request.FILES.getlist('file_field')
if form.is_valid():
for f in files:
with open(f.name, 'wb+') as destination:
for chunk in f.chunks():
destination.write(chunk)
im = get_thumbnail(f.name, '100x100', crop='center', quality=99)
return self.form_valid(form)
else:
return self.form_invalid(form)

As you have said in the question, you do not want to store in root and generate thumbnail. Then I would suggest something like this:
from PIL import Image
class FileFieldView(FormView):
form_class = FileFieldForm
template_name = 'app_workflow/upload.html' # Replace with your template.
success_url = '/photo' # Replace with your URL or reverse().
def form_valid(self, *args, **kwargs):
img_size = (100, 100)
files = self.request.FILES.getlist('file_field')
for f in files:
im = Image.open(f)
im.thumbnail(img_size)
# your thumbnail image is in memory now
# you can now store it in your model and use django-storages to upload it to gcloud
return super().form_valid(*args, **kwargs)
Here I am not storing images and directly loading it in the PIL.Image module to generate thumbnails. You can use django-storages to upload data from FileField to gcloud.
Storing in root:
Then you can change the code like this:
for f in files:
for chunk in f.chunks():
destination.write(chunk)
im = Image.open(f)
im.thumbnail(img_size)
im.save('thumb_{}'.format(f.name))

Error uploading image using postman in django rest framework

I'm trying to create an endpoint to upload images(using postman) to a specific folder using django rest framework. This is my settings for the folder,
MEDIA_URL = '/media/'
MEDIA_ROOT = os.path.join(BASE_DIR, 'media')
This is my model,
class UserMedia(models.Model):
user = models.OneToOneField(User, related_name='medias', on_delete=models.CASCADE, )
profile_image_web = models.FileField(null=True)
profile_image_android = models.FileField(null=True)
profile_image_ios = models.FileField(null=True)
thumbnail = models.FileField(null=True)
This is the Serializer,
class UserMediaSerializer(serializers.ModelSerializer):
class Meta:
model = models.UserMedia
fields = (
'profile_image_web', 'profile_image_ios', 'profile_image_android', 'thumbnail',
)
This is the api,
class CreateUpdateUserMedia(views.APIView):
parser_classes = (MultiPartParser, FormParser)
def post(self, request, **kwargs):
serializer = UserMediaSerializer(request.data)
if serializer.is_valid():
serializer.save()
return Response(serializer.data, status=status.HTTP_201_CREATED)
else:
return Response(serializer.errors, status=status.HTTP_400_BAD_REQUEST)
Now when I try to upload ONE image corresponding to one of the fields using POSTMAN, this is the error I get.
'Cannot call `.is_valid()` as no `data=` keyword argument was '
AssertionError: Cannot call `.is_valid()` as no `data=` keyword argument was passed when instantiating the serializer instance.
Which is perfectly understandable, but I dont know how to fix it.
Here are my questions,
How do I correctly upload images using django rest framework.
I don't expect this api to be called with 4 images together, but 4 times using one image at a time, how do I pass the relevant name and
modify the serializer accordingly.
How do I provide a subpath to the root media directory.
Finally I want the serializer to display the full image url, how do I do that?

You are using serializer = UserMediaSerializer(request.data) but you should call it by serializer = UserMediaSerializer(data=request.data)
For uploading images in Django rest framework you should either upload images on S3 and pass s3 url in DRF API or use base64 field in serializer and send base64 encoded value of image in API
import uuid
import base64
import imghdr
from django.utils.translation import ugettext_lazy as _
from django.core.files.base import ContentFile
from rest_framework import serializers
ALLOWED_IMAGE_TYPES = (
"jpeg",
"jpg",
"png",
"gif"
)
class Base64ImageField(serializers.ImageField):
"""
A django-rest-framework field for handling image-uploads through raw post data.
It uses base64 for en-/decoding the contents of the file.
"""
def to_internal_value(self, base64_data):
# Check if this is a base64 string
if not base64_data:
return None
if isinstance(base64_data, basestring):
# Try to decode the file. Return validation error if it fails.
try:
decoded_file = base64.b64decode(base64_data)
except TypeError:
raise serializers.ValidationError(_("Please upload a valid image."))
# Generate file name:
file_name = str(uuid.uuid4())[:12] # 12 characters are more than enough.
# Get the file name extension:
file_extension = self.get_file_extension(file_name, decoded_file)
if file_extension not in ALLOWED_IMAGE_TYPES:
raise serializers.ValidationError(_("The type of the image couldn't been determined."))
complete_file_name = file_name + "." + file_extension
data = ContentFile(decoded_file, name=complete_file_name)
return super(Base64ImageField, self).to_internal_value(data)
raise serializers.ValidationError('This is not an base64 string')
def to_representation(self, value):
# Return url including domain name.
return value.name
def get_file_extension(self, filename, decoded_file):
extension = imghdr.what(filename, decoded_file)
extension = "jpg" if extension == "jpeg" else extension
return extension
Updated
You should use ImageField (not FileField) for images.
You can use above field directly in serializer like any other field.
class UserMediaSerializer(serializers.ModelSerializer):
profile_image_web = Base64ImageField(required=False)
class Meta:
model = models.UserMedia
fields = ('profile_image_web', 'profile_image_ios', 'profile_image_android', 'thumbnail')

How to return generated file download with Django REST Framework?

I need to return generated file download as a Django REST Framework response. I tried the following:
def retrieve(self, request, *args, **kwargs):
template = webodt.ODFTemplate('test.odt')
queryset = Pupils.objects.get(id=kwargs['pk'])
serializer = StudentSerializer(queryset)
context = dict(serializer.data)
document = template.render(Context(context))
doc = converter().convert(document, format='doc')
res = HttpResponse(
FileWrapper(doc),
content_type='application/msword'
)
res['Content-Disposition'] = u'attachment; filename="%s_%s.zip"' % (context[u'surname'], context[u'name'])
return res
But it returns a msword document as json.
How do I make it start downloading as file instead?

Here's an example of returning a file download directly from DRF. The trick is to use a custom renderer so you can return a Response directly from the view:
from django.http import FileResponse
from rest_framework import viewsets, renderers
from rest_framework.decorators import action
class PassthroughRenderer(renderers.BaseRenderer):
"""
Return data as-is. View should supply a Response.
"""
media_type = ''
format = ''
def render(self, data, accepted_media_type=None, renderer_context=None):
return data
class ExampleViewSet(viewsets.ReadOnlyModelViewSet):
queryset = Example.objects.all()
#action(methods=['get'], detail=True, renderer_classes=(PassthroughRenderer,))
def download(self, *args, **kwargs):
instance = self.get_object()
# get an open file handle (I'm just using a file attached to the model for this example):
file_handle = instance.file.open()
# send file
response = FileResponse(file_handle, content_type='whatever')
response['Content-Length'] = instance.file.size
response['Content-Disposition'] = 'attachment; filename="%s"' % instance.file.name
return response
Note I'm using a custom endpoint download instead of the default endpoint retrieve, because that makes it easy to override the renderer just for this endpoint instead of for the whole viewset -- and it tends to make sense for list and detail to return regular JSON anyway. If you wanted to selectively return a file download you could add more logic to the custom renderer.

This may work for you:
file_path = file_url
FilePointer = open(file_path,"r")
response = HttpResponse(FilePointer,content_type='application/msword')
response['Content-Disposition'] = 'attachment; filename=NameOfFile'
return response.
For FrontEnd code refer this

I am using DRF and i found a view code to download file, which would be like
from rest_framework import generics
from django.http import HttpResponse
from wsgiref.util import FileWrapper
class FileDownloadListAPIView(generics.ListAPIView):
def get(self, request, id, format=None):
queryset = Example.objects.get(id=id)
file_handle = queryset.file.path
document = open(file_handle, 'rb')
response = HttpResponse(FileWrapper(document), content_type='application/msword')
response['Content-Disposition'] = 'attachment; filename="%s"' % queryset.file.name
return response
and url.py will be
path('download/<int:id>/',FileDownloadListAPIView.as_view())
I am using React.js in frontend and i get a response like
handleDownload(id, filename) {
fetch(`http://127.0.0.1:8000/example/download/${id}/`).then(
response => {
response.blob().then(blob => {
let url = window.URL.createObjectURL(blob);
let a = document.createElement("a");
console.log(url);
a.href = url;
a.download = filename;
a.click();
});
});
}
and after i got successful in downloading a file which also opens correctly and i hope this gonna work. Thanks

For me, using Python 3.6, Django 3.0, and DRF 3.10, The problem came from using the wrong type of response. I needed to use a django.http.HttpResponse, as seen below:
from django.http import HttpResponse
...
with open('file.csv', 'r') as file:
response = HttpResponse(file, content_type='text/csv')
response['Content-Disposition'] = 'attachment; filename=file.csv'
return response

I solved my problem by saving file in media folder and sending of the link of it to front-end.
#permission_classes((permissions.IsAdminUser,))
class StudentDocxViewSet(mixins.RetrieveModelMixin, viewsets.GenericViewSet):
def retrieve(self, request, *args, **kwargs):
template = webodt.ODFTemplate('test.odt')
queryset = Pupils.objects.get(id=kwargs['pk'])
serializer = StudentSerializer(queryset)
context = dict(serializer.data)
document = template.render(Context(context))
doc = converter().convert(document, format='doc')
p = u'docs/cards/%s/%s_%s.doc' % (datetime.now().date(), context[u'surname'], context[u'name'])
path = default_storage.save(p, doc)
return response.Response(u'/media/' + path)
And handled this like in my front-end (AngularJS SPA)
$http(req).success(function (url) {
console.log(url);
window.location = url;
})

In models.py
class Attachment(models.Model):
file = models.FileField(upload_to=attachment_directory_path, blank=True, null=True)
...
#property
def filename(self):
return self.file.name.split('/')[-1:][0]
in views.py
import mimetypes
from django.http import FileResponse
class AttachmentViewSet(ModelViewSet):
...
#action(methods=['GET'], detail=True)
def download(self, request, **kwargs):
att = self.get_object()
file_handle = att.file.open()
mimetype, _ = mimetypes.guess_type(att.file.path)
response = FileResponse(file_handle, content_type=mimetype)
response['Content-Length'] = att.file.size
response['Content-Disposition'] = "attachment; filename={}".format(att.filename)
return response
and in frontend, I used axios for download files. api is axios client.
export function fileDownload(url, filename){
return api.get(url, { responseType: 'blob' })
.then((response)=>{
const url = window.URL.createObjectURL(new Blob([response.data]));
const link = document.createElement('a');
link.href = url;
link.setAttribute('download', filename);
document.body.appendChild(link);
link.click();
})
}
hope that it helps

Using django-downloadview this can be done like so:
from rest_framework.decorators import action
from django_downloadview import ObjectDownloadView
class DocumentViewSet(viewsets.ReadOnlyModelViewSet):
#action(detail=True)
def download(self, request, pk):
return ObjectDownloadView.as_view(
model=, # your model here
)(request, pk=pk)
The viewset can then be registered via DRF routers.

TypeError uploading image file to Amazon S3 in Django using BOTO Library

I am a total beginner to programming and Django so I'd appreciate help that beginner can get his head round!
I was following a tutorial to show how to upload images to an Amazon S3 account with the Boto library but I think it is for an older version of Django (I'm on 1.1.2 and Python 2.65) and something has changed. I get an error:
Exception Type: TypeError
Exception Value: 'InMemoryUploadedFile' object is unsubscriptable
My code is:
Models.py:
from django.db import models
from django.contrib.auth.models import User
from django import forms
from datetime import datetime
class PhotoUrl(models.Model):
url = models.CharField(max_length=128)
uploaded = models.DateTimeField()
def save(self):
self.uploaded = datetime.now()
models.Model.save(self)
views.py:
import mimetypes
from django.http import HttpResponseRedirect
from django.shortcuts import render_to_response
from django.core.urlresolvers import reverse
from django import forms
from django.conf import settings
from boto.s3.connection import S3Connection
from boto.s3.key import Key
def awsdemo(request):
def store_in_s3(filename, content):
conn = S3Connection(settings.AWS_ACCESS_KEY_ID, settings.AWS_SECRET_ACCESS_KEY)
b = conn.create_bucket('almacmillan-hark')
mime = mimetypes.guess_type(filename)[0]
k = Key(b)
k.key = filename
k.set_metadata("Content-Type", mime)
k.set_contents_from_strong(content)
k.set_acl("public-read")
photos = PhotoUrl.objects.all().order_by("-uploaded")
if not request.method == "POST":
f = UploadForm()
return render_to_response('awsdemo.html', {'form':f, 'photos':photos})
f = UploadForm(request.POST, request.FILES)
if not f.is_valid():
return render_to_response('awsdemo.html', {'form':f, 'photos':photos})
file = request.FILES['file']
filename = file.name
content = file['content']
store_in_s3(filename, content)
p = PhotoUrl(url="http://almacmillan-hark.s3.amazonaws.com/" + filename)
p.save()
photos = PhotoUrl.objects.all().order_by("-uploaded")
return render_to_response('awsdemo.html', {'form':f, 'photos':photos})
urls.py:
(r'^awsdemo/$', 'harkproject.s3app.views.awsdemo'),
awsdemo.html:
<div class="form">
<strong>{{form.file.label}}</strong>
<form method="POST" action ="." enctype="multipart/form-data">
{{form.file}}<br/>
<input type="submit" value="Upload">
</form>
</div>
I'd really appreciate help. I hope I have provided enough code.
Kind regards
AL

I think your problem is this line:
content = file['content']
From the Django docs:
Each value in FILES is an UploadedFile object containing the following attributes:
read(num_bytes=None) -- Read a number of bytes from the file.
name -- The name of the uploaded file.
size -- The size, in bytes, of the uploaded file.
chunks(chunk_size=None) -- A generator that yields sequential chunks of data.
Try this instead:
content = file.read()

Have you tried Django Storages? That way you only have to specify a storage backend (s3boto in this case) either as the default storage backend, or supply it as an argument to a FileField or ImageField class.