Develop Reference

Django: Use JSON and requests in my django app

I am using JSON, re and requests to get the number of followers from an instagram page:
import json
import re
import requests
PROFILE = 'espn'
response = requests.get('https://www.instagram.com/' + PROFILE)
json_match = re.search(r'window\._sharedData = (.*);</script>', response.text)
profile_json = json.loads(json_match.group(1))['entry_data']['ProfilePage'][0]['graphql']['user']
print(profile_json['edge_followed_by']['count'])
I am wondering how I can then add this to my django project to use that number to display on my website, updating each time someone vists the page to reflect the dynamic number. I had a look around but I dont really understand where you would add this code. I would image it would be added in the views section to then be rendered in the html page but not sure what the format would be.
The view for the page I would like it to be used for example is
View.py:
def index(request):
post = Post.objects.all()[0]
recent_posts = Post.objects.all()[1:4]
latest_posts = Post.objects.all()[:5]
data = {"post":post, "recent_posts":recent_posts, "latest_posts":latest_posts}
return render(request, 'index.html', data)
Also as an extra question - how would the process differ if I was using a Generic Class-Based view vs a function view.
Thank you very much.

Just get the data as you are - but instead of printing the number, put it in a variable and include it in your template context:
def index(request):
post = Post.objects.all()[0]
recent_posts = Post.objects.all()[1:4]
latest_posts = Post.objects.all()[:5]
PROFILE = 'espn'
response = requests.get('https://www.instagram.com/' + PROFILE)
json_match = re.search(r'window\._sharedData = (.*);</script>', response.text)
profile_json = json.loads(json_match.group(1))['entry_data']['ProfilePage'][0]['graphql']['user']
page_visitors = profile_json['edge_followed_by']['count']
data = {"post":post, "recent_posts":recent_posts, "latest_posts":latest posts, "page_visitors":page_visitors}
return render(request, 'index.html', data)
In a class based view you would add this code to the get_context_data method.
Note though that this approach really isn't great - you appear to be scraping content from the Instagram webpage, which means you are heavily reliant on how that works, and your code could unexpectedly break at basically any time. I do not know if Instagram has an API that allows access to this information - but they likely do, and I would advise to make your requests to that instead.

You can give the JSON profile_json as context of your function view and just access each element of the JSON in the Django Template.
import json
import re
import requests
def index(request):
PROFILE = 'espn'
response = requests.get('https://www.instagram.com/' + PROFILE)
json_match = re.search(r'window\._sharedData = (.*);</script>', response.text)
profile_json = json.loads(json_match.group(1))['entry_data']['ProfilePage'][0]['graphql']['user']
context = {'profile': profile_json}
return render(request, 'index.html', context)
Now, assuming that profile_json has a structure of {'followers': 2000}, you can access it in the Django template as follows: {{ profile.followers }} since the JSON was given under the name profile to the context of the view.
If you want to use Generic Class-Based Views, I suggest to use a TemplateView.
from django.views.generic import TemplateView
class ProfileTemplateView(TemplateView):
template_name = 'index.html'
def get_context_data(self, **kwargs):
context = super().get_context_data(**kwargs)
PROFILE = 'espn'
response = requests.get('https://www.instagram.com/' + PROFILE)
json_match = re.search(r'window\._sharedData = (.*);</script>', response.text)
profile_json = json.loads(json_match.group(1))['entry_data']['ProfilePage'][0]['graphql']['user']
context['profile'] = profile_json
return context
Next, you can access the profile context the same way as before in your Django template.

Django JsonRequest in generic view always returns a string

(using Django2.0)
This is my first time trying to collaborate with a frond-end developer and I am trying to serialize a Django model from a generic ListView. Even though I manage to send a JsonResponse with my objects as json, they are always a string:
"[{\"model\": \"questions.question\", \"pk\": 9535, \"fields\": {\"created\": \"2018-04-14T17:02:38.559Z\", \"modified\": \"2018-04-14T18:04:14.264Z\", \"question\": \"TEST\", \"category\": \"Rules\", \"event\": \"Beyonce\", \"answer\": \"aergaergaergaer\", \"verified\": true, \"verified_by\": [\"someotheruser\"], \"count\": 0, \"user_created\": [\"someuser\"]}}]"
the way the front-end developer solved this issue is by calling a JSON.parse(). (see: https://www.w3schools.com/js/js_json_parse.asp).
Is this the correct way to do it or should I return the objects without a string?
If I am wrong and there is a way to do this without the strings here is my view and url:
views.py:
from events.models import Event
from django.core import serializers
from django.http import JsonResponse
class EventView(LoginRequiredMixin, ListView):
login_url = '/accounts/login/'
model = Question
template_name = 'faq/faq.html'
def get_context_data(self, **kwargs):
context = super().get_context_data(**kwargs)
events_list = Event.objects.all()
context['events_list'] = events_list
return context
def get_queryset(self):
event = Event.objects.get(event=self.kwargs['event'])
queryset = Question.objects.filter(event=event)
return queryset
def get(self, request, *args, **kwargs):
queryset = self.get_queryset()
data = serializers.serialize("json", queryset, use_natural_foreign_keys=True)
return JsonResponse(data, status=200, safe=False)
urls.py
urlpatterns += [
path('<str:event>/', EventView.as_view(), name='event'),
]
What I also tried:
def EventRequest(request, **kwargs):
event = Event.objects.get(event=kwargs['event'])
queryset = Question.objects.filter(event=event)
data = serializers.serialize("json", queryset, use_natural_foreign_keys=True)
dump = json.dumps(data)
return HttpResponse(dump, content_type='application/json')
and:
def EventRequest(request, **kwargs):
event = Event.objects.get(event=kwargs['event'])
queryset = Question.objects.filter(event=event)
data = serializers.serialize("json", queryset, use_natural_foreign_keys=True)
return JsonResponse(data, status=200, safe=False)
Once again this could be absolutely correct and the front-end developer should just do a JSON.parse(). Please let me know, thanks!

This is normal. JSON is a string notation. It is just a format to represent JavaScript objects and arrays (Python dicts and lists) as a string.
In the frontend you'll have to use JSON.parse() to convert it into a JavaScript array (list) or object (dict).
This also holds true when you send JSON from frontend to backend. You use JSON.stringify() to covert the JS object to string. Then in the backend you convert that string to a Python object using json.loads().

Django Rest Framework : Filtering against Table Field value

I'm improving my Django Web App with Django Rest API part and I have a question according to filtering against table field value.
I have my serializer class like this :
class IndividuResearchSerializer(serializers.ModelSerializer) :
class Meta :
model = Individu
fields = [
'id',
'NumeroIdentification',
'Nom',
'Prenom',
'VilleNaissance',
]
My views.py file with this class :
class IndividuResearchAPIView(ListAPIView) :
permission_classes = (IsAuthenticated,)
authentication_classes = (JSONWebTokenAuthentication,)
serializer_class = IndividuResearchSerializer
def get_queryset(self):
queryset = Individu.objects.all()
NIU = self.request.query_params.get('NumeroIdentification')
queryset = queryset.filter(NumeroIdentification=NIU)
return queryset
And my pythonic file which let to simulate connexion from another software based to API Rest :
import requests
mytoken = "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ1c2VyX2lkIjoxLCJ1c2VybmFtZSI6IkFkbWluIiwiZXhwIjoxNTE5NzMxOTAxLCJlbWFpbCI6InZhbGVudGluQGRhdGFzeXN0ZW1zLmZyIiwib3JpZ19pYXQiOjE1MTk3MjgzMDF9.493NzJ4OUEzTKu5bZsZ9UafMwQZHz9pESMsYgfd0RLc"
url = 'http://localhost:8000/Api/Identification/search/'
NIU = "I-19312-00001-305563-2"
response = requests.get(url, NIU = NIU, headers={'Authorization': 'JWT {}'.format(mytoken)})
print(response.text)
I would like to enter a NIU value into my request in order to filter my table and return the object according to this NIU.
For example, in my database I have this object :
I would like to return this object thanks to my API but I don't know if my function get_queryset is well-writen and How I can write my API request.
Into my urls.py file, I have :
url(r'^search/$', IndividuResearchAPIView.as_view() , name="Research"),
So I am not making a filtering by URL.
I read these posts in order to get more element :
Django REST framework - filtering against query param
django rest framework filter
and obviously DRF doc : http://www.django-rest-framework.org/api-guide/filtering/#filtering-against-the-current-user

You need to use this url to filter: http://localhost:8000/Api/Identification/search/?NumeroIdentification=NUA_value. With requests library try to pass it with params argument: response = requests.get(url, params={'NumeroIdentification': NIU}, headers={'Authorization': 'JWT {}'.format(mytoken)}).

How to return data from an API in Django?

Im trying to learn how to use APIs in Django and I want to return some simple data from one within a web page in html. The API is Mozscape and when running it in a terminal one can obtain the score of a website out of 100 like so:
from mozscape import Mozscape
client = Mozscape(
'api_user_id',
'secret_key')
url = 'http://www.google.com'
get_da = client.urlMetrics(url, cols=68719476736)
print(get_da)
and this prints the following
{u'pda': 100}
the '100' is all I want there. I want a user to enter a url into a form in a page in Django and to get that score int back so I have made the following models, views and form
class DomainAuthority(models.Model):
url = models.URLField(max_length=300)
def __str__(self):
return self.url
class Meta:
verbose_name = 'Domain'
verbose_name_plural = 'Domains'
views.py
def DomainAuthorityView(request):
form = DomainAuthorityForm(request.POST or None)
if form.is_valid():
new_domain = form.save(commit=False)
new_domain.save()
return render(request, 'domain_authority.html', {'form': form})
forms.py
class DomainAuthorityForm(forms.ModelForm):
class Meta:
model = DomainAuthority
fields = ['url']
so I have the form working and when a url is entered in the html form its saved in the admin backend but what I dont know how to do now is how to pass that url into the Mozscape API so that I can get the score back.
I took a look at the Django rest framework and installed it and followed some quick tutorial videos on Youtube and other places but in those examples they were taking saved Django objects such as blog posts and returning them as JSON data which is not what I want to do.
I tried import the API into the views file and then adding this line into then view
get_da = client.urlMetrics(new_domain, cols=68719476736)
but then I get this error after entering the url into the form in the web page
<DomainAuthority: https://www.google.com> is not JSON serializable
what do I need to do here to pass the user inputted urls to the API and return the correct response in a web page?
thanks
EDIT - UPDATED VIEW as of 19th Aug
def DomainAuthorityView(request):
form = DomainAuthorityForm(request.POST or None)
if form.is_valid():
new_domain = form.save(commit=False)
new_domain.save()
response = requests.get(new_domain.url, cols=68719476736)
#response = requests.get(client.urlMetrics(new_domain.url, cols=68719476736))
json_response = response.json()
score = json_response['pda']
return render(request, 'domain_authority_checked.html', {'score': score})
else:
return render(request, 'domain_authority.html', {'form': form})
so now it should redirect after successful form completion with url and the url is passed to the API to get the score and the redirects to 'domain_authority_checked.html' with just this
{{ score }}
so I have two outcomes here, if I pass in 'client.urlMetrics' into response I can load the 'domain_authority.html' but after a url his input into the form an error page returns with this
InvalidSchema at /domainauthority/
No connection adapters were found for '{'pda': 100}'
if I dont pass 'client.urlMetrics' to response then Django doesn't know what 'cols' is and returns this
TypeError at /domainauthority/
request() got an unexpected keyword argument 'cols'

I suggest this approach:
import requests
response = requests.get(url)
json_response = response.json()
score = json_response['key_name']
You can then simply render a template, add the score to the template context and display the value using {{ }}.
You may also want to define a rest_framework serializer (otherwise you don't need django_rest_framework) and verify the response against this serializer in order to ensure that you've received what you expected:
serializer = MySerializer(data=json_response)
if serializer.is_valid():
score = json_response['key_name']

You can use:
return HttpResponse(json.dumps(data), content_type='application/json')
instead of render the form. Only you need to import json in the header and create an empty dict named "data".

Django REST Framework file upload causing an "Unsupported media type 'multipart/form-data'" error

I am newbie in Django and Django REST Framework. I have the following serializer class which I am using to upload a file along other information. But, while I run the API endpoint with uploaded file, the result is something like this:
HTTP 415 Unsupported Media Type
Allow: POST, OPTIONS
Content-Type: application/json
Vary: Accept
{
"detail": "Unsupported media type \"multipart/form-data; boundary=----WebKitFormBoundaryybZ07gjZAqvcsZw3\" in request."
}
I tried hard by googling to solve this issue, but cannot come out in a solution, so here is my serializer and API views.
Serializer:
class ExampleSerializer(serializers.Serializer):
example_id = serializers.IntegerField()
description = serializers.CharField(allow_blank=True)
example_file = serializers.FileField(allow_empty_file=True)
def create_requirement_line(self):
request = self.context['request']
requirement_line = ExampleService().example_method(
example_id=self.validated_data['example_id'],
description=self.validated_data['description'],
example_file=self.validated_data['example_file']
)
return requirement_line
View:
class RequirementLineAPIView(BaseCreateAPIView):
serializer_class = ExampleSerializer
parser_classes = (FormParser,)
def post(self, request, format=None,*args, **kwargs):
serializer = self.get_serializer(data=request.data)
if serializer.is_valid():
try:
example_variable = serializer.example_method()
return Response(example_variable, status=status.HTTP_200_OK)
except ValidationError as e:
return Response(e.message, status=status.HTTP_400_BAD_REQUEST)
return Response(serializer.errors, status=status.HTTP_400_BAD_REQUEST)

You should use the MultiPartParser instead of the FormParser if you're sending multipart/form-data.

Raised if there are no parsers that can handle the content type of the request data when accessing request.DATA or request.FILES.
check Django REST Framework2 documentation
import suitable parser
from rest_framework.parsers import MultiPartParser, FormParser, JSONParser
class SampleView(APIView):
parser_classes = (MultiPartParser,FormParser,JSONParser)

Try Using FileField parser
Using Parsers in django rest