I am using JSON, re and requests to get the number of followers from an instagram page:
import json
import re
import requests
PROFILE = 'espn'
response = requests.get('https://www.instagram.com/' + PROFILE)
json_match = re.search(r'window\._sharedData = (.*);</script>', response.text)
profile_json = json.loads(json_match.group(1))['entry_data']['ProfilePage'][0]['graphql']['user']
print(profile_json['edge_followed_by']['count'])
I am wondering how I can then add this to my django project to use that number to display on my website, updating each time someone vists the page to reflect the dynamic number. I had a look around but I dont really understand where you would add this code. I would image it would be added in the views section to then be rendered in the html page but not sure what the format would be.
The view for the page I would like it to be used for example is
View.py:
def index(request):
post = Post.objects.all()[0]
recent_posts = Post.objects.all()[1:4]
latest_posts = Post.objects.all()[:5]
data = {"post":post, "recent_posts":recent_posts, "latest_posts":latest_posts}
return render(request, 'index.html', data)
Also as an extra question - how would the process differ if I was using a Generic Class-Based view vs a function view.
Thank you very much.
Just get the data as you are - but instead of printing the number, put it in a variable and include it in your template context:
def index(request):
post = Post.objects.all()[0]
recent_posts = Post.objects.all()[1:4]
latest_posts = Post.objects.all()[:5]
PROFILE = 'espn'
response = requests.get('https://www.instagram.com/' + PROFILE)
json_match = re.search(r'window\._sharedData = (.*);</script>', response.text)
profile_json = json.loads(json_match.group(1))['entry_data']['ProfilePage'][0]['graphql']['user']
page_visitors = profile_json['edge_followed_by']['count']
data = {"post":post, "recent_posts":recent_posts, "latest_posts":latest posts, "page_visitors":page_visitors}
return render(request, 'index.html', data)
In a class based view you would add this code to the get_context_data method.
Note though that this approach really isn't great - you appear to be scraping content from the Instagram webpage, which means you are heavily reliant on how that works, and your code could unexpectedly break at basically any time. I do not know if Instagram has an API that allows access to this information - but they likely do, and I would advise to make your requests to that instead.
You can give the JSON profile_json as context of your function view and just access each element of the JSON in the Django Template.
import json
import re
import requests
def index(request):
PROFILE = 'espn'
response = requests.get('https://www.instagram.com/' + PROFILE)
json_match = re.search(r'window\._sharedData = (.*);</script>', response.text)
profile_json = json.loads(json_match.group(1))['entry_data']['ProfilePage'][0]['graphql']['user']
context = {'profile': profile_json}
return render(request, 'index.html', context)
Now, assuming that profile_json has a structure of {'followers': 2000}, you can access it in the Django template as follows: {{ profile.followers }} since the JSON was given under the name profile to the context of the view.
If you want to use Generic Class-Based Views, I suggest to use a TemplateView.
from django.views.generic import TemplateView
class ProfileTemplateView(TemplateView):
template_name = 'index.html'
def get_context_data(self, **kwargs):
context = super().get_context_data(**kwargs)
PROFILE = 'espn'
response = requests.get('https://www.instagram.com/' + PROFILE)
json_match = re.search(r'window\._sharedData = (.*);</script>', response.text)
profile_json = json.loads(json_match.group(1))['entry_data']['ProfilePage'][0]['graphql']['user']
context['profile'] = profile_json
return context
Next, you can access the profile context the same way as before in your Django template.
Related
I am trying to create a "convert to pdf" button for a current page (service detail, which is a dynamic page) I tried to work with ReportLab (as Django documentation suggests, but I just can't make it work, and ReportLab documentation is not saying a word about such possibilities)
for now I can create a pdf file out of this view: (edit, got back to the code from django documentation for clarity)
views.py
#login_required
def service_detail(request, pk):
service = get_object_or_404(Service, pk=pk)
return render(request, 'detail.html', {'service':service, 'pk':pk})
#login_required
def render_to_pdf(request):
# Create the HttpResponse object with the appropriate PDF headers.
response = HttpResponse(content_type='application/pdf')
response['Content-Disposition'] = 'attachment; filename="service.pdf"''
# Create the PDF object, using the response object as its "file."
p = canvas.Canvas(response)
# Draw things on the PDF. Here's where the PDF generation happens.
p.drawString("Hello world.")
# Close the PDF object cleanly, and we're done.
p.showPage()
p.save()
return response
urls.py
url(r'^service/(?P<pk>\d+)/$', views.service_detail, name='service_detail'), #Services details
url(r'^render_to_pdf/', views.render_to_pdf, name='render_to_pdf'),
The template of the service detail includes dynamic elements, like:
Your location: {{ service.user_location }} <br><br>
Anyone knows what can I do, or what other technology I can use to create such PDF?
I am mostly working with Python, Django, HTML and CSS
content_type need to be application/pdf not service/pdf, also try to give pdf file name without spaces for example service.pdf, from docs, also you forgot to put attachment in Content-Disposition
try to write pdf to response like so and use BytesIO:
from io import BytesIO
#login_required
def render_to_pdf(request):
response = HttpResponse(content_type='application/pdf')
response['Content-Disposition'] = 'inline; filename="service.pdf'
buffer = BytesIO()
p = canvas.Canvas(buffer)
# Start writing the PDF here
p.drawString(100, 100, 'Hello world.')
# End writing
p.showPage()
p.save()
pdf = buffer.getvalue()
buffer.close()
response.write(pdf)
return response
You might want to consider the package django-wkhtmltopdf for generating PDF's in Django.
Using this, you can layout your PDF as an HTML-page (with CSS markup), like any other view, but it will render it as an PDF-file. Note that you will have to install the wkhtmltopdf binary to get it working.
By adding the ?as=html prefix in the url, you can view your page in the browser while developing.
The django-wkhtmltopdf package uses class based views. Here an example:
views.py
from django.utils.text import slugify
from wkhtmltopdf.views import PDFTemplateView
class Portfolio(PDFTemplateView):
template_name = 'frontend/portfolio.html'
def get_filename(self):
return slugify('portfolio ' + self.request.user.first_name + ' ' + self.request.user.last_name) + '.pdf'
#method_decorator(login_required)
def dispatch(self, *args, **kwargs):
return super().dispatch(*args, **kwargs)
urls.py
from django.urls import path
from . import views
app_name = 'frontend'
urlpatterns = [
path(
'<slug:group_slug>/<int:school_year_pk>/portfolio/',
views.Portfolio.as_view(),
name='portfolio'
),
...
Im trying to learn how to use APIs in Django and I want to return some simple data from one within a web page in html. The API is Mozscape and when running it in a terminal one can obtain the score of a website out of 100 like so:
from mozscape import Mozscape
client = Mozscape(
'api_user_id',
'secret_key')
url = 'http://www.google.com'
get_da = client.urlMetrics(url, cols=68719476736)
print(get_da)
and this prints the following
{u'pda': 100}
the '100' is all I want there. I want a user to enter a url into a form in a page in Django and to get that score int back so I have made the following models, views and form
class DomainAuthority(models.Model):
url = models.URLField(max_length=300)
def __str__(self):
return self.url
class Meta:
verbose_name = 'Domain'
verbose_name_plural = 'Domains'
views.py
def DomainAuthorityView(request):
form = DomainAuthorityForm(request.POST or None)
if form.is_valid():
new_domain = form.save(commit=False)
new_domain.save()
return render(request, 'domain_authority.html', {'form': form})
forms.py
class DomainAuthorityForm(forms.ModelForm):
class Meta:
model = DomainAuthority
fields = ['url']
so I have the form working and when a url is entered in the html form its saved in the admin backend but what I dont know how to do now is how to pass that url into the Mozscape API so that I can get the score back.
I took a look at the Django rest framework and installed it and followed some quick tutorial videos on Youtube and other places but in those examples they were taking saved Django objects such as blog posts and returning them as JSON data which is not what I want to do.
I tried import the API into the views file and then adding this line into then view
get_da = client.urlMetrics(new_domain, cols=68719476736)
but then I get this error after entering the url into the form in the web page
<DomainAuthority: https://www.google.com> is not JSON serializable
what do I need to do here to pass the user inputted urls to the API and return the correct response in a web page?
thanks
EDIT - UPDATED VIEW as of 19th Aug
def DomainAuthorityView(request):
form = DomainAuthorityForm(request.POST or None)
if form.is_valid():
new_domain = form.save(commit=False)
new_domain.save()
response = requests.get(new_domain.url, cols=68719476736)
#response = requests.get(client.urlMetrics(new_domain.url, cols=68719476736))
json_response = response.json()
score = json_response['pda']
return render(request, 'domain_authority_checked.html', {'score': score})
else:
return render(request, 'domain_authority.html', {'form': form})
so now it should redirect after successful form completion with url and the url is passed to the API to get the score and the redirects to 'domain_authority_checked.html' with just this
{{ score }}
so I have two outcomes here, if I pass in 'client.urlMetrics' into response I can load the 'domain_authority.html' but after a url his input into the form an error page returns with this
InvalidSchema at /domainauthority/
No connection adapters were found for '{'pda': 100}'
if I dont pass 'client.urlMetrics' to response then Django doesn't know what 'cols' is and returns this
TypeError at /domainauthority/
request() got an unexpected keyword argument 'cols'
I suggest this approach:
import requests
response = requests.get(url)
json_response = response.json()
score = json_response['key_name']
You can then simply render a template, add the score to the template context and display the value using {{ }}.
You may also want to define a rest_framework serializer (otherwise you don't need django_rest_framework) and verify the response against this serializer in order to ensure that you've received what you expected:
serializer = MySerializer(data=json_response)
if serializer.is_valid():
score = json_response['key_name']
You can use:
return HttpResponse(json.dumps(data), content_type='application/json')
instead of render the form. Only you need to import json in the header and create an empty dict named "data".
https://ultimatedjango.com/blog/how-to-consume-rest-apis-with-django-python-reques/
I am following this guide and trying to change API to work with semrush.com site.
Problem is that unlike in tutorial they dont return data in JSON format but like this:
domain;http://domain.com/fullUrl
I installed CSV render for rest framework and added this to settings:
REST_FRAMEWORK = {
# specifying the renderers
'DEFAULT_RENDERER_CLASSES': (
'rest_framework_csv.renderers.CSVRenderer',
),
}
Here is serialaizer:
class SemrushSerializer(serializers.ModelSerializer):
class Meta:
model = SavedSemrush
And views.py:
def save_semrush(request):
if request.method == "POST":
form = SubmitSemrush(request.POST)
if form.is_valid():
keyword = form.cleaned_data['keyword']
r = requests.get('http://api.semrush.com/?type=phrase_organic&key=' + settings.SEMRUSH_KEY + '&display_limit=10&export_columns=Dn,Ur&phrase=' + keyword + '&database=us')
serializer = SemrushSerializer(data=r)
if serializer.is_valid():
serializer.save()
else:
form = SubmitSemrush()
Since after submitting form it just reload(without adding data to orm) I can only guess that either serializer is wrong for this data type or I managed to mess up simple code in views :)
Any ideas ?
cheers
You're trying to instantiate the SemrushSerializer with a Request object.
You need to pass it JSON data.
r = requests.get(
'http://api.semrush.com/?type=phrase_organic&key={key}&'
'display_limit=10&export_columns=Dn,Ur&phrase={keyword}&'
'database=us'.format(
key=settings.SEMRUSH_KEY,
keyword=keyword
)
json = r.json()
serializer = SemrushSerializer(data=json)
I have created django form.py file :
from django import forms
from models import ResumeModel
class ResumeForm(forms.ModelForm):
username = forms.CharField(max_length = 200)
first_name = forms.CharField(max_length = 200)
last_name = forms.CharField(max_length = 200)
fathers_name = forms.CharField(max_length = 200)
email = forms.EmailField(required=True, label='Your e-mail address')
message = forms.CharField(widget=forms.Textarea)
class Meta():
model = ResumeModel
fields =('username','first_name','last_name','fathers_name','email','message')
views.py :
def save(request):
if 'submit' in request.POST:
form = ResumeForm(request.POST)
if form.is_valid():
form.save()
form = ResumeForm()
return render(request, '/success/', {'form': form})
else:
form = ResumeForm()
args = {}
args.update(csrf(request))
args['form'] =form
return render_to_response('create.html',args)
urls.py:
urlpatterns = patterns('',
url(r'^admin/', include(admin.site.urls)),
url(r'^save/', 'Resume.views.save'),
url(r'^success/$', TemplateView.as_view(template_name='success.html')),
)
Now all is working fine. problem is that as i submit form it moves to success page and when i go back to previous page that is /save/ page it holds all the values in fields that i submitted. How to clear the fields if i press back button to go back to that form page having url as "/save/ "
This is about the browsers behaviour. Browser shows u the page in its cache instead of calling a new page.
U may workaround this by using javascript.Unfortunately browser also dont run the javascript codes again.
But u may try :
$(window).bind("pageshow", function() {
// Clean form values
});
by JQuery
This is a browser implementation detail. You might be able to hack your way around it with JavaScript or break the cache etc. The subject is discussed in this stackoverflow question.
However a nicer UI approach might be to use AJAX to POST a serialised version of the form. If you successfully validate and process that form in your view (inserting a new row into your resume model table etc), you could then send a JSON response which might invoke some JavaScript to clear the form fields (maybe using .reset()). The user is then free to submit another form easily if that is the requirement.
Also note it is recommended that you use a HttpResponseRedirect after a successful POST (you have a render() response at the moment). This stops the users re-submitting the form again and potentially duplicating rows in your databases etc.
Hello I am working on a simple form. The form submits fine but if I refresh the page it resubmits the data. Seems the form is holding the data after submit and I assume since after the submit the request method is post. Question is what is the best way after the submit to clear the form in Django. After the submit the form variables should not be holding the values anymore. Thanks
def testimonials(request, template_name="testimonials.html"):
reviews = Reviews.objects.all()
if request.method == 'POST':
form = forms.ReviewsForm(data = request.POST)
# create a new item
if form.is_valid(): # All validation rules pass
# Process the data in form.cleaned_data
# ...
if form.is_valid():
nameIn = form.cleaned_data['name']
reviewIn = form.cleaned_data['review']
newReview = Reviews(name = nameIn, review = reviewIn)
newReview.save()
return render_to_response(template_name, locals(), context_instance=RequestContext(request))
else:
# This the the first page load, display a blank form
form = forms.ReviewsForm()
return render_to_response(template_name, locals(), context_instance=RequestContext(request))
Typically, you would issue a redirect after processing a form/POST request (this is common web development practice to avoid the resubmission issue you mentioned). So instead of a render_to_response, you might issue a HttpResponseRedirect like so:
if form.is_valid():
# Process form as desired here
# Simple example; use reverse() to avoid hard-coding URLs
return HttpResponseRedirect('/success/')
Check out the using a form in view for a skeleton of how forms are typically processed.
use reverse instead of render to response
if form.is_valid():
nameIn = form.cleaned_data['name']
reviewIn = form.cleaned_data['review']
newReview = Reviews(name = nameIn, review = reviewIn)
newReview.save()
return HttpResponseRedirect(reverse('app_name:url'))
You could also use the 'redirect' shortcut:
from django.shortcuts import redirect
...
return redirect(newReview)
This is assuming you have get_absolute_url defined in your Review Model.
See the docs for more info.