I'm trying to get the view count for a list of videos from a channel. I've written a function and when I try to run it with just 'video_id', 'title' & 'published date' I get the output. However, when I want the view count or anything from statistics part of API, then it is giving a Key Error.
Here's the code:
def get_video_details(youtube, video_ids):
all_video_stats = []
for i in range(0, len(video_ids), 50):
request = youtube.videos().list(
part='snippet,statistics',
id = ','.join(video_ids[i:i+50]))
response = request.execute()
for video in response['items']:
video_stats = dict(
Video_id = video['id'],
Title = video['snippet']['title'],
Published_date = video['snippet']['publishedAt'],
Views = video['statistics']['viewCount'])
all_video_stats.append(video_stats)
return all_video_stats
get_video_details(youtube, video_ids)
And this is the error message:
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_18748/3337790216.py in <module>
----> 1 get_video_details(youtube, video_ids)
~\AppData\Local\Temp/ipykernel_18748/1715852978.py in get_video_details(youtube, video_ids)
14 Title = video['snippet']['title'],
15 Published_date = video['snippet']['publishedAt'],
---> 16 Views = video['statistics']['viewCount'])
17
18 all_video_stats.append(video_stats)
KeyError: 'viewCount'
I was referencing this Youtube video to write my code.
Thanks in advance.
I got it.
I had to use .get() to avoid the KeyErrors. It will return None for KeyErrors.
Replaced this code to get the solution.
Views = video['statistics'].get('viewCount')
Related
I analyzed the data in the precedent and tried to use topic modeling. Here is a
syntax I am using:
According to the error, I think it means that the string should go in when
joining, but the tuple was found. I don't know how to fix this part.
class FacebookAccessException(Exception): pass
def get_profile(request, token=None):
...
response = json.loads(urllib_response)
if 'error' in response:
raise FacebookAccessException(response['error']['message'])
access_token = response['access_token'][-1]
return access_token
#Join the review
word_list = ",".join([",".join(i) for i in sexualhomicide['tokens']])
word_list = word_list.split(",")
This is Error
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
C:\Users\Public\Documents\ESTsoft\CreatorTemp\ipykernel_13792\3474859476.py in <module>
1 #Join the review
----> 2 word_list = ",".join([",".join(i) for i in sexualhomicide['tokens']])
3 word_list = word_list.split(",")
C:\Users\Public\Documents\ESTsoft\CreatorTemp\ipykernel_13792\3474859476.py in <listcomp>(.0)
1 #Join the review
----> 2 word_list = ",".join([",".join(i) for i in sexualhomicide['tokens']])
3 word_list = word_list.split(",")
TypeError: sequence item 0: expected str instance, tuple found
This is print of 'sexual homicide'
print(sexualhomicide['cleaned_text'])
print("="*30)
print(twitter.pos(sexualhomicide['cleaned_text'][0],Counter('word')))
I can't upload the results of this syntax. Error occurs because it is classified as spam during the upload process.
I am trying to do scraping from Google News with pygooglenews.
I am trying to scrape more than 100 articles at a time (as google sets limit at 100) by changing the target dates using for loop. The below is what I have so far but I keep getting error message
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-84-4ada7169ebe7> in <module>
----> 1 df = pd.DataFrame(get_news('Banana'))
2 writer = pd.ExcelWriter('My Result.xlsx', engine='xlsxwriter')
3 df.to_excel(writer, sheet_name='Results', index=False)
4 writer.save()
<ipython-input-79-c5266f97934d> in get_titles(search)
9
10 for date in date_list[:-1]:
---> 11 search = gn.search(search, from_=date, to_=date_list[date_list.index(date)])
12 newsitem = search['entries']
13
~\AppData\Roaming\Python\Python37\site-packages\pygooglenews\__init__.py in search(self, query, helper, when, from_, to_, proxies, scraping_bee)
140 if from_ and not when:
141 from_ = self.__from_to_helper(validate=from_)
--> 142 query += ' after:' + from_
143
144 if to_ and not when:
TypeError: unsupported operand type(s) for +=: 'dict' and 'str'
import pandas as pd
from pygooglenews import GoogleNews
import datetime
gn = GoogleNews()
def get_news(search):
stories = []
start_date = datetime.date(2021,3,1)
end_date = datetime.date(2021,3,5)
delta = datetime.timedelta(days=1)
date_list = pd.date_range(start_date, end_date).tolist()
for date in date_list[:-1]:
search = gn.search(search, from_=date.strftime('%Y-%m-%d'), to_=(date+delta).strftime('%Y-%m-%d'))
newsitem = search['entries']
for item in newsitem:
story = {
'title':item.title,
'link':item.link,
'published':item.published
}
stories.append(story)
return stories
df = pd.DataFrame(get_news('Banana'))
Thank you in advance.
It looks like you are correctly passing in a string into get_news() which is then passed on as the first argument (search) into gn.search().
However, you're reassigning search to the result of gn.search() in the line:
search = gn.search(search, from_=date.strftime('%Y-%m-%d'), to_=(date+delta).strftime('%Y-%m-%d'))
# ^^^^^^
# gets overwritten with the result of gn.search()
In the next iteration this reassigned search is passed into gn.search() which it doesn't like.
If you look at the code in pygooglenews, it looks like gn.search() is returning a dict which would explain the error.
To fix this, simply use a different variable, e.g.:
result = gn.search(search, from_=date.strftime('%Y-%m-%d'), to_=(date+delta).strftime('%Y-%m-%d'))
newsitem = result['entries']
I know that pygooglenews has a limit of 100 articles, so you must to make a loop in which it will scrape every day separately.
I'm trying to get a complete list of instances available in a region. The code will iterate over a number of pages but stops short with an error:
Traceback (most recent call last):
File "list_available_instance_offerings.py", line 29, in <module>
marker = page_iterator['Marker']
TypeError: 'PageIterator' object is not subscriptable
How can I iterate over all of the pages without erroring prematurely?
Here's my script:
import sys
import boto3
ec2 = boto3.client("ec2")
marker = None
while True:
paginator = ec2.get_paginator('describe_instance_type_offerings')
page_iterator = paginator.paginate(
LocationType='availability-zone',Filters=[{'Name': 'location', 'Values':['us-east-1a']}],
PaginationConfig={
'PageSize': 50,
'StartingToken': marker})
for page in page_iterator:
offerings = page['InstanceTypeOfferings']
for offer in offerings:
print(offer['InstanceType'])
try:
marker = page_iterator['Marker']
except KeyError:
sys.exit()
There is no such property as Marker. I belive that you are after NextToken from page. In this case, it should be:
try:
marker = page['NextToken']
except KeyError:
sys.exit()
When using a boto3 paginator, you don't need to worry about the Marker. The purpose of the paginator is to manage that for you.
client = boto3.client('route53')
paginator = client.get_paginator('list_health_checks')
response_iterator = paginator.paginate(
PaginationConfig={
'PageSize': 10
}
)
for page in response_iterator:
for healthcheck in page['HealthChecks']:
print(healthcheck["Id"])
If you have 87 health checks, this will list 8 pages of 10 and 1 page of 7.
You can also use MaxItems if you want to limit the output. For example, if you have MaxItems=80 you will get 8 pages of 10.
This code is pre-made in a Zapier forum to pull failed responses from another piece of software called iAuditor. When I plug in the code and update the API token and webhook URL this error pops up:
Traceback (most recent call last):
SyntaxError: invalid syntax (usercode.py, line 42)
Here is the code:
[code]
import json
import requests
auth_header = {'Authorization': 'a4fca847d3f203bd7306ef5d1857ba67a2b3d66aa455e06fac0ad0be87b9d226'}
webhook_url = 'https://hooks.zapier.com/hooks/catch/3950922/efka9n/'
api_url = 'https://api.safetyculture.io/audits/'
audit_id = input['audit_id']
audit_doc = requests.get(api_url + audit_id, headers=auth_header).json()
failed_items = []
audit_author = audit_doc['audit_data']['authorship']['author']
conducted_on = audit_doc['audit_data']['date_completed']
conducted_on = conducted_on[:conducted_on.index('T')]
audit_title = audit_doc['template_data']['metadata']['name']
for item in audit_doc['items']:
if item.get('responses') and item['responses'].get('failed') == True:
label = item.get('label')
if label is None:
label = 'no_label'
responses = item['responses']
response_label = responses['selected'][0]['label']
notes = responses.get('text')
if notes is None:
notes = ''
failed_items.append({'label': label,
'response_label': response_label,
'conducted_on': conducted_on,
'notes': notes,
'author': audit_author
})
for item in failed_items:
r = requests.post(webhook_url, data = item)
return response.json()
[/code]
This looks like an error from the platform. It looks like Zapier uses a script called usercode.py to bootstrap launching your script and the error seems to be coming from that part.
I am using postman to test if my django app call is working as expected but it is not.
I get the following error:
ValueError at /mobile/emulate_create</title>
Exception Value: No JSON object could be decoded
I have the following code:
#csrf_exempt
def emulate_create(request):
args = json.loads(request.body, object_hook=utils._datetime_decoder)
resourceId, count, hours = args['resourceId'], args['count'], args['hours']
now = datetime.now()
remainder = now.minute % 15
timeFrom = now + timedelta(minutes=remainder)
timeTo = timeFrom + timedelta(hours=hours)
reservation = ReservationProspect(byUser=request.user, forUser=request.user, resource=get_object_or_404(Resource, uuid=resourceId), modality=get_object_or_404(Modality, name="online"), timeFrom=timeFrom, timeTo=timeTo, count=count)
return HttpResponse(json.dumps[reservation.toDict()])
Does anyone see what might be happening here?