Develop Reference

Getting KeyError: 'viewCount' for using Youtube API in Python

I'm trying to get the view count for a list of videos from a channel. I've written a function and when I try to run it with just 'video_id', 'title' & 'published date' I get the output. However, when I want the view count or anything from statistics part of API, then it is giving a Key Error.
Here's the code:
def get_video_details(youtube, video_ids):
all_video_stats = []
for i in range(0, len(video_ids), 50):
request = youtube.videos().list(
part='snippet,statistics',
id = ','.join(video_ids[i:i+50]))
response = request.execute()
for video in response['items']:
video_stats = dict(
Video_id = video['id'],
Title = video['snippet']['title'],
Published_date = video['snippet']['publishedAt'],
Views = video['statistics']['viewCount'])
all_video_stats.append(video_stats)
return all_video_stats
get_video_details(youtube, video_ids)
And this is the error message:
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_18748/3337790216.py in <module>
----> 1 get_video_details(youtube, video_ids)
~\AppData\Local\Temp/ipykernel_18748/1715852978.py in get_video_details(youtube, video_ids)
14 Title = video['snippet']['title'],
15 Published_date = video['snippet']['publishedAt'],
---> 16 Views = video['statistics']['viewCount'])
17
18 all_video_stats.append(video_stats)
KeyError: 'viewCount'
I was referencing this Youtube video to write my code.
Thanks in advance.

I got it.
I had to use .get() to avoid the KeyErrors. It will return None for KeyErrors.
Replaced this code to get the solution.
Views = video['statistics'].get('viewCount')

How to resolve AttributeError when trying to set None as default method argument

I know there are loads of answers to this question but I'm still not getting it...
Following is sa_reporting.py
class saReport():
def __init__(self, body, to_table, normalise=False, date_col=None):
global directory
self.body = body
self.to_table = to_table
self.normalise = normalise
self.date_col = date_col if date_col is not None else []
directory = os.path.join('/Users','python', self.to_table)
if not os.path.exists(directory):
os.mkdir(directory)
def download_files(self, ...):
...
def download_reports(self, ...):
...
def get_files(self):
...
def read_file(self, file):
....
def load_to_db(self, sort_by=None): # THIS IS WHAT I THINK IS CAUSING THE ERROR
sort_by = sort_by if sort_by is not None else [] # THIS IS WHAT I TRIED TO FIX IT
def normalise_data(self, data):
dim_data = []
for row in data:
if row not in dim_data:
dim_data.append(row)
return dim_data
def convert_dates(self, data):
if self.date_col:
for row in data:
for index in self.date_col:
if len(row[index]) > 10:
row[index] = row[index][:-5].replace('T',' ')
row[index] = datetime.datetime.strptime(row[index], "%Y-%m-%d %H:%M:%S")
else:
row[index] = datetime.datetime.strptime(row[index], "%Y-%m-%d").date()
return data
print(f'\nWriting data to {self.to_table} table...', end='')
files = self.get_files()
for file in files:
print('Processing ' + file.split("sa360/",1)[1] + '...', end='')
csv_file = self.read_file(file)
csv_headers = ', '.join(csv_file[0])
csv_data = csv_file[1:]
if self.normalise:
csv_data = self.normalise_data(csv_data)
csv_data = self.convert_dates(csv_data)
if sort_by:
csv_data = sorted(csv_data, key=itemgetter(sort_by))
#...some other code that inserts into a database...
Executing the following script (sa_main.py):
import sa_reporting
from sa_body import *
dim_campaign_test = sa_reporting.saReport(
body=dim_campaign_body,
to_table='dimsa360CampaignTest',
normalise=True,
date_col=[4,5]
)
dim_campaign_test_download = dim_campaign_test.download_reports()
dim_campaign_test_download.load_to_db(sort_by=0) # THIS IS WHERE THE ERROR OCCURS
Output and error message:
Downloading reports...
The report is still generating...restarting
The report is ready
Processing...
Downloading fragment 0 for report AAAnOdc9I_GnxAB0
Files successfully downloaded
Traceback (most recent call last):
File "sa_main.py", line 43, in <module>
dim_campaign_test_download.load_to_db(sort_by=0)
AttributeError: 'NoneType' object has no attribute 'load_to_db'
Why am I getting this error? And how can I fix it?
I just want to make None be the default argument and if a user specifies the sort_by parameter then None will be replaced with whatever the user specifies (which should be an integer index)

This code would seem to suggest that dim_campaign_test_download is being set to None. As in the below line, you set it to the result of dim_campaign_test.download_reports(), it is likely that no reports are being found.
dim_campaign_test_download = dim_campaign_test.download_reports()
You might want to instead do the following, as dim_campaign_test is the saReport Object on which you probably want to operate:
dim_campaign_test.load_to_db(sort_by=0)

HTTP is not subscriptable issue

I am trying to integrate the 2capcha text solver in one of my applications but i am coming accross the error: 'HTTPResponse' object is not subscriptable
code:
import time
from urllib import request, parse
class CaptchaTextSolver(object):
def solve_text(self, captcha_api_key, phrase_to_solve):
try:
postUrl = "https://2captcha.com/in.php"
result = ""
values = {'key': captcha_api_key, 'method': 'POST', 'textcaptcha': phrase_to_solve}
data = parse.urlencode(values).encode()
req = request.Request(postUrl, data=data)
res = request.urlopen(req)
print(res)
if res[0:3] == 'OK|':
captchaID = res[3:]
print("--> " + captchaID)
for i in range(0, 24):
req = request.Request("https://2captcha.com/res.php?key=" + captcha_api_key + "&action=get&id=" + captchaID)
answerResp = request.urlopen(req)
answerResponse = answerResp.read()
if len(answerResponse) < 3:
result = answerResponse
break
elif answerResponse[0:3] == "OK|":
result = answerResponse[3:]
break
elif answerResponse != "CAPCHA_NOT_READY":
pass
time.sleep(5)
return result
except Exception as e:
print("2CAPTCHA.COM ERROR: " , e)
My code is fairly simple enough, after gooogling the error for a bit i'm still no where near to knowing the issue, any help would be appreciated.

The following line
if res[0:3] == 'OK|':
Throws an error because you are trying to grab elements 0,1,2 of res, which is an HTTPResponse object. An HTTPResponse is not subscriptable like a list is. You probably meant to do this
response = res.read()
if response[0:3] == 'OK|':

How to troubleshoot and resolve "TypeError: 'NoneType' object is not subscriptable" in python

I have a piece of code that fetches data from the ticketmaster API using a function I've named get_event_info. The first revision of the code worked as desired, subsequently I modified the original function to make use of header based authentication instead of URL based. I also added a few lines to the function which were intended to validate the response status code. After making these changes the code began producing the following TypeError:
Traceback (most recent call last):
File "ticketmaster_only_w_headers.py", line 146, in <module>
for event in ticket_search["_embedded"]["events"].items():
TypeError: 'NoneType' object is not subscriptable
I've read quite a bit about this type of error but I'm still unable to determine why my code is producing it in this instance. I would really appreciate an explanation on why my code is producing this error and what troubleshooting methods I should have used to uncover the source error. I'm fairly comfortable with programming but certainly no expert so the simpler the language used the better.
(Function Definition)
def get_event_info(search):
if search in CACHE_DICTION:
d = CACHE_DICTION[search]
else:
api_url = '{0}events/'.format(api_url_base)
payload = {"keyword": search, "apikey": api_token,
"format": "json", "dmaId": "366", "size": 200, "radius": "2"}
response = requests.get(api_url, headers=headers, params=payload)
if response.status_code == 200:
d = json.loads(response.text)
CACHE_DICTION[search] = d
f = open(CACHE_FNAME, 'w')
f.write(json.dumps(CACHE_DICTION))
f.close()
else:
d = None
return d
(Code snippet that triggers the error)
ticket_search = get_event_info("")
for event in ticket_search["_embedded"]["events"]:
a = event["id"]
b = event["name"]
if "dateTime" in event["dates"]["start"]:
c = event["dates"]["start"]["dateTime"].replace(
"T", " ").replace("Z", "")
else:
c = "NONE"
if "end" in event["dates"] and "dateTime" in event["dates"]["end"]:
j = event["dates"]["end"]["dateTime"].replace(
"T", " ").replace("Z", "")
else:
j = "NONE"
(Code that creates, opens, and writes to the cache used in the above code)
CACHE_FNAME = "ticketmaster_cache.json"
try:
cache_file = open(CACHE_FNAME, "r")
cache_contents = cache_file.read()
CACHE_DICTION = json.loads(cache_contents)
cache_file.close()
except:
CACHE_DICTION = {}
The previous revision of the get_event_info function shown below which does not produce any TypeError:
def get_event_info(search, ticketmaster_key = ticketmaster_key):
if search in CACHE_DICTION:
d = CACHE_DICTION[search]
else:
data = requests.get("https://app.ticketmaster.com/discovery/v2/events",
params = {"keyword": search, "apikey": ticketmaster_key,
"format":"json", "dmaId": "366", "size": 200, "radius": "2"})
print(data.url)
d = json.loads(data.text)
CACHE_DICTION[search] = d
f = open(CACHE_FNAME, 'w')
f.write(json.dumps(CACHE_DICTION))
f.close()
return d
Traceback & Error message I see when I run the latest revision of the code:
Traceback (most recent call last):
File "ticketmaster_only_w_headers.py", line 146, in <module>
for event in ticket_search["_embedded"]["events"]:
TypeError: 'NoneType' object is not subscriptable

Whenever you have a function that can explicitly return None, you should always check the return value first:
def func(a):
if a == 1:
return list(range(10)) # could return a list
else:
return None # or it could return None
a = 10
f = func(a)
f[1]
# raises TypeError: NoneType is not subscriptable
# check for NoneType first
if f is not None:
print(f[1])
# otherwise, kick out different result
else:
print('Got "None" for f!')
# Got "None" for f!
Your ticket_search is returned as None, but because your for loop is trying to do a key-lookup, it's failing, because None doesn't support that operation. Your logic, following from the above, should look like:
if ticket_search is not None:
for event in ticket_search["_embedded"]["events"]:
a = event["id"]
else:
raise TypeError
# or do something else

Well, the interpreter is explicitly telling you that you are trying to evaluate something like a[i], where a is None (instead of the intended type, like a list or a dict). In your case, it is either ticket_search itself, or ticket_search["_embedded"].
In any case, if you can rerun your code at all, putting a print(ticket_search) under ticket_search = get_event_info("") should make everything clear.

Multiple Term search by following multiple users using Streaming API

I am trying to Retrieve multiple keyword term tweets by following specific group of users. Using the code below:
I have posted one more code before that regarding issues for value error:
I figure it out somehow but again I am stuck because of this traceback
import tweepy
from tweepy.error import TweepError
consumer_key=('ABC'),
consumer_secret=('ABC'),
access_key=('ABC'),
access_secret=('ABC')
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_key, access_secret)
api=tweepy.API(auth)
class CustomStreamListener(tweepy.StreamListener):
def on_status(self, status):
try:
print "%s\t%s\t%s\t%s" % (status.text,
status.author.screen_name,
status.created_at,
status.source,)
except Exception, e:
print error
#def filter(self, follow=None, track=None, async=False, locations=None):
#self.parameters = {}
#self.headers['Content-type'] = "application/x-www-form-urlencoded"
#if self.running:
#raise TweepError('Stream object already connected!')
#self.url = '/%i/statuses/filter.json?delimited=length' % STREAM_VERSION
def filter(self, follow=None, track=None, async=False, locations=None):
self.parameters = {}
self.headers['Content-type'] = "application/x-www-form-urlencoded"
if self.running:
raise TweepError('Stream object already connected!')
self.url = '/%i/statuses/filter.json?delimited=length' % STREAM_VERSION
if obey:
self.parameters['follow'] = ' '.join(map(str, obey))
if track:
self.parameters['track'] = ' '.join(map(str, track))
if locations and len(locations) > 0:
assert len(locations) % 4 == 0
self.parameters['locations'] = ' '.join('%.2f' % l for l in locations)
self.body = urllib.urlencode(self.parameters)
self.parameters['delimited'] = 'length'
self._start(async)
def on_error(self, status_code):
return True
streaming_api = tweepy.streaming.Stream(auth, CustomStreamListener(), timeout=60)
list_users = ['17006157','59145948','157009365','16686144','68044757','33338729']#Some ids
list_terms = ['narendra modi','robotics']#Some terms
streaming_api.filter(follow=[list_users])
streaming_api.filter(track=[list_terms])
I am getting a traceback:
Traceback (most recent call last):
File "C:\Python27\nytimes\26052014\Multiple term search with multiple addreses.py", line 49, in <module>
streaming_api.filter(follow=[list_users])
File "build\bdist.win32\egg\tweepy\streaming.py", line 296, in filter
encoded_follow = [s.encode(encoding) for s in follow]
AttributeError: 'list' object has no attribute 'encode'
Please help me resolving the issue.

You define list_users here
list_users = ['17006157','59145948','157009365','16686144','68044757','33338729']
and then you pass it to streaming_api.filter like this
streaming_api.filter(follow=[list_users])
When the streaming_api.filter function is iterating over the value you pass as follow, it gives the error
AttributeError: 'list' object has no attribute 'encode'
The reason for this is as follows
You call streaming_api.filter like this
streaming_api.filter(follow=[list_users])
Here
streaming_api.filter(follow=[list_users])
you are trying to pass your list as value for follow, however because you put list_users in enclosing [] you are creating a list in a list. Then streaming_api.filter iterates over follow, calling .encode on each entry as we see here
[s.encode(encoding) for s in follow]
But the entry s is a list while it should be a string.
That is because you accidentally created a list in a list like you can see above.
The solution is simple. Change
streaming_api.filter(follow=[list_users])
to
streaming_api.filter(follow=list_users)
To pass a list to a function, you can just specify the name. No need to enclose it in []
Same applies to the last line. Change
streaming_api.filter(track=[list_terms])
to
streaming_api.filter(track=list_terms)