Why am I getting a python RecursionError - python

I am creating a web scraper to go over almost 400k records. Basically, it works like this, I have a CSV of part numbers that need to be searched on this site. The site has an exposed API so I am able to skip the frontend and make a direct request to the site after logging in. I created one function called GetPdcResults() which takes in a list of parts and a start number. The start argument is for if the scraper stops for any reason I can start it back up at the same point it left off on the parts list. Then the main loop of the scraper which enumerates over each part in the list builds a payload for that part and requests the information. Some error handling for if I have a network error or a cookie error which only happens when my user's session has expired. Then it calls the CleanPdcResults() function. This cleans the response returned from the site and saves the relevant information to a CSV for exporting.
To my understanding recursion is when a function calls itself repeatedly and there is a limit to this in python and is more resource intensive. Iteration is when you use a loop to repeat a set of actions.
I think I want iteration in my app, not recursion because currently I a, getting this error I have never seen before.
RecursionError: maximum recursion depth exceeded while calling a Python object
I'm assuming because there is recursion happening in my functions instead of iteration but I can't seem to point it out. The only time a function is calling itself is when there is a cookie error and the GetPdcResults() function is called again but that wouldn't be called so many times that a Limit is reached.
Can someone help me find where recursion is happening in my scrapper and how I can convert it to iteration to stop this error?? Any help is appreciated!
def GetPdcResults(parts, start=0):
logger = PartsLogger()
logger.log_cookie(headers['cookie'])
logger.log_num_parts(parts)
for (i, part) in tqdm(enumerate(parts[start:], start), total=len(parts[start:])):
if part == nan:
break
logger.log_cur_part(i, part)
payload = "{\"catalogId\":\"2\",\"locatorService\":\"Panda\""
payload += f',"partNumber":"{part}", "cacheKey":"{part}_2_en-US_7497fea0-4fb6-4b28-b0e8-62e3e4204cc5"{"}"}'
try:
response = requests.request("POST", url, headers=headers, data=payload)
except requests.exceptions.RequestException as e:
print('\n[-] Request Error')
print(e)
logger.log_error(str(e), part=part)
if response.status_code == 401:
logger.log_error('[-] Cookie Error', part=part)
print('\n[-] Cookie Error')
GetPdcResults(parts, start=i)
break
CleanPdcResults(response.json(), i, part, logger)
def CleanPdcResults(resp, index, part, logger):
try:
pdc_results = resp['d']['PdcResults']
pdc92 = {}
for pdc in pdc_results:
if '92' in pdc['LocationName']:
pdc92.update(pdc)
break
if(bool(pdc92)):
foundPart = [{'':index, 'Part':part, 'Qty':pdc92['Quantity']}]
df = pd.DataFrame(foundPart)
if not exists('Parts.csv'):
df.to_csv('Parts.csv', index=False)
df.to_csv('Parts.csv', mode='a', index=False, header=False)
else:
print('\n[-] Part Not Found')
except Exception as e:
logger.log_error(str(e), part=part, response=resp)
Traceback (most recent call last):
File "c:\Users\carte\OneDrive\Documents\GrayTeck\Chad S\CleanPartsCSV.py", line 30, in run
GetPdcResults(partsList, start=startIndex)
File "c:\Users\carte\OneDrive\Documents\GrayTeck\Chad S\GetPDCRes.py", line 57, in GetPdcResults
GetPdcResults(parts, start=i)
File "c:\Users\carte\OneDrive\Documents\GrayTeck\Chad S\GetPDCRes.py", line 57, in GetPdcResults
GetPdcResults(parts, start=i)
File "c:\Users\carte\OneDrive\Documents\GrayTeck\Chad S\GetPDCRes.py", line 57, in GetPdcResults
GetPdcResults(parts, start=i)
[Previous line repeated 973 more times]
File "c:\Users\carte\OneDrive\Documents\GrayTeck\Chad S\GetPDCRes.py", line 48, in GetPdcResults
response = requests.request("POST", url, headers=headers, data=payload)
File "C:\Python310\lib\site-packages\requests\api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "C:\Python310\lib\site-packages\requests\sessions.py", line 529, in request
resp = self.send(prep, **send_kwargs)
File "C:\Python310\lib\site-packages\requests\sessions.py", line 645, in send
r = adapter.send(request, **kwargs)
File "C:\Python310\lib\site-packages\requests\adapters.py", line 440, in send
resp = conn.urlopen(
File "C:\Python310\lib\site-packages\urllib3\connectionpool.py", line 703, in urlopen
httplib_response = self._make_request(
File "C:\Python310\lib\site-packages\urllib3\connectionpool.py", line 449, in _make_request
six.raise_from(e, None)
File "<string>", line 3, in raise_from
File "C:\Python310\lib\site-packages\urllib3\connectionpool.py", line 444, in _make_request
httplib_response = conn.getresponse()
File "C:\Python310\lib\http\client.py", line 1374, in getresponse
response.begin()
File "C:\Python310\lib\http\client.py", line 337, in begin
self.headers = self.msg = parse_headers(self.fp)
File "C:\Python310\lib\http\client.py", line 236, in parse_headers
return email.parser.Parser(_class=_class).parsestr(hstring)
File "C:\Python310\lib\email\parser.py", line 67, in parsestr
return self.parse(StringIO(text), headersonly=headersonly)
File "C:\Python310\lib\email\parser.py", line 56, in parse
feedparser.feed(data)
File "C:\Python310\lib\email\feedparser.py", line 176, in feed
self._call_parse()
File "C:\Python310\lib\email\feedparser.py", line 180, in _call_parse
self._parse()
File "C:\Python310\lib\email\feedparser.py", line 295, in _parsegen
if self._cur.get_content_maintype() == 'message':
File "C:\Python310\lib\email\message.py", line 594, in get_content_maintype
ctype = self.get_content_type()
File "C:\Python310\lib\email\message.py", line 578, in get_content_type
value = self.get('content-type', missing)
File "C:\Python310\lib\email\message.py", line 471, in get
return self.policy.header_fetch_parse(k, v)
File "C:\Python310\lib\email\_policybase.py", line 316, in header_fetch_parse
return self._sanitize_header(name, value)
File "C:\Python310\lib\email\_policybase.py", line 287, in _sanitize_header
if _has_surrogates(value):
File "C:\Python310\lib\email\utils.py", line 57, in _has_surrogates
s.encode()
RecursionError: maximum recursion depth exceeded while calling a Python object


Python's default maximum recursion depth is 1000, but you can check yours with print(sys.getrecursionlimit()) or set a new one with
# import sys
new_recursion_limit = 2000 # set as you prefer
sys.setrecursionlimit(new_recursion_limit)
# print('recursion limit is now', sys.getrecursionlimit())
but this is considered a dangerous method.
Instead you should consider setting extra parameters for GetPdcResults [or any other recursive function] - something like
def GetPdcResults(parts, start=0, maxDepth=999, curDepth=0):
and then increment curDepth with every recursive call, such as
# if response.status_code == 401:
# logger.log_error('[-] Cookie Error', part=part)
# print('\n[-] Cookie Error')
if curDepth < maxDepth:
GetPdcResults(parts, start=i, maxDepth=maxDepth, curDepth=curDepth+1)
## else: print(curDepth, 'is too deep') # if you want an alternate action...
# break

Related

Error: "Only unicode objects are escapable. Got None of type <class 'NoneType'>". Could someone please help me spot the mistake(s) in my code?

I am a beginner at Twitter Development and Python programming in general, but recently I have been trying to build a bot that, when tagged in reply to a Tweet, finds keywords and uses Google to deliver some info from reliable sources regarding the topic of the original Tweet. However, I have encountered one major problem while programming: API doesn't get created since the code triggers the error "only unicode objects are escapable". I have used module Config to set my Twitter API credentials as environmental variables and that seems to work fine on its own. Before trying to run the bot, I activate my virtual environment and export the env variables, so I do not think this issue has to do with incorrectly setting those variables, but I would not say I am certain about that either!
The code goes this way:
import tweepy
import logging
from config import create_api
import time
import re
from googlesearch import search
import sys
import io
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger()
api = tweepy.API
def check_mentions(api, keywords, since_id):
logger.info("Collecting info")
new_since_id = since_id
for tweet in tweepy.Cursor(api.mentions_timeline,
since_id=since_id).items():
new_since_id = max(tweet.id, new_since_id)
if tweet.in_reply_to_status_id is not None:
in_reply_to_status_id = previous_tweet.id
status_id = tweet.in_reply_to_status_id
tweet_u = api.get_status(status_id,tweet_mode='extended')
#to store the output -links- as a variable to use it later
old_stdout = sys.stdout
new_stdout = io.StringIO()
sys.stdout = new_stdout
output = new_stdout.getvalue()
sys.stdout = old_stdout
print(output)
text = output
# remove words that are between 1 and 3 characters long
shortword = re.compile(r'\W*\b\w{1,3}\b')
print(shortword.sub('', text))
keywords_search = print(shortword.sub('', text))
if keywords_search is not None:
mystring = search(keywords_search, num_results=500)
else:
mystring = search("error", num_results=1)
for word in mystring:
if "harvard" in word or "cornell" in word or "researchgate" in word or "yale" in word or "rutgers" in word or "caltech" in word or "upenn" in word or "princeton" in word or "columbia" in word or "journal" in word or "mit" in word or "stanford" in word or "gov" in word or "pubmed" in word:
print(word)
#to store the output -links- as a variable to use it later
old_stdout = sys.stdout
new_stdout = io.StringIO()
sys.stdout = new_stdout
for word in mystring:
if "harvard" in word or "cornell" in word or "researchgate" in word or "yale" in word or "rutgers" in word or "caltech" in word or "upenn" in word or "princeton" in word or "columbia" in word or "journal" in word or "mit" in word or "stanford" in word or "gov" in word or "pubmed" in word:
print(word)
#to print normally again
output = new_stdout.getvalue()
sys.stdout = old_stdout
print(output)
if output is not None:
status = "Hi there! This may be what you're looking for" and print(output),
len(status) <= 280
api.update_status(status, in_reply_to_status_id=tweet.id, auto_populate_reply_metadata=False),
else:
status = "Sorry, I cannot help you with that :(. You might want to try again with a distinctly sourced Tweet",
len(status) <= 280
api.update_status(status, in_reply_to_status_id=tweet.id, auto_populate_reply_metadata=False),
return new_since_id
def main():
api = create_api()
since_id = 1 #the last mention you have.
while True:
print(since_id)
since_id = check_mentions(api, ["help", "support"], since_id)
logger.info("Waiting...")
time.sleep(15)
if __name__ == "__main__":
main()
My Config module:
import tweepy
import logging
import os
logger = logging.getLogger()
def create_api():
consumer_key = os.getenv("XXXXXXXXXX")
consumer_secret = os.getenv("XXXXXXXXXX")
access_token = os.getenv("XXXXXXXXXX")
access_token_secret = os.getenv("XXXXXXXXXX")
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
api = tweepy.API(auth, wait_on_rate_limit=True,
wait_on_rate_limit_notify=True)
try:
api.verify_credentials()
except Exception as e:
logger.error("Error creating API", exc_info=True)
raise e
logger.info("API created")
return api
The error goes:
ERROR:root:Error creating API
Traceback (most recent call last):
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\tweepy\binder.py", line 184, in execute
resp = self.session.request(self.method,
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\requests\sessions.py", line 516, in request
prep = self.prepare_request(req)
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\requests\sessions.py", line 449, in prepare_request
p.prepare(
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\requests\models.py", line 318, in prepare
self.prepare_auth(auth, url)
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\requests\models.py", line 549, in prepare_auth
r = auth(self)
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\requests_oauthlib\oauth1_auth.py", line 108, in __call__
r.url, headers, _ = self.client.sign(
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\oauthlib\oauth1\rfc5849\__init__.py", line 313, in sign
('oauth_signature', self.get_oauth_signature(request)))
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\oauthlib\oauth1\rfc5849\__init__.py", line 127, in get_oauth_signature
uri, headers, body = self._render(request)
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\oauthlib\oauth1\rfc5849\__init__.py", line 209, in _render
headers = parameters.prepare_headers(
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\oauthlib\oauth1\rfc5849\utils.py", line 32, in wrapper
return target(params, *args, **kwargs)
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\oauthlib\oauth1\rfc5849\parameters.py", line 59, in prepare_headers
escaped_value = utils.escape(value)
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\oauthlib\oauth1\rfc5849\utils.py", line 56, in escape
raise ValueError('Only unicode objects are escapable. ' +
ValueError: Only unicode objects are escapable. Got None of type <class 'NoneType'>.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\maria\OneDrive\Documentos\Lara\Python\Factualbot\config.py", line 23, in create_api
api.verify_credentials()
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\tweepy\api.py", line 672, in verify_credentials
return bind_api(
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\tweepy\binder.py", line 253, in _call
return method.execute()
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\tweepy\binder.py", line 192, in execute
six.reraise(TweepError, TweepError('Failed to send request: %s' % e), sys.exc_info()[2])
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\six.py", line 702, in reraise
raise value.with_traceback(tb)
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\tweepy\binder.py", line 184, in execute
resp = self.session.request(self.method,
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\requests\sessions.py", line 516, in request
prep = self.prepare_request(req)
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\requests\sessions.py", line 449, in prepare_request
p.prepare(
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\requests\models.py", line 318, in prepare
self.prepare_auth(auth, url)
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\requests\models.py", line 549, in prepare_auth
r = auth(self)
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\requests_oauthlib\oauth1_auth.py", line 108, in __call__
r.url, headers, _ = self.client.sign(
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\oauthlib\oauth1\rfc5849\__init__.py", line 313, in sign
('oauth_signature', self.get_oauth_signature(request)))
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\oauthlib\oauth1\rfc5849\__init__.py", line 127, in get_oauth_signature
uri, headers, body = self._render(request)
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\oauthlib\oauth1\rfc5849\__init__.py", line 209, in _render
headers = parameters.prepare_headers(
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\oauthlib\oauth1\rfc5849\utils.py", line 32, in wrapper
return target(params, *args, **kwargs)
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\oauthlib\oauth1\rfc5849\parameters.py", line 59, in prepare_headers
escaped_value = utils.escape(value)
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\oauthlib\oauth1\rfc5849\utils.py", line 56, in escape
raise ValueError('Only unicode objects are escapable. ' +
tweepy.error.TweepError: Failed to send request: Only unicode objects are escapable. Got None of type <class 'NoneType'>.
Traceback (most recent call last):
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\tweepy\binder.py", line 184, in execute
resp = self.session.request(self.method,
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\requests\sessions.py", line 516, in request
prep = self.prepare_request(req)
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\requests\sessions.py", line 449, in prepare_request
p.prepare(
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\requests\models.py", line 318, in prepare
self.prepare_auth(auth, url)
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\requests\models.py", line 549, in prepare_auth
r = auth(self)
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\requests_oauthlib\oauth1_auth.py", line 108, in __call__
r.url, headers, _ = self.client.sign(
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\oauthlib\oauth1\rfc5849\__init__.py", line 313, in sign
('oauth_signature', self.get_oauth_signature(request)))
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\oauthlib\oauth1\rfc5849\__init__.py", line 127, in get_oauth_signature
uri, headers, body = self._render(request)
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\oauthlib\oauth1\rfc5849\__init__.py", line 209, in _render
headers = parameters.prepare_headers(
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\oauthlib\oauth1\rfc5849\utils.py", line 32, in wrapper
return target(params, *args, **kwargs)
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\oauthlib\oauth1\rfc5849\parameters.py", line 59, in prepare_headers
escaped_value = utils.escape(value)
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\oauthlib\oauth1\rfc5849\utils.py", line 56, in escape
raise ValueError('Only unicode objects are escapable. ' +
ValueError: Only unicode objects are escapable. Got None of type <class 'NoneType'>.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\maria\OneDrive\Documentos\Lara\Python\Factualbot\botstring20.py", line 114, in <module>
main()
File "C:\Users\maria\OneDrive\Documentos\Lara\Python\Factualbot\botstring20.py", line 105, in main
api = create_api()
File "C:\Users\maria\OneDrive\Documentos\Lara\Python\Factualbot\config.py", line 26, in create_api
raise e
File "C:\Users\maria\OneDrive\Documentos\Lara\Python\Factualbot\config.py", line 23, in create_api
api.verify_credentials()
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\tweepy\api.py", line 672, in verify_credentials
return bind_api(
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\tweepy\binder.py", line 253, in _call
return method.execute()
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\tweepy\binder.py", line 192, in execute
six.reraise(TweepError, TweepError('Failed to send request: %s' % e), sys.exc_info()[2])
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\six.py", line 702, in reraise
raise value.with_traceback(tb)
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\tweepy\binder.py", line 184, in execute
resp = self.session.request(self.method,
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\requests\sessions.py", line 516, in request
prep = self.prepare_request(req)
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\requests\sessions.py", line 449, in prepare_request
p.prepare(
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\requests\models.py", line 318, in prepare
self.prepare_auth(auth, url)
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\requests\models.py", line 549, in prepare_auth
r = auth(self)
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\requests_oauthlib\oauth1_auth.py", line 108, in __call__
r.url, headers, _ = self.client.sign(
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\oauthlib\oauth1\rfc5849\__init__.py", line 313, in sign
('oauth_signature', self.get_oauth_signature(request)))
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\oauthlib\oauth1\rfc5849\__init__.py", line 127, in get_oauth_signature
uri, headers, body = self._render(request)
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\oauthlib\oauth1\rfc5849\__init__.py", line 209, in _render
headers = parameters.prepare_headers(
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\oauthlib\oauth1\rfc5849\utils.py", line 32, in wrapper
return target(params, *args, **kwargs)
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\oauthlib\oauth1\rfc5849\parameters.py", line 59, in prepare_headers
escaped_value = utils.escape(value)
File "C:\Users\maria\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\oauthlib\oauth1\rfc5849\utils.py", line 56, in escape
raise ValueError('Only unicode objects are escapable. ' +
tweepy.error.TweepError: Failed to send request: Only unicode objects are escapable. Got None of type <class 'NoneType'>.
I know it is pretty long but I just wanted to know if anyone could tell me which direction to go with the bug-solving process, since I am kind of lost and I did not know where else to ask! Thank you in advance and sorry for how long this is!!!!<3

One or more of your credentials is None when it's used to initialize the instance of API.
It's very likely that when you're retrieving your environment variables with os.getenv, one or more of them is not found because there isn't an environment variable with that name/key.

ChunkedEncodingError while making an API query

This is a small API request that is throwing me a requests.exceptions.ChunkedEncodingError
import requests
def categories_list():
categories = []
response = requests.get("https://fr.openfoodfacts.org/categories&json=1")
data = response.json()
i = 0
for category in data["tags"]:
if category["products"] >= 1200:
name = category["name"]
categories.append(name)
i += 1
print("It's ok, imported %s" % i)
categories_list()
Error code:
File "exception.py", line 18, in <module>
categories_list()
File "exception.py", line 6, in categories_list
response = requests.get("https://fr.openfoodfacts.org/categories&json=1")
File "/home/pi/Documents/venv/lib/python3.7/site-packages/requests/api.py", line 76, in get
return request('get', url, params=params, **kwargs)
File "/home/pi/Documents/venv/lib/python3.7/site-packages/requests/api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "/home/pi/Documents/venv/lib/python3.7/site-packages/requests/sessions.py", line 530, in request
resp = self.send(prep, **send_kwargs)
File "/home/pi/Documents/venv/lib/python3.7/site-packages/requests/sessions.py", line 683, in send
r.content
File "/home/pi/Documents/venv/lib/python3.7/site-packages/requests/models.py", line 829, in content
self._content = b''.join(self.iter_content(CONTENT_CHUNK_SIZE)) or b''
File "/home/pi/Documents/venv/lib/python3.7/site-packages/requests/models.py", line 754, in generate
raise ChunkedEncodingError(e)
requests.exceptions.ChunkedEncodingError: ('Connection broken: IncompleteRead(3573 bytes read, 6667 more expected)', IncompleteRead(3573 bytes read, 6667 more expected))
Could it be possibly my internet connection? Similar queries worked for me yesterday...

Python: TypeError: inet_aton() argument 1 must be str, not None

I am working with a mobile game API and a Telegram Bot. It worked when I put in a fixed Clantag, but now I wanted to let the user write a tag and add it to the link. The app then should search for the clan and get the right stats. Everything is ok but I get this message now and cannot find my mistake. Would be happy if you could help!
def main():
last_update_id = None
message = ""
while True:
updates = get_updates(last_update_id)
if len(updates["result"]) > 0:
last_update_id = get_last_update_id(updates) + 1
message = get_last_update_Message(updates)
clan_stats(updates, message)
def get_last_update_Message(updates):
message = ""
for update in updates["result"]:
message = update["message"]
return message["text"]
def clan_stats(updates, ID):
#https://api.royaleapi.com/clan/1RLU78YU
Link = '"https://api.royaleapi.com/clan/' + ID + '"'
r=requests.get(Link, headers={"Accept":"application/json",
"authorization":"Bearer TOKENHERE"})
clan = r.json()
Full Traceback:
Traceback (most recent call last):
File "/home/Lee63225/clashroyaleclanbot.py", line 188, in <module>
main()
File "/home/Lee63225/clashroyaleclanbot.py", line 184, in main
clan_stats(updates, message)
File "/home/Lee63225/clashroyaleclanbot.py", line 80, in clan_stats
"authorization":"Bearer TOKENHERE"})
File "/usr/lib/python3.7/site-packages/requests/api.py", line 72, in get
return request('get', url, params=params, **kwargs)
File "/usr/lib/python3.7/site-packages/requests/api.py", line 58, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/lib/python3.7/site-packages/requests/sessions.py", line 503, in request
prep.url, proxies, stream, verify, cert
File "/usr/lib/python3.7/site-packages/requests/sessions.py", line 676, in merge_environment_settings
env_proxies = get_environ_proxies(url, no_proxy=no_proxy)
File "/usr/lib/python3.7/site-packages/requests/utils.py", line 760, in get_environ_proxies
if should_bypass_proxies(url, no_proxy=no_proxy):
File "/usr/lib/python3.7/site-packages/requests/utils.py", line 716, in should_bypass_proxies
if is_ipv4_address(parsed.hostname):
File "/usr/lib/python3.7/site-packages/requests/utils.py", line 640, in is_ipv4_address
socket.inet_aton(string_ip)
TypeError: inet_aton() argument 1 must be str, not None
Thank you!

I think it should rather be
Link = 'https://api.royaleapi.com/clan/' + ID
There is some surrounding " in your attempt.
But now as I look, the function is called as clan_stats(update,message). That "message" should be a clantag, make sure it is (as now it comes from get_last_update_Message(), and looks very suspicious.

requests.exceptions.TooManyRedirects while using Tagme in Python

My python code contains a method as follows:
import tagme
def concept_extraction ():
tagme.GCUBE_TOKEN = "702e87ce-3750-4069-900d-92d12a17cda4-843334162"
string = "Sachin is running in the Kolkata stadium."
print string
lunch_annotations = tagme.annotate(string)
#Print annotations with a score higher than 0.1
for ann in lunch_annotations.get_annotations(0.1):
print ann
return;
Where for a given string, I want to determine the entities present in the string. But I am getting an error as follows:
File "/home/krishnendu/Python_workspace/QG/concept_extraction.py", line 8, in concept_extraction
lunch_annotations = tagme.annotate(string)
File "/usr/local/lib/python2.7/dist-packages/tagme/__init__.py", line 201, in annotate
json_response = _issue_request(api, payload, gcube_token)
File "/usr/local/lib/python2.7/dist-packages/tagme/__init__.py", line 271, in _issue_request
res = requests.post(api, data=payload)
File "/usr/local/lib/python2.7/dist-packages/requests-1.2.3-py2.7.egg/requests/api.py", line 88, in post
return request('post', url, data=data, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests-1.2.3-py2.7.egg/requests/api.py", line 44, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests-1.2.3-py2.7.egg/requests/sessions.py", line 335, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests-1.2.3-py2.7.egg/requests/sessions.py", line 454, in send
history = [resp for resp in gen] if allow_redirects else []
File "/usr/local/lib/python2.7/dist-packages/requests-1.2.3-py2.7.egg/requests/sessions.py", line 87, in resolve_redirects
raise TooManyRedirects('Exceeded %s redirects.' % self.max_redirects)
requests.exceptions.TooManyRedirects: Exceeded 30 redirects.
Question: The python version is 2.7 and tagme version is 0.1.3. How to handle this issue?

python-instagram OAuthPermissionsException

So I'm trying to get Instagram photos that fit certain parameters and I'm getting the following stack:
Traceback (most recent call last):
File "instagram_find_shows.py", line 83, in <module>
if __name__ == "__main__": main()
File "instagram_find_shows.py", line 48, in main
get_instagram_posts(show_name, show_time, coordinates)
File "instagram_find_shows.py", line 73, in get_instagram_posts
str(coordinates[1]), min_time, max_time)
File "C:\Users\User Name\Anaconda3\lib\site-packages\instagram\bind.py", line 197, in _call
return method.execute()
File "C:\Users\User Name\Anaconda3\lib\site-packages\instagram\bind.py", line 189, in execute
content, next = self._do_api_request(url, method, body, headers)
File "C:\Users\User Name\Anaconda3\lib\site-packages\instagram\bind.py", line 163, in _do_api_request
raise InstagramAPIError(status_code, content_obj['meta']['error_type'], content_obj['meta']['error_message'])
instagram.bind.InstagramAPIError: (400) OAuthPermissionsException-This request requires scope=public_content, but this access token is not authorized with this scope. The user must re-authorize your application with scope=public_content to be granted this permissions.
The code is as follows:
def get_instagram_posts(name, time, coordinates):
max_time_dt = time + timedelta(hours=3)
min_time_dt = time - timedelta(hours=1)
max_time = str(calendar.timegm(max_time_dt.timetuple()))
min_time = str(calendar.timegm(min_time_dt.timetuple()))
dist_rad_str = str(insta_dist_radius_m)
count_str = str(insta_count)
api = InstagramAPI(access_token=insta_access_token,
client_secret=insta_client_secret)
r = api.media_search(name, count_str, str(coordinates[0]),
str(coordinates[1]), min_time, max_time)
photos = []
for media in r:
photos.append('<img src="%s"/>' % media.images['thumbnail'].url)
print(photos[0])
I can't figure out what to do... Literally I'm just trying to do a simple test, not trying to cripple their API. Is there any way to do this within Instagram's parameters? Thanks so much!

Fixed by going to the following URL in the browser:
https://www.instagram.com/oauth/authorize?client_id=[CLIENT_ID]&redirect_uri=[REDIRECT_URI]&response_type=code&scope=basic+public_content+follower_list+comments+relationships+likes

Categories

Resources