I'm trying to find the correct way to handle rate limits when iterating through a list of followers using the Cursor object. Here is what I'm trying:
while True:
try:
for follower in tweepy.Cursor(api.followers, id=root_usr).items():
print(follower.id)
except tweepy.TweepError:
# hit rate limit, sleep for 15 minutes
print('Rate limited. Sleeping for 15 minutes.')
time.sleep(15 * 60 + 15)
continue
except StopIteration:
break
This is probably incorrect, since an exception will make the for loop start from the beginning again. What's the right way to iterate through all of root_usr's followers while also handling the rate limit problems?
You can construct the Cursor instance on a variable and call next(cursor) inside a try-catch. You can handle StopIteration and TweepError's that way.
Like
cursor = tweepy.Cursor(api.followers, id=root_usr).items()
# or tweepy.Cursor(api.followers, id=root_usr).items().iter()
while True:
try:
follower = next(cursor)
print(follower.id)
except StopIteration:
break
except tweepy.TweepError:
# Handle this, sleep, log, etc.
pass
There's an example under the code snippets section of the documentation that suggests making a wrapper function to deal with error handling.
def limit_handled(cursor):
while True:
try:
yield next(cursor)
except tweepy.RateLimitError:
time.sleep(15 * 60)
except StopIteration:
print("Done")
break
for follower in limit_handled(tweepy.Cursor(api.followers, id=root_usr).items()):
print(follower.id)
Also, I suggest setting count to the max, tweepy.Cursor(api.followers, id=root_usr, count=200).items(), to make the most of each API call.
Related
If I want to call a function both in a single iteration and in a looping function, that reuses the same code, whats the proper way to create the two functions?
Here is an example single function DoStuff and 3 attempts to create a second way of calling it, in __name__=="__main__".
import time
def DoStuff(txt):
"""Do it one time"""
print(txt)
def LoopStuff(txt):
"""Function of a Function"""
try:
while True:
if time.time() % 5 < 1:
DoStuff(txt)
time.sleep(1)
except KeyboardInterrupt:
print("You killed it.")
except:
print("Loop Failed.")
def Loop(function):
"""This one's broken"""
try:
while True:
if time.time() % 5 < 1:
function()
time.sleep(1)
except KeyboardInterrupt:
print("You killed it.")
except:
print("Loop Failed.")
def Looped(function):
"""Wrapper"""
def wrap(*args,**kwargs):
try:
while True:
if time.time() % 5 < 1:
function(*args, **kwargs)
time.sleep(1)
except KeyboardInterrupt:
print("You killed it.")
except:
print("Loop Failed.")
return wrap
#Looped
def WrapStuff(txt):
"""Hows this work?"""
DoStuff(txt)
if __name__ == '__main__':
DoStuff("one time text")
LoopStuff("Function fuction")
Loop(DoStuff("what's your"))
WrapStuff("conjunction?")
I was trying to create a loop that ran every min, regardless of execution time, but I have shortened it to 5 seconds here for troubleshooting. Maybe there's a better way to do this with a chron type behavior, but that's not really the question I'm asking here.
The 1st and 3rd attempt work but the 2nd exits with
in <module> Loop(DoStuff("what's your"))
in Loop function()
TypeError: 'NoneType' is not callable
I just went trial and error until I found this code that worked. It seems like this will let me use Looped in a wrapper function and in this single line style. If you have a better answer, please share.
It's just a cleaner way of what I was trying to accomplish with WrapSuff.
ShortWrap = Looped(DoStuff)
ShortWrap('my text')
I can see, in hindsight, that the second example Loop(DoStuff("what's your")) is just a failed half-step between the first and third example.
I'm having trouble understanding the mechanism of my code. I want to retrieve the realtime price data of a crypto from an exchange and print it every second, while creating a retry mechanism that catches an exception when it occurs.
while True:
try_cnt = 10
while try_cnt > 0:
try:
price = get_current_price(symbol) # function imported from elsewhere
print(price)
time.sleep(1)
try_cnt = 0
break
except Exception as e:
print(e)
try_cnt -= 1
This is how I have it set up but I'm having trouble follow the logic. This is how I see it as of now. After the code has been run:
Price is retrieved
try_cnt is set to 0 and the second while loop is broken
The next second, try_cnt is reset to 10 and price is retrieved
try_cnt is set to 0 again and the second while loop is broken
repeat of this process
Am I understanding this correctly?
EDIT: the reason why I have a while loop outside is that I have other things that are going on at specific times besides the part I mentioned here. The part that I mentioned is supposed to be running all the time every second. For example,
while True:
if now.hour == 0 and now.minute == 0 and now.second == 0:
A = functionA(x,y)
try_cnt = 10
while try_cnt > 0:
try:
price = get_current_price(symbol) # function imported from elsewhere
print(price)
time.sleep(1)
try_cnt = 0
break
except Exception as e:
print(e)
try_cnt -= 1
I think that you are understanding everything just fine, but the code itself is a little silly and redundant. For example, setting try_cnt to 0 and then following that with break is pointless - both lines do the same thing. And unless there is some significance to the variable try_cnt that is not present within the code you posted, its existence seems pretty much useless to me. Here's code that does the exact same thing as the code above, but without the redundancies:
while True:
try:
price = get_current_price(symbol)
print(price)
time.sleep(1)
except Exception as e:
print(e)
In response to your edit, the while True loop encompassing everything makes a lot more sense. However, this means that your code isn't doing what you want it to do. The outer while loop ensures that even if try_cnt reaches 0, try_cnt will just be reset to 10 on the next pass and it will be as if no Exceptions ever occurred (besides the ten error messages in the console). If you want that inner while loop to stop executing when ten Exceptions occur, you need to place the try_cnt = 10 statement outside of both while loops like so:
try_cnt = 10
while True:
if now.hour == 0 and now.minute == 0 and now.second == 0:
A = functionA(x,y)
while try_cnt > 0:
try:
price = get_current_price(symbol)
print(price)
time.sleep(1)
try_cnt = 10
break
except Exception as e:
print(e)
try_cnt -= 1
Python - I am trying to get some code to check whether a webpage has loaded correctly, and handle a couple of exception (but not error) scenarios.
So the code would do the following
While the webpage is not loaded correctly (I have set some variables to control this)
try to check whether the page had loaded correctly
try to check whether it hasn't loaded correctly as the account is logged out
try to check whether it hasn't loaded correctly as it needs an update
otherwise wait 5 seconds and loop through the block again, until the maximum number of retries has been hit
I tried doing Try, except, except but as the errors are not specific (TypeError etc) then I get an 'Only one catch all' code rejection
I also considered an If Elif etc but then I don't know how to get it to loop back to the start if it doesn't match any of the scenarios and needs to wait and then try again.
The code I have tried
while not loaded and attempts < maxattempts and not loggedout: #confirm that the page is not loaded, not too many attempts or logged out.
try:
x1, y1=pygu.center(pygu.locateOnScreen("/whatsappopened.png", confidence=0.8))
time.sleep(2)
pygu.moveTo(x1,y1)
current_time = now.strftime("%H-%M-%S")
loaded =True
except:
x1, y1=pygu.center(pygu.locateOnScreen("whatsapploggedout.png", confidence=0.8))
time.sleep(2)
pygu.moveTo(x1,y1)
loggedout =True
except:
x1, y1=pygu.center(pygu.locateOnScreen("/whatsappupdate.png", confidence=0.8))
time.sleep(2)
pygu.hotkey('ctrl', 'w')
loggedout =True
except:
time.sleep(5)
attempts += +1
print("page not loaded after %s attempts" %(attempts))
Any guidance appreciated!
I came up with a potential solution with comments that may let you think about how to continue with the program and suit it to your needs.
attempts = 1
maxattempts = 5
loaded = False
loggedout = False
update = False
time_now = datetime.now()
while not loaded and attempts < maxattempts and not loggedout: #confirm that the page is not loaded, not too many attempts or logged out.
# Check if WhatsApp is open
try:
x1, y1=pygu.center(pygu.locateOnScreen("whatsappopened.png", confidence=0.8))
print("WhatsApp opened")
time.sleep(2)
pygu.moveTo(x1, y1)
time_now = datetime.now().strftime("%H:%M:%S")
loaded = True
except Exception as e:
# Here because WhatsApp is not open or logged out
# Check if WhatsApp is logged out
try:
x2, y2=pygu.center(pygu.locateOnScreen("whatsapploggedout.png", confidence=0.8))
print("WhatsApp logged out")
time.sleep(2)
pygu.moveTo(x2, y2)
loggedout = True
except Exception as e:
# Check if needs update
try:
x3, y3=pygu.center(pygu.locateOnScreen("whatsappupdate.png", confidence=0.8))
print("WhatsApp update")
time.sleep(2)
pygu.moveTo(x3, y3)
update = True
except Exception as e:
# Not open
print("WhatsApp not open")
time.sleep(2)
attempts += 1
time_now = datetime.now().strftime("%H:%M:%S")
loaded = False
loggedout = False
update = False
print("Attempts: " + str(attempts))
In the case I am looping into a list, where for value msg_list = 0 it will execute action(0), usr). This action can fail for a determined user, I should choose aother user if it happens, perform all the actions related to the user
How can I do to repeat action[0] if it fails?
for msg in range(len(msg_list)):
# in this case msg = 0
usr = select_random_user()
multiple_actions_for(usr) # This are lots of code lines I don't want to repeat!!
try:
action(msg, usr)
more_actions(usr.related_stuff)
except Exception as e:
go_back_to(msg =0 instead of looping into msg=1) # this is what I want to do
How can I do to get that? Repeat the loop for msg = i instead of passing to msg = i + 1?
Put your code into the endless while-loop with exiting from it if try was successful:
for msg in range(len(msg_list)):
while True:
usr = select_random_user()
multiple_actions_for(usr)
try:
action(msg, usr)
more_actions(usr.related_stuff)
except Exception as e:
continue
else:
break
It really depends. Is it okay to go back to the beginning of the loop? If so, you can call "continue", which stops the current iteration, and restarts the loop. Otherwise, I don't think there's something similar to a goto in Python, no. It's a very structured language.
Try using while loop instead of for loop. The idea is as shown bellow:
bool still_looping = True
msg = 0
while still_looping:
usr = select_random_user()
multiple_actions_for(usr)
try:
action(msg, usr)
more_actions(usr.related_stuff)
if (msg < len (msg_list)):
msg += 1
except Exception as e:
# Do whatever you want here. msg was not incremented
if (msg == len(msg_list))
still_looping = False #verify if this was the last execution
So there's this website that posts something I want to buy at a random time of day for a limited amount of time and I want to write something to send a message to my phone when a new url is posted to that webpage.
I planned on doing this by counting the number of links on the page (since it's rarely updated) and checking it every 5 minutes against what it was 5 minutes before that, then 5 minutes later check it against what it was 10 minutes before that, 5 minutes later check what it was 15 minutes before that... and if it's greater than what it originally was, send a message to my phone. Here's what I have so far:
class url_alert:
url = ''
def link_count(self):
notifyy=True
while notifyy:
try:
page = urllib.request.urlopen(self.url)
soup = bs(page, "lxml")
links=[]
for link in soup.findAll('a'):
links.append(link.get('href'))
notifyy=False
print('found', int(len(links)), 'links')
except:
print('Stop making so many requests')
time.sleep(60*5)
return len(links)
def phone(self):
self= phone
phone.message = client.messages.create(to="", from_="",body="")
print('notified')
def looper(self):
first_count = self.link_count()
print('outside while')
noty = True
while noty:
try:
second_count = self.link_count()
print('before compare')
if second_count == first_count:
self.phone()
noty = False
except:
print('not quite...')
time.sleep(60)
alert = url_alert()
alert.looper()
As a test, I decided to set the if statement that determines whether or not to send a message as equal but the loop kept on running. Am I calling the functions within the looper function the right way?
It looks like you need to eliminate the try block, as it is now, if self.phone() takes an exception you will never leave the loop
def looper(self):
first_count = self.link_count()
while True:
if first_count != self.link_count():
self.phone()
break
time.sleep(60)