I am trying to find a way in which I can re run my code if there is a timeout or an error.
This is something that happens if the internet connection drops - So I'd need to delay for a few seconds whilst it comes back online and try again..
Would there be a way in which I could run my code in another python script and tell it to re run if it times out or disconnects?
Thanks in advance
If you are talking about http request connection error and if you are using the requests library you can use this urllib retry
FYI https://docs.python-requests.org/en/master/api/#requests.adapters.HTTPAdapter
Of course, other libraries will have their own retry function.
If you just want simple retry code, use the below code
retries_count = 3 # hard value
delay = 3 # hard value
while True:
try:
... run some code
return or break
except {Your Custom Error}:
if retries_count <= 0:
raise
retries_count -= 1
time.sleep(delay)
A Google search for "python throttling" will give you a lot of reference.
You can use a try-catch in an infinite loop like this:
while True:
try:
# your code
# call another python script
except:
time.sleep(10)
also, you can check if the error and run whatever you need to run, depend on the error type. for example:
while True:
try:
# your code
# call another python script
# break
except Exception as e:
if e == 'error type':
time.sleep(10)
else:
pass
You actually can do it with try except block.
You can find all about it here: https://docs.python.org/3/tutorial/errors.html
You just make one script, that has something like that:
while True:
try:
another_script_name.main()
except:
time.sleep(20)
Of course, you need to import both time and the other script you made.
What these lines are doing is just an infinite loop, that always tries to run the main function on the other script you made, and if some sort of error occurs, the system will sleep for 20 seconds and then will try again, because it is in the infinite loop.
Related
I have a function that is on a schedule to run every minute. The first step is to pull data from an API. Sometimes this works and sometimes it times out. If it works, I want to do a bunch of stuff to the data and then save it. If it doesn't work, I just want to skip to the end of the function and not do anything. So here is how it works:
def job():
try:
a = requests.get("API address that sometimes work and sometimes doesn't")
except:
print('connection error')
#a bunch of code that transforms the data and then saves it
and here is the scheduler:
schedule.every().minute.at(":01").do(job)
while 1:
schedule.run_pending()
time.sleep(1)
I can kind of achieve what I want by moving the #a bunch of code line into the try, but then other potential errors are going to be caught in the "connection error" as well.
Is there a way where I can make the exception skip to the end of the function? And then because its on a one minute scheduler it just tries again later?
I know its not reproducible code but this is simple enough that it shouldn't be necessary.
You can simply use return.
def job():
try:
a = requests.get("API address that sometimes work and sometimes doesn't")
except:
print('connection error')
return
I'm in trouble about how to end a 'try' loop, which is occurred since I have the 'try', here is the code:
import time
class exe_loc:
mem = ''
lib = ''
main = ''
def wizard():
while True:
try:
temp_me = input('Please specify the full directory of the memory, usually it will be a folder called "mem"> ' )
if temp_me is True:
exe_loc.mem = temp_me
time.sleep(1)
else:
print('Error value! Please run this configurator again!')
sys.exit()
temp_lib = input('Please specify the full directory of the library, usually it will be a folder called "lib"> ')
if temp_lib is True:
exe_loc.lib = temp_lib
time.sleep(1)
else:
print('Invalid value! Please run this configurator again!')
sys.exit()
temp_main = input('Please specify the full main executable directory, usually it will be app main directory> ')
if temp_main is True:
exe_loc.main = temp_main
time.sleep(1)
I tried end it by using break, pass, and I even leaves it empty what I get is Unexpected EOF while parsing, I searched online and they said it is caused when the code blocks were not completed. Please show me if any of my code is wrong, thanks.
Btw, I'm using python 3 and I don't know how to be more specific for this question, kindly ask me if you did not understand. Sorry for my poor english.
EDIT: Solved by removing the try because I'm not using it, but I still wanna know how to end a try loop properly, thanks.
Your problem isn't the break, it's the overall, high-level shape of your try clause.
A try requires either an except or a finally block. You have neither, which means your try clause is never actually complete. So python keeps looking for the next bit until it reaches EOF (End Of File), at which point it complains.
The python docs explain in more detail, but basically you need either:
try:
do_stuff_here()
finally:
do_cleanup_here() # always runs, even if the above raises an exception
or
try:
do_stuff_here()
except SomeException:
handle_exception_here() # if do_stuff_here raised a SomeException
(You can also have both the except and finally.) If you don't need either the cleanup or the exception handling, that's even easier: just get rid of the try altogether, and have the block go directly under that while True.
Finally, as a terminology thing: try is not a loop. A loop is a bit of code that gets executed multiple times -- it loops. The try gets executed once. It's a "clause," not a "loop."
You have to also 'catch' the exception with the except statement, otherwise the try has no use.
So if you do something like:
try:
# some code here
except Exception:
# What to do if specified error is encountered
This way if anywhere in your try block an exception is raised it will not break your code, but it will be catched by your except.
So I have a situation where I am to use internet connection for 12 hours straight and make calls to an api. But Light keeps going off after every 10 minutes. Is is possible to write a try, except function that will cause a delay of 10 minutes in case an error of timed out is generated. It is hopeful that the electricity will come back in 10 minutes.|
This is what I am currently using:
try:
a=translator.translate(str(x1),dest='hi')
b=translator.translate(str(x2),dest='hi')
except:
sleep(60*10)
You can use the retry module for these kind of retrying on exception. This makes the code to look much cleaner. pip install retry should install the module
from retry import retry
#retry(Exception, delay=10*60, tries=-1)
def my_code_that_needs_to_be_retried_for_ever():
a=translator.translate(str(x1),dest='hi')
b=translator.translate(str(x2),dest='hi')
# Call the function
my_code_that_needs_to_be_retried_for_ever()
With the above code, when my_code_that_needs_to_be_retried_for_ever is invoked it would be retried every 60*10 seconds (10 mins) forever (as tries is set to -1) everytime the code inside the fuction block raises an Exception
Use try and except to catch the exception and then time.sleep to make your Python script sleep for the desired amount of time. You can then put everything inside an endless while loop and break out of it once everything finished.
while True:
try:
# put everything here which might produce exception
pass
# if this point is reached everything worked fine, so exit loop
break
except:
time.sleep(10*60)
You can run the following example to see the general idea:
import random
import time
print("Before loop")
while True:
try:
print("Try to execute commands")
# your commands here
if random.random() > 0.3:
print("Randomly simulate timeout")
raise Exception("Timeout")
print("Everything done")
break
except:
print("Timeout: sleep for 2 seconds and try again")
time.sleep(2)
print("After loop")
Instead of real commands, we randomly decide to raise an exception to simulate the timeout. The result might look something like this:
Before loop
Try to execute commands
Randomly simulate timeout
Timeout: sleep for 2 seconds and try again
Try to execute commands
Randomly simulate timeout
Timeout: sleep for 2 seconds and try again
Try to execute commands
Randomly simulate timeout
Timeout: sleep for 2 seconds and try again
Try to execute commands
Everything done
After loop
I have a scraping code within a for loop, but it would take several hours to complete, and the program stops when my Internet connection breaks. What I (think I) need is a condition at the beginning of the scraper that tells Python to keep trying at that point.
I tried to use the answer from here:
for w in wordlist:
#some text processing, works fine, returns 'textresult'
if textresult == '___': #if there's nothing in the offline resources
bufferlist = list()
str1=str()
mlist=list() # I use these in scraping
br = mechanize.Browser()
tried=0
while True:
try:
br.open("http://the_site_to_scrape/")
# scraping, with several ifs. Each 'for w' iteration results with scrape_result string.
except (mechanize.HTTPError, mechanize.URLError) as e:
tried += 1
if isinstance(e,mechanize.HTTPError):
print e.code
else:
print e.reason.args
if tried > 4:
exit()
time.sleep(120)
continue
break
Works while I'm online. When the connection breaks, Python writes the 403 code and skips that word from wordlist, moves on to the next and does the same. How can I tell Python to wait for connection within the iteration?
EDIT: I would appreciate it if you could write at least some of the necessary commands and tell me where they should be placed in my code, because I've never dealt with exception loops.
EDIT - SOLUTION I applied Abhishek Jebaraj's modified solution. I just added a very simple exception handling command:
except:
print "connection interrupted"
time.sleep(30)
Also, Jebaraj's getcode command will raise an error. Before r.getcode, I used this:
import urllib
r = urllib.urlopen("http: the site ")
The top answer to this question helped me as well.
Write another while loop inside which will keep trying to connect to the internet.
It will break only when it receives status code of 200 and then you can continue with your program.
Kind of like
retry = True
while retry:
try:
r = br.open(//your site)
if r.getcode()/10==20:
retry = False
except:
// code to handle any exception
// rest of your code
I have a Python program that deals with a webpage with Selenium every 5 minutes from a cron job. cron sends me an email for every exception raised (cron just catches all stderr). Sometimes the website has some persistent internal errors, and I end up with hundreds of system emails to check. I then started ignoring exceptions, but I found it would be more sane if the program would send me an email if the problem persists for one hour.
I have written the following code:
#!/usr/bin/env python3
import sys, os, time
name = os.path.basename(sys.argv[0])
def send_email(server, message):
# just print for testing
print('Sending email...')
def handle_exception(exception, function):
# save a state file in /dev/shm with the name of the exception
warn_state = '/dev/shm/{}.{}'.format(name, exception.__name__)
try:
function
except(exception) as exception:
if os.path.exists(warn_state):
if time.time() - os.path.getmtime(warn_state) > 3600:
send_email('localhost', exception.args[1])
else:
open(warn_state, 'w').close()
else:
if os.path.exists(warn_state):
os.remove(warn_state)
def my_function():
# try raising a NameError
print(undefined_variable)
# run my_function() catching exceptions
handle_exception(NameError, my_function)
When I run the code above, I have checked that the else: part is being executed, indicating that function is not failing at all. I'm new to programming, so I don't really know if this would be the proper way to do this, but I created this function because I have to deal with at least five different types of exceptions on this script, that are raised randomly by Selenium due to server or network problems.
you are not actually calling function
function
does nothing. you need to do
function()