I'm trying this simple code, but the damn break doesn't work... what is wrong?
while True:
for proxy in proxylist:
try:
h = urllib.urlopen(website, proxies = {'http': proxy}).readlines()
print 'worked %s' % proxy
break
except:
print 'error %s' % proxy
print 'done'
It's supposed to leave the while when the connection work, and go back and try another proxy if it didn't
ok, here is what I'm doing
I'm trying to check a website and if it changed, it has to break out of the while to continue to the rest of the script, but when the proxy doesn't connect, I get error from the variable, as it's null, so what I want is this for work as loop to try a proxy, and if it work, continue the script, and the end of the script, go back and try the next proxy, and if the next doesn't work, it will be back to the beginning to try the third proxy, and so on....
I'm trying something like this
while True:
for proxy in proxylist:
try:
h = urllib.urlopen(website, proxies = {'http': proxy})
except:
print 'error'
check_content = h.readlines()
h.close()
if check_before != '' and check_before != check_content:
break
check_before = check_content
print 'everything the same'
print 'changed'
You just break out of for loop -- not while loop:
running = True
while running:
for proxy in proxylist:
try:
h = urllib.urlopen(website, proxies = {'http': proxy}).readlines()
print 'worked %s' % proxy
running = False
except:
print 'error %s' % proxy
print 'done'
You can use a custom exception and then catch it:
exit_condition = False
try:
<some code ...>
if exit_conditon is True:
raise UnboundLocalError('My exit condition was met. Leaving try block')
<some code ...>
except UnboundLocalError, e:
print 'Here I got out of try with message %s' % e.message
pass
except Exception, e:
print 'Here is my initial exception'
finally:
print 'Here I do finally only if I want to'
You break out of the for loop only, so you never leave the while loop and restart iterating over the proxylist over and over again. Just omit the surrounding while loop, I actually don't understand why you enclosed the code in a while True in the first place.
break is breaking the innermost loop, which is the for loop in your case. To break from more than one loop you have few options:
Introduce a condition
Create a sub and use return
but in your case you actually don't need the outer while loop at all. Just remove it.
Related
Currently I have a program that has proxies and makes a request to get my ip address with that proxy and returns it back in json.
An example request back is this:
Got Back: {'ip': '91.67.240.45', 'country': 'Germany', 'cc': 'DE'}
I want my program to try and make a request to the url and if it does not get the request because the proxy is down I want to try again 5 times before moving onto the next ip address.
I thought this except block would work but it is not breaking out of the loop when the 5 iterations are over and I am not sure why.
My program does however work when the proxy is up for the first try as it breaks after the first attempt and then moves onto the next ip address.
Here is what I currently have:
import requests
import time
proxies = [
"95.87.220.19:15600",
"91.67.240.45:3128",
"85.175.216.32:53281",
"91.236.251.131:8118",
"91.236.251.131:8118",
"88.99.10.249:1080",
]
def sol(ip):
max_tries = 5
for i in range(1, max_tries+1):
try:
print(f"Using Proxy: {ip}")
r = requests.get('https://api.myip.com', proxies={"https": ip})
print(f"Got Back: {r.json()}")
break
except OSError:
time.sleep(5)
print(f"Retrying...: {i}")
break
for i in proxies:
sol(i)
How can I make it s my loop has 5 tries before moving onto the next ip address.
My program does however work when the proxy is up for the first try as it breaks after the first attempt and then moves onto the next ip address.
It does this unconditionally, because you have an unconditional break after the except block. Code keeps going past a try/except when the except is entered, assuming it doesn't have an abnormal exit of its own (another exception, return etc.).
So,
I thought this except block would work but it is not breaking out of the loop when the 5 iterations are over and I am not sure why.
It doesn't break out "after the 5 iterations are over" because it breaks out after the first iteration, whether or not that attempt was successful.
If I understand correctly, you can just remove break from the last line. If you have an unconditional break in a loop, it will always iterate once.
def sol(ip):
max_tries = 5
for i in range(1, max_tries+1):
try:
print(f"Using Proxy: {ip}")
r = requests.get('https://api.myip.com', proxies={"https": ip})
print(f"Got Back: {r.json()}")
break
except OSError:
time.sleep(5)
print(f"Retrying...: {i}")
# break <---- Remove this line
Using retrying would look something like the following:
from retrying import retry
import requests
#retry(stop_max_attempt_number=5)
def bad_host():
print('trying get from bad host')
return requests.get('https://bad.host.asdqwerqweo79ooo/')
try:
bad_host()
except IOError as ex:
print(f"Couldn't connect because: {str(ex)}")
...which give the following output:
trying get from bad host
trying get from bad host
trying get from bad host
trying get from bad host
trying get from bad host
Couldn't connect to bad host because: HTTPSConnectionPool(host='bad.host.asdqwerqweo79ooo', port=443): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x10b6bd910>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known'))
Getting fancy
If you want to get fancy, you could also add things like exponential backoff and selectively retrying certain exceptions.
Here's an example:
import random
import time
def retry_ioerror(exception):
return isinstance(exception, IOError)
#retry(
wait_exponential_multiplier=100,
wait_exponential_max=1000,
retry_on_exception=retry_ioerror,
stop_max_attempt_number=10)
def do_something():
t = time.time()
print(f'trying {t}')
r = random.random()
if r > 0.9:
return 'yay!'
if r > 0.8:
raise RuntimeError('Boom!')
else:
raise IOError('Bang!')
try:
result = do_something()
print(f'Success! {result}')
except RuntimeError as ex:
print(f"Failed: {str(ex)}")
With Kubernetes Container running a Python script:
import time
while True:
try:
for i in range(10):
if i==0:
raise Exception('Exception occurred!')
except:
pass
time.sleep(1)
I would like to pass the Exception's message 'Exception occurred!' down to the Container so this error message could be seen with:
kubectl describe pod pod_id
Would it be possible?
Anything you print() will be visible in kubectl logs. (You may need to set an environment variable PYTHONUNBUFFERED=1 in your pod spec.)
Your code as you've written it will never print anything. The construct
try:
...
except:
pass
silently ignores any and all exceptions out of the try block. The bare except: even captures some system-level exceptions like SystemExit or KeyboardInterrupt; this is almost always wrong. Often you want your except blocks to be as tightly scoped as you can, and the Python tutorial on user-defined exceptions is a helpful pattern.
(The exception to this, particularly in a Kubernetes context, is that you will often want a very broad exception handler to do something like return an HTTP 500 error to a network request, rather than crashing the application.)
A better example might look like:
import time
class OneException(Exception):
pass
def iteration():
for i in range(10):
try:
if i == 1:
raise OneException("it is one")
print(i, math.sqrt(i), math.sqrt(-i))
# will work when i==0 but fail when i==2
except OneException as e:
print(i, repr(e))
# and proceed to the next iteration
if __name__ == '__main__':
while True:
# The top-level loop. We want a very broad catch here.
try:
iteration()
except Exception as e:
print('iteration failed', repr(e))
time.sleep(1)
After a server upgrade, I'm having an issue with a python script eating up all the server connections because it appears that after a timeout it doesn't actually end the loop. The code looks like:
if os.name == 'posix':
signal.signal(signal.SIGALRM, self.handle_timeout)
signal.alarm(__TIMEOUT__)
try:
self.inputline = self.rfile.readline()
except IOError:
continue
if os.name == 'posix':
signal.alarm(0)
The signal occurs and all that does is set a terminated variable to 1 and print a log about the time out. The while looks like while not self.terminated:. My guess is that because it has except IOError: the except doesn't occur and it's still sitting on the readline(). So the question is, what is the proper way to ensure the SIGALRM will cause the continue which will end the loop and exit the script?
TIA!!
In handle_timeout() you can raise an exception to get your try to wake up. Then catch that exception in your try/except:
def handle_timeout(signum, frame):
raise KeyboardInterrupt('received signal to exit')
while not terminated:
try:
self.inputline = self.rfile.readline()
except IOError, KeyboardInterrupt:
terminated = True
continue
I don't have enough reputation yet to comment on another post so I'll ask it here. I am getting the "ConnectionError 10054 An existing connection was forcibly closed by the remote host" while doing a 'requests.get'. Now, I read this code on the other post but not sure if it is complete; can you please edit it so that it keeps retrying my 'requests.get' when the error comes up then exit of course when my 'requests.get' is successful. Thank you
import socket
retry_count = 5 # this is configured somewhere
for retries in range(retry_count):
try:
data1 = requests.get(url)
return True
except (error_reply, error_perm, error_temp):
return False
except socket.gaierror, e:
if e.errno != 10054:
return False
reconnect()
return False
Getting error;
return True
^
SyntaxError: 'return' outside function
The reason why you are having an error is simple. You haven't made a function, so you can't call return. The simplest way to fix this is using a break which allows you to jump out of the loop. Here's an example of using break in your code.
import socket
import requests
retry_count = 5
for retries in range(retry_count):
try:
data1 = requests.get(url)
#Jumps Out Of Loop
break
except (socket.gaierror, requests.ConnectionError) as e:
if e.errno != 10054:
continue
reconnect()
#Does Something If Loop Never Breaks
else:
print("Couldn't Connect")
I have the following code that is part of my email class that I use in my programs. Currently I am running the quit function whether or not a connection to the SMTP server was made in the connect function. I know I could put the quit function inside of the try statement after the email is sent, but I would like to figure out how to write the code to say the equivalent of "if a connection to the server is open, close it." What is the best way to write that in Python?
Thanks!
def connect(self, headers, msg):
try:
self.server.starttls()
try:
self.server.login(self.usrname,self.pswd)
try:
self.server.sendmail(self.sendfrom, self.sendto, headers + "\r\n\r\n" + msg)
except Exception as sendmailfail:
print(sendmailfail)
except Exception as emailfail:
print (emailfail)
except Exception as error:
print(error)
def quit(self):
self.server.quit()
print("The SMTP connection is closed")
first = GmailSmpt('x','y','z','zz')
x , y = first.message()
first.connect(x,y)
first.quit()
You need to finish the "Errors and Exceptions" section of the tutorial.
try:
possibly_fail()
except ...:
handle_exception()
else:
no_exceptions()
finally:
always_run_this()