Develop Reference

What is the correct way to attach messages to an error?

I have a project which is structured something like this, with multiple functionality and often more possible sources of error. One functionality may also call something else that raises an error.
def functionality_one(arguments) -> str:
try:
status_feedback = attempt_functionality_one(arguments)
# this would usually be multiple lines
except ValueError as e:
return "known-failure-code"
except ConnectionError as e:
raise ConnectionError("Some user-friendly message for unexpected error") from e
else:
return status_feedback
def main():
## when the relevant CLI argument is passed:
try:
status = functionality_one(arguments)
except Exception as e:
send_notification_to_user(e.args[0])
else:
send_notification_to_user(USER_FRIENDLY_SUCCESS_MESSAGES.get(status, "Success!"))
if __name__ == "__main__":
main()
Focus on this bit about re-raising errors:
except ConnectionError as e:
raise ConnectionError("Some user-friendly message for unexpected error") from e
I do this to attach a user-friendly message in the error that I can later display to the user. Is there a better way to accomplish this?
In particular, normally error tracebacks just state errors that propogate. With this method, it gives a message like "... was the direct cause of the following exception ..." and I don't know whether this is the norm in Python. Here's an example from the log file:
Traceback (most recent call last):
File "D:\username\Documents\tech-projects\project-name\src\auth.py", line 157, in login
login_request = post(
File "D:\username\Documents\tech-projects\project-name\.venv\lib\site-packages\requests\api.py", line 115, in post
return request("post", url, data=data, json=json, **kwargs)
File "D:\username\Documents\tech-projects\project-name\.venv\lib\site-packages\requests\api.py", line 59, in request
return session.request(method=method, url=url, **kwargs)
File "D:\username\Documents\tech-projects\project-name\.venv\lib\site-packages\requests\sessions.py", line 587, in request
resp = self.send(prep, **send_kwargs)
File "D:\username\Documents\tech-projects\project-name\.venv\lib\site-packages\requests\sessions.py", line 701, in send
r = adapter.send(request, **kwargs)
File "D:\username\Documents\tech-projects\project-name\.venv\lib\site-packages\requests\adapters.py", line 565, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='wifi-login.university-website.domain', port=80): Max retries exceeded with url: /cgi-bin/authlogin (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x000001FF681B27D0>: Failed to establish a new connection: [Errno 11001] getaddrinfo failed'))
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "D:\username\Documents\tech-projects\project-name\login_cli.py", line 269, in main
status_message: str = parsed_namespace.func(parsed_namespace)
File "D:\username\Documents\tech-projects\project-name\login_cli.py", line 197, in connect
return src.auth.login(credentials)
File "D:\username\Documents\tech-projects\project-name\src\auth.py", line 164, in login
raise ConnectionError(f"Server-side error. Contact IT support or wait until morning.") from e
requests.exceptions.ConnectionError: Server-side error. Contact IT support or wait until morning.
So what's the right way to do this? Feel free to suggest a change that completely changes the structure of the program too, if you feel that's necessary.

How to check the connectivity for a URL in python

My requirement is almost same as Requests — how to tell if you're getting a success message?
But I need to print error whenever I could not reach the URL..Here is my try..
# setting up the URL and checking the conection by printing the status
url = 'https://www.google.lk'
try:
page = requests.get(url)
print(page.status_code)
except requests.exceptions.HTTPError as err:
print("Error")
The issue is rather than printing just "Error" it prints a whole error msg as below.
Traceback (most recent call last):
File "testrun.py", line 22, in <module>
page = requests.get(url)
File "/root/anaconda3/envs/py36/lib/python3.6/site-packages/requests/api.py", line 76, in get
return request('get', url, params=params, **kwargs)
File "/root/anaconda3/envs/py36/lib/python3.6/site-packages/requests/api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "/root/anaconda3/envs/py36/lib/python3.6/site-packages/requests/sessions.py", line 530, in request
resp = self.send(prep, **send_kwargs)
File "/root/anaconda3/envs/py36/lib/python3.6/site-packages/requests/sessions.py", line 643, in send
r = adapter.send(request, **kwargs)
File "/root/anaconda3/envs/py36/lib/python3.6/site-packages/requests/adapters.py", line 516, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='learn.microsoft.com', port=443): Max retries exceeded with url: /en-us/microsoft-365/enterprise/urls-and-ip-address-ranges?view=o365-worldwide (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7ff91a543198>: Failed to establish a new connection: [Errno 110] Connection timed out',))
Can someone show me how should I modify my code to just print "Error" only if there is any issue? Then I can extend it to some other requirement.

You're not catching the correct exception.
import requests
url = 'https://www.googlggggggge.lk'
try:
page = requests.get(url)
print(page.status_code)
except (requests.exceptions.HTTPError, requests.exceptions.ConnectionError):
print("Error")
you can also do except Exception, however note that Exception is too broad and is not recommended in most cases since it traps all errors

You need to either use a general exception except or catch all exceptions that requests module might throw, e.g. except (requests.exceptions.HTTPError, requests.exceptions.ConnectionError).
For full list see: Correct way to try/except using Python requests module?

Handeling errors in Python requests [duplicate]

This question already has answers here:
Python requests.exceptions.SSLError: EOF occurred in violation of protocol
(9 answers)
Closed 1 year ago.
I am learning to use requests in Python and I need a way to get a meaningful output if the site does not exist at all.
I looked at this question, but it is unclear if the OP of the question actually wants to check if the site exists, or if it just returns an error. The problem with all of the answers that question is that if the site does not exist at all we cannot really use HTTP response headers, because no response is returned from a server that does not exist.
Here is an example.
If I use this code I will not get any errors because the site exists.
import requests
r = requests.get('https://duckduckgo.com')
However, if I enter a web page I know does not exist I will get an error
import requests
r = requests.get('https://thissitedoesnotexist.com')
if r.status_code == requests.codes.ok:
print('Site good')
else:
print('Site bad')
This error is super long and I would prefer to have a more meaningful and short error if the site does not exist.
Traceback (most recent call last):
File "C:\Users\ADMIN\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connectionpool.py", line 699, in urlopen
httplib_response = self._make_request(
File "C:\Users\ADMIN\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connectionpool.py", line 382, in _make_request
self._validate_conn(conn)
File "C:\Users\ADMIN\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connectionpool.py", line 1010, in _validate_conn
conn.connect()
File "C:\Users\ADMIN\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connection.py", line 416, in connect
self.sock = ssl_wrap_socket(
File "C:\Users\ADMIN\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\util\ssl_.py", line 449, in ssl_wrap_socket
ssl_sock = _ssl_wrap_socket_impl(
File "C:\Users\ADMIN\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\util\ssl_.py", line 493, in _ssl_wrap_socket_impl
return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
File "C:\Users\ADMIN\AppData\Local\Programs\Python\Python310\lib\ssl.py", line 512, in wrap_socket
return self.sslsocket_class._create(
File "C:\Users\ADMIN\AppData\Local\Programs\Python\Python310\lib\ssl.py", line 1070, in _create
self.do_handshake()
File "C:\Users\ADMIN\AppData\Local\Programs\Python\Python310\lib\ssl.py", line 1341, in do_handshake
self._sslobj.do_handshake()
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:997)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\ADMIN\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\adapters.py", line 439, in send
resp = conn.urlopen(
File "C:\Users\ADMIN\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\connectionpool.py", line 755, in urlopen
retries = retries.increment(
File "C:\Users\ADMIN\AppData\Local\Programs\Python\Python310\lib\site-packages\urllib3\util\retry.py", line 574, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='234876.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:997)')))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\ADMIN\Desktop\tetst.py", line 2, in <module>
r = requests.get('https://234876.com')
File "C:\Users\ADMIN\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\api.py", line 75, in get
return request('get', url, params=params, **kwargs)
File "C:\Users\ADMIN\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "C:\Users\ADMIN\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\sessions.py", line 542, in request
resp = self.send(prep, **send_kwargs)
File "C:\Users\ADMIN\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\sessions.py", line 655, in send
r = adapter.send(request, **kwargs)
File "C:\Users\ADMIN\AppData\Local\Programs\Python\Python310\lib\site-packages\requests\adapters.py", line 514, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='234876.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:997)')))
Is it possible to make a function that returns, for example print('The site probably does not exist') or at least does not give an EOF error?

Normally the desirable thing to do is trap Exceptions from requests
You also can use .raise_for_status() on the Response to get a meaningful Exception for non-OK requests
However, you want to watch out for where you want to handle an Exception
immediately? can your program handle it meaningfully or should it exit?
should the caller handle a specific Exception (such as requests.exceptions.Timeout) or a more general one?
do you have many functions which call each other? should any handle some subset of possible Exceptions? and which?
See Python Exception Hierarchy for how the first-party Exceptions inheritance structure
import sys
import requests
def some_function_which_makes_requests():
r = requests.get("https://example.com", timeout=(2,10))
r.raise_for_status() # raise for non-OK
return r.json() # interpret response via some method (for example as JSON)
def main():
...
try:
result_json = some_function_which_makes_requests
except requests.exceptions.Timeout:
print("WARNING: request timed out")
result_json = None # still effectively handled for later program?
except requests.exceptions.RequestException as ex:
sys.exit(f"something wrong with Request: {repr(ex)}")
except Exception:
sys.exit(f"something wrong around Request: {repr(ex)}")
# now you can use result_json

Did some more research and just learned that I need to use a Python Try Except as mentioned by #Anand Sowmithiran. Here is a video explaining it for beginners: https://www.youtube.com/watch?v=NIWwJbo-9_8
import requests
try:
r = requests.get("http://www.duckduckgo.com")
except requests.exceptions.ConnectionError:
print('\n\tSorry. There was a network problem getting the URL. Perhaps it does not exist?\n\tCheck the URL, DNS issues or if you are being rejected by the server.')
else:
print(r)

Connection Reset by Peer with Python module urllib

What I'm tryng to do is to download images through Google Custom Search API using Python. It's my first time and maybe I'm making some banal error catching exceptions with Python.
It all works fine even when I get into some error which is not 10054 Connection Reset By Peer. The code is something like this, I've just taken out the useless part:
try:
urllib.request.urlretrieve(myUrl['link'],str(count)+'.jpg')
except URLError as e:
print(e.reason)
Sometimes it happens that connection is reset by peer and the console shows this error.
urllib.request.urlretrieve(myUrl['link'],str(count)+'.jpg')
File "C:\Python33\lib\urllib\request.py", line 210, in urlretrieve
block = fp.read(bs)
File "C:\Python33\lib\http\client.py", line 503, in read
return super(HTTPResponse, self).read(amt)
File "C:\Python33\lib\http\client.py", line 542, in readinto
n = self.fp.readinto(b)
File "C:\Python33\lib\socket.py", line 297, in readinto
return self._sock.recv_into(b)
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host
Press any key to continue . . .
Now I'm not really interested in make that URL work, but I just want my loop to keep going and not to stop. How can I catch that exception?
Edit:
I also noticed that sometimes the code correctly catches the error and print(e.reason) correctly outputs [WinError 10054] without stopping the loop. That's very strange.

If you don't know the exact problem, you can catch all exceptions as so:
try:
urllib.request.urlretrieve(myUrl['link'],str(count)+'.jpg')
except URLError as e:
print(e.reason)
except KeyboardInterrupt as ki:
raise ki
except:
print("Unknown Error")

Having Trouble Raising Exception for urllib2.urlopen()

I'm using the following code and I can't figure out why it's not raising an exception when the urlopen() is failing..
In my particular case, I know why it's failing.. My url's don't have http:// in front of them... but I want to catch those cases so the script can continue to run my code rather than exiting.
req = urllib2.Request(link)
try:
url = urllib2.urlopen(req)
except urllib2.URLError, e:
print e.code
print e.read()
return False
and I'm getting..
Traceback (most recent call last):
File "./getURLs.py", line 141, in <module>
main()
File "./getURLs.py", line 82, in main
Process(args).get_children()
File "./getURLs.py", line 65, in get_children
self.get_links(link)
File "./getURLs.py", line 46, in get_links
data = urllib2.urlopen(req)
File "/usr/local/lib/python2.7/urllib2.py", line 126, in urlopen
return _opener.open(url, data, timeout)
File "/usr/local/lib/python2.7/urllib2.py", line 383, in open
protocol = req.get_type()
File "/usr/local/lib/python2.7/urllib2.py", line 244, in get_type
raise ValueError, "unknown url type: %s" % self.__original
ValueError: unknown url type: /
.
.
.
Solution
for anyone else interested in my particular solution.. I'm using the following to catch both exceptions.
req = urllib2.Request(link)
try:
url = urllib2.urlopen(req)
except (ValueError,urllib2.URLError) as e:
print e
return False

From what you've pasted, it looks like you're catching the wrong type of exception. The code should say
try:
url=urllib2.urlopen(req)
except ValueError:
# etc etc etc.
If it's critical that the entirety of your code run, you can also have a plain except: with an unspecified exception type, or even a finally. See: http://docs.python.org/tutorial/errors.html