I have been struggling with this problem since I installed python on a new PC.
Basically, I have the https request:
import requests
import ssl
proxies = {
'https': "https://myproxyhere"
}
r = requests.get('https://example.com', proxies=proxies, timeout=10)
print(r.text)
On the other PC it works fine, but on this one it gives that error:
```(Caused by SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1122)')))```
I had openssl 1.1.1h installed and python 3.9 had openssl version 1.1.1g. However, I downgraded my openssl to 1.1.1g, and still the error persists, I'm not able to do SSL requests, using proxies.
Keep in mind that the proxies are not the cause, as i have the same exact piece of code working fine on my other laptop, however i bought a Windows Surface Pro and when i did this fresh install, im not able to make this work. Ive tried everything so far, from trying other versions of openssl and python.
Does someone have a clue on how to fix this?
Full Error Traceback:
Traceback (most recent call last):
File "C:\Users\gonca\Desktop\debug.py", line 9, in <module>
r = requests.get('https://example.com', proxies=proxies, timeout=10)
File "C:\Users\gonca\AppData\Local\Programs\Python\Python39\lib\site-packages\requests\api.py", line 76, in get
return request('get', url, params=params, **kwargs)
File "C:\Users\gonca\AppData\Local\Programs\Python\Python39\lib\site-packages\requests\api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "C:\Users\gonca\AppData\Local\Programs\Python\Python39\lib\site-packages\requests\sessions.py", line 542, in request
resp = self.send(prep, **send_kwargs)
File "C:\Users\gonca\AppData\Local\Programs\Python\Python39\lib\site-packages\requests\sessions.py", line 655, in send
r = adapter.send(request, **kwargs)
File "C:\Users\gonca\AppData\Local\Programs\Python\Python39\lib\site-packages\requests\adapters.py", line 514, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='example.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1122)')))
Edit:
For people that face this error, the solution was down-grading the requests module, it was bugged.
There is a bug in requests-2.25.1. I have to downgrade to 2.24.0
pip install requests==2.24.0
Related
I am trying to use and install a python library according to the instructions at: https://pypi.org/project/ai4bharat-transliteration/. My system is behind a corporate proxy and I am able to use pip and other libraries including urllib3 to access the internet when I am writing code from scratch.
However, in this case, the library wants to access some files over internet when running:
`e = XlitEngine("hi", beam_width=10, rescore=True)`
And that results in a wall of urllib3 proxy related errors, the first of which is:
Downloading Multilingual model for transliteration
SSL certificate not verified...
/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py:1045: InsecureRequestWarning: Unverified HTTPS request is being made to host '172.xx.yy.zzz'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#ssl-warnings
warnings.warn(
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 700, in urlopen
self._prepare_proxy(conn)
File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 996, in _prepare_proxy
conn.connect()
File "/usr/local/lib/python3.8/dist-packages/urllib3/connection.py", line 369, in connect
self._tunnel()
File "/usr/lib/python3.8/http/client.py", line 901, in _tunnel
(version, code, message) = response._read_status()
File "/usr/lib/python3.8/http/client.py", line 277, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File "/usr/lib/python3.8/socket.py", line 669, in readinto
return self._sock.recv_into(b)
socket.timeout: timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/requests/adapters.py", line 439, in send
resp = conn.urlopen(
File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 787, in urlopen
retries = retries.increment(
File "/usr/local/lib/python3.8/dist-packages/urllib3/util/retry.py", line 592, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='objects.githubusercontent.com', port=443): Max retries exceeded with url: /github-production-release-asset-2e65be/487173539/4ef3b62d-385b-4a3a-9ab1-a3cc55764ef3?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20221220%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20221220T101158Z&X-Amz-Expires=300&X-Amz-Signature=d24db49d92188df3dbf8a0f1a05126bdaae8bf42289befe734331a41b336f11c&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=487173539&response-content-disposition=attachment%3B%20filename%3Dindicxlit-en-indic-v1.0.zip&response-content-type=application%2Foctet-stream (Caused by ProxyError('Cannot connect to proxy.', timeout('timed out')))
Where 172.xx.yy.zzz is the url to my proxy and I am guessing that the warning regarding the SSL certificate is just a warning and the problem is the proxy configuration of urllib3.
If so, is there a way to set a universal proxy which will be honored by urllib3 before the library is called by the XlitEngine package installed above. I am reluctant to attempt any changes to the XlitEngine package installed above. I tried posting the issue on Github for XlitEngine but have not received any response so far.
If it is of any consequence, I am using Python 3.8.10 on a headless Ubuntu 20.04 Server Virtual Machine.
Cheers!
I'm trying to use Python requests to access a URL from https://dadosabertos.bndes.gov.br, but it fails in CentOS. It works fine in Windows.
Here is the error:
>>> import requests
>>> requests.__version__
'2.26.0'
>>> requests.get('https://dadosabertos.bndes.gov.br')Traceback (most recent call last): File "/opt/python3/lib64/python3.6/site-packages/urllib3/connectionpool.py", line 696, in urlopen
self._prepare_proxy(conn)
File "/opt/python3/lib64/python3.6/site-packages/urllib3/connectionpool.py", line 964, in _prepare_proxy
conn.connect()
File "/opt/python3/lib64/python3.6/site-packages/urllib3/connection.py", line 426, in connect
tls_in_tls=tls_in_tls,
File "/opt/python3/lib64/python3.6/site-packages/urllib3/util/ssl_.py", line 450, in ssl_wrap_socket
sock, context, tls_in_tls, server_hostname=server_hostname
File "/opt/python3/lib64/python3.6/site-packages/urllib3/util/ssl_.py", line 493, in _ssl_wrap_socket_impl
return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
File "/usr/lib64/python3.6/ssl.py", line 365, in wrap_socket
_context=self, _session=session)
File "/usr/lib64/python3.6/ssl.py", line 776, in __init__
self.do_handshake()
File "/usr/lib64/python3.6/ssl.py", line 1036, in do_handshake
self._sslobj.do_handshake()
File "/usr/lib64/python3.6/ssl.py", line 648, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:897)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/python3/lib64/python3.6/site-packages/requests/adapters.py", line 449, in send
timeout=timeout
File "/opt/python3/lib64/python3.6/site-packages/urllib3/connectionpool.py", line 756, in urlopen
method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
File "/opt/python3/lib64/python3.6/site-packages/urllib3/util/retry.py", line 574, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='dadosabertos.bndes.gov.br', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:897)'),))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/opt/python3/lib64/python3.6/site-packages/requests/api.py", line 75, in get
return request('get', url, params=params, **kwargs)
File "/opt/python3/lib64/python3.6/site-packages/requests/api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "/opt/python3/lib64/python3.6/site-packages/requests/sessions.py", line 542, in request
resp = self.send(prep, **send_kwargs)
File "/opt/python3/lib64/python3.6/site-packages/requests/sessions.py", line 655, in send
r = adapter.send(request, **kwargs)
File "/opt/python3/lib64/python3.6/site-packages/requests/adapters.py", line 514, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='dadosabertos.bndes.gov.br', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:897)'),))
My CentOS version: CentOS Linux release 8.4.2105. It works in Windows 10. I'm using requests lib version 2.26.0.
I tried to download the certificate from the site and validate with it using this command:
requests.get('https://dadosabertos.bndes.gov.br', verify=True,
cert='./bndes-gov-br.pem')
but got a similar exception. Here is the stack trace:
>>> requests.get('https://dadosabertos.bndes.gov.br', verify=True, cert='./bndes-gov-br.pem')
Traceback (most recent call last):
File "/home/xxxxx/lib/python3.7/site-packages/urllib3/connectionpool.py", line 594, in urlopen
self._prepare_proxy(conn)
File "/home/xxxxx/lib/python3.7/site-packages/urllib3/connectionpool.py", line 805, in _prepare_proxy
conn.connect()
File "/home/xxxxx/lib/python3.7/site-packages/urllib3/connection.py", line 344, in connect
ssl_context=context)
File "/home/xxxxx/lib/python3.7/site-packages/urllib3/util/ssl_.py", line 338, in ssl_wrap_socket
context.load_cert_chain(certfile, keyfile)
ssl.SSLError: [SSL] PEM lib (_ssl.c:3854)
Note that this error happens also in Python 3.7.
I exported the certificate following these instructions
Initially I tried to configure my machine global certificates, but it looks like Python and Requests lib uses its own. Another question gave me a lot of valuable info to configure my certificate.
Since I couldn't make requests lib use my certificate, I believe there is an error in the downloaded certificate or in the validation lib.
Here is its contents of my bndes-gov-br.pem file downloaded using the browser (I got the same error trying with the complete certificate chain):
-----BEGIN CERTIFICATE-----
MIIGjzCCBXegAwIBAgIMdIDfTRbWNDjcygdHMA0GCSqGSIb3DQEBCwUAMFAxCzAJ
BgNVBAYTAkJFMRkwFwYDVQQKExBHbG9iYWxTaWduIG52LXNhMSYwJAYDVQQDEx1H
bG9iYWxTaWduIFJTQSBPViBTU0wgQ0EgMjAxODAeFw0yMDAyMTMxNzM3MDBaFw0y
MjAyMTMxNzM3MDBaMIGlMQswCQYDVQQGEwJCUjEXMBUGA1UECBMOUmlvIGRlIEph
bmVpcm8xFzAVBgNVBAcTDlJpbyBkZSBKYW5laXJvMQwwCgYDVQQLEwNBVEkxPTA7
BgNVBAoTNEJhbmNvIE5hY2lvbmFsIGRlIERlc2Vudm9sdmltZW50byBFY29ub21p
Y28gZSBTb2NpYWwxFzAVBgNVBAMMDiouYm5kZXMuZ292LmJyMIIBIjANBgkqhkiG
9w0BAQEFAAOCAQ8AMIIBCgKCAQEAsqNBHzLfEWeYk5cxF+hT3ZV9Ki6u7WjGXOx4
c6HMB7tDrbyp8wbmaJPNo8yWDAJ0eL4N+QVJ6IG2rJ7DLU65+76qcv8iLG5OcsnZ
K9o1NfnEaNWIy8Vf0edO7bkalXD8YYf5QQMSZ+TqPIA3cJnFKibNTbqaBRbjvwF9
QBaCATZnl0xg3/kD2Wdjtzdrg0JXBcRcrDeQOV/22/O2JMjbjRpoMeuqR9O8OwfE
JTT3tJxTE6LWKSIZR8nc+rMLW4sqw+QZPGMdS85m9eStUrHxQUHEBpScAPN9fN4c
u2L0U51nedZgfHEfqyjYVCOY0zoVEv5MW0UV5+mbObcy2v/d5QIDAQABo4IDETCC
Aw0wDgYDVR0PAQH/BAQDAgWgMIGOBggrBgEFBQcBAQSBgTB/MEQGCCsGAQUFBzAC
hjhodHRwOi8vc2VjdXJlLmdsb2JhbHNpZ24uY29tL2NhY2VydC9nc3JzYW92c3Ns
Y2EyMDE4LmNydDA3BggrBgEFBQcwAYYraHR0cDovL29jc3AuZ2xvYmFsc2lnbi5j
b20vZ3Nyc2FvdnNzbGNhMjAxODBWBgNVHSAETzBNMEEGCSsGAQQBoDIBFDA0MDIG
CCsGAQUFBwIBFiZodHRwczovL3d3dy5nbG9iYWxzaWduLmNvbS9yZXBvc2l0b3J5
LzAIBgZngQwBAgIwCQYDVR0TBAIwADAnBgNVHREEIDAegg4qLmJuZGVzLmdvdi5i
coIMYm5kZXMuZ292LmJyMB0GA1UdJQQWMBQGCCsGAQUFBwMBBggrBgEFBQcDAjAf
BgNVHSMEGDAWgBT473/yzXhnqN5vjySNiPGHAwKz6zAdBgNVHQ4EFgQUr8ZwKoFq
XqEty6FDsn6fsqeGm+kwggF9BgorBgEEAdZ5AgQCBIIBbQSCAWkBZwB1AKS5CZC0
GFgUh7sTosxncAo8NZgE+RvfuON3zQ7IDdwQAAABcD+gIlMAAAQDAEYwRAIgR/oA
SKJK0xqLbAJGCVnSP5IyLeXHkEYA9XsQGsISa3kCIBwZ4jMOyZYdZD7WzRF7Zq9G
/xxH9V8NzJcu5Sn6iKo5AHYAb1N2rDHwMRnYmQCkURX/dxUcEdkCwQApBo2yCJo3
2RMAAAFwP6AimgAABAMARzBFAiEAwo4mDeGUqOCWdgHBoPsjgq4RnjA2e/o4tSpb
dLWIzYUCIHUhbmk9jH8kx0W0t5SOLI/tBAJRyWlaC3GEAUSh5sW4AHYAVYHUwhaQ
NgFK6gubVzxT8MDkOHhwJQgXL6OqHQcT0wwAAAFwP6AiegAABAMARzBFAiEA+Lku
wDF2G9QAVuCSd85xFUkoAV8MO0Cv2nle4ZbzgeECIE6SdOMLinYiX4YUZzl/jzql
ZT3/XeNQ4XvCO5Fa7i9tMA0GCSqGSIb3DQEBCwUAA4IBAQBTn7kU8YF+N0uWrUJj
89vrq2OSXI8ShkimdziYNmciH9+Qvle1X/utcfng8SGa0xiSAcNSlEYRskq6D3pv
uSkXRO/9/r5+7WNRYE4wb/b1AbMQYINPqEd6SXW139Em7WPrq5M8nzzAXZ7Qy+ii
7cq4K7E0VPMCDsK948iUf+Nr7BBNlaD5J5/cWPm1p/EHi6pG6RUdTWTLnPjt40G9
6K7HivIvGkMq7HcEs2An+Y9yTmjzV1YhCIV/BzuFbc97z8vpfeF738K9N6bPkbFt
CcjkGVLQHiw0sld6uL75u+Z4gq8JFRd1OJFYT2EgJQFpl3zFQBVVuBMQivM9/QHO
xY6d
-----END CERTIFICATE-----
How do I configure Python 3.6 in CentOS so it access files in https://dadosabertos.bndes.gov.br without turning off SSL?
As specified in the document: https://docs.python-requests.org/en/latest/api/, the cert option is used to specify the client cert instead of server cert. Client cert is what you (as client) provides to the web server, so it believes who you are, thus it's not what you need. Instead, server cert is what https://dadosabertos.bndes.gov.br provides for you so you know that you are talking to the real website. It's provided by the server during the SSL handshake so you don't need to mannually speicify it. Question here is that requests fail to validate the server cert.
I try to reproduce your result in Docker centos environment but it works without any problems. requests use root certificates provided by the certifi package. It's possible that your certifi package is out-of-data. So I guess you may uninstall certifi and requests package and reinstall to get a latest copy of trusted root certificates.
My requests and dependencies versions:
certifi-2021.10.8
charset-normalizer-2.0.7
idna-3.3
requests-2.26.0
urllib3-1.26.7
In Python using MacOS X, made an attempt to make a POST request to a website but I got the following error post_response = session.post(post_url, data=post_payload, headers=post_headers):
Traceback (most recent call last):
File "web_requests.py", line 424, in <module>
main()
File "web_requests.py", line 340, in main
post_response = session.post(post_url, data=post_payload, headers=post_headers)
File "/Library/Python/2.7/site-packages/requests/sessions.py", line 535, in post
return self.request('POST', url, data=data, json=json, **kwargs)
File "/Library/Python/2.7/site-packages/requests/sessions.py", line 488, in request
resp = self.send(prep, **send_kwargs)
File "/Library/Python/2.7/site-packages/requests/sessions.py", line 609, in send
r = adapter.send(request, **kwargs)
File "/Library/Python/2.7/site-packages/requests/adapters.py", line 497, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:590)
What could be the issue?
Thank you in advance and will be sure to upvote/accept answer
EDIT
Specific to POST requests in Python
According to the addititional information in the comment SSLLabs reports for this site that it support TLS 1.2 only:
TLS 1.2 Yes
TLS 1.1 No
TLS 1.0 No
SSL 3 No
And according to another information in the comments the OpenSSL version is 0.9.8:
.. OpenSSL 0.9.8zg 14 July 2015
Since OpenSSL 0.9.8 only support up to TLS 1.0 and especially not TLS 1.2 there is no common TLS protocol spoken between the client and the server. Thus, the handshake fails.
The way to fix the problem is to upgrade the version of OpenSSL as used by Python away from the very old version to at least OpenSSL 1.0.1. See for example Updating openssl in python 2.7 for more information or use Anaconda Python which comes preinstalled with a lot of modules and also includes a current version of OpenSSL.
I'm trying to get the content of Presto public documentation for some analysis using Python requests:
import requests
requests.get('https://prestodb.io/docs/current/functions/logical.html')
This returns the following error:
File "/Users/my_user/PycharmProjects/untitled/test.py", line 3, in <module>
requests.get('https://prestodb.io/docs/current/functions/logical.html')
File "/Library/Python/2.7/site-packages/requests/api.py", line 71, in get
return request('get', url, params=params, **kwargs)
File "/Library/Python/2.7/site-packages/requests/api.py", line 57, in request
return session.request(method=method, url=url, **kwargs)
File "/Library/Python/2.7/site-packages/requests/sessions.py", line 475, in request
resp = self.send(prep, **send_kwargs)
File "/Library/Python/2.7/site-packages/requests/sessions.py", line 585, in send
r = adapter.send(request, **kwargs)
File "/Library/Python/2.7/site-packages/requests/adapters.py", line 477, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: ("bad handshake: Error([('SSL routines', 'SSL23_GET_SERVER_HELLO', 'sslv3 alert handshake failure')],)",)
Tried diagnose using openssl and I found that the site requires Server Name Indication:
openssl s_client -connect prestodb.io:443 -servername prestodb.io // works
openssl s_client -connect prestodb.io:443 // doesn't work
CONNECTED(00000003)
140735204868176:error:14077438:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert internal error:s23_clnt.c:769:
I have done a lot of search and did try something else:
I have tried using a custom HTTPAdapter as described here to specify the protocol to be TLSv1 but that doesn't help.
I have also tried directly use SSL context and connection objects and set_tlsext_host_name but it doesn't help either.
It seems to me the requirement is to use TLSv1 and SNI. But I'm not able to get that through requests. Could someone please help me solve this issue? Thanks in advance.
Thanks to Lukas Graf and I resolved this based on https://github.com/kennethreitz/requests/issues/2022. What I actually did is to install OpenSSL from Homebrew, then install a new version of Python 2 from Homebrew which will automatically link against the Homebrew-provided OpenSSL:
brew install openssl
brew install python
Using the newly installed python I'm able to get the presto document page.
I have written Robot Framework test scripts in .tsv format to test web-services/APIs. Everything was working fine until today (probably because of the new updates of Robot Framework) when I started to get the following error:
SSLError: ("bad handshake: SysCallError(-1, 'Unexpected EOF')",)
This error keeps popping up for the following code in a test script:
${headers}= Create Dictionary Content-Type application/json Accept application/json
RequestsKeywords.Get Request httpbin ${url} headers=${headers} //ERROR SHOWS FOR THIS STATEMENT
I did get a detailed traceback for this error which is as follows:
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/RequestsLibrary/RequestsKeywords.py", line 298, in get_request
session, uri, params, headers, redir, timeout)
File "/Library/Python/2.7/site-packages/RequestsLibrary/RequestsKeywords.py", line 801, in _get_request
cookies=self.cookies)
File "/Library/Python/2.7/site-packages/requests/sessions.py", line 480, in get
return self.request('GET', url, **kwargs)
File "/Library/Python/2.7/site-packages/requests/sessions.py", line 468, in request
resp = self.send(prep, **send_kwargs)
File "/Library/Python/2.7/site-packages/requests/sessions.py", line 576, in send
r = adapter.send(request, **kwargs)
File "/Library/Python/2.7/site-packages/requests/adapters.py", line 447, in send
raise SSLError(e, request=request)
My system configuration:
Mac OS X (10.11.3)
Python (2.7.10)
openssl (1.0.2f)
requests (2.9.1)
robotframework (3.0)
robotframework-httplibrary (0.4.2)
robotframework-requests (0.4.4)
robotframework-ride (1.5.2.1)
robotframework-sshlibrary (2.1.2)
pyOpenSSL (0.15.1)
How do I resolve this issue?
You're all up to date, so there are two possibilities:
The handshake isn't going smoothly because of a break in the trust chain. Start from the bottom and work up. Are you testing this on a local area network? Is the certificate up to date? Can you access the site from your location and others without an error? SSL labs showing anything up? What about firewall rules?
There's a bug in the recent update. OpenSSL reports in my experience are caused by connection issues rather than obscure software problems - this is the less likely of the two.
Try installing requests[security] instead of requests. It uses PyOpenSSL, which is better than OpenSSL: pip install requests[security] vs pip install requests: Difference