Develop Reference

Python wrong SSL version using requests

I have been struggling with this problem since I installed python on a new PC.
Basically, I have the https request:
import requests
import ssl
proxies = {
'https': "https://myproxyhere"
}
r = requests.get('https://example.com', proxies=proxies, timeout=10)
print(r.text)
On the other PC it works fine, but on this one it gives that error:
```(Caused by SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1122)')))```
I had openssl 1.1.1h installed and python 3.9 had openssl version 1.1.1g. However, I downgraded my openssl to 1.1.1g, and still the error persists, I'm not able to do SSL requests, using proxies.
Keep in mind that the proxies are not the cause, as i have the same exact piece of code working fine on my other laptop, however i bought a Windows Surface Pro and when i did this fresh install, im not able to make this work. Ive tried everything so far, from trying other versions of openssl and python.
Does someone have a clue on how to fix this?
Full Error Traceback:
Traceback (most recent call last):
File "C:\Users\gonca\Desktop\debug.py", line 9, in <module>
r = requests.get('https://example.com', proxies=proxies, timeout=10)
File "C:\Users\gonca\AppData\Local\Programs\Python\Python39\lib\site-packages\requests\api.py", line 76, in get
return request('get', url, params=params, **kwargs)
File "C:\Users\gonca\AppData\Local\Programs\Python\Python39\lib\site-packages\requests\api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "C:\Users\gonca\AppData\Local\Programs\Python\Python39\lib\site-packages\requests\sessions.py", line 542, in request
resp = self.send(prep, **send_kwargs)
File "C:\Users\gonca\AppData\Local\Programs\Python\Python39\lib\site-packages\requests\sessions.py", line 655, in send
r = adapter.send(request, **kwargs)
File "C:\Users\gonca\AppData\Local\Programs\Python\Python39\lib\site-packages\requests\adapters.py", line 514, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='example.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1122)')))
Edit:
For people that face this error, the solution was down-grading the requests module, it was bugged.

There is a bug in requests-2.25.1. I have to downgrade to 2.24.0
pip install requests==2.24.0

Python requests - Attempt to get presto documentation returns sslv3 alert handshake failure

I'm trying to get the content of Presto public documentation for some analysis using Python requests:
import requests
requests.get('https://prestodb.io/docs/current/functions/logical.html')
This returns the following error:
File "/Users/my_user/PycharmProjects/untitled/test.py", line 3, in <module>
requests.get('https://prestodb.io/docs/current/functions/logical.html')
File "/Library/Python/2.7/site-packages/requests/api.py", line 71, in get
return request('get', url, params=params, **kwargs)
File "/Library/Python/2.7/site-packages/requests/api.py", line 57, in request
return session.request(method=method, url=url, **kwargs)
File "/Library/Python/2.7/site-packages/requests/sessions.py", line 475, in request
resp = self.send(prep, **send_kwargs)
File "/Library/Python/2.7/site-packages/requests/sessions.py", line 585, in send
r = adapter.send(request, **kwargs)
File "/Library/Python/2.7/site-packages/requests/adapters.py", line 477, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: ("bad handshake: Error([('SSL routines', 'SSL23_GET_SERVER_HELLO', 'sslv3 alert handshake failure')],)",)
Tried diagnose using openssl and I found that the site requires Server Name Indication:
openssl s_client -connect prestodb.io:443 -servername prestodb.io // works
openssl s_client -connect prestodb.io:443 // doesn't work
CONNECTED(00000003)
140735204868176:error:14077438:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert internal error:s23_clnt.c:769:
I have done a lot of search and did try something else:
I have tried using a custom HTTPAdapter as described here to specify the protocol to be TLSv1 but that doesn't help.
I have also tried directly use SSL context and connection objects and set_tlsext_host_name but it doesn't help either.
It seems to me the requirement is to use TLSv1 and SNI. But I'm not able to get that through requests. Could someone please help me solve this issue? Thanks in advance.

Thanks to Lukas Graf and I resolved this based on https://github.com/kennethreitz/requests/issues/2022. What I actually did is to install OpenSSL from Homebrew, then install a new version of Python 2 from Homebrew which will automatically link against the Homebrew-provided OpenSSL:
brew install openssl
brew install python
Using the newly installed python I'm able to get the presto document page.

Robot Framework test scripts fail with SSLError

I have written Robot Framework test scripts in .tsv format to test web-services/APIs. Everything was working fine until today (probably because of the new updates of Robot Framework) when I started to get the following error:
SSLError: ("bad handshake: SysCallError(-1, 'Unexpected EOF')",)
This error keeps popping up for the following code in a test script:
${headers}= Create Dictionary Content-Type application/json Accept application/json
RequestsKeywords.Get Request httpbin ${url} headers=${headers} //ERROR SHOWS FOR THIS STATEMENT
I did get a detailed traceback for this error which is as follows:
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/RequestsLibrary/RequestsKeywords.py", line 298, in get_request
session, uri, params, headers, redir, timeout)
File "/Library/Python/2.7/site-packages/RequestsLibrary/RequestsKeywords.py", line 801, in _get_request
cookies=self.cookies)
File "/Library/Python/2.7/site-packages/requests/sessions.py", line 480, in get
return self.request('GET', url, **kwargs)
File "/Library/Python/2.7/site-packages/requests/sessions.py", line 468, in request
resp = self.send(prep, **send_kwargs)
File "/Library/Python/2.7/site-packages/requests/sessions.py", line 576, in send
r = adapter.send(request, **kwargs)
File "/Library/Python/2.7/site-packages/requests/adapters.py", line 447, in send
raise SSLError(e, request=request)
My system configuration:
Mac OS X (10.11.3)
Python (2.7.10)
openssl (1.0.2f)
requests (2.9.1)
robotframework (3.0)
robotframework-httplibrary (0.4.2)
robotframework-requests (0.4.4)
robotframework-ride (1.5.2.1)
robotframework-sshlibrary (2.1.2)
pyOpenSSL (0.15.1)
How do I resolve this issue?

You're all up to date, so there are two possibilities:
The handshake isn't going smoothly because of a break in the trust chain. Start from the bottom and work up. Are you testing this on a local area network? Is the certificate up to date? Can you access the site from your location and others without an error? SSL labs showing anything up? What about firewall rules?
There's a bug in the recent update. OpenSSL reports in my experience are caused by connection issues rather than obscure software problems - this is the less likely of the two.

Try installing requests[security] instead of requests. It uses PyOpenSSL, which is better than OpenSSL: pip install requests[security] vs pip install requests: Difference

Why pip doesn't work in virtual enviroment?

I am trying to use pip from virtual enviroment on Linux Ubuntu 14.04 x64.
While using pip from shell as:
pip search bla
it works fine.
But using from virtualenv it shows error:
(proj_testing)uxu#box:~$ pip search bla
/home/uxu/.virtualenvs/proj_testing/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
/home/uxu/.virtualenvs/proj_testing/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:90: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Exception:
Traceback (most recent call last):
File "/home/uxu/.virtualenvs/proj_testing/local/lib/python2.7/site-packages/pip/basecommand.py", line 223, in main
status = self.run(options, args)
File "/home/uxu/.virtualenvs/proj_testing/local/lib/python2.7/site-packages/pip/commands/search.py", line 43, in run
pypi_hits = self.search(query, options)
File "/home/uxu/.virtualenvs/proj_testing/local/lib/python2.7/site-packages/pip/commands/search.py", line 60, in search
hits = pypi.search({'name': query, 'summary': query}, 'or')
File "/usr/lib/python2.7/xmlrpclib.py", line 1233, in __call__
return self.__send(self.__name, args)
File "/usr/lib/python2.7/xmlrpclib.py", line 1587, in __request
verbose=self.__verbose
File "/home/uxu/.virtualenvs/proj_testing/local/lib/python2.7/site-packages/pip/download.py", line 785, in request
headers=headers, stream=True)
File "/home/uxu/.virtualenvs/proj_testing/local/lib/python2.7/site-packages/pip/_vendor/requests/sessions.py", line 508, in post
return self.request('POST', url, data=data, json=json, **kwargs)
File "/home/uxu/.virtualenvs/proj_testing/local/lib/python2.7/site-packages/pip/download.py", line 373, in request
return super(PipSession, self).request(method, url, *args, **kwargs)
File "/home/uxu/.virtualenvs/proj_testing/local/lib/python2.7/site-packages/pip/_vendor/requests/sessions.py", line 465, in request
resp = self.send(prep, **send_kwargs)
File "/home/uxu/.virtualenvs/proj_testing/local/lib/python2.7/site-packages/pip/_vendor/requests/sessions.py", line 573, in send
r = adapter.send(request, **kwargs)
File "/home/uxu/.virtualenvs/proj_testing/local/lib/python2.7/site-packages/pip/_vendor/cachecontrol/adapter.py", line 46, in send
resp = super(CacheControlAdapter, self).send(request, **kw)
File "/home/uxu/.virtualenvs/proj_testing/local/lib/python2.7/site-packages/pip/_vendor/requests/adapters.py", line 431, in send
raise SSLError(e, request=request)
SSLError: [Errno 1] _ssl.c:510: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
It doesn't make sense, anyone got idea?

Try execute pip search bla like superuser
For ubuntu the command is:
sudo pip search bla

From https://urllib3.readthedocs.org/en/latest/security.html:
Certain Python platforms (specifically, versions of Python earlier than 2.7.9) have restrictions in their ssl module that limit the configuration that urllib3 can apply. In particular, this can cause HTTPS requests that would succeed on more featureful platforms to fail, and can cause certain security features to be unavailable. If you encounter this warning, it is strongly recommended you upgrade to a newer Python version, or that you use pyOpenSSL as described in the OpenSSL / PyOpenSSL section.
You probably have a less recent version of python in your virtual environment. Perhaps this is why this error only occurs in the virtual environment.

requests package raises SSLError when connecting to github

I want to access https://api.github.com/ with the requests package. I tried requests.get('https://api.github.com/'), but got the following SSL error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/requests/api.py", line 55, in get
return request('get', url, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/requests/api.py", line 44, in request
return session.request(method=method, url=url, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/requests/sessions.py", line 456, in request
resp = self.send(prep, **send_kwargs)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/requests/sessions.py", line 559, in send
r = adapter.send(request, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/requests/adapters.py", line 382, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: [Errno 1] _ssl.c:507: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
I'm using OSX Yosemite Beta 3, python 2.7.6 and requests 2.3.0.
I know that I could pass verify=False to get to ignore the error. However, I can't do that because I can't modify the code that requests is being called from.
Using openssl, openssl s_client -connect github.com:443, does not seem to cause a problem.
Should requests work for SSL no matter what? Is my operating system causing the issue? How do I make requests trust GitHub's certificate?

Requests hard codes the default list of trusted certificates. It doesn't matter what your os trusts, only what requests trusts. If the site you're trying to access is not signed by one of the defaults, you need to tell requests what to trust manually.
You can tell requests what to trust by passing verify=<some file>, or by setting the env var REQUESTS_CA_BUNDLE=<some file> if you do not have access to the library code.
So call your program with REQUESTS_CA_BUNDLE=my_verify.pem python my_program.py
or some other way to set the env var.
If you're trying to access, for example, https://api.github.com/ and you can reach it with curl but not requests, you can pass requests the bundle used by your os. On most Linux systems it is found at /etc/ssl/certs/ca-certificates.crt. Of you can pass it a file containing just GitHub's public cert.
To get the cert for a given server use the command openssl s_client -showcerts -connect api.github.com:443 < /dev/null | openssl x509 -outform PEM > github.crt. See https://stackoverflow.com/a/16797458/400617.
Note that although pip uses requests, it also hard codes the list of trusted defaults. To override pip, you need to set the env var PIP_CERT=<some file>. So be aware that whatever library you're using might have it's own way to override this.