I recently got an SSL certificate for my site:
https://ram.rachum.com/
It works great in browsers. But it fails for requests:
>>> import requests
>>> requests.get('https://ram.rachum.com')
Traceback (most recent call last):
File "<pyshell#1>", line 1, in <module>
requests.get('https://ram.rachum.com')
File "C:\Python27\lib\site-packages\requests\api.py", line 55, in get
return request('get', url, **kwargs)
File "C:\Python27\lib\site-packages\requests\api.py", line 44, in request
return session.request(method=method, url=url, **kwargs)
File "C:\Python27\lib\site-packages\requests\sessions.py", line 354, in request
resp = self.send(prep, **send_kwargs)
File "C:\Python27\lib\site-packages\requests\sessions.py", line 460, in send
r = adapter.send(request, **kwargs)
File "C:\Python27\lib\site-packages\requests\adapters.py", line 250, in send
raise SSLError(e)
SSLError: hostname 'ram.rachum.com' doesn't match either of '*.webfaction.com', 'webfaction.com'
Why? Why does requests look at the webfaction certificate rather than my own certificate, which is valid for ram.rachum.com?
You are using a requests library without support for SNI (server name indication), but you have multiple SSL certificates behind the same IP address which requires SNI. You can verify this with openssl s_client. Without given a name for SNI the server just gives the default certificate for this IP, which is *.webfaction.com:
openssl s_client -connect ram.rachum.com:443
...
0 ...CN=*.webfaction.com
But if you specify a hostname for SNI it returns the expected certificate:
openssl s_client -connect ram.rachum.com:443 -servername ram.rachum.com
...
0 ...CN=ram.rachum.com...
Maybe you need to upgrade your requests library and other modules too, see using requests with TLS doesn't give SNI support
Related
I would like to authenticate to server from my client using certificate that is generated from server.I have a server-ca.crt and below is the CURL command that is working.How to send similar request using python requests module .
$ curl -X GET -u sat_username:sat_password \
-H "Accept:application/json" --cacert katello-server-ca.crt \
https://satellite6.example.com/katello/api/organizations
I have tried following way and it is getting some exception, can someone help in resolving this issue.
python requestsCert.py
Traceback (most recent call last):
File "requestsCert.py", line 2, in <module>
res=requests.get('https://satellite6.example.com/katello/api/organizations', cert='/certificateTests/katello-server-ca.crt', verify=True)
File "/usr/lib/python2.7/site-packages/requests/api.py", line 68, in get
return request('get', url, **kwargs)
File "/usr/lib/python2.7/site-packages/requests/api.py", line 50, in request
response = session.request(method=method, url=url, **kwargs)
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 464, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 576, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 431, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: [SSL] PEM lib (_ssl.c:2554)
res=requests.get('https://...', cert='/certificateTests/katello-server-ca.crt', verify=True)
The cert argument in requests.get is used to specify the client certificate and key which should be used for mutual authentication. It is not used to specify the trusted CA as the --cacert argument in curl does. Instead you should use the verify argument:
res=requests.get('https://...', verify='/certificateTests/katello-server-ca.crt')
For more information see SSL Cert Verification and Client Side Certificates in the documentation for requests.
Here iam using a request module to download the api information from github and code is shown below.
# Creation of Github request
# Import requests
import requests
r = requests.get('https://api.github.com/user', auth=('user','pass'))
print(r.status_code)
print(r.headers['content-type'])
print(r.encoding)
print(r.text)
print(r.json())
While using this module there is a error
python github.py
Traceback (most recent call last):
File "github.py", line 6, in <module>
r = requests.get('https://api.github.com/user', auth=('user','pass'))
File "/home/ubuntu/.local/lib/python2.7/site-packages/requests/api.py", line 70, in get
return request('get', url, params=params, **kwargs)
File "/home/ubuntu/.local/lib/python2.7/site-packages/requests/api.py", line 56, in request
return session.request(method=method, url=url, **kwargs)
File "/home/ubuntu/.local/lib/python2.7/site-packages/requests/sessions.py", line 488, in request
resp = self.send(prep, **send_kwargs)
File "/home/ubuntu/.local/lib/python2.7/site-packages/requests/sessions.py", line 609, in send
r = adapter.send(request, **kwargs)
File "/home/ubuntu/.local/lib/python2.7/site-packages/requests/adapters.py", line 497, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: ("bad handshake: Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')],)",)
What i have tried:
Replaced the username and password with my authentation details and it gave the following error.
401
application/json; charset=utf-8
utf-8
{"message":"Bad credentials","documentation_url":"https://developer.github.com/v3"}
{u'documentation_url': u'https://developer.github.com/v3', u'message': u'Bad credentials'}
Tried to work with access token in the settings tab of github but it was no use.
Any help? Please help me solve this problem.
Also requests modules is installed on my system.
import requests
r = requests.get('https://api.github.com/user', auth=('user','pass'), verify=False)
print(r.status_code)
print(r.headers['content-type'])
print(r.encoding)
print(r.text)
print(r.json())
This might work, if that's the case.. Then you have trouble verifying the certificate sent from the server (seeing as you're using HTTPS).
There's a whole section about this
Requests verifies SSL certificates for HTTPS requests, just like a web browser. By default, SSL verification is enabled, and Requests will throw a SSLError if it's unable to verify the certificate:
Note that you shouldn't disable TLS/SSL verification for production code, rather investigate why the certificate isn't valid and follow the guide lines from the official documentation.
Manual verification
You can always export the github certificate from say a browser and place the cert in the same directory as your script. Now this should normally not be needed, but as a test.. This should work:
r.get('https://api.github.com', verify='./github.crt')
And again, make sure you've exported the certificate and placed it as github.crt in the same directory as your script.
I am having trouble using pyVmomi with python 2.7.5. I get SSL certificate errors when trying to run the sample scripts from the SDK. I tried all the solutions mentioned on this post but none of them worked for me.
Below is the complete console output.
/usr/lib/python2.7/site-packages/requests/packages/urllib3/util/ssl_.py:315: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#snimissingwarning. SNIMissingWarning /usr/lib/python2.7/site-packages/requests/packages/urllib3/util/ssl_.py:120: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning. InsecurePlatformWarning
Traceback (most recent call last):
File "hello_world_vcenter.py", line 105, in <module>
main()
File "hello_world_vcenter.py", line 80, in main
port=int(args.port))
File "/usr/lib/python2.7/site-packages/pyVim/connect.py", line 663, in SmartConnect
sslContext)
File "/usr/lib/python2.7/site-packages/pyVim/connect.py", line 552, in __FindSupportedVersion
sslContext)
File "/usr/lib/python2.7/site-packages/pyVim/connect.py", line 472, in __GetServiceVersionDescription
tree = __GetElementTreeFromUrl(url, sslContext)
File "/usr/lib/python2.7/site-packages/pyVim/connect.py", line 440, in __GetElementTreeFromUrl
sock = requests.get(url)
File "/usr/lib/python2.7/site-packages/requests/api.py", line 67, in get
return request('get', url, params=params, **kwargs)
File "/usr/lib/python2.7/site-packages/requests/api.py", line 53, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 468, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 576, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 447, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: [Errno 1] _ssl.c:504: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
That looks like you're using a self-signed certificate. While connecting via SmartConnect use your own sslContext and disable certificate verification.
from pyVim.connect import SmartConnect
import ssl
context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
context.verify_mode = ssl.CERT_NONE
si = SmartConnect(host=somehost.com, port=443, user=someone, pwd=secret, sslContext=context)
... or use a signed ssl certificate.
There are a few similiar questions on here (e.g. here).
I'm trying to use the requests module to make a post request to an endpoint that requires ssl auth. My pem file is in the specified path and contains the client cert, and private key. However, I keep getting the Certificate Verified Failed exception. I see in the nginx logs that the request never even made it there. Anyone have any ideas why? I know the certs should work.
params = {
"param_2" : "32100",
"param_1" : "abc"
}
headers = {
"Content-Type" : "application/json"
}
body = json.dumps(params)
r = requests.post(
https://somesite.com/somepath,
data=body,
headers=headers,
timeout=10,
verify="/path/to/cert.pem"
)
Traceback (most recent call last):
File "./somefile.py", line 264, in <module>
start()
File "./somefile.py", line 149, in start
verify="/path/to/cert.pem"
File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 88, in post
return request('post', url, data=data, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 44, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 448, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 554, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 417, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: [Errno 1] _ssl.c:504: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
From the requests docs:
Requests can also ignore verifying the SSL certificate if you set
verify to False.
requests.get('https://kennethreitz.com', verify=False)
By default, verify is set to True. Option verify only applies to host
certs.
You can also specify a local cert to use as client side certificate,
as a single file (containing the private key and the certificate) or
as a tuple of both file’s path:
requests.get('https://kennethreitz.com', cert=('/path/server.crt', '/path/key'))
So it seems like you've just got the args wrong. Try it with 'cert' instead of verify.
I am using Requests 1.2.3 on Windows 7 x64 and am trying to connect to (any) site via HTTPS using a HTTPS proxy by passing the proxies argument to the request.
I don't experience this error when using urllib2's ProxyHandler, so I don't think it's on my proxy's side.
>>> opener = urllib2.build_opener(urllib2.ProxyHandler({'https': 'IP:PORT'}))
>>> resp = opener.open('https://www.google.com')
>>> resp.url
'https://www.google.co.uk/'
>>> resp = requests.get('https://www.google.com', proxies={'https': 'IP:PORT'})
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python27\lib\site-packages\requests\api.py", line 55, in get
return request('get', url, **kwargs)
File "C:\Python27\lib\site-packages\requests\api.py", line 44, in request
return session.request(method=method, url=url, **kwargs)
File "C:\Python27\lib\site-packages\requests\sessions.py", line 335, in request
resp = self.send(prep, **send_kwargs)
File "C:\Python27\lib\site-packages\requests\sessions.py", line 438, in send
r = adapter.send(request, **kwargs)
File "C:\Python27\lib\site-packages\requests\adapters.py", line 331, in send
raise SSLError(e)
requests.exceptions.SSLError: [Errno 8] _ssl.c:504: EOF occurred in violation of protocol
I should probably note that the same error still happens if I pass verify=False to the request.
Any suggestions? I've looked at related questions but there was nothing that worked for me.
I suspect your proxy is a http proxy over which you can use https (the common case)
The problem is, that requests uses https to talk to proxies if the request itself https.
Using an explicit protocol (http) for your proxy should fix things: proxies={'https': 'http://IP:PORT'}
Also have a look at https://github.com/kennethreitz/requests/issues/1182