I'm trying to use the requests module to make a post request to an endpoint that requires ssl auth. My pem file is in the specified path and contains the client cert, and private key. However, I keep getting the Certificate Verified Failed exception. I see in the nginx logs that the request never even made it there. Anyone have any ideas why? I know the certs should work.
params = {
"param_2" : "32100",
"param_1" : "abc"
}
headers = {
"Content-Type" : "application/json"
}
body = json.dumps(params)
r = requests.post(
https://somesite.com/somepath,
data=body,
headers=headers,
timeout=10,
verify="/path/to/cert.pem"
)
Traceback (most recent call last):
File "./somefile.py", line 264, in <module>
start()
File "./somefile.py", line 149, in start
verify="/path/to/cert.pem"
File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 88, in post
return request('post', url, data=data, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 44, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 448, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 554, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 417, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: [Errno 1] _ssl.c:504: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
From the requests docs:
Requests can also ignore verifying the SSL certificate if you set
verify to False.
requests.get('https://kennethreitz.com', verify=False)
By default, verify is set to True. Option verify only applies to host
certs.
You can also specify a local cert to use as client side certificate,
as a single file (containing the private key and the certificate) or
as a tuple of both file’s path:
requests.get('https://kennethreitz.com', cert=('/path/server.crt', '/path/key'))
So it seems like you've just got the args wrong. Try it with 'cert' instead of verify.
Related
I have an API, where I need to get the bearer token. When I use 'Postman' application, I get the bearer token correctly. I have written below python code for the same but I get below errors. Please help. I need to send username and password in the body as a form data.
import requests
url = "https://322.286.24.01/ach/ach_api/login"
payload={'username': 'test',
'password': 'test12'}
response = requests.post( url,data=payload)
print(response.text)
ERROR:
Traceback (most recent call last):
File "/tmp/test.py", line 13, in <module>
response = requests.post(url,data=payload)
File "/usr/local/lib/python3.9/site-packages/requests/api.py", line 119, in post
return request('post', url, data=data, json=json, **kwargs)
File "/usr/local/lib/python3.9/site-packages/requests/api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 542, in request
resp = self.send(prep, **send_kwargs)
File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 655, in send
r = adapter.send(request, **kwargs)
File "/usr/local/lib/python3.9/site-packages/requests/adapters.py", line 514, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='322.286.24.01', port=443): Max retries exceeded with url: /ach/ach_api/login (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate (_ssl.c:1123)')))
The error is because requests is trying to check for the ca cert, try the following:
response = requests.post(url, data=payload, verify=False)
Or, if you have a ca.crt somewhere, usually pem format, you can try:
response = requests.post(url, data=payload, verify='/path/to/pem')
Also the IP address looks funny, although since the client connects I suspect you just changed that to anonymise your post?
I would like to authenticate to server from my client using certificate that is generated from server.I have a server-ca.crt and below is the CURL command that is working.How to send similar request using python requests module .
$ curl -X GET -u sat_username:sat_password \
-H "Accept:application/json" --cacert katello-server-ca.crt \
https://satellite6.example.com/katello/api/organizations
I have tried following way and it is getting some exception, can someone help in resolving this issue.
python requestsCert.py
Traceback (most recent call last):
File "requestsCert.py", line 2, in <module>
res=requests.get('https://satellite6.example.com/katello/api/organizations', cert='/certificateTests/katello-server-ca.crt', verify=True)
File "/usr/lib/python2.7/site-packages/requests/api.py", line 68, in get
return request('get', url, **kwargs)
File "/usr/lib/python2.7/site-packages/requests/api.py", line 50, in request
response = session.request(method=method, url=url, **kwargs)
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 464, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 576, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 431, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: [SSL] PEM lib (_ssl.c:2554)
res=requests.get('https://...', cert='/certificateTests/katello-server-ca.crt', verify=True)
The cert argument in requests.get is used to specify the client certificate and key which should be used for mutual authentication. It is not used to specify the trusted CA as the --cacert argument in curl does. Instead you should use the verify argument:
res=requests.get('https://...', verify='/certificateTests/katello-server-ca.crt')
For more information see SSL Cert Verification and Client Side Certificates in the documentation for requests.
I'm using urllib3 against private services that have self signed certificates. Is there any way to have urllib3 ignore the certificate errors and make the request anyways?
import urllib3
c = urllib3.HTTPSConnectionPool('10.0.3.168', port=9001)
c.request('GET', '/')
When using the following:
import urllib3
c = urllib3.HTTPSConnectionPool('10.0.3.168', port=9001, cert_reqs='CERT_NONE')
c.request('GET', '/')
The following error is raised:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3/dist-packages/urllib3/request.py", line 67, in request
**urlopen_kw)
File "/usr/lib/python3/dist-packages/urllib3/request.py", line 80, in request_encode_url
return self.urlopen(method, url, **urlopen_kw)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 415, in urlopen
body=body, headers=headers)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 267, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/lib/python3.3/http/client.py", line 1061, in request
self._send_request(method, url, body, headers)
File "/usr/lib/python3.3/http/client.py", line 1099, in _send_request
self.endheaders(body)
File "/usr/lib/python3.3/http/client.py", line 1057, in endheaders
self._send_output(message_body)
File "/usr/lib/python3.3/http/client.py", line 902, in _send_output
self.send(msg)
File "/usr/lib/python3.3/http/client.py", line 840, in send
self.connect()
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 103, in connect
match_hostname(self.sock.getpeercert(), self.host)
File "/usr/lib/python3/dist-packages/urllib3/packages/ssl_match_hostname/__init__.py", line 32, in match_hostname
raise ValueError("empty or no certificate")
ValueError: empty or no certificate
Using cURL I'm able to get the expected response from the service
$ curl -k https://10.0.3.168:9001/
Please read the documentation for API endpoints
Try following code:
import urllib3
c = urllib3.HTTPSConnectionPool('10.0.3.168', port=9001, cert_reqs='CERT_NONE',
assert_hostname=False)
c.request('GET', '/')
See Setting assert_hostname to False will disable SSL hostname verification
In this question I see many answers but, IMHO, too much unnecessary information that can lead to confusion.
Just add the cert_reqs='CERT_NONE' parameter
import urllib3
http = urllib3.PoolManager(cert_reqs='CERT_NONE')
I found the answer to my problem. The urllib3 documentation does not, in fact, completely explain how to suppress SSL certificate validation. What is missing is a reference to ssl.CERT_NONE.
My code has a boolean, ssl_verify, to indicate whether or not I want SSL validation. The code now looks like this:
import ssl
import urllib3
#
#
#
if (ssl_verify):
cert_reqs = ssl.CERT_REQUIRED
else:
cert_reqs = ssl.CERT_NONE
urllib3.disable_warnings()
http = urllib3.PoolManager(cert_reqs = cert_reqs)
auth_url = f'https://{fmc_ip}/api/fmc_platform/v1/auth/generatetoken'
type = {'Content-Type': 'application/json'}
auth = urllib3.make_headers(basic_auth=f'{username}:{password}')
headers = { **type, **auth }
resp = http.request('POST',
auth_url,
headers=headers,
timeout=10.0)
Try to instanciate your connection pool this way:
HTTPSConnectionPool(self.host, self.port, cert_reqs=ssl.CERT_NONE)
or this way:
HTTPSConnectionPool(self.host, self.port, cert_reqs='CERT_NONE')
Source: https://github.com/shazow/urllib3/blob/master/test/with_dummyserver/test_https.py
EDIT (after seeing your edit):
It looks like the remote host didn't send a certificate (is it possible?).
This is the code (from urllib3) which raised an exception:
def match_hostname(cert, hostname):
"""Verify that *cert* (in decoded format as returned by
SSLSocket.getpeercert()) matches the *hostname*. RFC 2818 rules
are mostly followed, but IP addresses are not accepted for *hostname*.
CertificateError is raised on failure. On success, the function
returns nothing.
"""
if not cert:
raise ValueError("empty or no certificate")
So it looks like cert is empty, which means that self.sock.getpeercert() returned an empty string.
I'm calling a REST API with requests in python and so far have been successful when I set verify=False.
Now, I have to use client side cert that I need to import for authentication and I'm getting this error everytime I'm using the cert (.pfx). cert.pfx is password protected.
r = requests.post(url, params=payload, headers=headers,
data=payload, verify='cert.pfx')
This is the error I'm getting:
Traceback (most recent call last):
File "C:\Users\me\Desktop\test.py", line 65, in <module>
r = requests.post(url, params=payload, headers=headers, data=payload, verify=cafile)
File "C:\Python33\lib\site-packages\requests\api.py", line 88, in post
return request('post', url, data=data, **kwargs)
File "C:\Python33\lib\site-packages\requests\api.py", line 44, in request
return session.request(method=method, url=url, **kwargs)
File "C:\Python33\lib\site-packages\requests\sessions.py", line 346, in request
resp = self.send(prep, **send_kwargs)
File "C:\Python33\lib\site-packages\requests\sessions.py", line 449, in send
r = adapter.send(request, **kwargs)
File "C:\Python33\lib\site-packages\requests\adapters.py", line 322, in send
raise SSLError(e)
requests.exceptions.SSLError: unknown error (_ssl.c:2158)
I've also tried openssl to get .pem and key but with .pem and getting SSL: CERTIFICATE_VERIFY_FAILED
Can someone please direct me on how to import the certs and where to place it? I tried searching but still faced with the same issue.
I had this same problem. The verify parameter refers to the server's certificate. You want the cert parameter to specify your client certificate.
import requests
cert_file_path = "cert.pem"
key_file_path = "key.pem"
url = "https://example.com/resource"
params = {"param_1": "value_1", "param_2": "value_2"}
cert = (cert_file_path, key_file_path)
r = requests.get(url, params=params, cert=cert)
I had the same problem and to resolve this, I came to know that we have to send RootCA along with certificate and its key as shown below,
response = requests.post(url, data=your_data, cert=('path_client_certificate_file', 'path_certificate_key_file'), verify='path_rootCA')
I am using Requests 1.2.3 on Windows 7 x64 and am trying to connect to (any) site via HTTPS using a HTTPS proxy by passing the proxies argument to the request.
I don't experience this error when using urllib2's ProxyHandler, so I don't think it's on my proxy's side.
>>> opener = urllib2.build_opener(urllib2.ProxyHandler({'https': 'IP:PORT'}))
>>> resp = opener.open('https://www.google.com')
>>> resp.url
'https://www.google.co.uk/'
>>> resp = requests.get('https://www.google.com', proxies={'https': 'IP:PORT'})
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python27\lib\site-packages\requests\api.py", line 55, in get
return request('get', url, **kwargs)
File "C:\Python27\lib\site-packages\requests\api.py", line 44, in request
return session.request(method=method, url=url, **kwargs)
File "C:\Python27\lib\site-packages\requests\sessions.py", line 335, in request
resp = self.send(prep, **send_kwargs)
File "C:\Python27\lib\site-packages\requests\sessions.py", line 438, in send
r = adapter.send(request, **kwargs)
File "C:\Python27\lib\site-packages\requests\adapters.py", line 331, in send
raise SSLError(e)
requests.exceptions.SSLError: [Errno 8] _ssl.c:504: EOF occurred in violation of protocol
I should probably note that the same error still happens if I pass verify=False to the request.
Any suggestions? I've looked at related questions but there was nothing that worked for me.
I suspect your proxy is a http proxy over which you can use https (the common case)
The problem is, that requests uses https to talk to proxies if the request itself https.
Using an explicit protocol (http) for your proxy should fix things: proxies={'https': 'http://IP:PORT'}
Also have a look at https://github.com/kennethreitz/requests/issues/1182