I am trying to freeze a python app that depends on requests, but I am getting the following error:
Traceback (most recent call last):
File "c:\Python33\lib\site-packages\requests\packages\urllib3\util.py", line 630, in ssl_wrap_socket
context.load_verify_locations(ca_certs)
FileNotFoundError: [Errno 2] No such file or directory
Looks like it is having trouble finding the ssl certificate with the executable. I found this which seems to be the same problem, but I am not able to figure out how they got it to work. The main problem seems to be that the certificate bundled by requests is not copied over to the compressed library. So it seems that I will have to force cx_freeze to bundle the certificates and then point to it from my script.
Starting with this simple script everything works fine:
import requests
r = requests.get("https://yourapihere.com")
print(r.json())
Then if I add the certificate file I stat getting errors:
import requests
r = requests.get("https://yourapihere.com", cert=requests.certs.where())
print(r.json())
-
Traceback (most recent call last):
File "c:\Python33\lib\site-packages\requests\packages\urllib3\connectionpool.py", line 480, in urlopen
body=body, headers=headers)
File "c:\Python33\lib\site-packages\requests\packages\urllib3\connectionpool.py", line 285, in _make_request
conn.request(method, url, **httplib_request_kw)
File "c:\Python33\lib\http\client.py", line 1065, in request
self._send_request(method, url, body, headers)
File "c:\Python33\lib\http\client.py", line 1103, in _send_request
self.endheaders(body)
File "c:\Python33\lib\http\client.py", line 1061, in endheaders
self._send_output(message_body)
File "c:\Python33\lib\http\client.py", line 906, in _send_output
self.send(msg)
File "c:\Python33\lib\http\client.py", line 844, in send
self.connect()
File "c:\Python33\lib\site-packages\requests\packages\urllib3\connection.py", line 164, in connect
ssl_version=resolved_ssl_version)
File "c:\Python33\lib\site-packages\requests\packages\urllib3\util.py", line 637, in ssl_wrap_socket
context.load_cert_chain(certfile, keyfile)
ssl.SSLError: [SSL] PEM lib (_ssl.c:2155)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "c:\Python33\lib\site-packages\requests\adapters.py", line 330, in send
timeout=timeout
File "c:\Python33\lib\site-packages\requests\packages\urllib3\connectionpool.py", line 504, in urlopen
raise SSLError(e)
requests.packages.urllib3.exceptions.SSLError: [SSL] PEM lib (_ssl.c:2155)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "example.py", line 10, in <module>
r = requests.get("https://yourapihere.com", cert=requests.certs.where())
File "c:\Python33\lib\site-packages\requests\api.py", line 55, in get
return request('get', url, **kwargs)
File "c:\Python33\lib\site-packages\requests\api.py", line 44, in request
return session.request(method=method, url=url, **kwargs)
File "c:\Python33\lib\site-packages\requests\sessions.py", line 383, in request
resp = self.send(prep, **send_kwargs)
File "c:\Python33\lib\site-packages\requests\sessions.py", line 486, in send
r = adapter.send(request, **kwargs)
File "c:\Python33\lib\site-packages\requests\adapters.py", line 385, in send
raise SSLError(e)
requests.exceptions.SSLError: [SSL] PEM lib (_ssl.c:2155)
I guess I am using it correctly, but cant really figure out why it is not working. I guess that after fixing this I can continue and add the certificate to the cx_freeze bundle, something like:
example.py:
import os
import requests
cert = os.path.join(os.path.dirname(requests.__file__),'cacert.pem')
r = requests.get("https://yourapihere.com", cert=cert)
print(r.json())
setup.py:
from cx_Freeze import setup, Executable
import requests.certs
build_exe_options = {"zip_includes":[(requests.certs.where(),'requests/cacert.pem')]}
executables = [
Executable('example.py')
]
setup(
executables=executables
)
if someone could give me a tip it would be much appreciated.
I found this comment from another thread that worked for me:
https://stackoverflow.com/a/25239701/3935084
Summary below:
You can also use environment variable "REQUESTS_CA_BUNDLE" (as said http://docs.python-requests.org/en/latest/user/advanced/#ssl-cert-verification)
It's much simpler than correcting all your requests:
os.environ["REQUESTS_CA_BUNDLE"] = os.path.join(os.getcwd(), "cacert.pem")
Steps to get this to work:
Explicitly tell requests where the certificate are
Tell cx_freeze to also get the certificate file when "building"
have the code to use the proper certificate file when frozen
test.py:
import os
import sys
import requests
# sets the path to the certificate file
if getattr(sys, 'frozen', False):
# if frozen, get embeded file
cacert = os.path.join(os.path.dirname(sys.executable), 'cacert.pem')
else:
# else just get the default file
cacert = requests.certs.where()
# remember to use the verify to set the certificate to be used
# I guess it could also work with REQUESTS_CA_BUNDLE, but I have not tried
r = requests.get('https://www.google.com', verify=cacert)
print(r)
setup.py:
from cx_Freeze import setup, Executable
import requests
import sys
executable = Executable( script = "test.py" )
# Add certificate to the build
options = {
"build_exe": {
'include_files' : [(requests.certs.where(), 'cacert.pem')]
}
}
setup(
version = "0",
requires = ["requests"],
options = options,
executables = [executable]
)
to build it, just:
$ python setup.py build
If successful you should see:
$ test
<Response [200]>
Related
I have an URL in the config file which I parsed using ConfigParser to get the requests
config.ini
[default]
root_url ='https://reqres.in/api/users?page=2'
FetchFeeds.py
import requests
from configparser import ConfigParser
import os
import requests
config = ConfigParser()
config.read(os.path.join(os.path.dirname(__file__), '../Config', 'config.ini'))
rootUrl = (config['default']['root_url'])
print(rootUrl)
response = requests.get(rootUrl)
Even the URL is printed properly 'https://reqres.in/api/users?page=2'but I am getting the following error on the requests module
Traceback (most recent call last):
File "C:/Users/sam/PycharmProjects/testProject/GET_Request/FetchFeeds.py", line 11, in <module>
response = requests.get(rootUrl)
File "C:\Users\sam\PycharmProjects\testProject\venv\lib\site-packages\requests\api.py", line 76, in get
return request('get', url, params=params, **kwargs)
File "C:\Users\sam\PycharmProjects\testProject\venv\lib\site-packages\requests\api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "C:\Users\sam\PycharmProjects\testProject\venv\lib\site-packages\requests\sessions.py", line 530, in request
resp = self.send(prep, **send_kwargs)
File "C:\Users\sam\PycharmProjects\testProject\venv\lib\site-packages\requests\sessions.py", line 637, in send
adapter = self.get_adapter(url=request.url)
File "C:\Users\sam\PycharmProjects\testProject\venv\lib\site-packages\requests\sessions.py", line 730, in get_adapter
raise InvalidSchema("No connection adapters were found for {!r}".format(url))
requests.exceptions.InvalidSchema: No connection adapters were found for "'https://reqres.in/api/users?page=2'"
Process finished with exit code 1
Please note I checked carefully there are no white spaces
The main issue is it tries to find a URL like 'url.com' which doesn't exist (correct would be url.com), so the solution is to not put apostrophes in config.ini files:
[default]
root_url = https://reqres.in/api/users?page=2
Also think about if you configure the parameters rather in your code than in the config file and use this instead:
requests.get(config['default']['root_url'], params={'page': 2})
I am trying to run a Dashku python script on my Amazon EC2 instance running Amazon Linux.
This is the error I get:
[ec2-user#ip-10-231-47-166 ~]$ python dashku_53dc123bc0f9ac740b009af9.py
Traceback (most recent call last):
File "dashku_53dc123bc0f9ac740b009af9.py", line 16, in <module>
requests.post('https://dashku.com/api/transmission', data=json.dumps(payload),headers=headers)
File "/usr/lib/python2.6/site-packages/requests/api.py", line 88, in post
return request('post', url, data=data, **kwargs)
File "/usr/lib/python2.6/site-packages/requests/api.py", line 44, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 335, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python2.6/site-packages/requests/sessions.py", line 438, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python2.6/site-packages/requests/adapters.py", line 331, in send
raise SSLError(e)
requests.exceptions.SSLError: [Errno 1] _ssl.c:493: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
Note that I calling sudo yum install requests did not do much, but calling sudo yum upgrade python-requests did install it.
The script works find on my local machine and has not been modified at all:
# Instructions
#
# easy_install requests
# python dashku_53dc123bc0f9ac740b009af9.py
#
import requests
import json
payload = {
"bigNumber": 500,
"_id": "XXXXXX",
"apiKey": "XXXXXX"
}
headers = {'content-type': 'application/json'}
requests.post('https://dashku.com/api/transmission', data=json.dumps(payload),headers=headers)
Running it on my local machine works fine and Dashku is updated accordingly.
Any ideas? Thanks.
Not sure if this is the appropriate solution, but adding verify=False to the requests.post call "fixes" the problem.
requests.post('https://dashku.com/api/transmission', data=json.dumps(payload),headers=headers, verify=False)
I can't figure out why all of a sudden the below code that uses Asana's API generates the below SSL error. Something must have changed on my laptop, since it runs perfectly on my other computer.
from asana import asana
class Login(object):
def __init__(self):
api = 'API'
self.asana_api = asana.AsanaAPI(api, debug=False)
self.user_id = 7359085011308L
class Test(Login):
def Test(self):
Id = 2467584555313L
print self.asana_api.list_tasks(Id,self.user_id)
Traceback (most recent call last):
File "/Users/Chris/Dropbox/AsanaPullPush.py", line 75, in <module>
if __name__ == "__main__": main()
File "/Users/Chris/Dropbox/AsanaPullPush.py", line 72, in main
print Test().Test()
File "/Users/Chris/Dropbox/AsanaPullPush.py", line 15, in Test
print self.asana_api.list_tasks(Id,self.user_id)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/asana/asana.py", line 174, in list_tasks
return self._asana(target)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/asana/asana.py", line 74, in _asana
r = requests.get(target, auth=(self.apikey, ""))
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/requests/api.py", line 55, in get
return request('get', url, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/requests/api.py", line 44, in request
return session.request(method=method, url=url, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/requests/sessions.py", line 383, in request
resp = self.send(prep, **send_kwargs)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/requests/sessions.py", line 486, in send
r = adapter.send(request, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/requests/adapters.py", line 389, in send
raise SSLError(e)
requests.exceptions.SSLError: [Errno 1] _ssl.c:507: error:0D0890A1:asn1 encoding routines:ASN1_verify:unknown message digest algorithm
We recently changed our SSL key in response to the Heartbleed bug you may have heard about. http://blog.asana.com/2014/04/heartbleed/
It looks like your laptop may not have the right SSL. See https://github.com/pypa/pip/issues/829 for discussion of a similar issue.
You should be able to check SSL version on the two machines with python -c "import ssl; print ssl.OPENSSL_VERSION". If indeed the laptop is behind, you'll need to update your python's SSL.
I'm trying to access a journal article hosted by an academic service provider (SP), using a Python script.
The server authenticates using a Shibboleth login. I read Logging into SAML/Shibboleth authenticated server using python and tried to implement a login with Python Requests.
The script starts by querying the SP for the link leading to my IDP institution, and is supposed then to authenticate automatically with the IDP. The first part works, but when following the link to the IDP, it chokes on an SSL error.
Here is what I used:
import requests
import lxml.html
LOGINLINK = 'https://www.jsave.org/action/showLogin?redirectUri=%2F'
USERAGENT = 'Mozilla/5.0 (X11; Linux x86_64; rv:28.0) Gecko/20100101 Firefox/28.0'
s = requests.session()
s.headers.update({'User-Agent' : USERAGENT})
# getting the page where you can search for your IDP
# need to get the cookies so we can continue
response = s.get(LOGINLINK)
rtext = response.text
print('Don\'t see your school?' in rtext) # prints True
# POSTing the name of my institution
data = {
'institutionName' : 'tubingen',
'submitForm' : 'Search',
'currUrl' : '%2Faction%2FshowBasicSearch',
'redirectUri' : '%2F',
'activity' : 'isearch'
}
response = s.post(BASEURL + '/action/showLogin', data=data)
rtext = response.text
print('university of tubingen' in rtext) # prints True
# get the link that leads to the IDP
tree = lxml.html.fromstring(rtext)
loginlinks = tree.cssselect('a.extLogin')
if (loginlinks):
loginlink = loginlinks[0].get('href')
else:
exit(1)
print('continuing to IDP')
response = s.get(loginlink)
rtext = response.text
print('zentrale Anmeldeseite' in rtext)
This yields:
continuing to IDP...
2014-04-04 10:04:06,010 - INFO - Starting new HTTPS connection (1): idp.uni-tuebingen.de
Traceback (most recent call last):
File "/usr/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 480, in urlopen
body=body, headers=headers)
File "/usr/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 285, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/lib/python3.4/http/client.py", line 1066, in request
self._send_request(method, url, body, headers)
File "/usr/lib/python3.4/http/client.py", line 1104, in _send_request
self.endheaders(body)
File "/usr/lib/python3.4/http/client.py", line 1062, in endheaders
self._send_output(message_body)
File "/usr/lib/python3.4/http/client.py", line 907, in _send_output
self.send(msg)
File "/usr/lib/python3.4/http/client.py", line 842, in send
self.connect()
File "/usr/lib/python3.4/site-packages/requests/packages/urllib3/connection.py", line 164, in connect
ssl_version=resolved_ssl_version)
File "/usr/lib/python3.4/site-packages/requests/packages/urllib3/util.py", line 639, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "/usr/lib/python3.4/ssl.py", line 344, in wrap_socket
_context=self)
File "/usr/lib/python3.4/ssl.py", line 540, in __init__
self.do_handshake()
File "/usr/lib/python3.4/ssl.py", line 767, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: TLSV1_ALERT_INTERNAL_ERROR] tlsv1 alert internal error (_ssl.c:598)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.4/site-packages/requests/adapters.py", line 330, in send
timeout=timeout
File "/usr/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 504, in urlopen
raise SSLError(e)
requests.packages.urllib3.exceptions.SSLError: [SSL: TLSV1_ALERT_INTERNAL_ERROR] tlsv1 alert internal error (_ssl.c:598)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "./try.py", line 154, in <module>
response = s.get(loginlink)
File "/usr/lib/python3.4/site-packages/requests/sessions.py", line 395, in get
return self.request('GET', url, **kwargs)
File "/usr/lib/python3.4/site-packages/requests/sessions.py", line 383, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python3.4/site-packages/requests/sessions.py", line 486, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python3.4/site-packages/requests/adapters.py", line 385, in send
raise SSLError(e)
requests.exceptions.SSLError: [SSL: TLSV1_ALERT_INTERNAL_ERROR] tlsv1 alert internal error (_ssl.c:598)
Using s.get(loginlink, verify=False) yields exactly the same error. Simply using urllib.request.urlopen(loginlink) does so, too.
Printing and pasting the link into Firefox, on the other hand, works fine.
After trying with openssl s_client it looks like the destination idp.uni-tuebingen.de:443 is only support SSLv3 and misbehaving on anything newer. With forcing SSLv3 one gets:
$ openssl s_client -connect idp.uni-tuebingen.de:443 -ssl3
CONNECTED(00000003)
depth=3 C = DE, O = Deutsche Telekom AG, OU = T-TeleSec Trust Center, CN = Deutsche Telekom Root CA 2
...
But with default setup or forcing TLv1 (-tls1) it only returns an alert:
openssl s_client -connect idp.uni-tuebingen.de:443
CONNECTED(00000003)
140493591938752:error:14077438:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert internal error:s23_clnt.c:741:
So you need to find a way to force SSLv3 for this connection. I'm not familiar with the python at this point but maybe http://docs.python-requests.org/en/latest/user/advanced/ chapter "Example: Specific SSL Version" helps.
And why it works with firefox: the browsers usually retry with a downgraded SSL version if the connects with the safer versions fail. E.g. everybody is trying to work around broken stuff so that the owner of the broken stuff has no intention to fix it :(
I'm using urllib3 against private services that have self signed certificates. Is there any way to have urllib3 ignore the certificate errors and make the request anyways?
import urllib3
c = urllib3.HTTPSConnectionPool('10.0.3.168', port=9001)
c.request('GET', '/')
When using the following:
import urllib3
c = urllib3.HTTPSConnectionPool('10.0.3.168', port=9001, cert_reqs='CERT_NONE')
c.request('GET', '/')
The following error is raised:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3/dist-packages/urllib3/request.py", line 67, in request
**urlopen_kw)
File "/usr/lib/python3/dist-packages/urllib3/request.py", line 80, in request_encode_url
return self.urlopen(method, url, **urlopen_kw)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 415, in urlopen
body=body, headers=headers)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 267, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/lib/python3.3/http/client.py", line 1061, in request
self._send_request(method, url, body, headers)
File "/usr/lib/python3.3/http/client.py", line 1099, in _send_request
self.endheaders(body)
File "/usr/lib/python3.3/http/client.py", line 1057, in endheaders
self._send_output(message_body)
File "/usr/lib/python3.3/http/client.py", line 902, in _send_output
self.send(msg)
File "/usr/lib/python3.3/http/client.py", line 840, in send
self.connect()
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 103, in connect
match_hostname(self.sock.getpeercert(), self.host)
File "/usr/lib/python3/dist-packages/urllib3/packages/ssl_match_hostname/__init__.py", line 32, in match_hostname
raise ValueError("empty or no certificate")
ValueError: empty or no certificate
Using cURL I'm able to get the expected response from the service
$ curl -k https://10.0.3.168:9001/
Please read the documentation for API endpoints
Try following code:
import urllib3
c = urllib3.HTTPSConnectionPool('10.0.3.168', port=9001, cert_reqs='CERT_NONE',
assert_hostname=False)
c.request('GET', '/')
See Setting assert_hostname to False will disable SSL hostname verification
In this question I see many answers but, IMHO, too much unnecessary information that can lead to confusion.
Just add the cert_reqs='CERT_NONE' parameter
import urllib3
http = urllib3.PoolManager(cert_reqs='CERT_NONE')
I found the answer to my problem. The urllib3 documentation does not, in fact, completely explain how to suppress SSL certificate validation. What is missing is a reference to ssl.CERT_NONE.
My code has a boolean, ssl_verify, to indicate whether or not I want SSL validation. The code now looks like this:
import ssl
import urllib3
#
#
#
if (ssl_verify):
cert_reqs = ssl.CERT_REQUIRED
else:
cert_reqs = ssl.CERT_NONE
urllib3.disable_warnings()
http = urllib3.PoolManager(cert_reqs = cert_reqs)
auth_url = f'https://{fmc_ip}/api/fmc_platform/v1/auth/generatetoken'
type = {'Content-Type': 'application/json'}
auth = urllib3.make_headers(basic_auth=f'{username}:{password}')
headers = { **type, **auth }
resp = http.request('POST',
auth_url,
headers=headers,
timeout=10.0)
Try to instanciate your connection pool this way:
HTTPSConnectionPool(self.host, self.port, cert_reqs=ssl.CERT_NONE)
or this way:
HTTPSConnectionPool(self.host, self.port, cert_reqs='CERT_NONE')
Source: https://github.com/shazow/urllib3/blob/master/test/with_dummyserver/test_https.py
EDIT (after seeing your edit):
It looks like the remote host didn't send a certificate (is it possible?).
This is the code (from urllib3) which raised an exception:
def match_hostname(cert, hostname):
"""Verify that *cert* (in decoded format as returned by
SSLSocket.getpeercert()) matches the *hostname*. RFC 2818 rules
are mostly followed, but IP addresses are not accepted for *hostname*.
CertificateError is raised on failure. On success, the function
returns nothing.
"""
if not cert:
raise ValueError("empty or no certificate")
So it looks like cert is empty, which means that self.sock.getpeercert() returned an empty string.