UnificationEngine: Unable to send Requests via Python using TLS or SSL - python

I am attempting to use a Raspberry Pi 3 (Model B) to send Requests by Python to the Unification Engine. With SSL/TLS verification disabled, the request happens normally, but I need to get SSL/TLS working with it.
The below code is meant to force a session to use TLSv1 for the purpose of sending Python Requests to UnificationEngine:
class ForceTLSV1Adapter(HTTPAdapter):
def init_poolmanager(self, connection, maxsize, block=False):
self.poolmanager = PoolManager(num_pools=connection,maxsize=maxsize,block=block,ssl_version=ssl.PROTOCOL_TLSv1)
def proxy_manager_for(self, proxy, **proxy_kwargs):
proxy_kwargs['ssl_version'] = ssl.PROTOCOL_TLSv1
return super(ForceTLVS1Adapter, self).proxy_manager_for(proxy, **proxy_kwargs)
----Some Code here----
s = requests.Session()
s.mount('https://apiv2.unificationengine.com', ForceTLSV1Adapter())
----Some Code Here----
However, this error pops up after I send the request.
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 516, in urlopen
body=body, headers=headers)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 304, in _make_request
self._validate_conn(conn)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 724, in _validate_conn
conn.connect()
File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 237, in connect
ssl_version=resolved_ssl_version)
File "/usr/lib/python3/dist-packages/urllib3/util/ssl_.py", line 123, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "/usr/lib/python3.4/ssl.py", line 364, in wrap_socket
_context=self)
File "/usr/lib/python3.4/ssl.py", line 577, in __init__
self.do_handshake()
File "/usr/lib/python3.4/ssl.py", line 804, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:600)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/requests/adapters.py", line 362, in send
timeout=timeout
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 543, in urlopen
raise SSLError(e)
urllib3.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:600)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/pi/Documents/HumidityRequest.py", line 134, in <module>
sendEmail(round(temp,3))
File "/home/pi/Documents/HumidityRequest.py", line 92, in sendEmail
r = requests.post('https://apiv2.unificationengine.com/v2/message/send', auth=(key,secret),data=userMessage)
File "/usr/lib/python3/dist-packages/requests/api.py", line 94, in post
return request('post', url, data=data, json=json, **kwargs)
File "/usr/lib/python3/dist-packages/requests/api.py", line 49, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 457, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 569, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python3/dist-packages/requests/adapters.py", line 420, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:600)
I'm not sure what is the issue right now.

Looks like the certificate verification isn't done correctly.
Have a look at:
https://the.randomengineer.com/2014/01/29/using-ssl-wrap_socket-for-secure-sockets-in-python/
https://urllib3.readthedocs.io/en/latest/user-guide.html#ssl
you could also try the httplib.

Related

Pandas_datareader issue

When I try to read data from stooq through pandas_datareader.data this error keeps coming through:
Traceback (most recent call last):
File "/Users/matteo/PycharmProjects/pythonProject/venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 700, in urlopen
self._prepare_proxy(conn)
File "/Users/matteo/PycharmProjects/pythonProject/venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 996, in _prepare_proxy
conn.connect()
File "/Users/matteo/PycharmProjects/pythonProject/venv/lib/python3.11/site-packages/urllib3/connection.py", line 414, in connect
self.sock = ssl_wrap_socket(
^^^^^^^^^^^^^^^^
File "/Users/matteo/PycharmProjects/pythonProject/venv/lib/python3.11/site-packages/urllib3/util/ssl_.py", line 449, in ssl_wrap_socket
ssl_sock = _ssl_wrap_socket_impl(
^^^^^^^^^^^^^^^^^^^^^^
File "/Users/matteo/PycharmProjects/pythonProject/venv/lib/python3.11/site-packages/urllib3/util/ssl_.py", line 493, in _ssl_wrap_socket_impl
return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/ssl.py", line 517, in wrap_socket
return self.sslsocket_class._create(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/ssl.py", line 1075, in _create
self.do_handshake()
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/ssl.py", line 1346, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:992)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/matteo/PycharmProjects/pythonProject/venv/lib/python3.11/site-packages/requests/adapters.py", line 489, in send
resp = conn.urlopen(
^^^^^^^^^^^^^
File "/Users/matteo/PycharmProjects/pythonProject/venv/lib/python3.11/site-packages/urllib3/connectionpool.py", line 787, in urlopen
retries = retries.increment(
^^^^^^^^^^^^^^^^^^
File "/Users/matteo/PycharmProjects/pythonProject/venv/lib/python3.11/site-packages/urllib3/util/retry.py", line 592, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='stooq.com', port=443): Max retries exceeded with url: /q/d/l/?s=%5EDJI&i=d&d1=20180216&d2=20230215 (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:992)')))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/matteo/PycharmProjects/pythonProject/stock_data.py", line 3, in <module>
f = web.DataReader('^DJI', 'stooq')
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/matteo/PycharmProjects/pythonProject/venv/lib/python3.11/site-packages/pandas/util/_decorators.py", line 211, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/Users/matteo/PycharmProjects/pythonProject/venv/lib/python3.11/site-packages/pandas_datareader/data.py", line 432, in DataReader
).read()
^^^^^^
File "/Users/matteo/PycharmProjects/pythonProject/venv/lib/python3.11/site-packages/pandas_datareader/base.py", line 253, in read
df = self._read_one_data(self.url, params=self._get_params(self.symbols))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/matteo/PycharmProjects/pythonProject/venv/lib/python3.11/site-packages/pandas_datareader/base.py", line 108, in _read_one_data
out = self._read_url_as_StringIO(url, params=params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/matteo/PycharmProjects/pythonProject/venv/lib/python3.11/site-packages/pandas_datareader/base.py", line 119, in _read_url_as_StringIO
response = self._get_response(url, params=params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/matteo/PycharmProjects/pythonProject/venv/lib/python3.11/site-packages/pandas_datareader/base.py", line 155, in _get_response
response = self.session.get(
^^^^^^^^^^^^^^^^^
File "/Users/matteo/PycharmProjects/pythonProject/venv/lib/python3.11/site-packages/requests/sessions.py", line 600, in get
return self.request("GET", url, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/matteo/PycharmProjects/pythonProject/venv/lib/python3.11/site-packages/requests/sessions.py", line 587, in request
resp = self.send(prep, **send_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/matteo/PycharmProjects/pythonProject/venv/lib/python3.11/site-packages/requests/sessions.py", line 701, in send
r = adapter.send(request, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/matteo/PycharmProjects/pythonProject/venv/lib/python3.11/site-packages/requests/adapters.py", line 563, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='stooq.com', port=443): Max retries exceeded with url: /q/d/l/?s=%5EDJI&i=d&d1=20180216&d2=20230215 (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:992)')))
I tried to run the same code on jupiter lab and the error is the same, contrary in google colab I get the correct result. Here is the code from google colab:
Code:
import pandas_datareader.data as web
f = web.DataReader('^DJI', 'stooq')
f[:10]

I Cant Use Pyngrok(certificate verify failed: certificate has expired)

I can't Use Pyngrok
Every time I run a script with the module pyngrok it displays the error:
Downloading ngrok ...
Traceback (most recent call last):
File "C:\Users\hp\AppData\Local\Programs\Python\Python310\lib\urllib\request.py", line 1348, in do_open
h.request(req.get_method(), req.selector, req.data, headers,
File "C:\Users\hp\AppData\Local\Programs\Python\Python310\lib\http\client.py", line 1282, in request
self._send_request(method, url, body, headers, encode_chunked)
File "C:\Users\hp\AppData\Local\Programs\Python\Python310\lib\http\client.py", line 1328, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "C:\Users\hp\AppData\Local\Programs\Python\Python310\lib\http\client.py", line 1277, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "C:\Users\hp\AppData\Local\Programs\Python\Python310\lib\http\client.py", line 1037, in _send_output
self.send(msg)
File "C:\Users\hp\AppData\Local\Programs\Python\Python310\lib\http\client.py", line 975, in send
self.connect()
File "C:\Users\hp\AppData\Local\Programs\Python\Python310\lib\http\client.py", line 1454, in connect
self.sock = self._context.wrap_socket(self.sock,
File "C:\Users\hp\AppData\Local\Programs\Python\Python310\lib\ssl.py", line 512, in wrap_socket
return self.sslsocket_class._create(
File "C:\Users\hp\AppData\Local\Programs\Python\Python310\lib\ssl.py", line 1070, in _create
self.do_handshake()
File "C:\Users\hp\AppData\Local\Programs\Python\Python310\lib\ssl.py", line 1341, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:997)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\hp\AppData\Local\Programs\Python\Python310\lib\site-packages\pyngrok\installer.py", line 94, in install_ngrok
download_path = _download_file(url, **kwargs)
File "C:\Users\hp\AppData\Local\Programs\Python\Python310\lib\site-packages\pyngrok\installer.py", line 215, in _download_file
response = urlopen(url, **kwargs)
File "C:\Users\hp\AppData\Local\Programs\Python\Python310\lib\urllib\request.py", line 216, in urlopen
return opener.open(url, data, timeout)
File "C:\Users\hp\AppData\Local\Programs\Python\Python310\lib\urllib\request.py", line 519, in open
response = self._open(req, data)
File "C:\Users\hp\AppData\Local\Programs\Python\Python310\lib\urllib\request.py", line 536, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
File "C:\Users\hp\AppData\Local\Programs\Python\Python310\lib\urllib\request.py", line 496, in _call_chain
result = func(*args)
File "C:\Users\hp\AppData\Local\Programs\Python\Python310\lib\urllib\request.py", line 1391, in https_open
return self.do_open(http.client.HTTPSConnection, req,
File "C:\Users\hp\AppData\Local\Programs\Python\Python310\lib\urllib\request.py", line 1351, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:997)>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\hp\AppData\Local\Programs\Python\Python310\server1.py", line 12, in <module>
public_url = ngrok.connect(port, "tcp", options={"remote_addr": "{}:{}".format(host, port)})
File "C:\Users\hp\AppData\Local\Programs\Python\Python310\lib\site-packages\pyngrok\ngrok.py", line 251, in connect
api_url = get_ngrok_process(pyngrok_config).api_url
File "C:\Users\hp\AppData\Local\Programs\Python\Python310\lib\site-packages\pyngrok\ngrok.py", line 160, in get_ngrok_process
install_ngrok(pyngrok_config)
File "C:\Users\hp\AppData\Local\Programs\Python\Python310\lib\site-packages\pyngrok\ngrok.py", line 98, in install_ngrok
installer.install_ngrok(pyngrok_config.ngrok_path)
File "C:\Users\hp\AppData\Local\Programs\Python\Python310\lib\site-packages\pyngrok\installer.py", line 98, in install_ngrok
raise PyngrokNgrokInstallError("An error occurred while downloading ngrok from {}: {}".format(url, e))
pyngrok.exception.PyngrokNgrokInstallError: An error occurred while downloading ngrok from https://bin.equinox.io/c/4VmDzA7iaHb/ngrok-stable-windows-amd64.zip: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:997)>
Please, Help!
OS: Windows11 Home Single Language
Cpu: Intel i5
python version:3.10.2(I Tried it on 3.9.7 too)
I even tried updating python and my os but it won't work!
[SSL: CERTIFICATE_VERIFY_FAILED]
Currently having the same issue, one way is to use pyopenssl
import requests
import json
from requests.adapters import HTTPAdapter
from requests.packages.urllib3.poolmanager import PoolManager
import ssl
class MyAdapter(HTTPAdapter):
def init_poolmanager(self, connections, maxsize, block=False):
self.poolmanager = PoolManager(num_pools=connections,
maxsize=maxsize,
block=block,
ssl_version=ssl.PROTOCOL_TLSv1_2)
url = 'https://example.com'
data = {'key': 'value'}
session = requests.Session()
session.mount('https://', MyAdapter())
response = session.post(url, data=json.dumps(data))
but I wouldn't recommend because you're not using ssl auth

Python requests throwing SSLEOFError

I am trying to connect a Python Requests POC to this tool. I am specifically looking at the test cases for FIA-X509 Ext 1.1. However, the Python POC is throwing the following error:
Traceback (most recent call last):
File "/home/hussain/Desktop/pythonVenv/python36Venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 601, in urlopen
chunked=chunked)
File "/home/hussain/Desktop/pythonVenv/python36Venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 346, in _make_request
self._validate_conn(conn)
File "/home/hussain/Desktop/pythonVenv/python36Venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 850, in _validate_conn
conn.connect()
File "/home/hussain/Desktop/pythonVenv/python36Venv/lib/python3.6/site-packages/urllib3/connection.py", line 326, in connect
ssl_context=context)
File "/home/hussain/Desktop/pythonVenv/python36Venv/lib/python3.6/site-packages/urllib3/util/ssl_.py", line 329, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "/usr/lib/python3.6/ssl.py", line 407, in wrap_socket
_context=self, _session=session)
File "/usr/lib/python3.6/ssl.py", line 814, in __init__
self.do_handshake()
File "/usr/lib/python3.6/ssl.py", line 1068, in do_handshake
self._sslobj.do_handshake()
File "/usr/lib/python3.6/ssl.py", line 689, in do_handshake
self._sslobj.do_handshake()
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:777)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/hussain/Desktop/pythonVenv/python36Venv/lib/python3.6/site-packages/requests/adapters.py", line 440, in send
timeout=timeout
File "/home/hussain/Desktop/pythonVenv/python36Venv/lib/python3.6/site-packages/urllib3/connectionpool.py", line 639, in urlopen
_stacktrace=sys.exc_info()[2])
File "/home/hussain/Desktop/pythonVenv/python36Venv/lib/python3.6/site-packages/urllib3/util/retry.py", line 388, in increment
raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='10.0.0.221', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:777)'),))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "req.py", line 91, in <module>
r = session.request('GET', 'https://10.0.0.221', verify=True)
File "/home/hussain/Desktop/pythonVenv/python36Venv/lib/python3.6/site-packages/requests/sessions.py", line 508, in request
resp = self.send(prep, **send_kwargs)
File "/home/hussain/Desktop/pythonVenv/python36Venv/lib/python3.6/site-packages/requests/sessions.py", line 618, in send
r = adapter.send(request, **kwargs)
File "/home/hussain/Desktop/pythonVenv/python36Venv/lib/python3.6/site-packages/requests/adapters.py", line 506, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='10.0.0.221', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLEOFError(8, 'EOF occurred in violation of protocol (_ssl.c:777)'),))
This is the POC code that i am using to connect to it:
import ssl
import requests
from requests.adapters import HTTPAdapter
from requests.packages.urllib3.poolmanager import PoolManager
from requests.packages.urllib3.util import ssl_
CIPHERS = (
'RSA+AES'
)
class TlsAdapter(HTTPAdapter):
def __init__(self, ssl_options=0, **kwargs):
self.ssl_options = ssl_options
super(TlsAdapter, self).__init__(**kwargs)
def init_poolmanager(self, *pool_args, **pool_kwargs):
ctx = ssl_.create_urllib3_context(ciphers=CIPHERS, cert_reqs=ssl.CERT_REQUIRED, options=self.ssl_options)
self.poolmanager = PoolManager(*pool_args,
ssl_context=ctx,
**pool_kwargs)
session = requests.session()
adapter = TlsAdapter(ssl.OP_NO_TLSv1 | ssl.OP_NO_TLSv1_1)
session.mount("https://", adapter)
r = session.request('GET', 'https://10.0.0.221', verify=True)
print(r)
Can anyone please help me in figuring out what i am missing here and how can this be resolved? I tried looking up the problem on the requests github issue page as well as on stackoverflow but none of them seemed to work for me.
Thanks!

How can I login through 'https' login page using python3.4?

I'm trying to login certain page with python3.4 requests, bs4 module.
When I'm trying to login with below code, ssl errors occurs :
import requests
userid = "myid"
password = "mypassword"
loginURL = "https://example.com/member/login"
params = {'userid': userid, 'password':password}
res = requests.post(loginURL, params=params, verify=True)
Errors:
choejeong-us-MacBook-Pro:Desktop kris$ python a.py
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 559, in urlopen
body=body, headers=headers)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 345, in _make_request
self._validate_conn(conn)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 782, in _validate_conn
conn.connect()
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/connection.py", line 250, in connect
ssl_version=resolved_ssl_version)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/util/ssl_.py", line 285, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/ssl.py", line 365, in wrap_socket
_context=self)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/ssl.py", line 583, in __init__
self.do_handshake()
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/ssl.py", line 810, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:600)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/adapters.py", line 370, in send
timeout=timeout
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/packages/urllib3/connectionpool.py", line 588, in urlopen
raise SSLError(e)
requests.packages.urllib3.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:600)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "a.py", line 10, in <module>
res = requests.post(loginURL, params=params, verify=True)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/api.py", line 109, in post
return request('post', url, data=data, json=json, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/api.py", line 50, in request
response = session.request(method=method, url=url, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/sessions.py", line 468, in request
resp = self.send(prep, **send_kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/sessions.py", line 576, in send
r = adapter.send(request, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/requests/adapters.py", line 433, in send
raise SSLError(e, request=request)
requests.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:600)
How can I login the site with ssl page?.
I'm looked some books about python requests module and web scraping, there are no books dealing this problem.
p.s I wonder if python is really strong enough to deal with 'all' web pages ? I mean, Can I crawl all sites in the web? or there are some limitations?
The last line of the traceback tells you why:
requests.exceptions.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:600)
requests failed to verify the server's SSL certificate. You can disable cert verification with the verify option:
res = requests.post(loginURL, params=params, verify=False)
There are security implications in doing this, so whether you should do that is a matter for you to decide based on your application's requirements.

Troubleshooting Python API Error Message (Highrise API)

I ran this script (https://github.com/mikehking/highrise-analysis/blob/master/highrise-analysis.py) to calculate usage by user for the Highrise CRM system (www.highrisehq.com). Recently, the script has been returning this error; but I'm not sure what it is telling me -- can anyone help with where I should be looking to fix this?
Thanks
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 557, in urlopen
body=body, headers=headers)
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 382, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/lib/python3.4/http/client.py", line 1065, in request
self._send_request(method, url, body, headers)
File "/usr/lib/python3.4/http/client.py", line 1103, in _send_request
self.endheaders(body)
File "/usr/lib/python3.4/http/client.py", line 1061, in endheaders
self._send_output(message_body)
File "/usr/lib/python3.4/http/client.py", line 906, in _send_output
self.send(msg)
File "/usr/lib/python3.4/http/client.py", line 841, in send
self.connect()
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 133, in connect
ssl_version=resolved_ssl_version)
File "/usr/lib/python3/dist-packages/urllib3/util.py", line 619, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "/usr/lib/python3.4/ssl.py", line 364, in wrap_socket
_context=self)
File "/usr/lib/python3.4/ssl.py", line 578, in __init__
self.do_handshake()
File "/usr/lib/python3.4/ssl.py", line 805, in do_handshake
self._sslobj.do_handshake()
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:598)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/requests/adapters.py", line 330, in send
timeout=timeout
File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 590, in urlopen
raise SSLError(e)
urllib3.exceptions.SSLError: EOF occurred in violation of protocol (_ssl.c:598)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/mikehking/workspace/highrise-analysis/highrise-analysis.py", line 188, in <module>
Create_Notes_Backup(PROD_API_KEY, PROD_API_USR, 'highrise-production-notes.bak', 'highrise-production-users.bak', 'highrise-production-people.bak', 'highrise-production-cases.bak', trailing_days = 365) # Production Environment
File "/home/mikehking/workspace/highrise-analysis/highrise-analysis.py", line 71, in Create_Notes_Backup
tmp_notes = high.get_person_notes(person.highrise_id)
File "/usr/local/lib/python3.4/dist-packages/highton/highton.py", line 436, in get_person_notes
return self._get_notes(subject_id, 'people')
File "/usr/local/lib/python3.4/dist-packages/highton/highton.py", line 433, in _get_notes
highrise_type, subject_id)), Note)
File "/usr/local/lib/python3.4/dist-packages/highton/highton.py", line 115, in _get_data
content = self._get_request(endpoint, params).content
File "/usr/local/lib/python3.4/dist-packages/highton/highton.py", line 44, in _get_request
params=params,
File "/usr/lib/python3/dist-packages/requests/api.py", line 55, in get
return request('get', url, **kwargs)
File "/usr/lib/python3/dist-packages/requests/api.py", line 44, in request
return session.request(method=method, url=url, **kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 455, in request
resp = self.send(prep, **send_kwargs)
File "/usr/lib/python3/dist-packages/requests/sessions.py", line 558, in send
r = adapter.send(request, **kwargs)
File "/usr/lib/python3/dist-packages/requests/adapters.py", line 385, in send
raise SSLError(e)
requests.exceptions.SSLError: EOF occurred in violation of protocol (_ssl.c:598)
The last traceback suggests that the script is trying to Create_Notes_Backup, but encounters an error when communicating with the Highrise API.
The error message:
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:598)
is consistent with a server that accepts your connection but then closes it immediately or prematurely. This might be a firewall issue.
If you have OpenSSL installed on this machine, you can try something like this to check the connection:
openssl s_client -crlf -connect USER.highrisehq.com:443
where USER is your Highrise user. You should normally get a long TLS handshake trace full of technical details, ending with --- and then it waits for your input.

Categories

Resources