I'm trying to access a table on factfinder.census.gov via their API.
I've tried the solutions listed here and tried using http, all sorts of variants of this code, etc. In fact, I can't even seem to do a simple get (I'm using python requests) on http://factfinder.census.gov/ at all.
E.g.,
https://factfinder.census.gov/service/data/v1/en/programs/DEC/datasets/10_SF1/tables/GCTPH1/data/0100000US.04000
I had to tack on a user access key (from https://factfinder.census.gov/service/UserAccessKey.html) to get access, and then I could just paste https://factfinder.census.gov/service/data/v1/en/programs/DEC/datasets/10_SF1/tables/GCTPH1/data/0100000US.04000?key=MYKEY and it worked fine in private/incognito mode by just pasting it into my browser url bar)
However, I'm getting
Error
requests.exceptions.SSLError: HTTPSConnectionPool(host='factfinder.census.gov', port=443): Max retries exceeded with url: /service/data/v1/en/programs/DEC/datasets/10_SF1/tables/GCTPH1/data/0100000US.04000?key=MYKEY (Caused by SSLError(SSLError(1, '[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:598)'),))
I'd hazard a guess that you're querying the link you posted (http://factfinder.census.gov/) using http not https (notice if you click that link it redirects to https), the site uses SSL encryption and is rejecting the unencrypted connection attempt.
Related
I am trying to connect to the server doing:
nexus_config = nexuscli.nexus_config.NexusConfig(username=NEXUS_USER,
password=NEXUS_PASSWORD,
url=NEXUS_URL,
x509_verify=True)
nexus_client = nexuscli.nexus_client.NexusClient(config=nexus_config)
print(nexus_client.repositories.list)
But, I get the error:
nexuscli.exception.NexusClientConnectionError: HTTPSConnectionPool(host='my_site.net', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1091)')))
https://nexus3-cli.readthedocs.io/en/latest/nexuscli.html#module-nexuscli.nexus_config
checking the URL using the API endpoing with the requests library and setting validate=False I get a warning and does work.
So I change my code to: x509_verify=True hoping that the same from requests will happen that i will be getting warnings but instead i got.
Is there something I am missing? or maybe there is a bug in the library?
thanks guys.
nexuscli.exception.NexusClientAPIError: <exception str() failed>
You are getting the exception: NexusClientAPIError. According to the documents:
exception nexuscli.exception.NexusClientAPIError Bases: Exception
Unexpected response from Nexus service.
I think your code is fine and the issue with x.509 certificate is resolved and the problem might be on the server's side.
I'm using urllib3 to get the html content of a website via a socks proxy.
Unfortunately this does not work on any website. Running this on a few sites will give me an error.
import urllib3
from urllib3.contrib.socks import SOCKSProxyManager
proxy = SOCKSProxyManager('socks5://myProxyIP:8080')
r = proxy.request('GET', "https://urlofwebsite", preload_content=False)
urllib3.exceptions.MaxRetryError:
SOCKSHTTPSConnectionPool(host='urlofwebsite', port=443): Max retries
exceeded with url: /index.php (Caused by SSLError(SSLError(1, u'[SSL:
UNSUPPORTED_PROTOCOL] unsupported protocol (_ssl.c:727)'),))
I assume this might be an issue with the SSL (or TLS) version the site is using but as I am not the owner of the server I have to edit my script to handle with this.
Is it possible to change the setting in urllib3 to accept this connection?
Thanks!
How do I keep my script running after encountering this error?
requests.exceptions.SSLError:
HTTPSConnectionPool(host='www.funcate.org.br', port=443): Max retries
exceeded with url: /pt/portal-de-compras?file=../../../index.php%250A
(Caused by SSLError(SSLError(1, u'[SSL: CERTIFICATE_VERIFY_FAILED]
certificate verify failed (_ssl.c:718)'),))
You can switch off SSL certificate verification by passing verify=False as an extra argument to the requests.get() function:
response = requests.get('https://foobar.com.br/', verify=False)
Be advised that this will make you susceptible to all sorts of man in the middle attacks. SSL certificates are used for a reason :-) Although I realize that you are not necessarily in a position to enforce this.
I use python-requests to talk to HTTPS web services, some of which present incomplete certificate X509 chains. I'm having trouble figuring out how to access the invalid/incomplete certificates in order to explain the error to the user.
Here's an example illustrated by https://ssllabs.com/ssltest, where the server sends only the leaf certificate, and not the intermediate certificate which is necessary for validation, but missing from certifi's root CA store:
When I try to connect with python-requests, I get an exception that isn't very useful:
request.get('https://web.service.com/path')
SSLError: HTTPSConnectionPool(host='web.service.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",),))
Obviously, I can use separate tools to figure out what's wrong in any particular case (e.g. gnutls-cli, openssl s_client, SSLLabs, etc.).
However, what I really want to be able to do is to be able to catch and diagnose the problem with the certificate chain in my Python code, so that I can present a more specific error message to the user. This answer suggests a monkey-patch to the response object; it's not particularly elegant, but it works—though only when the response object is returned successfully, and not in the case of an exception.
What are the cleanest ways to instrument requests to save the peer's certificate chain in the exception object returned when requests fails to validate the certificate chain itself?
Take requests.get("https://151.101.1.69") # stackoverflow's ip as an example:
try:
requests.get("https://151.101.1.69")
except requests.exceptions.SSLError as e:
cert = e.args[0].reason.args[0]._peer_cert
Then cert is a dict contains the peer's certificate. As I'm not very familiar with SSL, I don't know if it is enough for your case.
BTW, in this case the error is "hostname '151.101.1.69' doesn't match either of '*.stackexchange.com', ...omitted. I'm not sure about the structure of exception in your real case, so you may need to find it on your own. I think it should have the same name _peer_cert.
update
The above method doesn't work when handshake fails... But it still can be done:
try:
requests.get("https://fpslinux1.finalphasesystems.com/")
except requests.exceptions.SSLError:
import ssl
import OpenSSL
cert = ssl.get_server_certificate(('fpslinux1.finalphasesystems.com', 443))
cert = OpenSSL.crypto.load_certificate(OpenSSL.crypto.FILETYPE_PEM, cert)
print(cert.get_issuer())
print(cert.get_subject().get_components())
Yes it is a little dirty but I don't have a better method as a ssl socket doesn't
even return invalid certs from C level :/
To use OpenSSL, you need to install pyopenssl.
I've looked at and tried several stack solutions regarding the https requests using a proxy with python, and I've seen the discussions on GitHub.
My impression was that the requests library in python 3 now has support for https requests using a proxy and so I don't understand why mine doesn't work:
import requests
proxydict = {
'http':'http://xx.xx.xx.xxx:5555/',
'https':'https://xx.xx.xxx.xx:5555/'
}
requests.get('https://www.google.co.uk',proxies = proxydict).
When I run this code I get:
ConnectionError: HTTPSConnectionPool(host='www.python.org', port=443):
Max retries exceeded with url: / (Caused by SSLError(SSLError("bad handshake:
Error([('SSL routines', 'ssl3_get_server_certificate', 'certificate verify failed')],)",),))
I was used a postman proxy to do this.
My http requests with a proxy work fine. Can someone help?
Edit:
I wanted to do this with a corporate proxy so I can't simply use a different proxy. Also, the https addresses work fine in the browser, it's simply when I do a python request with the proxy as described in the documentation that this error occurs.
Thanks.
Hi because your proxy is not working with ssl or your proxy is used try other
from here
https://www.sslproxies.org/
scroll down and you see https proxies