I have a program that runs successfully in my user/virtualenv. The program access an API using requests. For the purposes of this post, the whole program can be read as:
requests.get("https://example.com")
This works totally fine when I call it from the command line. However I'm trying to get it to run under supervisord and for whatever reason when I do it this way it is failing with an SSL Error like the below:
SSLError("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)")
It is using the same user, python environment, directory etc. Any idea what else to check / what else could be causing this?
Edit: I think this might be a firewall rule type of thing. Exploring that option.
Are the SSL certificates available in the supervisord environment? I assume you use requests.certs, so are the certificates there where expected requests.certs.where()?
So it turns out this was a network proxy thing. The machine I was running on uses a squid proxy and I had to add the following line to set the right environment variables in my supervisor config for it to work:
environment=http_proxy=http://proxy.server:3128/,https_proxy=http://proxy.server:3128/
Related
I built an app using Streamlit (on Mac) which uses data from Yahoo Finance and I'm trying to run it locally. It was working fine last week, but now when I try to run it I get the following error:
SSLError: HTTPSConnectionPool(host='finance.yahoo.com', port=443): Max retries exceeded with url: /quote/SLY (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1125)')))
This is also preventing me from installing packages as well, so it is completely halting my project progress. I can't do anything until this is solved.
'I have never really developed an app before, so I have no idea what to do in this situation (I'm really a noob in this area so please forgive my ignorance). I have tried researching this problem online and believe I need to "add a certificate" to my list of certificates but don't understand how to do that. I am not using "requests" in my code. In fact, my code is literally just a simple script in VScode that I run in my terminal with streamlit run myapp.py. I have thought about just disabling SSL certification, but apparently that is a bad idea (I don't even understand how to do that anyway).
I will try my best to show the steps generally, if you have problems feel free to ask.
Step 1:
Obtaining an SSL certificate.
You can purchase one easily from any trusted website.
Step 2:
Install the SSL certificate on your server.
The exact steps for doing this will depend on your hosting environment and the type of certificate you have. But do not fear most of them have a guide to how to add them
For example if you only want to use it in local I believe there is a trick such as this example
Step 3:
Configure your Streamlit app to use the SSL certificate.
In the .streamlit/config.toml file, add the following lines:
[server]
enableCORS = false
headless = true
port = 8080
enableXsrfProtection = true
certfile = /path/to/certificate.crt
keyfile = /path/to/privatekey.key
(I wrote path but those should be the path to given files)
When sending a request to a specific URL I get an SSL error and I am not sure why. First please see the error message I am presented with:
requests.exceptions.SSLError: HTTPSConnectionPool(host='dicmedia.korean.go.kr', port=443): Max retries exceeded with url: /multimedia/naver/2016/40000/35000/14470_byeon-gyeong.wav (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)')))
I searched unsuccessfully to different Stackoverflow questions for the last two days:
I already tried:
https://github.com/Unbabel/COMET/issues/29 (This seems to be related with an internal update Python received relating to the use of specific SSL certificates (not an expert here)
Downloading the certificate in question and directly linking to it with verify="private/etc/ssl/certs"
I am honestly at loss why I receive this error. As the error message itself indicates it seems that the server in question could get my local certificates somehow. The script worked until a week before. I did not update Python before then. Right now I use python 3.10.2 downloaded from the official website.
I don't want to set verify=False as this just skips the verification process and leaves me vulnerable as numerous people already pointed out at different questions. Besides that it really bothers me that I can't resolve the error.
Any help is much appreciated. See the specific request:
import requests
def request(url):
response = requests.get(url, verify="/private/etc/ssl/certs")
print(response)
request("https://dicmedia.korean.go.kr/multimedia/naver/2016/40000/35000/14470_byeon-
gyeong.wav")
After a lot of googling I figured out the solution myself:
The problem - so it seems - was not all certificates needed where included in Pythons cacert.pem file. As I indicated in my question above to tackle this I downloaded the certifi module at first. As this didn't work out as well I suppose certifi missed the necessary certificates as well.
But I suppose not all certificates in the certificate where missing. As answers to similar questions indicated as well mostly what is missing is not the entire chain, but only the intermediate certificates.
After:
1. downloading the necessary certificates (see the lock symbol in your browser; if you're on OSX you need to drag and drop the big images of the certificates to your finder or desktop etc.),
2. converting them to .perm files and bundling them together: cat first_cert.pem second_cert.pem > combined_cert.pem
and
3. providing the specific path of the bundled certificates as indicated in my question: verify="private/etc/ssl/certs (you may of course choose a different file path).
my request got accepted by the server.
I guess my mistake when trying this solution was that I didn't download the entire chain at first, but only the last certificate.
I really hope this helps someone else as a point of reference.
What I am still dying to know though, is why the error popped up in the first place. I didn't change my script at all and use it on a regular basis, but suddenly got presented with said error. Was the reason that the server I tried to reach change its certificates?
Apologies if my terminology is incorrect.
I faced the same issue on MAC OS. My resolution: The issue was with my python installation. I went to the applications/ python folder and clicked on the install certificate command it got fixed after that.
I have installed Openstack CLI and when I try to use any command say
openstack server list
it is throwing the below error
Failed to discover available identity versions when contacting
https://44.128.19.51:5000/v3. Attempting to parse version from URL.
SSL exception connecting to https://44.128.19.51:5000/v3/auth/tokens:
[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed
(_ssl.c:765)
I tried setting the export OS_CACERT=/path/to/ca.crt, but it is not working.
You must provide a proper authorization url. Sometimes the port of the url can be wrong. Like in my case, the authorization url had port 1300 instead of 5000.
And have you sourced your RC file?
Other than for proper authorization url, proper CACERT path or proper authorization certificates it should show this error.
A long time has passed since the question, but if someone like myself faces the problem, enter the "OpenStack" command with the flag "--insecure".
Here's the related documentation.
I followed Matt Wrock's very helpful guide: Understanding and troubleshooting WinRM connection and authentication,
(http://www.hurryupandwait.io/blog/understanding-and-troubleshooting-winrm-connection-and-authentication-a-thrill-seekers-guide-to-adventure) and I was able to set up an SSL connection on my remote machine, which I verified using:
Test-WSMan -ComputerName "My DNS" -UseSSL
... which returned a non-error message. And I can also connect from powershell:
Enter-PSSession -ComputerName "My DNS" -Credential $cred -UseSSL
However, when I run the following python code:
import winrm
s = winrm.Session('My DNS', auth=('Remote Username', 'Remote Password'), transport='ssl')
r = s.run_cmd('ipconfig', ['/all'])
... I get receive the following error code:
winrm.exceptions.WinRMTransportError: 500 WinRMTransport. [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)
The certificate verify failed error makes me think that I improperly configured the SSL configuration; however, I seem to be able to connect from Powershell.
Can anybody tell me what I am doing wrong or how to properly connect using ssl?
Thanks for your time
Problem found with the pywinrm version.
Use python version 2.7.10 and pywinrm version (0.1.1). It Worked fine without any error.
Latest version forcibly verifying the ssl certificate even we put exception handlers to ignore certificate in the script.
The code below:
import urllib2
file = urllib2.urlopen("http://foo.bar.com:82")
works just fine on my mac (OS X 10.8.4 running Python 2.7.1. It opens the URL and I can parse the file with no problems.
When I try the EXACT same code (these two lines) in GoDaddy Python 2.7.3 (or 2.4) I receive an error:
urllib2.URLError: <urlopen error (111, 'Connection refused')
The problem has something to do with the port :82 that is an essential part of the address. I have tried using a forwarding address with masking, etc., and nothing works.
Any idea why it would work in one environment and not in the other (ostensibly similar) environment? Any ideas how to get around this? I also tried Mechanize to no avail. Previous posts have suggested focusing on urllib2.HTTPBasicAuthHandler, but it works fine on my OS X environment without anything special.
Ideas are welcome.
Connection refused means that your operating system tried to contact the remote host, but got a "closed port" message.
Most likely, this is because of a firewall between GoDaddy and foo.bar.com. Most likely, foo.bar.com is only reachable from your computer or your local network, but it also could be GoDaddy preventing access to strange ports.
From a quick look at the GoDaddy support forums, it looks like they only support outgoing requests to ports 80 (HTTP) and 443 (HTTPS) on their shared hosts. See e.g.
http://support.godaddy.com/groups/web-hosting/forum/topic/curl-to-ports-other-than-80/