How do you fix an SSL error caused by a CNAME DNS record?
I have an API hosted on AWS accessible from a URL like https://sd098fs0f98s9f0s.execute-api.us-east-1.amazonaws.com.
To shorten this and rebrand it a little, I setup a CNAME record to map myapi.mydomain.com to sd098fs0f98s9f0s.execute-api.us-east-1.amazonaws.com.
Using the original URL, this simple Python is able to access the API perfectly:
import requests
r = requests.get(url='https://sd098fs0f98s9f0s.execute-api.us-east-1.amazonaws.com')
However, using the new URL:
import requests
r = requests.get(url='https://myapi.mydomain.com')
results in the error:
Traceback (most recent call last):
File ".env/lib/python3.7/site-packages/urllib3/connectionpool.py", line 603, in urlopen
chunked=chunked)
File ".env/lib/python3.7/site-packages/urllib3/connectionpool.py", line 344, in _make_request
self._validate_conn(conn)
File ".env/lib/python3.7/site-packages/urllib3/connectionpool.py", line 843, in _validate_conn
conn.connect()
File ".env/lib/python3.7/site-packages/urllib3/connection.py", line 370, in connect
ssl_context=context)
File ".env/lib/python3.7/site-packages/urllib3/util/ssl_.py", line 355, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "/usr/lib/python3.7/ssl.py", line 423, in wrap_socket
session=session
File "/usr/lib/python3.7/ssl.py", line 870, in _create
self.do_handshake()
File "/usr/lib/python3.7/ssl.py", line 1139, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1076)
What's causing this and how do I fix it?
I see that you're using https. The server certificate is issued to specific hostname (Common name attribute). This name has to match the address you're trying to access with HTTPS. If you added DNS CNAME you have to update the certificate. If I recall correctly you have to add CNAME as Alternative Subject Name attribute, so you'll need a new certificate.
The error "WRONG_VERSION_NUMBER" might be somewhat misleading. It means that the server presented a wrong TLS version. In the TLS handshake, client and the server negotiate the TLS version. If the server supports only 1.0 and your client accept only 1.2+ then "WRONG_VERSION_NUMBER" will appear. But it might also happen if the client gets some unexpected data, like a plain HTTP instead of HTTPS. To check what exactly was the unexpected data, you'll have to capture wireshark / tcpdump network traffic logs. I wasted a lot of time debugging wrong_version_number recently. Only after looking at wireshark logs it became clear that it was the China firewall, because we got HTTP Forbidden to the initial Client Hello.
Related
i need to do a script for imap backup but when i'm trying to connect to the imap server with my script i'm getting that error:
File "c:\Users\Lenovo\Desktop\python\progettoscuola.py", line 5, in <module>
imapSrc = imaplib.IMAP4_SSL('mail.safemail.it')
File "C:\Program Files\Python310\lib\imaplib.py", line 1323, in __init__
IMAP4.__init__(self, host, port, timeout)
File "C:\Program Files\Python310\lib\imaplib.py", line 202, in __init__
self.open(host, port, timeout)
File "C:\Program Files\Python310\lib\imaplib.py", line 1336, in open
IMAP4.open(self, host, port, timeout)
File "C:\Program Files\Python310\lib\imaplib.py", line 312, in open
self.sock = self._create_socket(timeout)
File "C:\Program Files\Python310\lib\imaplib.py", line 1327, in _create_socket
return self.ssl_context.wrap_socket(sock,
File "C:\Program Files\Python310\lib\ssl.py", line 512, in wrap_socket
return self.sslsocket_class._create(
File "C:\Program Files\Python310\lib\ssl.py", line 1070, in _create
self.do_handshake()
File "C:\Program Files\Python310\lib\ssl.py", line 1341, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:997)```
Python 3.10 increased the default security settings of the TLS stack by among other things prohibiting any ciphers which still use the RSA key exchange. RSA key exchange is long considered inferior since it does not provide forward secrecy and is therefore also no longer available in TLS 1.3. So in general the change in Python 3.10 can be considered an improvement.
But, some servers still require this obsolete key exchange and mail.safemail.it seems to be among these. Connecting to such servers with the newly hardened TLS settings will now fail, even if it succeeded with older versions of Python.
To make connections possible again it is necessary to use weaker security settings. For this specific server it can be done by falling back to the DEFAULT ciphers used by OpenSSL. The following code will create a new SSL context and use it for connecting to the host. The important part here is to use weaker settings using ctx.set_ciphers('DEFAULT') .
import imaplib
import ssl
ctx = ssl.create_default_context()
ctx.set_ciphers('DEFAULT')
imapSrc = imaplib.IMAP4_SSL('mail.safemail.it', ssl_context = ctx)
The title really is the question — how do I get urllib3 to download the SSL cert from the remote server when trying to make an HTTPS connection?
Background
Over in ServerVault land, I'm trying to get to the bottom of a problem with AWS autoscaling groups failing to be able to download phpMyAdmin as part of their init process. As part of trying to diagnose that, I'm now over in StackOverflow land, working with python scripts. Here's where I am:
#! /usr/bin/python3
import cfnbootstrap
from cfnbootstrap.packages import requests
from requests import utils
from requests.utils import DEFAULT_CA_BUNDLE_PATH
from requests.packages import urllib3
conn = urllib3.connection_from_url("https://dev.mysql.com/", retries=False)
conn.cert_reqs = 'CERT_REQUIRED'
conn.ca_certs = DEFAULT_CA_BUNDLE_PATH
response = conn.request("GET", "/get/")
conn.close()
print(response.status)
That works fine:
$ ./urltest.py
404
However, if I change it to be
conn = urllib3.connection_from_url("https://www.phpmyadmin.net", retries=False)
response = conn.request("GET", "/downloads/")
I run into my problem:
$ ./urltest.py
Traceback (most recent call last):
File "/usr/lib/python3.7/site-packages/cfnbootstrap/packages/requests/packages/urllib3/connectionpool.py", line 544, in urlopen
body=body, headers=headers)
File "/usr/lib/python3.7/site-packages/cfnbootstrap/packages/requests/packages/urllib3/connectionpool.py", line 341, in _make_request
self._validate_conn(conn)
File "/usr/lib/python3.7/site-packages/cfnbootstrap/packages/requests/packages/urllib3/connectionpool.py", line 762, in _validate_conn
conn.connect()
File "/usr/lib/python3.7/site-packages/cfnbootstrap/packages/requests/packages/urllib3/connection.py", line 238, in connect
ssl_version=resolved_ssl_version)
File "/usr/lib/python3.7/site-packages/cfnbootstrap/packages/requests/packages/urllib3/util/ssl_.py", line 265, in ssl_wrap_socket
return context.wrap_socket(sock, server_hostname=server_hostname)
File "/usr/lib64/python3.7/ssl.py", line 423, in wrap_socket
session=session
File "/usr/lib64/python3.7/ssl.py", line 870, in _create
self.do_handshake()
File "/usr/lib64/python3.7/ssl.py", line 1139, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1091)
It seems very unlikely that the certificate has expired, as if I go check manually by using a web browser, all is good with the world; however, I'd like to see exactly what certificate urllib3 is seeing as having expired. How do I do this?
NB: I'm not calling certify.where() directly, as I'm not sure what shananigans AWS is performing in the background, but since their scripts are auto-installed as part of booting the instance, there's not much I can do about it, so I'm trying to use their internal processes.
The problem is the expiration of the Let's Encrypt DST Root CA X3 that happened on Sept.30 -- https://letsencrypt.org/docs/dst-root-ca-x3-expiration-september-2021/. The requests package doesn't trust the ISRG Root X1 cert and is trying to verify it with the expired cert.
Download the ISRG X1 PEM from https://letsencrypt.org/certificates/ and add it to the cfnbootstrap library's cacert.pem file.
cat isgrootx1.pem >> /path/to/your/python/env/lib/python3.9/site-packages/cfnbootstrap/packages/requests/cacert.pem
You can find the location of your cacert.pem with the following:
from cfnbootstrap.packages.requests.certs import where
print(where())
Also... I would change up your code to make sure you're actually using the cfn packaged requests module rather than the standalone requests. Otherwise, you probably will have to update it in both spots.
from cfnbootstrap.packages.requests.utils import DEFAULT_CA_BUNDLE_PATH
from cfnbootstrap.packages.requests.packages import urllib3
conn = urllib3.connection_from_url("https://www.phpmyadmin.net/", retries=False)
conn.cert_reqs = 'CERT_REQUIRED'
conn.ca_certs = DEFAULT_CA_BUNDLE_PATH
response = conn.request("GET", "/get/")
conn.close()
print(response.status)
If you do want to update the standalone requests and the one packaged with cfnbootstrap, use the following to find and update both cacert paths:
import requests
import cfnbootstrap.packages.requests
print(requests.certs.where())
print(cfnbootstrap.packages.requests.certs.where())
I am attempting to use python sockets to make an Extensible Provisioning Protocol (EPP) request to a domain registrar, which only accepts requests over ssl.
Certificate file: www.myDomain.se.crt
Key File: mydomain.pem
openssl s_client -connect epptestv3.iis.se:700 -cert www.myDomain.se.crt -key mydomain.pem
When I try making request using openssl client I successfully get greeting response from registrar, but when I use following code in python i get ssl certificate error.
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.settimeout(15)
sock.connect(('epptestv3.iis.se', 700))
sock.settimeout(60) # regular timeout
ssl_keyfile='myDomain.pem'
ssl_certfile='www.myDomain.se.crt'
ssl_ciphers='AES256-GCM-SHA384'
ssl_version=ssl.PROTOCOL_TLSv1_2
sock = ssl.wrap_socket(sock,
ssl_keyfile,
ssl_certfile,
ssl_version=ssl_version,
ciphers=ssl_ciphers,
server_side=False,
cert_reqs=ssl.CERT_REQUIRED,
ca_certs=None
)
After executing script I get following error:
Traceback (most recent call last):
File "server_connect.py", line 54, in <module>
ca_certs=ssl_keyfile
File "/usr/lib/python2.7/ssl.py", line 933, in wrap_socket
ciphers=ciphers)
File "/usr/lib/python2.7/ssl.py", line 601, in __init__
self.do_handshake()
File "/usr/lib/python2.7/ssl.py", line 830, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)
Any idea what's wrong here?
From your code:
cert_reqs=ssl.CERT_REQUIRED,
ca_certs=None
From the documentation of wrap_socket:
If the value of this parameter is not CERT_NONE, then the ca_certs parameter must point to a file of CA certificates.
Essentially you are asking in your code to validate the certificate from the server (CERT_REQUIRED) but specify at the same time that you have no trusted root (ca_certs=None). But without trusted root certificates no validation can be done.
Note that changing your code to use CERT_NONE instead would be a bad idea. It would probably work since no certificate validation will be done but it would be open to man in the middle attacks.
I have a Python SSL server that uses a self-signed certificate. I start my server like this:
httpd = BaseHTTPServer.HTTPServer(('', 443), MyHTTPHandler)
httpd.socket = ssl.wrap_socket (httpd.socket, certfile='server.pem', server_side=True, cert_reqs=0)
httpd.serve_forever()
I get the following error when I connect using Firefox:
----------------------------------------
Exception happened during processing of request from ('127.0.0.1', 51194)
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 295, in _handle_request_noblock
self.process_request(request, client_address)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 321, in process_request
self.finish_request(request, client_address)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 334, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 655, in __init__
self.handle()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/BaseHTTPServer.py", line 340, in handle
self.handle_one_request()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/BaseHTTPServer.py", line 310, in handle_one_request
self.raw_requestline = self.rfile.readline(65537)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 480, in readline
data = self._sock.recv(self._rbufsize)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 734, in recv
return self.read(buflen)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 621, in read
v = self._sslobj.read(len or 1024)
SSLError: [SSL: TLSV1_ALERT_UNKNOWN_CA] tlsv1 alert unknown ca (_ssl.c:1751)
----------------------------------------
I do not see this behavior with Chrome or another client. It only happens on the first connection (complaints about certificate) until I accept the certificate. The exception actually does not cause the program to quit.
Why am I getting an error on the server? How can I avoid the exception?
The TLv1 unknown CA alert is sent by some clients if they cannot verify the certificate of the server because it is signed by an unknown issuer CA. You can avoid this kind of exception if you use a certificate which is already trusted by the client or which can be validated against a root CA of the client (don't forget to include the chain certificates too).
If you cannot avoid this error (for instance because you are using a self-signed certificate) then you have to catch the exception and deal with it by closing the connection. You might need to do this by using handle_request to handle each request by itself and catch exceptions instead of using serve_forever.
I had the same error as you, even though I had a signed certificate from Sectigo. Turns out, you need a certificate chain rather than only your domain's certificate itself.
Source
While referring to this site, and the following video: https://www.youtube.com/watch?v=_YjX7rtiAsk
, I found that I need to create a new file called certificate-chain.pem, and manually/with the help of scripts concatenate (join) three certificates - domain certificate, CA certificate and the USERTrust certificate, one after the other.
Then, in the file you need, point to this certificate bundle/chain. This is a solution which I wept on for 7 hours.
I'm trying to send an http get request via the httplib, but I'm facing issues.
conn = httplib.HTTPConnection("10.30.111.13/View")
conn.request("GET", "/Default.aspx")
res = conn.getresponse()
if res.status == 200:
print(res.status)
else:
print("Something went terribly wrong")
I get the following error:
TypeError (cannot concatenate 'str' and 'int' objects).
If put the next line of codes, it works no problem:
conn = httplib.HTTPConnection("www.google.com")
conn.request("GET", "/")
EDIT, here is a more detailed log I managed to pull out of my third party software (it restricts me in turn of python usability):
File "<string>", line 3248, in initialization
File "C:\python22\lib\httplib.py", line 701, in request
self._send_request(method, url, body, headers)
File "C:\python22\lib\httplib.py", line 723, in _send_request
self.endheaders()
File "C:\python22\lib\httplib.py", line 695, in endheaders
self._send_output()
File "C:\python22\lib\httplib.py", line 581, in _send_output
self.send(msg)
File "C:\python22\lib\httplib.py", line 548, in send
self.connect()
File "C:\python22\lib\httplib.py", line 516, in connect
socket.SOCK_STREAM):
gaierror: (7, 'getaddrinfo failed')
I'm not someplace where I can test this now, but here's what I think:
You're passing only an IP address to a host field that's expecting a DNS address, not an IP address. That's why your second error listing says 'getaddrinfo' failed.
That said, I'm not sure how to use an IP address with httplib. Maybe try "http://10.30.111.13" instead. A good way to test it would be to replace your IP address above with Google's and see if you still get the error.
Maybe this will help -- sorry I can't say more!
I have changed the IP address for a DNS address. I also removed any path/URI that were in the HTTPConnection() parameter. Now it works. Sorry for such an obvious question guys.