I am writing a tool to monitor server certificate expiration. I'm using python3 ssl and socket modules to get the server cert using a pretty basic method of creating a default context, disabling hostname validation and certificate verification, calling SSLSocket.connect(), then SSLSocket.getpeercert(), with the sole purpose of grabbing the server certificate, and that is all.
This is all within a private network and I am not concerned with validation.
I have some devices that require client certs signed by a private CA (which my tool doesn't have), so the handshake fails on SSLSocket.connect(), making SSLSocket.getpeercert() impossible.
I know that the server certificate is indeed being provided to my client (along with that pesky Certificate Request) during the handshake. I can see it in a packet capture, as well as just using the openssl s_client command line.
Here is my code.
def get_cert(self, host, port):
ctx = ssl.create_default_context()
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE
with ctx.wrap_socket(socket.socket(), server_hostname=host) as s:
s.settimeout(10)
s.connect((host, port))
binary_cert = s.getpeercert(True)
cert = OpenSSL.crypto.load_certificate(OpenSSL.crypto.FILETYPE_ASN1, binary_cert)
pem_cert = OpenSSL.crypto.dump_certificate(OpenSSL.crypto.FILETYPE_PEM, cert).decode()
return pem_cert
Is there any way to get a little lower into the handshake messages to get the server cert, even though the handshake ultimately fails?
My current solution is to just run openssl s_client -connect host:port using subprocess.run() in the event of a ssl.SSLError.
You may catch exception that do_handshake() produced and then continue to process server certificate.
import OpenSSL
import socket
dst = ('10.10.10.10', 443)
sock = socket.create_connection(dst)
context = OpenSSL.SSL.Context(OpenSSL.SSL.SSLv23_METHOD)
connection = OpenSSL.SSL.Connection(context, sock)
connection.set_connect_state()
try:
connection.do_handshake()
except:
print(connection.get_peer_cert_chain())
Tested on python 2.7.17 and 3.8.5
It looks like there's unfortunately no way to do it with python's ssl module in versions < 3.10. In those versions, the only way to get the peer certificate that I can see is through the low-level _ssl.SSLSocket.getpeercert() method and that immediately throws exception if the handshake is not complete.
Since python 3.10, there's a new _ssl.SSLSocket.get_unverified_chain() method that does not do the handshake check, so perhaps something like this abomination could work?
ssock = context.wrap_socket(sock, do_handshake_on_connect=False)
try:
ssock.do_handshake()
except ssl.SSLError as e:
pass
certs = ssock._sslobj._sslobj.get_unverified_chain()
... but I have not tested it.
Related
I am trying to create a connection to a TLS (TLSv1) secured MQTT Broker(Rabbitmq with MQTT Plugin enabled) with the python implementation of the eclipse paho client. The same works fine with the MQTTFX application which is based on the java implementation of paho. For this i am using self signed certificates.
Java version uses:
CA-File: ca_certificate.crt
Client Certificate client_cert.crt
Client Key File: client_key.key
Python Version should use:
CA-File: ca_certificate.pem
Client Certificate: client_cert.pem
Client key file: client_key.key
I tried to establish a connection like this:
import ssl
import paho.mqtt.client as paho
# Locations of CA Authority, client certificate and client key file
ca_cert = "ca_certificate.pem"
client_cert = "client_certificate.pem"
client_key = "client_key.pem"
# Create ssl context with TLSv1
context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
context.load_verify_locations(ca_cert)
context.load_cert_chain(client_cert, client_key)
# Alternative to using ssl context but throws the exact same error
# client.tls_set(ca_certs=ca_cert, certfile=client_cert, keyfile=client_key, tls_version=ssl.PROTOCOL_TLSv1)
client = paho.Client()
client.username_pw_set(username="USER", password="PASSWORD")
client.tls_set_context(context)
client.tls_insecure_set(False)
client.connect_async(host="HOSTNAME", port="PORT")
client.loop_forever()
Which results in the following error:
ssl.SSLError: [SSL: NO_CIPHERS_AVAILABLE] no ciphers available (_ssl.c:997)
Could it be that I need to explicitly pass a cipher that the broker supports or could it be due of an older openssl version? I am a little bit lost right now, maybe someone has a clue on how to solve this.
Edit: I got it to work by myself but still not sure why exactly it works now.
Changed context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
to context = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
Changed client.tls_insecure_set(False)
to client.tls_insecure_set(True)
PROTOCOL_TLSv1 forces the client to only use TLS v1.0 which is old and unless you have explicitly forced your broker to only use the same version unlikely to match.
Using PROTOCOL_TLS_CLIENT will allow Python to negotiate across the full range of TLS v1.0 to TLS v1.3 until it finds one that both the client and the broker support.
Why you are having to set client.tls_insecure_set(True) is hard to answer without knowing more about the certificates you are using with the broker. Does it container a CA/SAN entry that matches the HOSTNAME you are using to connect? The documentation says it will explicitly enforce the hostname check.
ssl.PROTOCOL_TLS_CLIENT
Auto-negotiate the highest protocol version that both the client and
server support, and configure the context client-side connections. The
protocol enables CERT_REQUIRED and check_hostname by default.
I have a MySQL server hosted at a particular IP and port. Now, I need to validate this connectivity whether the server is up or down. Also this server uses a self-signed SSL certificate.
Does anyone have any reference python snippet that would comply this connectivity validation along with SSL certificate verification?
Trye the same thing you'd do from a command line. E.g. when you ``telnetserver port, then you get "connection refused" if a server is not listening on the given port, or a timeout if server:port is behind a firewall, or "No route to host" etc. For a successful connection you get something.
Self signed certs can't be validate, AFAIK, and this will work for ssl connections too.
import telnetlib
server_ip = '192.168.2.1'
server_port = 80
timeout = 5
conn_ok = False
try:
tn = telnetlib.Telnet(server_ip, server_port, timeout)
conn_ok = True
except Exception as e:
print(e)
tn.close()
I have a simple TLS client in python running in Ubuntu 18.04 and openssl version 1.1.0g. The client supports a single ciphersuite. I get an error when trying to connect to a TLS 1.0 server. The cipher suite is not supported by the server. I know that the reason for the error is most likely due to lack of ciphersuite mismatch but I am looking for a more meaningful error for the user in this case. The error I am getting at the moment is pointing to SSLv3 which neither the client nor the server has anything to do with SSLv3. The client disables SSLv3 and the server as well. This is the error :
[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:833)
My question is: I need a better error message says for example (lack of ciphersuite mismatch or something like that is relevant to ciphersuite issue). Is there any? Of course I could write my own message but the socket connection can fail for various reasons and I can not make a general error that always says "ciphersuite mismatch".
This is the client script:
import socket,ssl
import itertools
context = ssl.SSLContext()
context.verify_mode = ssl.CERT_NONE
context.check_hostname = False
ciphers = "ECDHE-ECDSA-AES128-GCM-SHA256"
context.set_ciphers(ciphers)
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
domainName = "privatedomain.com"
sslSocket = context.wrap_socket(s, server_hostname = domainName)
try:
sslSocket.connect((domainName, 443))
except (ssl.SSLError, ssl.SSLEOFError, ssl.CertificateError,ssl.SSLSyscallError, ssl.SSLWantWriteError, ssl.SSLWantReadError,ssl.SSLZeroReturnError) as e:
print("Error: ",e)
sslSocket.close()
From the client's view, it is not possible to get another message than the one sent by the server, which is handshake failure in your case. The error message are, for example, documented in RFC 2246 7.2.
The reason why you see SSLv3 in your message, is that you probably send a SSLv3 Hello, which is something allowed to negotiate a TLS 1.0 or later protocol.
Late answer but hopefully helpful . . .
Both client and server must agree on the transport layer version for the connection to be successful. Consider meeting a person for the first time. The person (client) extends their hand to you (server) in a gesture of greeting. If you just saw the person come out of the latrine without washing hands and you see (and/or smell) something undesirable, you will not extend your hand in return.
It is similar with an SSL handshake. The client says "Hey I'd like to communicate via TLS v1.0". The savvy admin for the server knows TLS v1.0 is not secure and they have disabled it on the server--so the server responds to the client, "No, but how about version 1.3?" (ie: "Go wash your hands first"). If the client accepts (washes hands), the handshake is accepted and the connection is established. If the client refuses, the server keeps asking for lower versions ("How about a gallon of Purell then?") until the client accepts or the server has no other versions to offer (walks away).
Basically, the handshake is designed to use the highest version that both the client and server support.
This page has a nice table of versions for client & server (about half way down in the "SSL Contexts" section:
https://docs.python.org/3/library/ssl.html
Note that TLS v1.0 is no longer considered secure (Google "POODLE attack"). If your server supports it, disable it ASAP.
For me this:
urllib.error.URLError: <urlopen error [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:1123)>
meant I was doing this
cipherstr = 'MEDIUM:!aNULL:!eNULL'
context = ssl._create_unverified_context()
context.set_ciphers(cipherstr)
commenting out the set_ciphers and it works now.
Other thing to check: make sure your version of OpenSSL is new enough.
I am trying to get the server certificate of badssl.com subdomains (ex. https://expired.badssl.com).
import ssl
ssl.get_server_certificate(('expired.badssl.com', 443))
But when examining the above generated certificate I see that the certificate has
Identity: badssl-fallback-unknown-subdomain-or-no-sni
which means SNI is failing. How can I get the server certificate of different subdomains of badssl.com? (I am using python 2.7.12)
Found the answer.
import ssl
hostname = "expired.badssl.com"
port = 443
conn = ssl.create_connection((hostname, port))
context = ssl.SSLContext(ssl.PROTOCOL_SSLv23)
sock = context.wrap_socket(conn, server_hostname=hostname)
certificate = ssl.DER_cert_to_PEM_cert(sock.getpeercert(True))
Searching for "Python ssl.get_server_certificate SNI" brought me easily to this answer. Although the OP himself answer is correct, I would like to provide a little more insight for future reference.
With some [hostname]s the fallowing call using Python 3.7:
ssl.get_server_certificate(("example.com", 443)
will complain with a traceback that ends with:
ssl.SSLError: [SSL: TLSV1_ALERT_INTERNAL_ERROR] tlsv1 alert internal error (_ssl.c:1045)
Doing some further investigation, making use of the openssl s_client utility, allows to discover that those same [hostname]s which made get_server_certificate to fail, also makes the fallowing command:
openssl s_client -showcerts -connect example.com:443
to fail with this error:
SSL23_GET_SERVER_HELLO:tlsv1 alert internal error:s23_clnt.c:802
Note that the error message is similar to the one returned by the python code.
Using the -servername switch did the trick:
openssl s_client -showcerts -connect example.com:443 -servername example.com
leading to the conclusion that the investigated hostname refers to a secure server that makes use of SNI (a good explanation on what that means is given by the SNI Wikipedia article).
So, switching again to Python and looking at the get_server_certificate method, examining the ssl module source (here for convenience), you can discover that the function includes this call:
context.wrap_socket(sock)
without the server_hostname=hostname key argument, which of course should mean that get_server_certificate cannot be used querying a SNI server. A little more effort is required:
hostname = "example.com"
port = 443
context = ssl.create_default_context()
with socket.create_connection((hostname, port)) as sock:
with context.wrap_socket(sock, server_hostname=hostname) as sslsock:
der_cert = sslsock.getpeercert(True)
# from binary DER format to PEM
pem_cert = ssl.DER_cert_to_PEM_cert(der_cert)
print(pem_cert)
I try to connect to a FTP Server which only supports TLS 1.2
Using Python 3.4.1
My Code:
import ftplib
import ssl
ftps = ftplib.FTP_TLS()
ftps.ssl_version = ssl.PROTOCOL_TLSv1_2
print (ftps.connect('108.61.166.122',31000))
print(ftps.login('test','test123'))
ftps.prot_p()
print (ftps.retrlines('LIST'))
Error on client side:
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:598)
Error on server side:
Failed TLS negotiation on control channel, disconnected. (SSL_accept():
error:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol)
The credentials in the example are working for testing.
See the end of this post for the final solution. The rest are the steps needed to debug the problem.
I try to connect to a FTP Server which only supports TLS 1.2 Using Python 3.4.1
How do you know?
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:598)
I would suggest one of the many SSL problems between client and server, like the server not supporting TLS 1.2, no common ciphers etc. These problems are hard to debug because you either get only some SSL alert or the server will simply close the connection without any obvious reason. If you have access to the server look for error messages on the server side.
You may also try to not to enforce an SSL version but use the default instead, so that client and server will agree to the best SSL version both support. If this will still not work try with a client which is known to work with this server and make a packet capture of the good and bad connections and compare. If you need help with that post the packet captures to cloudshark.org.
Edit#1: just tried it with python 3.4.0 and 3.4.2 against a test server:
python 3.4.0 does a TLS 1.0 handshake, i.e. ignores the setting
python 3.4.2 does a successful TLS 1.2 handshake
In both versions ftplib has the minor bug, that it sends AUTH SSL instead of AUTH TLS if ftps.ssl_version is something else then TLS 1.0, i.e. SSLv3 or TLS1.1.+. While I doubt that this is the origin of the problem it might actually be if the FTP server handles AUTH TLS and AUTH SSL differently.
Edit#2 and Solution:
A packet capture shows that setting ftps.ssl_version has no effect and the SSL handshake will still be done with TLS 1.0 only. Looking at the source code of ftplib in 3.4.0 gives:
ssl_version = ssl.PROTOCOL_TLSv1
def __init__(self, host='', user='', passwd='', acct='', keyfile=None,
certfile=None, context=None,
timeout=_GLOBAL_DEFAULT_TIMEOUT, source_address=None):
....
if context is None:
context = ssl._create_stdlib_context(self.ssl_version,
certfile=certfile,
keyfile=keyfile)
self.context = context
Since __init__ is called when ftplib.FTP_TLS() is called the SSL context will be created with the default ssl_version used by ftplib (ssl.PROTOCOL_TLSv1) and not with your own version. To enforce another SSL version you must to provide your own context with the needed SSL version. The following works for me:
import ftplib
import ssl
ctx = ssl._create_stdlib_context(ssl.PROTOCOL_TLSv1_2)
ftps = ftplib.FTP_TLS(context=ctx)
print (ftps.connect('108.61.166.122',31000))
print(ftps.login('test','test123'))
ftps.prot_p()
print (ftps.retrlines('LIST'))
Alternatively you could set the protocol version globally instead of only for this FTP_TLS object:
ftplib.FTP_TLS.ssl_version = ssl.PROTOCOL_TLSv1_2
ftps = ftplib.FTP_TLS()
And just a small but important observation: it looks like that ftplib does not do any kind of certificate validation, since it accepts this self-signed certificate which does not match the name without complaining. This makes a active man-in-the-middle attack possible. Hopefully they will fix this insecure behavior in the future, in which case the code here will fail because of an invalid certificate.
Firstly AFAIK no ftp supports SSL directly, for which ftps is introduced. Also sftp and ftps are two different concepts: http://en.wikipedia.org/wiki/FTPS .Now, your problem is regarding the programming and not related to SSL or FTPs or any such client-server communication
import ftplib
import ssl
ftps = ftplib.FTP_TLS()
#ftps.ssl_version = ssl.PROTOCOL_TLSv1_2
print (ftps.connect('108.61.166.122',31000))
print(ftps.login('test','test123'))
ftps.prot_p()
print (ftps.retrlines('LIST'))
as ftplib has no attribute PROTOCOL_TLSv1_2 besides which it works fine. and well, your host is not responding !
Hopefully it helps !