Develop Reference

Python Eclipse Paho Client - TLS Connection to MQTT Broker Exception: No ciphers available

I am trying to create a connection to a TLS (TLSv1) secured MQTT Broker(Rabbitmq with MQTT Plugin enabled) with the python implementation of the eclipse paho client. The same works fine with the MQTTFX application which is based on the java implementation of paho. For this i am using self signed certificates.
Java version uses:
CA-File: ca_certificate.crt
Client Certificate client_cert.crt
Client Key File: client_key.key
Python Version should use:
CA-File: ca_certificate.pem
Client Certificate: client_cert.pem
Client key file: client_key.key
I tried to establish a connection like this:
import ssl
import paho.mqtt.client as paho
# Locations of CA Authority, client certificate and client key file
ca_cert = "ca_certificate.pem"
client_cert = "client_certificate.pem"
client_key = "client_key.pem"
# Create ssl context with TLSv1
context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
context.load_verify_locations(ca_cert)
context.load_cert_chain(client_cert, client_key)
# Alternative to using ssl context but throws the exact same error
# client.tls_set(ca_certs=ca_cert, certfile=client_cert, keyfile=client_key, tls_version=ssl.PROTOCOL_TLSv1)
client = paho.Client()
client.username_pw_set(username="USER", password="PASSWORD")
client.tls_set_context(context)
client.tls_insecure_set(False)
client.connect_async(host="HOSTNAME", port="PORT")
client.loop_forever()
Which results in the following error:
ssl.SSLError: [SSL: NO_CIPHERS_AVAILABLE] no ciphers available (_ssl.c:997)
Could it be that I need to explicitly pass a cipher that the broker supports or could it be due of an older openssl version? I am a little bit lost right now, maybe someone has a clue on how to solve this.
Edit: I got it to work by myself but still not sure why exactly it works now.
Changed context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
to context = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
Changed client.tls_insecure_set(False)
to client.tls_insecure_set(True)

PROTOCOL_TLSv1 forces the client to only use TLS v1.0 which is old and unless you have explicitly forced your broker to only use the same version unlikely to match.
Using PROTOCOL_TLS_CLIENT will allow Python to negotiate across the full range of TLS v1.0 to TLS v1.3 until it finds one that both the client and the broker support.
Why you are having to set client.tls_insecure_set(True) is hard to answer without knowing more about the certificates you are using with the broker. Does it container a CA/SAN entry that matches the HOSTNAME you are using to connect? The documentation says it will explicitly enforce the hostname check.
ssl.PROTOCOL_TLS_CLIENT
Auto-negotiate the highest protocol version that both the client and
server support, and configure the context client-side connections. The
protocol enables CERT_REQUIRED and check_hostname by default.

Get a server certificate despite handshake failure in Python

I am writing a tool to monitor server certificate expiration. I'm using python3 ssl and socket modules to get the server cert using a pretty basic method of creating a default context, disabling hostname validation and certificate verification, calling SSLSocket.connect(), then SSLSocket.getpeercert(), with the sole purpose of grabbing the server certificate, and that is all.
This is all within a private network and I am not concerned with validation.
I have some devices that require client certs signed by a private CA (which my tool doesn't have), so the handshake fails on SSLSocket.connect(), making SSLSocket.getpeercert() impossible.
I know that the server certificate is indeed being provided to my client (along with that pesky Certificate Request) during the handshake. I can see it in a packet capture, as well as just using the openssl s_client command line.
Here is my code.
def get_cert(self, host, port):
ctx = ssl.create_default_context()
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE
with ctx.wrap_socket(socket.socket(), server_hostname=host) as s:
s.settimeout(10)
s.connect((host, port))
binary_cert = s.getpeercert(True)
cert = OpenSSL.crypto.load_certificate(OpenSSL.crypto.FILETYPE_ASN1, binary_cert)
pem_cert = OpenSSL.crypto.dump_certificate(OpenSSL.crypto.FILETYPE_PEM, cert).decode()
return pem_cert
Is there any way to get a little lower into the handshake messages to get the server cert, even though the handshake ultimately fails?
My current solution is to just run openssl s_client -connect host:port using subprocess.run() in the event of a ssl.SSLError.

You may catch exception that do_handshake() produced and then continue to process server certificate.
import OpenSSL
import socket
dst = ('10.10.10.10', 443)
sock = socket.create_connection(dst)
context = OpenSSL.SSL.Context(OpenSSL.SSL.SSLv23_METHOD)
connection = OpenSSL.SSL.Connection(context, sock)
connection.set_connect_state()
try:
connection.do_handshake()
except:
print(connection.get_peer_cert_chain())
Tested on python 2.7.17 and 3.8.5

It looks like there's unfortunately no way to do it with python's ssl module in versions < 3.10. In those versions, the only way to get the peer certificate that I can see is through the low-level _ssl.SSLSocket.getpeercert() method and that immediately throws exception if the handshake is not complete.
Since python 3.10, there's a new _ssl.SSLSocket.get_unverified_chain() method that does not do the handshake check, so perhaps something like this abomination could work?
ssock = context.wrap_socket(sock, do_handshake_on_connect=False)
try:
ssock.do_handshake()
except ssl.SSLError as e:
pass
certs = ssock._sslobj._sslobj.get_unverified_chain()
... but I have not tested it.

SSL: SSLV3_ALERT_HANDSHAKE_FAILURE sslv3 alert handshake failure (_ssl.c:833)

I have a simple TLS client in python running in Ubuntu 18.04 and openssl version 1.1.0g. The client supports a single ciphersuite. I get an error when trying to connect to a TLS 1.0 server. The cipher suite is not supported by the server. I know that the reason for the error is most likely due to lack of ciphersuite mismatch but I am looking for a more meaningful error for the user in this case. The error I am getting at the moment is pointing to SSLv3 which neither the client nor the server has anything to do with SSLv3. The client disables SSLv3 and the server as well. This is the error :
[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:833)
My question is: I need a better error message says for example (lack of ciphersuite mismatch or something like that is relevant to ciphersuite issue). Is there any? Of course I could write my own message but the socket connection can fail for various reasons and I can not make a general error that always says "ciphersuite mismatch".
This is the client script:
import socket,ssl
import itertools
context = ssl.SSLContext()
context.verify_mode = ssl.CERT_NONE
context.check_hostname = False
ciphers = "ECDHE-ECDSA-AES128-GCM-SHA256"
context.set_ciphers(ciphers)
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
domainName = "privatedomain.com"
sslSocket = context.wrap_socket(s, server_hostname = domainName)
try:
sslSocket.connect((domainName, 443))
except (ssl.SSLError, ssl.SSLEOFError, ssl.CertificateError,ssl.SSLSyscallError, ssl.SSLWantWriteError, ssl.SSLWantReadError,ssl.SSLZeroReturnError) as e:
print("Error: ",e)
sslSocket.close()

From the client's view, it is not possible to get another message than the one sent by the server, which is handshake failure in your case. The error message are, for example, documented in RFC 2246 7.2.
The reason why you see SSLv3 in your message, is that you probably send a SSLv3 Hello, which is something allowed to negotiate a TLS 1.0 or later protocol.

Late answer but hopefully helpful . . .
Both client and server must agree on the transport layer version for the connection to be successful. Consider meeting a person for the first time. The person (client) extends their hand to you (server) in a gesture of greeting. If you just saw the person come out of the latrine without washing hands and you see (and/or smell) something undesirable, you will not extend your hand in return.
It is similar with an SSL handshake. The client says "Hey I'd like to communicate via TLS v1.0". The savvy admin for the server knows TLS v1.0 is not secure and they have disabled it on the server--so the server responds to the client, "No, but how about version 1.3?" (ie: "Go wash your hands first"). If the client accepts (washes hands), the handshake is accepted and the connection is established. If the client refuses, the server keeps asking for lower versions ("How about a gallon of Purell then?") until the client accepts or the server has no other versions to offer (walks away).
Basically, the handshake is designed to use the highest version that both the client and server support.
This page has a nice table of versions for client & server (about half way down in the "SSL Contexts" section:
https://docs.python.org/3/library/ssl.html
Note that TLS v1.0 is no longer considered secure (Google "POODLE attack"). If your server supports it, disable it ASAP.

For me this:
urllib.error.URLError: <urlopen error [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:1123)>
meant I was doing this
cipherstr = 'MEDIUM:!aNULL:!eNULL'
context = ssl._create_unverified_context()
context.set_ciphers(cipherstr)
commenting out the set_ciphers and it works now.
Other thing to check: make sure your version of OpenSSL is new enough.

Forcing requests library to use TLSv1.1 or TLSv1.2 in Python

I am trying to send a POST call using requests library in python to a server. Earlier I was able to successfully send POST calls but recently, the server deprecated TLSv1.0 and now only supports TLSv1.1 and TLSv1.2. Now the same code throws me a "requests.exceptions.SSLError: EOF occurred in violation of protocol (_ssl.c:590)" error.
I found this thread on stackoverflow Python Requests requests.exceptions.SSLError: [Errno 8] _ssl.c:504: EOF occurred in violation of protocol which says that we need to subclass the HTTPAdapter after which the session object will use TLSv1. I changed my code accordingly and here is my new code
class MyAdapter(HTTPAdapter):
def init_poolmanager(self, connections, maxsize, block=False):
self.poolmanager = PoolManager(
num_pools=connections,
maxsize=maxsize,
block=block,
ssl_version=ssl.PROTOCOL_TLSv1
)
url="https://mywebsite.com/ui/"
headers={"Cookie":"some_value","X-CSRF-Token":"some value","Content-Type":"application/json"}
payload={"name":"some value","Id":"some value"}
s = requests.Session()
s.mount('https://', MyAdapter())
r=s.post(url,json=payload,headers=headers)
html=r.text
print html
But even after using this, I get the same error EOF occurred in violation of protocol (_ssl.c:590).
My first question is that, I somewhere read that requests by default uses ssl. I know that my server used TLSv1.0 then, so was my code working because TLSv1.0 has backward compatibility with ssl3.0 ?
My second question is that, the stackoverflow thread that I mentioned above using which I changed my code to subclass HTTPAdapter, said that this will work for TLSv1. But since TLSv1.0 is deprecated in my server, will this code still work?

The TLS stack will use the best version available automatically. If it does not work any longer when TLS 1.0 support is disabled at the server it usually means that your local TLS stack simply does not support newer protocol version like TLS 1.2. This is often the case on Mac OS X since it ships with a rotten old version of OpenSSL (0.9.8). In this case no python code will help you to work around the problem, but you need to get a python which uses a newer version of OpenSSL.
To check which openssl version you are using execute the following within python:
import ssl
print(ssl.OPENSSL_VERSION)
To have support for TLS 1.2 you need OpenSSL version 1.0.2 or 1.0.1. If you have only 1.0.0 or 0.9.8 you need to upgrade your python+OpenSSL. See Updating openssl in python 2.7 for more information on how to do this.

I was getting random connection errors from very old server (it's rated F by https://www.ssllabs.com) until I wasn't start using this code in my HTTPAdapter:
def init_poolmanager(self, *args, **kwargs):
ssl_context = ssl.create_default_context()
# Sets up old and insecure TLSv1.
ssl_context.options &= ~ssl.OP_NO_TLSv1_3 & ~ssl.OP_NO_TLSv1_2 & ~ssl.OP_NO_TLSv1_1
ssl_context.minimum_version = ssl.TLSVersion.TLSv1
# Also you could try to set ciphers manually as it was in my case.
# On other ciphers their server was reset the connection with:
# [Errno 104] Connection reset by peer
# ssl_context.set_ciphers("ECDHE-RSA-AES256-SHA")
# See urllib3.poolmanager.SSL_KEYWORDS for all available keys.
kwargs["ssl_context"] = ssl_context
return super().init_poolmanager(*args, **kwargs)

Python TLS1.2 socket server

I have an assignment to create a secure socket server using TLS version 1.1 or 1.2. I'm using python 3.4 (as that's the only version with native TLS 1.1/1.2 support). I've made a self-signed CA and signed both the client and the server. A snippet of the code is as follows:
In my server:
tls_server = ssl.wrap_socket(server, ssl_version=ssl.PROTOCOL_TLSv1_2,
cert_reqs=ssl.CERT_NONE, server_side=True,
keyfile='./server.key', certfile='./server.crt',
ca_certs='./SigningCA/signing-ca.crt')
and in the client:
tls_client = ssl.wrap_socket(client, keyfile="./client.key",
certfile="./client.crt", ssl_version=ssl.PROTOCOL_TLSv1_2,
cert_reqs=ssl.CERT_REQUIRED, ca_certs='./SigningCA/signing-ca.crt')
The connection works fine, I get a request and response. But when I print out the results of the client or server cipher() method, I get the following:
('AES256-SHA', 'TLSv1/SSLv3', 256)
which seems to indicate I'm still running TLSv1/SSLv3. Does anyone have some insight into this? Any help would be appreciated.