Develop Reference

Python Eclipse Paho Client - TLS Connection to MQTT Broker Exception: No ciphers available

I am trying to create a connection to a TLS (TLSv1) secured MQTT Broker(Rabbitmq with MQTT Plugin enabled) with the python implementation of the eclipse paho client. The same works fine with the MQTTFX application which is based on the java implementation of paho. For this i am using self signed certificates.
Java version uses:
CA-File: ca_certificate.crt
Client Certificate client_cert.crt
Client Key File: client_key.key
Python Version should use:
CA-File: ca_certificate.pem
Client Certificate: client_cert.pem
Client key file: client_key.key
I tried to establish a connection like this:
import ssl
import paho.mqtt.client as paho
# Locations of CA Authority, client certificate and client key file
ca_cert = "ca_certificate.pem"
client_cert = "client_certificate.pem"
client_key = "client_key.pem"
# Create ssl context with TLSv1
context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
context.load_verify_locations(ca_cert)
context.load_cert_chain(client_cert, client_key)
# Alternative to using ssl context but throws the exact same error
# client.tls_set(ca_certs=ca_cert, certfile=client_cert, keyfile=client_key, tls_version=ssl.PROTOCOL_TLSv1)
client = paho.Client()
client.username_pw_set(username="USER", password="PASSWORD")
client.tls_set_context(context)
client.tls_insecure_set(False)
client.connect_async(host="HOSTNAME", port="PORT")
client.loop_forever()
Which results in the following error:
ssl.SSLError: [SSL: NO_CIPHERS_AVAILABLE] no ciphers available (_ssl.c:997)
Could it be that I need to explicitly pass a cipher that the broker supports or could it be due of an older openssl version? I am a little bit lost right now, maybe someone has a clue on how to solve this.
Edit: I got it to work by myself but still not sure why exactly it works now.
Changed context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
to context = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
Changed client.tls_insecure_set(False)
to client.tls_insecure_set(True)

PROTOCOL_TLSv1 forces the client to only use TLS v1.0 which is old and unless you have explicitly forced your broker to only use the same version unlikely to match.
Using PROTOCOL_TLS_CLIENT will allow Python to negotiate across the full range of TLS v1.0 to TLS v1.3 until it finds one that both the client and the broker support.
Why you are having to set client.tls_insecure_set(True) is hard to answer without knowing more about the certificates you are using with the broker. Does it container a CA/SAN entry that matches the HOSTNAME you are using to connect? The documentation says it will explicitly enforce the hostname check.
ssl.PROTOCOL_TLS_CLIENT
Auto-negotiate the highest protocol version that both the client and
server support, and configure the context client-side connections. The
protocol enables CERT_REQUIRED and check_hostname by default.

RetriesExhaustedError on connecting to HPE iLO 5 through Python iLO REST client

Following is a Python based RESTful library client (recommended by HP https://developer.hpe.com/platform/ilo-restful-api/home) that uses Redfish REST API (https://github.com/HewlettPackard/python-ilorest-library) to connect to the remote HPE iLO5 server of ProLiant DL360 Gen10 based hardware
#! /usr/bin/python
import redfish
iLO_host = "https://xx.xx.xx.xx"
username = "admin"
password = "xxxxxx"
# Create a REST object
REST_OBJ = redfish.redfish_client(base_url=iLO_host,username=username, password=password, default_prefix='/redfish/v1')
# Login into the server and create a session
REST_OBJ.login(auth="session")
# HTTP GET request
response = REST_OBJ.get("/redfish/v1/systems/1", None)
print response
REST_OBJ.logout()
I am getting RetriesExhaustedError when creating REST object. However, I can successfully do SSH to the server from the VM (RHEL7.4) where I am running this script. The authentication details are given correctly. I verified that the Web Server is enabled (both port 443 and 80) in the iLO Security - Access settings. Also, in my VM box the Firewalld service has been stopped and IPTables is flushed. But still connection could not be established. What other possibilities I can try yet?

I found the root cause. The issue is with SSL Certificate verification being done by the Python code.
This can be turned off by setting the environment variable PYTHONHTTPSVERIFY=0 before running the code solved the problem.

This is a very old topic, but perhaps for other people that have a similar issue when accessing the iLO in any way, and not just over Python:
You most likely need to update the firmware in your server, so that the TLS is updated. You will most likely need to use an old browser to do this, as modern versions of Mozilla/Chrome will not work with old TLS. I have had luck with Konqueror.

Forcing requests library to use TLSv1.1 or TLSv1.2 in Python

I am trying to send a POST call using requests library in python to a server. Earlier I was able to successfully send POST calls but recently, the server deprecated TLSv1.0 and now only supports TLSv1.1 and TLSv1.2. Now the same code throws me a "requests.exceptions.SSLError: EOF occurred in violation of protocol (_ssl.c:590)" error.
I found this thread on stackoverflow Python Requests requests.exceptions.SSLError: [Errno 8] _ssl.c:504: EOF occurred in violation of protocol which says that we need to subclass the HTTPAdapter after which the session object will use TLSv1. I changed my code accordingly and here is my new code
class MyAdapter(HTTPAdapter):
def init_poolmanager(self, connections, maxsize, block=False):
self.poolmanager = PoolManager(
num_pools=connections,
maxsize=maxsize,
block=block,
ssl_version=ssl.PROTOCOL_TLSv1
)
url="https://mywebsite.com/ui/"
headers={"Cookie":"some_value","X-CSRF-Token":"some value","Content-Type":"application/json"}
payload={"name":"some value","Id":"some value"}
s = requests.Session()
s.mount('https://', MyAdapter())
r=s.post(url,json=payload,headers=headers)
html=r.text
print html
But even after using this, I get the same error EOF occurred in violation of protocol (_ssl.c:590).
My first question is that, I somewhere read that requests by default uses ssl. I know that my server used TLSv1.0 then, so was my code working because TLSv1.0 has backward compatibility with ssl3.0 ?
My second question is that, the stackoverflow thread that I mentioned above using which I changed my code to subclass HTTPAdapter, said that this will work for TLSv1. But since TLSv1.0 is deprecated in my server, will this code still work?

The TLS stack will use the best version available automatically. If it does not work any longer when TLS 1.0 support is disabled at the server it usually means that your local TLS stack simply does not support newer protocol version like TLS 1.2. This is often the case on Mac OS X since it ships with a rotten old version of OpenSSL (0.9.8). In this case no python code will help you to work around the problem, but you need to get a python which uses a newer version of OpenSSL.
To check which openssl version you are using execute the following within python:
import ssl
print(ssl.OPENSSL_VERSION)
To have support for TLS 1.2 you need OpenSSL version 1.0.2 or 1.0.1. If you have only 1.0.0 or 0.9.8 you need to upgrade your python+OpenSSL. See Updating openssl in python 2.7 for more information on how to do this.

I was getting random connection errors from very old server (it's rated F by https://www.ssllabs.com) until I wasn't start using this code in my HTTPAdapter:
def init_poolmanager(self, *args, **kwargs):
ssl_context = ssl.create_default_context()
# Sets up old and insecure TLSv1.
ssl_context.options &= ~ssl.OP_NO_TLSv1_3 & ~ssl.OP_NO_TLSv1_2 & ~ssl.OP_NO_TLSv1_1
ssl_context.minimum_version = ssl.TLSVersion.TLSv1
# Also you could try to set ciphers manually as it was in my case.
# On other ciphers their server was reset the connection with:
# [Errno 104] Connection reset by peer
# ssl_context.set_ciphers("ECDHE-RSA-AES256-SHA")
# See urllib3.poolmanager.SSL_KEYWORDS for all available keys.
kwargs["ssl_context"] = ssl_context
return super().init_poolmanager(*args, **kwargs)

Connect to FTP TLS 1.2 Server with ftplib

I try to connect to a FTP Server which only supports TLS 1.2
Using Python 3.4.1
My Code:
import ftplib
import ssl
ftps = ftplib.FTP_TLS()
ftps.ssl_version = ssl.PROTOCOL_TLSv1_2
print (ftps.connect('108.61.166.122',31000))
print(ftps.login('test','test123'))
ftps.prot_p()
print (ftps.retrlines('LIST'))
Error on client side:
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:598)
Error on server side:
Failed TLS negotiation on control channel, disconnected. (SSL_accept():
error:140760FC:SSL routines:SSL23_GET_CLIENT_HELLO:unknown protocol)
The credentials in the example are working for testing.

See the end of this post for the final solution. The rest are the steps needed to debug the problem.
I try to connect to a FTP Server which only supports TLS 1.2 Using Python 3.4.1
How do you know?
ssl.SSLEOFError: EOF occurred in violation of protocol (_ssl.c:598)
I would suggest one of the many SSL problems between client and server, like the server not supporting TLS 1.2, no common ciphers etc. These problems are hard to debug because you either get only some SSL alert or the server will simply close the connection without any obvious reason. If you have access to the server look for error messages on the server side.
You may also try to not to enforce an SSL version but use the default instead, so that client and server will agree to the best SSL version both support. If this will still not work try with a client which is known to work with this server and make a packet capture of the good and bad connections and compare. If you need help with that post the packet captures to cloudshark.org.
Edit#1: just tried it with python 3.4.0 and 3.4.2 against a test server:
python 3.4.0 does a TLS 1.0 handshake, i.e. ignores the setting
python 3.4.2 does a successful TLS 1.2 handshake
In both versions ftplib has the minor bug, that it sends AUTH SSL instead of AUTH TLS if ftps.ssl_version is something else then TLS 1.0, i.e. SSLv3 or TLS1.1.+. While I doubt that this is the origin of the problem it might actually be if the FTP server handles AUTH TLS and AUTH SSL differently.
Edit#2 and Solution:
A packet capture shows that setting ftps.ssl_version has no effect and the SSL handshake will still be done with TLS 1.0 only. Looking at the source code of ftplib in 3.4.0 gives:
ssl_version = ssl.PROTOCOL_TLSv1
def __init__(self, host='', user='', passwd='', acct='', keyfile=None,
certfile=None, context=None,
timeout=_GLOBAL_DEFAULT_TIMEOUT, source_address=None):
....
if context is None:
context = ssl._create_stdlib_context(self.ssl_version,
certfile=certfile,
keyfile=keyfile)
self.context = context
Since __init__ is called when ftplib.FTP_TLS() is called the SSL context will be created with the default ssl_version used by ftplib (ssl.PROTOCOL_TLSv1) and not with your own version. To enforce another SSL version you must to provide your own context with the needed SSL version. The following works for me:
import ftplib
import ssl
ctx = ssl._create_stdlib_context(ssl.PROTOCOL_TLSv1_2)
ftps = ftplib.FTP_TLS(context=ctx)
print (ftps.connect('108.61.166.122',31000))
print(ftps.login('test','test123'))
ftps.prot_p()
print (ftps.retrlines('LIST'))
Alternatively you could set the protocol version globally instead of only for this FTP_TLS object:
ftplib.FTP_TLS.ssl_version = ssl.PROTOCOL_TLSv1_2
ftps = ftplib.FTP_TLS()
And just a small but important observation: it looks like that ftplib does not do any kind of certificate validation, since it accepts this self-signed certificate which does not match the name without complaining. This makes a active man-in-the-middle attack possible. Hopefully they will fix this insecure behavior in the future, in which case the code here will fail because of an invalid certificate.

Firstly AFAIK no ftp supports SSL directly, for which ftps is introduced. Also sftp and ftps are two different concepts: http://en.wikipedia.org/wiki/FTPS .Now, your problem is regarding the programming and not related to SSL or FTPs or any such client-server communication
import ftplib
import ssl
ftps = ftplib.FTP_TLS()
#ftps.ssl_version = ssl.PROTOCOL_TLSv1_2
print (ftps.connect('108.61.166.122',31000))
print(ftps.login('test','test123'))
ftps.prot_p()
print (ftps.retrlines('LIST'))
as ftplib has no attribute PROTOCOL_TLSv1_2 besides which it works fine. and well, your host is not responding !
Hopefully it helps !

Python TLS1.2 socket server

I have an assignment to create a secure socket server using TLS version 1.1 or 1.2. I'm using python 3.4 (as that's the only version with native TLS 1.1/1.2 support). I've made a self-signed CA and signed both the client and the server. A snippet of the code is as follows:
In my server:
tls_server = ssl.wrap_socket(server, ssl_version=ssl.PROTOCOL_TLSv1_2,
cert_reqs=ssl.CERT_NONE, server_side=True,
keyfile='./server.key', certfile='./server.crt',
ca_certs='./SigningCA/signing-ca.crt')
and in the client:
tls_client = ssl.wrap_socket(client, keyfile="./client.key",
certfile="./client.crt", ssl_version=ssl.PROTOCOL_TLSv1_2,
cert_reqs=ssl.CERT_REQUIRED, ca_certs='./SigningCA/signing-ca.crt')
The connection works fine, I get a request and response. But when I print out the results of the client or server cipher() method, I get the following:
('AES256-SHA', 'TLSv1/SSLv3', 256)
which seems to indicate I'm still running TLSv1/SSLv3. Does anyone have some insight into this? Any help would be appreciated.