Can anyone explain me how to add SSL to the Python STOMP client I'm using.
I added the stomp+ssl transport connector in the ActiveMQ configuration file and my basic Python STOMP client is below:
import time
import sys
import stomp
class MyListener(stomp.ConnectionListener):
def on_error(self, headers, message):
print('received an error "%s"' % message)
def on_message(self, headers, message):
print('received a message "%s"' % message)
conn = stomp.Connection()
conn.set_listener('', MyListener())
conn.start()
conn.connect('admin', 'password', wait=True)
conn.subscribe(destination='/queue/test', id=1, ack='auto')
conn.send(body=' '.join(sys.argv[1:]), destination='/queue/test')
time.sleep(2)
conn.disconnect()
I created the key store and trust store given in the http://activemq.apache.org/how-do-i-use-ssl.html docs and added them to the SSL_OPTS environment variable in the broker but I'm unable to find how to initialize the Python STOMP client with the key store and trust store. Should I use the SSL paraments given in the stomp.Connection() method, and if yes how to do so?
Can anyone please explain if there is any other way to add SSL over STOMP?
The Python STOMP client (as of version 4.1.20) uses an SSLContext to process its key pair/certificate, so there is no reason to produce a Java KeyStore for the client.
With this in mind, let us go through the entire process of setting up ApacheMQ to support SSL-wrapped STOMP connections. The process below has been tested on ApacheMQ 5.15.4. We explicitly set up two-way trust by manually moving self-signed certificates between the broker and client; using a certificate authority is also possible but how to do so is a different question.
Create a client certificate
As mentioned above, on the Python side of things, a KeyStore will have little use, and since SSLContext expects PEM encoded certificates, we might as well create the key pair and certificate by hand (that is, using openssl). First, on the client machine, let us create a 4096-bit RSA key:
openssl genrsa -out client.key 4096
Using this, turn the public key part into a certificate and sign it with the key itself; since we will be manually moving the certificate to the broker, self-signing the certificate is not an issue:
openssl req -new -out client.csr -key client.key
openssl x509 -req -days 365 -in client.csr -signkey client.key -out client.pem
rm client.csr
The STOMP client will need both the signed certificate, client.pem, and the private key, client.key, while the broker will only need the certificate.
Create a broker certificate
On the broker, we can follow the first part of the Apache guide and use the Java keytool to create a KeyStore with a key for the server:
keytool -genkeypair -alias broker -keyalg RSA -keysize 4096 -keystore broker.ks
When prompted for "first and last name", provide the hostname of the server, which in our example we will take simply to be localhost; if the broker and client are running on different servers, make sure that this is set to whatever the Python client will end up using to identify the broker:
What is your first and last name?
[Unknown]: localhost
All other input values can be left as "Unknown".
At the end of the day, we will only want to allow connections to the broker from clients with certificates that we know, so at this point copy the client.pem generated above to the broker and add it to a trust store through
keytool -import -alias client -keystore broker.ts -file client.pem
If the broker is to allow connections from any client, then this final step can be skipped.
Setting up ApacheMQ
By default, all connections through STOMP (and indeed all connections) are plaintext ones, and in order to enable STOMP connections over SSL, add the following <transportConnector /> to conf/apachemq.xml:
<transportConnectors>
<transportConnector name="stomp+ssl" uri="stomp+nio+ssl://0.0.0.0:61613?transport.enabledProtocols=TLSv1.2&needClientAuth=true" />
</transportConnectors>
Make sure to remove any existing plaintext connectors such as the default STOMP connector as otherwise clients will be able to simply use those and bypass the SSL requirement. Note also that needClientAuth=true is what forces client certificate validation; without this, clients are able to connect without providing a trusted certificate.
To configure ApacheMQ to use the key and trust stores defined above, define the environment variable ACTIVEMQ_SSL_OPTS through (on Unix)
export ACTIVEMQ_SSL_OPTS = -Djavax.net.ssl.keyStore=/path/to/broker.ks -Djavax.net.ssl.trustStore=/path/to/broker.ts -Djavax.net.ssl.keyStorePassword=passwordForBrokerKs -Djavax.net.ssl.trustStorePassword=passwordForBrokerTs
or (on Windows)
set ACTIVEMQ_SSL_OPTS=-Djavax.net.ssl.keyStore=C:\path\to\broker.ks -Djavax.net.ssl.trustStore=C:\path\to\broker.ts -Djavax.net.ssl.keyStorePassword=passwordForBrokerKs -Djavax.net.ssl.trustStorePassword=passwordForBrokerTs
Here, the two passwords are those chosen after running keytool in the previous step. If client certificate validation is not desired, simply leave out the configuration of trustStore and trustStorePassword.
With this, ActiveMQ can be started as usual through bin/activemq start. To make sure that the SSL configuration matches expectation, pay attention to the JVM args part of the output printed when starting the server.
Testing the STOMP client
With the broker properly set up, we can configure the client as well. Here, we provide stomp.Connection.set_ssl with references to the key and certificate created in the first step. Assuming that the ActiveMQ server is running on localhost:61613, your test script simply becomes
import time
import sys
import stomp
class MyListener(stomp.ConnectionListener):
def on_error(self, headers, message):
print('received an error "%s"' % message)
def on_message(self, headers, message):
print('received a message "%s"' % message)
host = 'localhost'
port = 61613
conn = stomp.Connection([(host, port)])
conn.set_ssl(for_hosts=[(host, port)], key_file='/path/to/client.key', cert_file='/path/to/client.pem')
conn.set_listener('', MyListener())
conn.start()
conn.connect('admin', 'password', wait=True)
conn.subscribe(destination='/queue/test', id=1, ack='auto')
conn.send(body='test message', destination='/queue/test')
time.sleep(2)
conn.disconnect()
To make sure that ApacheMQ is indeed validating the client certificate, we could repeat step 1 and create a new pair, client2.key/client2.pem say, and use that instead. Doing so should result in the following non-sensical error message being printed by ApacheMQ:
ERROR | Could not accept connection from null : {}
java.io.IOException: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
Validating the broker certificate
Now, the attentive reader will have noticed that we never actually moved the broker certificate to the client, and yet things seem to work regardless. As it turns out, the default behavior of stomp.py is to perform no certificate validation at all, allowing an (active) attacker to MITM the connection.
As we are rolling self-signed certificates, all we need to do to fix this situation is to provide the actual broker certificate to the Python client. On the broker, export the certificate through
keytool -exportcert -rfc -alias broker -keystore broker.ks -file broker.pem
and move broker.pem to the Python client. Now, in the test script above, include the certificate by replacing the SSL configuration with
conn.set_ssl(for_hosts=[(host, port)], key_file='/path/to/client.key', cert_file='/path/to/client.pem', ca_certs='/path/to/broker.pem')
As above, we can test that this is indeed performing the proper validation by repeating the broker certificate generation process to create a broker2.pem, use that in the test script, and note that it will fail with an
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:833)
Try this.
conn = stomp.Connection([(host, port)])
conn.set_listener('', MyListener())
conn.set_ssl(for_hosts=[(host, port)], ssl_version=ssl.PROTOCOL_TLS)
conn.start()
conn.connect(login, password, wait=True)
conn.send(body=message, destination=queue)
conn.disconnect()
or
conn.set_ssl(for_hosts=[(host, port)], ssl_version=_ssl.PROTOCOL_TLS)
The key point is understanding the key/cert files must be there - otherwise even stomp.py tests in their repository will not work. TLS will not work in stomp.py if the file names are not specified (correctly).
See the code & key generation instructions here: https://gist.github.com/borislitvak/6ccea503abf1b2f9c89e87309d6dab88
Related
I need your help.
I want to generate a certificate for a client for authorization on my api.
In my use case, I have an API that is hosted with python hypercorn and fastapi. Then I have multiple clients (also python (httpx)) that should request data from this api. For authentication between the client and the server, I want to use certificates. I want to provide the client with a certificate with which it can authorize itself with the server.
For generating the certificates i used this instruction: https://www.makethenmakeinstall.com/2014/05/ssl-client-authentication-step-by-step/
What did I do wrong, or how can I implement my use case?
server:
async def main():
config = Config.from_mapping(dict(
worker_class='trio',
certfile='server.cer',
keyfile='server.key',
verify_mode=VerifyMode.CERT_REQUIRED,
bind=f"0.0.0.0:{8000}"))
async with trio.open_nursery() as nursery:
nursery.start_soon(serve, app, config)
client:
import httpx
res = httpx.get("https://localhost:8000/", verify=True, cert=("client.cer", "client.key"))
res
When I execute the request in the client I get the following error:
ConnectError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate (_ssl.c:1123)
let's simplify how certificates work.
certificates used for authentication and authorization and this is what you're exactly doing in your case. your backend-server has its own certificate, and your client has his own certificate. your applications is set to recognize each other using these certificates and you're doing it in a great way.
but certificates are vulnerable and can be created locally, so I myself can create a certificate that says that I'm facebook or google. and here is the validation's role come; a well know authority that's already implemented in all browsers and OS - and we call it CA for a certification authority - signs your certificates, so when your server send its cert to a client, the client's browser would recognize the signature and tells you that the certificate is valid.
in your case here you have a self signed certificate, which means that for development purpose you've signed your cert by yourself "instead of the recognized CAs".
Overcoming such an issue would be in two ways.
you disable the validation in your code and in your client side as well. suits for development purpose or intra network.
sign your certificate and your client's certificate with a recognized authority like digicert, and add your CA in your trust-store.
I am trying to create a connection to a TLS (TLSv1) secured MQTT Broker(Rabbitmq with MQTT Plugin enabled) with the python implementation of the eclipse paho client. The same works fine with the MQTTFX application which is based on the java implementation of paho. For this i am using self signed certificates.
Java version uses:
CA-File: ca_certificate.crt
Client Certificate client_cert.crt
Client Key File: client_key.key
Python Version should use:
CA-File: ca_certificate.pem
Client Certificate: client_cert.pem
Client key file: client_key.key
I tried to establish a connection like this:
import ssl
import paho.mqtt.client as paho
# Locations of CA Authority, client certificate and client key file
ca_cert = "ca_certificate.pem"
client_cert = "client_certificate.pem"
client_key = "client_key.pem"
# Create ssl context with TLSv1
context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
context.load_verify_locations(ca_cert)
context.load_cert_chain(client_cert, client_key)
# Alternative to using ssl context but throws the exact same error
# client.tls_set(ca_certs=ca_cert, certfile=client_cert, keyfile=client_key, tls_version=ssl.PROTOCOL_TLSv1)
client = paho.Client()
client.username_pw_set(username="USER", password="PASSWORD")
client.tls_set_context(context)
client.tls_insecure_set(False)
client.connect_async(host="HOSTNAME", port="PORT")
client.loop_forever()
Which results in the following error:
ssl.SSLError: [SSL: NO_CIPHERS_AVAILABLE] no ciphers available (_ssl.c:997)
Could it be that I need to explicitly pass a cipher that the broker supports or could it be due of an older openssl version? I am a little bit lost right now, maybe someone has a clue on how to solve this.
Edit: I got it to work by myself but still not sure why exactly it works now.
Changed context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
to context = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
Changed client.tls_insecure_set(False)
to client.tls_insecure_set(True)
PROTOCOL_TLSv1 forces the client to only use TLS v1.0 which is old and unless you have explicitly forced your broker to only use the same version unlikely to match.
Using PROTOCOL_TLS_CLIENT will allow Python to negotiate across the full range of TLS v1.0 to TLS v1.3 until it finds one that both the client and the broker support.
Why you are having to set client.tls_insecure_set(True) is hard to answer without knowing more about the certificates you are using with the broker. Does it container a CA/SAN entry that matches the HOSTNAME you are using to connect? The documentation says it will explicitly enforce the hostname check.
ssl.PROTOCOL_TLS_CLIENT
Auto-negotiate the highest protocol version that both the client and
server support, and configure the context client-side connections. The
protocol enables CERT_REQUIRED and check_hostname by default.
Currently my client side HTTPS request code looks like this:
resp = requests.post(endpoint_predict_v100, files={'image': open(file_path, 'rb')}, verify=client_auth,
cert=(client_cert_path, client_secret_path))
This is an HTTPS request, the server root CA is client_auth and the client certificate and key are client_cert_path and client_secret_path.
because my server code is deployed and I cannot change much on that side, I am wondering if I can do the following on the client side:
enable server authentication only when the server root CA is provided by the client
enable client authentication only when the client cert and key are provided by the client.
I have found out that I am able to do the first thing, "enable server authentication only when the server root CA is provided by the client", by passing verify=False if the server root CA is not presented on the client side., referring to the Python requests docs. However, when I try to remove the cert=(client_cert_path, client_secret_path), the request will fail, throwing this error:
ssl.SSLError: [SSL] tlsv13 alert certificate required (_ssl.c:2570)
It seems that this error is suggesting I must pass in the client cert and key to make the server accept my request. but is there a way to disable client authentication on the client side, when the client cert and key are not present?
ssl.SSLError: [SSL] tlsv13 alert certificate required (_ssl.c:2570)
... is there a way to disable client authentication on the client side, when the client cert and key are not present?
The alert you get is because the server is requiring the client certificate. Since it is required by the server it is not sufficient to change some client side behavior only. This is similar to a server requesting authentication by password - this cannot be simply skipped in the client too.
With many servers there is a way to have client certificates optional, i.e. request a client certificate but don't insist that one is sent. How to do this with your specific server and if it is supported by your server at all is unknown though.
I know in python-requests you can send your certificate via verify parameter.
Something like this:
url = requests.get('someurl', verify='certifacate.pem')
So I understand TLS handshake protocol does follow:
The 'client hello' message
The client initiates the handshake by sending a "hello" message to the server. The message will include which TLS version the client supports, the cipher suites supported, and a string of random bytes known as the "client random.
Then we have the server hello message that lets client know what cipher suite to use.
The 'server hello' message: In reply to the client hello message, the server sends a message containing the server's SSL certificate, the server's chosen cipher suite, and the "server random," another random string of bytes that's generated by the server.
In my code using wireshark I've identified that server requests TLS_AES_256_GCM_SHA384
server hello screenshot
I am confused, what are we sending in .certificate.pem file
From my understanding premaster secret it when the client sends one more random string of bytes, the "premaster secret." The premaster secret is encrypted with the public key and can only be decrypted with the private key by the server. (The client gets the public key from the server's SSL certificate.)
and as for session keys it is when both client and server generate session keys from the client random, the server random, and the premaster secret. They should arrive at the same results.
is it the premaster secret, or is it the session keys that we are sending as certificate.pem in verify parameter with request?
Do we generate these by using crypto algorithm since server is requesting TLS_AES_256_GCM_SHA384 - we could replicate the output using the same algorithm using crypto library and pass it into requests? Is this the purpose of verify parameter?
The site I am currently trying to reach has SSL Certificates etc and it only accepts certain client certificate. I am trying to understand if verify is the correct parameter I should be using.
No, the "verify" argument has nothing to do with client-side certificates. It's your root certificate which can be used to verify the authenticity of the server-side certificate. If you don't want to verify it, you can use on your own risk verify=False .
For client-side certificates, you need to use the "cert" argument: https://requests.readthedocs.io/en/master/user/advanced/#client-side-certificates . Example:
requests.get('https://example.org', cert=('/path/client.cert', '/path/client.key'))
I have a twisted webserver with TLS authentication, and it appears to hang when I connect to it over SMTP. Here is the block of twisted code to start the server:
(Note: certificateData is our private key and public key concatenated together, that appeared to be the only way to get a self signed certificate to work)
customFactory = CustomSMTPFactory(portal)
certificate = PrivateCertificate.loadPEM(certificateData)
contextFactory = certificate.options(certificate)
tlsFactory = TLSMemoryBIOFactory(contextFactory, False, customFactory)
a = service.Application("Custom Server")
internet.TCPServer(5870, tlsFactory).setServiceParent(a)
On the client, this line just hangs waiting to read data:
smtplib.SMTP('localhost',5870)
Any ideas? How do I setup TLS authentication on a twisted webserver?
Your server starts TLS from the beginning of the connection. Try smtplib.SMTP_SSL instead, so your client expects this.