I need your help.
I want to generate a certificate for a client for authorization on my api.
In my use case, I have an API that is hosted with python hypercorn and fastapi. Then I have multiple clients (also python (httpx)) that should request data from this api. For authentication between the client and the server, I want to use certificates. I want to provide the client with a certificate with which it can authorize itself with the server.
For generating the certificates i used this instruction: https://www.makethenmakeinstall.com/2014/05/ssl-client-authentication-step-by-step/
What did I do wrong, or how can I implement my use case?
server:
async def main():
config = Config.from_mapping(dict(
worker_class='trio',
certfile='server.cer',
keyfile='server.key',
verify_mode=VerifyMode.CERT_REQUIRED,
bind=f"0.0.0.0:{8000}"))
async with trio.open_nursery() as nursery:
nursery.start_soon(serve, app, config)
client:
import httpx
res = httpx.get("https://localhost:8000/", verify=True, cert=("client.cer", "client.key"))
res
When I execute the request in the client I get the following error:
ConnectError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate (_ssl.c:1123)
let's simplify how certificates work.
certificates used for authentication and authorization and this is what you're exactly doing in your case. your backend-server has its own certificate, and your client has his own certificate. your applications is set to recognize each other using these certificates and you're doing it in a great way.
but certificates are vulnerable and can be created locally, so I myself can create a certificate that says that I'm facebook or google. and here is the validation's role come; a well know authority that's already implemented in all browsers and OS - and we call it CA for a certification authority - signs your certificates, so when your server send its cert to a client, the client's browser would recognize the signature and tells you that the certificate is valid.
in your case here you have a self signed certificate, which means that for development purpose you've signed your cert by yourself "instead of the recognized CAs".
Overcoming such an issue would be in two ways.
you disable the validation in your code and in your client side as well. suits for development purpose or intra network.
sign your certificate and your client's certificate with a recognized authority like digicert, and add your CA in your trust-store.
Currently my client side HTTPS request code looks like this:
resp = requests.post(endpoint_predict_v100, files={'image': open(file_path, 'rb')}, verify=client_auth,
cert=(client_cert_path, client_secret_path))
This is an HTTPS request, the server root CA is client_auth and the client certificate and key are client_cert_path and client_secret_path.
because my server code is deployed and I cannot change much on that side, I am wondering if I can do the following on the client side:
enable server authentication only when the server root CA is provided by the client
enable client authentication only when the client cert and key are provided by the client.
I have found out that I am able to do the first thing, "enable server authentication only when the server root CA is provided by the client", by passing verify=False if the server root CA is not presented on the client side., referring to the Python requests docs. However, when I try to remove the cert=(client_cert_path, client_secret_path), the request will fail, throwing this error:
ssl.SSLError: [SSL] tlsv13 alert certificate required (_ssl.c:2570)
It seems that this error is suggesting I must pass in the client cert and key to make the server accept my request. but is there a way to disable client authentication on the client side, when the client cert and key are not present?
ssl.SSLError: [SSL] tlsv13 alert certificate required (_ssl.c:2570)
... is there a way to disable client authentication on the client side, when the client cert and key are not present?
The alert you get is because the server is requiring the client certificate. Since it is required by the server it is not sufficient to change some client side behavior only. This is similar to a server requesting authentication by password - this cannot be simply skipped in the client too.
With many servers there is a way to have client certificates optional, i.e. request a client certificate but don't insist that one is sent. How to do this with your specific server and if it is supported by your server at all is unknown though.
I'm trying to implement mutual authentication on a ftps connection using ftplib module.
Here is my code:
Context = ssl.SSLContext(ssl.PROTOCOL_TLSv1_2)
Context.load_verify_locations(cafile=trusted.txt,capath=path)
Context.load_cert_chain(certfile=mycert.txt,keyfile=mikey.txt,password=xxxx)
Context.verify_mode=True
Ftp = ftplib.FTP_TLS(Context=Context)
Ftp.connect(host, port)
Ftp.auth()
Ftp.prot_p()
Ftp.set_pasv(True)
Ftp.cwd(dest_dir)
Ftp.storlines(xx,xx)
Ftp.close()
However above works fine only with client authentication set as no on ftps server side. When we try with client Auth yes
Error code is as below.
Ssl.SSLError: [SSL:SSLV3_ALERT_CERTIFICATE_UNKNOWN] sslv3 alert certificate unknown (_ssl.c:777)
I have the servers cert chain on ca file defined.
I have my trusted on servers side defined.
Still connection doesn't work well. And it works well if client Auth is disabled on server side.
Any suggestions on what could be wrong. Could it be ciphers?
I tried setting up ciphers but don't know how exchange happens in realtime. Or could this be that ftplib does not support fully mutually authentication at all??
Ssl.SSLError: [SSL:SSLV3_ALERT_CERTIFICATE_UNKNOWN] sslv3 alert certificate unknown (_ssl.c:777)
If you get this error in the client then the server failed to validate the client certificate, i.e. your mycert.txt and mikey.txt.
Since validation of the client certificate is done by the server you have to look at the server configuration and logs for more information of why your client certificate was not accepted. Typical problems are that the client certificate is a self-signed certificate, that the CA which issued the client certificate is not trusted in the server or that intermediate certificates are required to verify the certificate but the client is not sending these.
Can anyone explain me how to add SSL to the Python STOMP client I'm using.
I added the stomp+ssl transport connector in the ActiveMQ configuration file and my basic Python STOMP client is below:
import time
import sys
import stomp
class MyListener(stomp.ConnectionListener):
def on_error(self, headers, message):
print('received an error "%s"' % message)
def on_message(self, headers, message):
print('received a message "%s"' % message)
conn = stomp.Connection()
conn.set_listener('', MyListener())
conn.start()
conn.connect('admin', 'password', wait=True)
conn.subscribe(destination='/queue/test', id=1, ack='auto')
conn.send(body=' '.join(sys.argv[1:]), destination='/queue/test')
time.sleep(2)
conn.disconnect()
I created the key store and trust store given in the http://activemq.apache.org/how-do-i-use-ssl.html docs and added them to the SSL_OPTS environment variable in the broker but I'm unable to find how to initialize the Python STOMP client with the key store and trust store. Should I use the SSL paraments given in the stomp.Connection() method, and if yes how to do so?
Can anyone please explain if there is any other way to add SSL over STOMP?
The Python STOMP client (as of version 4.1.20) uses an SSLContext to process its key pair/certificate, so there is no reason to produce a Java KeyStore for the client.
With this in mind, let us go through the entire process of setting up ApacheMQ to support SSL-wrapped STOMP connections. The process below has been tested on ApacheMQ 5.15.4. We explicitly set up two-way trust by manually moving self-signed certificates between the broker and client; using a certificate authority is also possible but how to do so is a different question.
Create a client certificate
As mentioned above, on the Python side of things, a KeyStore will have little use, and since SSLContext expects PEM encoded certificates, we might as well create the key pair and certificate by hand (that is, using openssl). First, on the client machine, let us create a 4096-bit RSA key:
openssl genrsa -out client.key 4096
Using this, turn the public key part into a certificate and sign it with the key itself; since we will be manually moving the certificate to the broker, self-signing the certificate is not an issue:
openssl req -new -out client.csr -key client.key
openssl x509 -req -days 365 -in client.csr -signkey client.key -out client.pem
rm client.csr
The STOMP client will need both the signed certificate, client.pem, and the private key, client.key, while the broker will only need the certificate.
Create a broker certificate
On the broker, we can follow the first part of the Apache guide and use the Java keytool to create a KeyStore with a key for the server:
keytool -genkeypair -alias broker -keyalg RSA -keysize 4096 -keystore broker.ks
When prompted for "first and last name", provide the hostname of the server, which in our example we will take simply to be localhost; if the broker and client are running on different servers, make sure that this is set to whatever the Python client will end up using to identify the broker:
What is your first and last name?
[Unknown]: localhost
All other input values can be left as "Unknown".
At the end of the day, we will only want to allow connections to the broker from clients with certificates that we know, so at this point copy the client.pem generated above to the broker and add it to a trust store through
keytool -import -alias client -keystore broker.ts -file client.pem
If the broker is to allow connections from any client, then this final step can be skipped.
Setting up ApacheMQ
By default, all connections through STOMP (and indeed all connections) are plaintext ones, and in order to enable STOMP connections over SSL, add the following <transportConnector /> to conf/apachemq.xml:
<transportConnectors>
<transportConnector name="stomp+ssl" uri="stomp+nio+ssl://0.0.0.0:61613?transport.enabledProtocols=TLSv1.2&needClientAuth=true" />
</transportConnectors>
Make sure to remove any existing plaintext connectors such as the default STOMP connector as otherwise clients will be able to simply use those and bypass the SSL requirement. Note also that needClientAuth=true is what forces client certificate validation; without this, clients are able to connect without providing a trusted certificate.
To configure ApacheMQ to use the key and trust stores defined above, define the environment variable ACTIVEMQ_SSL_OPTS through (on Unix)
export ACTIVEMQ_SSL_OPTS = -Djavax.net.ssl.keyStore=/path/to/broker.ks -Djavax.net.ssl.trustStore=/path/to/broker.ts -Djavax.net.ssl.keyStorePassword=passwordForBrokerKs -Djavax.net.ssl.trustStorePassword=passwordForBrokerTs
or (on Windows)
set ACTIVEMQ_SSL_OPTS=-Djavax.net.ssl.keyStore=C:\path\to\broker.ks -Djavax.net.ssl.trustStore=C:\path\to\broker.ts -Djavax.net.ssl.keyStorePassword=passwordForBrokerKs -Djavax.net.ssl.trustStorePassword=passwordForBrokerTs
Here, the two passwords are those chosen after running keytool in the previous step. If client certificate validation is not desired, simply leave out the configuration of trustStore and trustStorePassword.
With this, ActiveMQ can be started as usual through bin/activemq start. To make sure that the SSL configuration matches expectation, pay attention to the JVM args part of the output printed when starting the server.
Testing the STOMP client
With the broker properly set up, we can configure the client as well. Here, we provide stomp.Connection.set_ssl with references to the key and certificate created in the first step. Assuming that the ActiveMQ server is running on localhost:61613, your test script simply becomes
import time
import sys
import stomp
class MyListener(stomp.ConnectionListener):
def on_error(self, headers, message):
print('received an error "%s"' % message)
def on_message(self, headers, message):
print('received a message "%s"' % message)
host = 'localhost'
port = 61613
conn = stomp.Connection([(host, port)])
conn.set_ssl(for_hosts=[(host, port)], key_file='/path/to/client.key', cert_file='/path/to/client.pem')
conn.set_listener('', MyListener())
conn.start()
conn.connect('admin', 'password', wait=True)
conn.subscribe(destination='/queue/test', id=1, ack='auto')
conn.send(body='test message', destination='/queue/test')
time.sleep(2)
conn.disconnect()
To make sure that ApacheMQ is indeed validating the client certificate, we could repeat step 1 and create a new pair, client2.key/client2.pem say, and use that instead. Doing so should result in the following non-sensical error message being printed by ApacheMQ:
ERROR | Could not accept connection from null : {}
java.io.IOException: javax.net.ssl.SSLHandshakeException: General SSLEngine problem
Validating the broker certificate
Now, the attentive reader will have noticed that we never actually moved the broker certificate to the client, and yet things seem to work regardless. As it turns out, the default behavior of stomp.py is to perform no certificate validation at all, allowing an (active) attacker to MITM the connection.
As we are rolling self-signed certificates, all we need to do to fix this situation is to provide the actual broker certificate to the Python client. On the broker, export the certificate through
keytool -exportcert -rfc -alias broker -keystore broker.ks -file broker.pem
and move broker.pem to the Python client. Now, in the test script above, include the certificate by replacing the SSL configuration with
conn.set_ssl(for_hosts=[(host, port)], key_file='/path/to/client.key', cert_file='/path/to/client.pem', ca_certs='/path/to/broker.pem')
As above, we can test that this is indeed performing the proper validation by repeating the broker certificate generation process to create a broker2.pem, use that in the test script, and note that it will fail with an
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:833)
Try this.
conn = stomp.Connection([(host, port)])
conn.set_listener('', MyListener())
conn.set_ssl(for_hosts=[(host, port)], ssl_version=ssl.PROTOCOL_TLS)
conn.start()
conn.connect(login, password, wait=True)
conn.send(body=message, destination=queue)
conn.disconnect()
or
conn.set_ssl(for_hosts=[(host, port)], ssl_version=_ssl.PROTOCOL_TLS)
The key point is understanding the key/cert files must be there - otherwise even stomp.py tests in their repository will not work. TLS will not work in stomp.py if the file names are not specified (correctly).
See the code & key generation instructions here: https://gist.github.com/borislitvak/6ccea503abf1b2f9c89e87309d6dab88
I have a twisted webserver with TLS authentication, and it appears to hang when I connect to it over SMTP. Here is the block of twisted code to start the server:
(Note: certificateData is our private key and public key concatenated together, that appeared to be the only way to get a self signed certificate to work)
customFactory = CustomSMTPFactory(portal)
certificate = PrivateCertificate.loadPEM(certificateData)
contextFactory = certificate.options(certificate)
tlsFactory = TLSMemoryBIOFactory(contextFactory, False, customFactory)
a = service.Application("Custom Server")
internet.TCPServer(5870, tlsFactory).setServiceParent(a)
On the client, this line just hangs waiting to read data:
smtplib.SMTP('localhost',5870)
Any ideas? How do I setup TLS authentication on a twisted webserver?
Your server starts TLS from the beginning of the connection. Try smtplib.SMTP_SSL instead, so your client expects this.