A python deployment our team has fails on
import socket
import ssl
context = ssl.wrap_socket(
sock,
ca_certs='root_ca_bundle.pem',
keyfile='pk.pem',
certfile='cert.pem',
do_handshake_on_connect=True,
cert_reqs=ssl.CERT_REQUIRED,
)
which is traced to failing on load_cert chain function with
if certfile:
context.load_cert_chain(certfile, keyfile)
The error I see is this
[X509: KEY_VALUES_MISMATCH] key values mismatch (_ssl.c:4067)
The real issue is that I copied a wrap_socket call using the same certfile,keyfile and ca_certs, and I don't see this issue.
It only appears on deployment.
To my understanding, load_cert_chain just verifies that the certfile and keyfile are mathematically aligned, using something like this:
$ openssl x509 -noout -modulus -in cert.pem
$ openssl rsa -noout -modulus -in key-no-pass.pem
Tried this on key and cert and verified they're the same.
What I am asking is :
Is there an explanation other than wrong keyfile or certfile being used in deployment?
Is there a way I can find source code for load_cert_chain function.
Related
I am trying to get the server certificate of badssl.com subdomains (ex. https://expired.badssl.com).
import ssl
ssl.get_server_certificate(('expired.badssl.com', 443))
But when examining the above generated certificate I see that the certificate has
Identity: badssl-fallback-unknown-subdomain-or-no-sni
which means SNI is failing. How can I get the server certificate of different subdomains of badssl.com? (I am using python 2.7.12)
Found the answer.
import ssl
hostname = "expired.badssl.com"
port = 443
conn = ssl.create_connection((hostname, port))
context = ssl.SSLContext(ssl.PROTOCOL_SSLv23)
sock = context.wrap_socket(conn, server_hostname=hostname)
certificate = ssl.DER_cert_to_PEM_cert(sock.getpeercert(True))
Searching for "Python ssl.get_server_certificate SNI" brought me easily to this answer. Although the OP himself answer is correct, I would like to provide a little more insight for future reference.
With some [hostname]s the fallowing call using Python 3.7:
ssl.get_server_certificate(("example.com", 443)
will complain with a traceback that ends with:
ssl.SSLError: [SSL: TLSV1_ALERT_INTERNAL_ERROR] tlsv1 alert internal error (_ssl.c:1045)
Doing some further investigation, making use of the openssl s_client utility, allows to discover that those same [hostname]s which made get_server_certificate to fail, also makes the fallowing command:
openssl s_client -showcerts -connect example.com:443
to fail with this error:
SSL23_GET_SERVER_HELLO:tlsv1 alert internal error:s23_clnt.c:802
Note that the error message is similar to the one returned by the python code.
Using the -servername switch did the trick:
openssl s_client -showcerts -connect example.com:443 -servername example.com
leading to the conclusion that the investigated hostname refers to a secure server that makes use of SNI (a good explanation on what that means is given by the SNI Wikipedia article).
So, switching again to Python and looking at the get_server_certificate method, examining the ssl module source (here for convenience), you can discover that the function includes this call:
context.wrap_socket(sock)
without the server_hostname=hostname key argument, which of course should mean that get_server_certificate cannot be used querying a SNI server. A little more effort is required:
hostname = "example.com"
port = 443
context = ssl.create_default_context()
with socket.create_connection((hostname, port)) as sock:
with context.wrap_socket(sock, server_hostname=hostname) as sslsock:
der_cert = sslsock.getpeercert(True)
# from binary DER format to PEM
pem_cert = ssl.DER_cert_to_PEM_cert(der_cert)
print(pem_cert)
UPDATED: fixed/working code at bottom of question. Re-implemented to use wrap_socket() instead of SSL.Context(). Negotiated cipher seems unaffected by ssl.PROTOCOL_
I've got an XML service which listens on port 8388. The service has it's own CA built in and will issue a P12 file for any client configured on it's admin interface (I need to include a CA cert in my connections).
I've configured a client and downloaded the P12 file; also exported PEM files (for easier debugging) from the P12 and exported a PEM file for the server CA (which was a Java keystore).
If I use openssl s_client and feed it the client-side cert/key and CA file, things seem to work correctly; ie: verify return:1 in the truncated output below. The openssl process does not generate any certificate errors on the server (as does my Python script). From what I've read, this is supposed to mean the certs are all OK and valid.
# openssl s_client -connect srv.domain.net:8388 -CAfile server_ca.pem -cert client_cert.pem -key client_key.pem
CONNECTED(00000003)
depth=1 ... emailAddress = ca#SERVERsystems.com
verify return:1
depth=0 ...CN = srv.domain.net
verify return:1
---
Certificate chain
0 s:/emailAddress=help#mydomain.net...CN=srv.domain.net
i:/C=US/...emailAddress=ca#SERVERsystems.com
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIJRDCCByygAwIBAgIGAVGIopaoMA0GCSqGSIb3DQEBDQUAMIG...rV
-----END CERTIFICATE-----
subject=/emailAddress=help#mydomain.net...CN=srv.domain.net
issuer=/C=US/...emailAddress=ca#SERVERsystems.com
---
Acceptable client certificate CA names
/C=US/...emailAddress=ca#SERVERsystems.com
/.../CN=srv.domain.net
---
SSL handshake has read 3369 bytes and written 5705 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
Protocol : TLSv1.2
Cipher : ECDHE-RSA-AES256-GCM-SHA384
Session-ID: ...DF6572FA00D62D83CF0B1A82F
Session-ID-ctx:
Master-Key: ...7F685C0B163A7739C271E9722FC0554108175C4
Key-Arg : None
PSK identity: None
PSK identity hint: None
SRP username: None
Start Time: 1461337153
Timeout : 300 (sec)
Verify return code: 0 (ok)
---
I am now attempting to hook up a Python (2.7.9) script to the XML service using the same cert, key and CA file but I can't get it working. Python is complaining about SSLv3 errors and the server says it can't verify the client. So, the connection works, but the handshake, certs or ciphers or something aren't right.
I've searched out numerous examples and implementations and this one seemed to be the simplest one so I started with it for a template. SSL3 is not the protocol I would stick with (POODLE), but it was supposed to be from a working example so I wanted to make as few changes as possible to get things working and then tweak from there. Anyone know what I've got wrong here? ouput/errors/logs are posted at the far bottom.
#!/usr/bin/python
# -*- coding: utf-8 -*-
import socket
import OpenSSL
from OpenSSL import *
import sys
serverName = sys.argv[1]
print "Using server : " + serverName
ctx = SSL.Context(SSL.SSLv3_METHOD)
ctx.use_privatekey_file('client_key.pem')
ctx.use_certificate_file('client_cert.pem')
ctx.load_verify_locations(cafile='server_ca.pem')
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((serverName, 8388))
sslSocket = socket.ssl(s)
print repr(sslSocket.server())
print repr(sslSocket.issuer())
print ("writing socket..")
sslSocket.write('<transaction><data>14</data></transaction>\n')
s.close()
Python output:
Using server : localhost
Traceback (most recent call last):
File "./test3.py", line 29, in <module>
sslSocket = socket.ssl(s)
File "/usr/lib/python2.7/socket.py", line 64, in ssl
return _realssl.sslwrap_simple(sock, keyfile, certfile)
File "/usr/lib/python2.7/ssl.py", line 993, in sslwrap_simple
ssl_sock.do_handshake()
ssl.SSLError: [SSL: SSLV3_ALERT_BAD_CERTIFICATE] sslv3 alert bad certificate (_ssl.c:581)
Server logs after connection above:
Apr 22 10:39:56 srv.domain.net ERROR server.auth [Thread-183,run:233] Couldn't validate the client certificate. Verify the validity and dates of the client cert.
Apr 22 10:39:56 srv.domain.net javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
Apr 22 10:39:56 srv.domain.net at sun.security.ssl.SSLSessionImpl.getPeerCertificates(SSLSessionImpl.java:431)
Apr 22 10:39:56 srv.domain.net at com.xml.server.auth.run(auth.java:226)
Apr 22 10:39:56 srv.domain.net at java.lang.Thread.run(Thread.java:745)
Working code:
#!/usr/bin/python
# -*- coding: utf-8 -*-
import socket
import ssl
import sys
import os
serverName = sys.argv[1]
print "Using server : " + serverName
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# ssl.PROTOCOL_xxx does not seem to affect negotiated cipher??
wrapper = ssl.wrap_socket(s,
ca_certs = "server_ca.pem",
certfile = "client_cert.pem",
keyfile = "client_key.pem",
cert_reqs=ssl.CERT_REQUIRED,
ssl_version=ssl.PROTOCOL_TLSv1)
wrapper.connect((serverName, 8388))
# some info on the connnection/cipher in use
print repr(wrapper.getpeername())
print repr(wrapper.cipher())
print repr(wrapper.getpeercert())
# send server command
print ("writing socket..")
wrapper.send ("<transaction><data>14</data></transaction>\n")
# read server reply
print "server reply: " + wrapper.recv(4096)
wrapper.close()
s.close()
ctx.use_privatekey_file('client_key.pem')
ctx.use_certificate_file('client_cert.pem')
...
sslSocket = socket.ssl(s)
While you create an SSL context with the client certificate you don't use this context within your SSL connection. This means no client certificate will be send and the server complains accordingly.
A much simpler way to use client certificates is by using the certfile and keyfile parameter of ssl.wrap_socket, see the documentation.
I'm trying to update a new valid SSL cert provided by DigiCert on my python flask. The same code with the old expired cert works just fine. However for the renewed cert, it did not recognise my certificate chain and giving error:
[root#test~]# openssl s_client -connect somedomain.com:9443
CONNECTED(00000003)
140340901406536:error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure:s23_clnt.c:744:
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 7 bytes and written 249 bytes
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
---
The weird thing is, when i'm using openssl command to verify my cert chain, everything is working well as follow:
[root#test ~]$ openssl verify -verbose -CAfile CertChain.crt Server.crt
Server.crt: OK
This is my python code
from OpenSSL import SSL
from flask import Flask, send_file, make_response, request, jsonify, Response
from flask.ext import restful
from flask.ext.restful import reqparse, abort, Api, Resource
#------------#
context = SSL.Context(SSL.SSLv23_METHOD)
context.use_certificate_file("Server.crt")
context.use_privatekey_file("Server.key")
context.use_certificate_chain_file("CertChain.crt")
context.set_options(SSL.OP_NO_SSLv3)
context.set_options(SSL.OP_NO_SSLv2)
context.set_cipher_list('ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS')
app = Flask(__name__)
#app.route("/")
def hello():
return "Hello World!"
if __name__ == "__main__":
app.run(host='0.0.0.0', port=9443, debug=True, use_reloader=False, ssl_context=context)
And for your info key and certificate is matching with no problem
[root#test]# openssl x509 -noout -modulus -in Server.crt | openssl md5
(stdin)= adc3de807ec6a02c5ba9da89f3fe5dd5
[root#test]# openssl rsa -noout -modulus -in Server.key | openssl md5
(stdin)= adc3de807ec6a02c5ba9da89f3fe5dd5
Is there anything affecting this? Python version? OpenSSL version? My cipher? Any body can help? Thanks
SSL handshake has read 7 bytes and written 249 bytes
These are not problems of the validation of the certificate.
The clients starts the SSL handshake but the server sends only 7 bytes back, which might be an SSL alert that something is wrong. The server never sends the certificate back so it cannot be a problem of the client side validation.
This means you should better check the server side for problems and not the client side. And since you say that exactly the same code works with the old certificate it can not be a problem of protocol version or ciphers but must relate to the certificate. My guess is thus that the private key you use does not match the certificate you use, which gets not checked when you do the openssl verify.
I got the solution. Previously my certificate chain only contains the Intermediate and Root certificate which is not complete.
The solution is to also include the server certificate into the chain which is weird since context.use_certificate_file is already defined.
Anyways, the handshake is now working fine. Thanks!
I'm trying to use imaplib.IMAP4_SSL function but I'm stuck with this error:
[Errno 336265225] _ssl.c:356: error:140B0009:SSL routines:SSL_CTX_use_PrivateKey_file:PEM lib
try:
mail = imaplib.IMAP4_SSL(host='imap.gmail.com', port="993", certfile="certIMAP.pem")
except Exception as e:
print e
sys.exit()
The certificate has been downloaded with:
echo | openssl s_client -connect imap.gmail.com:993 2>/dev/null | openssl x509 > certIMAP.pem
Permission on the file are ok.
If I use the keyfile option, they ask me the cert, and then the same error is obtained.
I could not find example for this function with cert specification.
Thank you.
While it is not obvious from the documentation I would suggest that the parameters certfile and keyfile are not used to verify the servers certificate (that's how you use it - and keyfile would not make any sense here) but are the optional certificate for the client and it's private key, which some server might require to identify the client (e.g. instead or additionally to password).
To verify the server you usually have something called CA file or CA path (CA = certificate agency) and you don't need a private key here. There seem to be no obvious way for given the CA certificates here, but Certificate Authority for imaplib and poplib python might help you with this.
Just trying to set up a simple SSL server. I have never had anything SSL work for me in the past. I have a loose understanding of how SSL certificates and signing.
The code is simple
import socket, ssl
context = ssl.SSLContext(ssl.PROTOCOL_TLSv1)
context.load_cert_chain(certfile="mycertfile") ###############
bindsocket = socket.socket()
bindsocket.bind(('', 2099))
bindsocket.listen(5)
while True:
newsocket, fromaddr = bindsocket.accept()
sslsoc = context.wrap_socket(newsocket, server_side=True)
request = sslsoc.read()
print(request)
The line in there with the ###s after it is the one that isnt working. I don't know what I have to do with openssl to generate a PEM file that will work here.
Can anyone enlighten me as to how to make this simple socket work.
By the way, this is NOT used for HTTP.
you can use this command to generate a self-signed certificate
openssl req -new -x509 -days 365 -nodes -out cert.pem -keyout cert.pem
the openssl framework will ask you to enter some information, such as your country, city, etc. just follow the instruction, and you will get a cert.pem file. the output file will have both your RSA private key, with which you can generate your public key, and the certificate.
the output file looks like this:
-----BEGIN RSA PRIVATE KEY-----
# your private key
-----END RSA PRIVATE KEY-----
-----BEGIN CERTIFICATE-----
# your certificate
-----END CERTIFICATE-----
just load it, and the ssl module will handle the rest for you:
context.load_cert_chain(certfile="cert.pem", keyfile="cert.pem")
btw, there is no "SSLContext" in python2. for guys who are using python2, just assign the pem file when wrapping socket:
newsocket, fromaddr = bindsocket.accept()
connstream = ssl.wrap_socket(newsocket,
server_side=True,
certfile="cert.pem",
keyfile="cert.pem",
ssl_version=YOUR CHOICE)
available ssl version: ssl.PROTOCOL_TLSv1, ssl.PROTOCOL_SSLv2, ssl.PROTOCOL_SSLv3, ssl.PROTOCOL_SSLv23. if you have no idea, ssl.PROTOCOL_SSLv23 may be your choice as it provides the most compatibility with other versions.
In your example, you provide a certfile, but no keyfile. Both are required.