I am trying to get the server certificate of badssl.com subdomains (ex. https://expired.badssl.com).
import ssl
ssl.get_server_certificate(('expired.badssl.com', 443))
But when examining the above generated certificate I see that the certificate has
Identity: badssl-fallback-unknown-subdomain-or-no-sni
which means SNI is failing. How can I get the server certificate of different subdomains of badssl.com? (I am using python 2.7.12)
Found the answer.
import ssl
hostname = "expired.badssl.com"
port = 443
conn = ssl.create_connection((hostname, port))
context = ssl.SSLContext(ssl.PROTOCOL_SSLv23)
sock = context.wrap_socket(conn, server_hostname=hostname)
certificate = ssl.DER_cert_to_PEM_cert(sock.getpeercert(True))
Searching for "Python ssl.get_server_certificate SNI" brought me easily to this answer. Although the OP himself answer is correct, I would like to provide a little more insight for future reference.
With some [hostname]s the fallowing call using Python 3.7:
ssl.get_server_certificate(("example.com", 443)
will complain with a traceback that ends with:
ssl.SSLError: [SSL: TLSV1_ALERT_INTERNAL_ERROR] tlsv1 alert internal error (_ssl.c:1045)
Doing some further investigation, making use of the openssl s_client utility, allows to discover that those same [hostname]s which made get_server_certificate to fail, also makes the fallowing command:
openssl s_client -showcerts -connect example.com:443
to fail with this error:
SSL23_GET_SERVER_HELLO:tlsv1 alert internal error:s23_clnt.c:802
Note that the error message is similar to the one returned by the python code.
Using the -servername switch did the trick:
openssl s_client -showcerts -connect example.com:443 -servername example.com
leading to the conclusion that the investigated hostname refers to a secure server that makes use of SNI (a good explanation on what that means is given by the SNI Wikipedia article).
So, switching again to Python and looking at the get_server_certificate method, examining the ssl module source (here for convenience), you can discover that the function includes this call:
context.wrap_socket(sock)
without the server_hostname=hostname key argument, which of course should mean that get_server_certificate cannot be used querying a SNI server. A little more effort is required:
hostname = "example.com"
port = 443
context = ssl.create_default_context()
with socket.create_connection((hostname, port)) as sock:
with context.wrap_socket(sock, server_hostname=hostname) as sslsock:
der_cert = sslsock.getpeercert(True)
# from binary DER format to PEM
pem_cert = ssl.DER_cert_to_PEM_cert(der_cert)
print(pem_cert)
Related
I'm trying to use SSH through Websocket Reverse-Proxy, the indication of successful request would be 101. The target server only accepts HTTPS which means I had to use SNI Spoofing to connect. Not to mention that the target also protected with CloudFlare, that's another reason to use TLS/SNI Spoofing.
Actually, I have tried websockets module that specifically handle this but always return 403 or 404 response. Using conventional requests also result into this. In the end, I have to combine ssl module with socket manually, it works but below codes has a problem with SSL Protocol Version:
cx = ssl.create_default_context()
sli = cx.wrap_socket(socket.socket(), server_hostname='unpkg.com')
sli.connect(('blog.clova.line.me', 443))
sli.sendall(b'''GET wss://unpkg.com/ HTTP/1.1\r
Host: identity.o2.co.uk.zainvps.tk\r
User-Agent: cpprestsdk/2.9.0\r
Upgrade: websocket\r
Connection: Upgrade\r
Sec-WebSocket-Version: 13\r\n\r
''')
Above codes unable to do Handshake for SSLv3 and triggers UnsafeLegacyRenegotiation. However, it does work for some domain but not for most of it that mainly use SSLv3.
Target Domain: unpkg.com
SSH: identity.o2.co.uk.zainvps.tk
To fix unsafelegacy, I had to use custom openssl config:
openssl_conf = openssl_init
[openssl_init]
ssl_conf = ssl_sect
[ssl_sect]
system_default = system_default_sect
[system_default_sect]
Options = UnsafeLegacyRenegotiation
I wanted it able to use any protocol version. Is there's some auto option that allow to use any SSL Protocol version; SSLv23, TLSv1, and SSLv3?
I intend to connect to the remote host example.com over TLS but I have to connect through a proxy IP address with DNS name example-proxy.com.
I don't have control over the SSL certificate and I cannot ask the admin at example.com to add example-proxy.com to its certificate's SAN.
Using example-prxoy.com would cause OpenSSL to error out because the host name does not match the name in the certificate. How can I split the host parameter into two: (1) domain name for the network connection and (2) domain name for the certificate verification.
I don't have the resources to modify the OpenSSL library but I can make changes to the Python libraries. According to this doc, I could have modified the match_hostname method to implement this feature but it is no longer available as of Python 3.7+.
Asks
How can I use Python 3.7+ to specify both a host name and a certificate name?
From the security standpoint, How could my implementation go wrong?
Just give a different hostname for TCP connection and TLS handshake, i.e. set server_hostname in wrap_socket. To modify the example from the official documentation for this:
import socket
import ssl
tls_hostname = 'www.example.com'
context = ssl.create_default_context()
with socket.create_connection(('127.0.0.1',8443)) as sock:
with context.wrap_socket(sock, server_hostname=tls_hostname) as ssock:
print(ssock.version())
This will connect to ('127.0.0.1',8443) but do the TLS handshake with www.example.com.
Note that this will use tls_hostname for both SNI extension in the TLS handshake and for validating the certificate. But this seems to be what you need based on your question anyway: connect to IP:port but do TLS handshake and validation with a specific hostname.
I am writing a tool to monitor server certificate expiration. I'm using python3 ssl and socket modules to get the server cert using a pretty basic method of creating a default context, disabling hostname validation and certificate verification, calling SSLSocket.connect(), then SSLSocket.getpeercert(), with the sole purpose of grabbing the server certificate, and that is all.
This is all within a private network and I am not concerned with validation.
I have some devices that require client certs signed by a private CA (which my tool doesn't have), so the handshake fails on SSLSocket.connect(), making SSLSocket.getpeercert() impossible.
I know that the server certificate is indeed being provided to my client (along with that pesky Certificate Request) during the handshake. I can see it in a packet capture, as well as just using the openssl s_client command line.
Here is my code.
def get_cert(self, host, port):
ctx = ssl.create_default_context()
ctx.check_hostname = False
ctx.verify_mode = ssl.CERT_NONE
with ctx.wrap_socket(socket.socket(), server_hostname=host) as s:
s.settimeout(10)
s.connect((host, port))
binary_cert = s.getpeercert(True)
cert = OpenSSL.crypto.load_certificate(OpenSSL.crypto.FILETYPE_ASN1, binary_cert)
pem_cert = OpenSSL.crypto.dump_certificate(OpenSSL.crypto.FILETYPE_PEM, cert).decode()
return pem_cert
Is there any way to get a little lower into the handshake messages to get the server cert, even though the handshake ultimately fails?
My current solution is to just run openssl s_client -connect host:port using subprocess.run() in the event of a ssl.SSLError.
You may catch exception that do_handshake() produced and then continue to process server certificate.
import OpenSSL
import socket
dst = ('10.10.10.10', 443)
sock = socket.create_connection(dst)
context = OpenSSL.SSL.Context(OpenSSL.SSL.SSLv23_METHOD)
connection = OpenSSL.SSL.Connection(context, sock)
connection.set_connect_state()
try:
connection.do_handshake()
except:
print(connection.get_peer_cert_chain())
Tested on python 2.7.17 and 3.8.5
It looks like there's unfortunately no way to do it with python's ssl module in versions < 3.10. In those versions, the only way to get the peer certificate that I can see is through the low-level _ssl.SSLSocket.getpeercert() method and that immediately throws exception if the handshake is not complete.
Since python 3.10, there's a new _ssl.SSLSocket.get_unverified_chain() method that does not do the handshake check, so perhaps something like this abomination could work?
ssock = context.wrap_socket(sock, do_handshake_on_connect=False)
try:
ssock.do_handshake()
except ssl.SSLError as e:
pass
certs = ssock._sslobj._sslobj.get_unverified_chain()
... but I have not tested it.
I have a simple TLS client in python running in Ubuntu 18.04 and openssl version 1.1.0g. The client supports a single ciphersuite. I get an error when trying to connect to a TLS 1.0 server. The cipher suite is not supported by the server. I know that the reason for the error is most likely due to lack of ciphersuite mismatch but I am looking for a more meaningful error for the user in this case. The error I am getting at the moment is pointing to SSLv3 which neither the client nor the server has anything to do with SSLv3. The client disables SSLv3 and the server as well. This is the error :
[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:833)
My question is: I need a better error message says for example (lack of ciphersuite mismatch or something like that is relevant to ciphersuite issue). Is there any? Of course I could write my own message but the socket connection can fail for various reasons and I can not make a general error that always says "ciphersuite mismatch".
This is the client script:
import socket,ssl
import itertools
context = ssl.SSLContext()
context.verify_mode = ssl.CERT_NONE
context.check_hostname = False
ciphers = "ECDHE-ECDSA-AES128-GCM-SHA256"
context.set_ciphers(ciphers)
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
domainName = "privatedomain.com"
sslSocket = context.wrap_socket(s, server_hostname = domainName)
try:
sslSocket.connect((domainName, 443))
except (ssl.SSLError, ssl.SSLEOFError, ssl.CertificateError,ssl.SSLSyscallError, ssl.SSLWantWriteError, ssl.SSLWantReadError,ssl.SSLZeroReturnError) as e:
print("Error: ",e)
sslSocket.close()
From the client's view, it is not possible to get another message than the one sent by the server, which is handshake failure in your case. The error message are, for example, documented in RFC 2246 7.2.
The reason why you see SSLv3 in your message, is that you probably send a SSLv3 Hello, which is something allowed to negotiate a TLS 1.0 or later protocol.
Late answer but hopefully helpful . . .
Both client and server must agree on the transport layer version for the connection to be successful. Consider meeting a person for the first time. The person (client) extends their hand to you (server) in a gesture of greeting. If you just saw the person come out of the latrine without washing hands and you see (and/or smell) something undesirable, you will not extend your hand in return.
It is similar with an SSL handshake. The client says "Hey I'd like to communicate via TLS v1.0". The savvy admin for the server knows TLS v1.0 is not secure and they have disabled it on the server--so the server responds to the client, "No, but how about version 1.3?" (ie: "Go wash your hands first"). If the client accepts (washes hands), the handshake is accepted and the connection is established. If the client refuses, the server keeps asking for lower versions ("How about a gallon of Purell then?") until the client accepts or the server has no other versions to offer (walks away).
Basically, the handshake is designed to use the highest version that both the client and server support.
This page has a nice table of versions for client & server (about half way down in the "SSL Contexts" section:
https://docs.python.org/3/library/ssl.html
Note that TLS v1.0 is no longer considered secure (Google "POODLE attack"). If your server supports it, disable it ASAP.
For me this:
urllib.error.URLError: <urlopen error [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:1123)>
meant I was doing this
cipherstr = 'MEDIUM:!aNULL:!eNULL'
context = ssl._create_unverified_context()
context.set_ciphers(cipherstr)
commenting out the set_ciphers and it works now.
Other thing to check: make sure your version of OpenSSL is new enough.
UPDATED: fixed/working code at bottom of question. Re-implemented to use wrap_socket() instead of SSL.Context(). Negotiated cipher seems unaffected by ssl.PROTOCOL_
I've got an XML service which listens on port 8388. The service has it's own CA built in and will issue a P12 file for any client configured on it's admin interface (I need to include a CA cert in my connections).
I've configured a client and downloaded the P12 file; also exported PEM files (for easier debugging) from the P12 and exported a PEM file for the server CA (which was a Java keystore).
If I use openssl s_client and feed it the client-side cert/key and CA file, things seem to work correctly; ie: verify return:1 in the truncated output below. The openssl process does not generate any certificate errors on the server (as does my Python script). From what I've read, this is supposed to mean the certs are all OK and valid.
# openssl s_client -connect srv.domain.net:8388 -CAfile server_ca.pem -cert client_cert.pem -key client_key.pem
CONNECTED(00000003)
depth=1 ... emailAddress = ca#SERVERsystems.com
verify return:1
depth=0 ...CN = srv.domain.net
verify return:1
---
Certificate chain
0 s:/emailAddress=help#mydomain.net...CN=srv.domain.net
i:/C=US/...emailAddress=ca#SERVERsystems.com
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIJRDCCByygAwIBAgIGAVGIopaoMA0GCSqGSIb3DQEBDQUAMIG...rV
-----END CERTIFICATE-----
subject=/emailAddress=help#mydomain.net...CN=srv.domain.net
issuer=/C=US/...emailAddress=ca#SERVERsystems.com
---
Acceptable client certificate CA names
/C=US/...emailAddress=ca#SERVERsystems.com
/.../CN=srv.domain.net
---
SSL handshake has read 3369 bytes and written 5705 bytes
---
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
Protocol : TLSv1.2
Cipher : ECDHE-RSA-AES256-GCM-SHA384
Session-ID: ...DF6572FA00D62D83CF0B1A82F
Session-ID-ctx:
Master-Key: ...7F685C0B163A7739C271E9722FC0554108175C4
Key-Arg : None
PSK identity: None
PSK identity hint: None
SRP username: None
Start Time: 1461337153
Timeout : 300 (sec)
Verify return code: 0 (ok)
---
I am now attempting to hook up a Python (2.7.9) script to the XML service using the same cert, key and CA file but I can't get it working. Python is complaining about SSLv3 errors and the server says it can't verify the client. So, the connection works, but the handshake, certs or ciphers or something aren't right.
I've searched out numerous examples and implementations and this one seemed to be the simplest one so I started with it for a template. SSL3 is not the protocol I would stick with (POODLE), but it was supposed to be from a working example so I wanted to make as few changes as possible to get things working and then tweak from there. Anyone know what I've got wrong here? ouput/errors/logs are posted at the far bottom.
#!/usr/bin/python
# -*- coding: utf-8 -*-
import socket
import OpenSSL
from OpenSSL import *
import sys
serverName = sys.argv[1]
print "Using server : " + serverName
ctx = SSL.Context(SSL.SSLv3_METHOD)
ctx.use_privatekey_file('client_key.pem')
ctx.use_certificate_file('client_cert.pem')
ctx.load_verify_locations(cafile='server_ca.pem')
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((serverName, 8388))
sslSocket = socket.ssl(s)
print repr(sslSocket.server())
print repr(sslSocket.issuer())
print ("writing socket..")
sslSocket.write('<transaction><data>14</data></transaction>\n')
s.close()
Python output:
Using server : localhost
Traceback (most recent call last):
File "./test3.py", line 29, in <module>
sslSocket = socket.ssl(s)
File "/usr/lib/python2.7/socket.py", line 64, in ssl
return _realssl.sslwrap_simple(sock, keyfile, certfile)
File "/usr/lib/python2.7/ssl.py", line 993, in sslwrap_simple
ssl_sock.do_handshake()
ssl.SSLError: [SSL: SSLV3_ALERT_BAD_CERTIFICATE] sslv3 alert bad certificate (_ssl.c:581)
Server logs after connection above:
Apr 22 10:39:56 srv.domain.net ERROR server.auth [Thread-183,run:233] Couldn't validate the client certificate. Verify the validity and dates of the client cert.
Apr 22 10:39:56 srv.domain.net javax.net.ssl.SSLPeerUnverifiedException: peer not authenticated
Apr 22 10:39:56 srv.domain.net at sun.security.ssl.SSLSessionImpl.getPeerCertificates(SSLSessionImpl.java:431)
Apr 22 10:39:56 srv.domain.net at com.xml.server.auth.run(auth.java:226)
Apr 22 10:39:56 srv.domain.net at java.lang.Thread.run(Thread.java:745)
Working code:
#!/usr/bin/python
# -*- coding: utf-8 -*-
import socket
import ssl
import sys
import os
serverName = sys.argv[1]
print "Using server : " + serverName
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# ssl.PROTOCOL_xxx does not seem to affect negotiated cipher??
wrapper = ssl.wrap_socket(s,
ca_certs = "server_ca.pem",
certfile = "client_cert.pem",
keyfile = "client_key.pem",
cert_reqs=ssl.CERT_REQUIRED,
ssl_version=ssl.PROTOCOL_TLSv1)
wrapper.connect((serverName, 8388))
# some info on the connnection/cipher in use
print repr(wrapper.getpeername())
print repr(wrapper.cipher())
print repr(wrapper.getpeercert())
# send server command
print ("writing socket..")
wrapper.send ("<transaction><data>14</data></transaction>\n")
# read server reply
print "server reply: " + wrapper.recv(4096)
wrapper.close()
s.close()
ctx.use_privatekey_file('client_key.pem')
ctx.use_certificate_file('client_cert.pem')
...
sslSocket = socket.ssl(s)
While you create an SSL context with the client certificate you don't use this context within your SSL connection. This means no client certificate will be send and the server complains accordingly.
A much simpler way to use client certificates is by using the certfile and keyfile parameter of ssl.wrap_socket, see the documentation.