Develop Reference

Python Socket SSL Failure

I'm trying to use SSH through Websocket Reverse-Proxy, the indication of successful request would be 101. The target server only accepts HTTPS which means I had to use SNI Spoofing to connect. Not to mention that the target also protected with CloudFlare, that's another reason to use TLS/SNI Spoofing.
Actually, I have tried websockets module that specifically handle this but always return 403 or 404 response. Using conventional requests also result into this. In the end, I have to combine ssl module with socket manually, it works but below codes has a problem with SSL Protocol Version:
cx = ssl.create_default_context()
sli = cx.wrap_socket(socket.socket(), server_hostname='unpkg.com')
sli.connect(('blog.clova.line.me', 443))
sli.sendall(b'''GET wss://unpkg.com/ HTTP/1.1\r
Host: identity.o2.co.uk.zainvps.tk\r
User-Agent: cpprestsdk/2.9.0\r
Upgrade: websocket\r
Connection: Upgrade\r
Sec-WebSocket-Version: 13\r\n\r
''')
Above codes unable to do Handshake for SSLv3 and triggers UnsafeLegacyRenegotiation. However, it does work for some domain but not for most of it that mainly use SSLv3.
Target Domain: unpkg.com
SSH: identity.o2.co.uk.zainvps.tk
To fix unsafelegacy, I had to use custom openssl config:
openssl_conf = openssl_init
[openssl_init]
ssl_conf = ssl_sect
[ssl_sect]
system_default = system_default_sect
[system_default_sect]
Options = UnsafeLegacyRenegotiation
I wanted it able to use any protocol version. Is there's some auto option that allow to use any SSL Protocol version; SSLv23, TLSv1, and SSLv3?

Disable Client Authentication for python HTTPS requests

Currently my client side HTTPS request code looks like this:
resp = requests.post(endpoint_predict_v100, files={'image': open(file_path, 'rb')}, verify=client_auth,
cert=(client_cert_path, client_secret_path))
This is an HTTPS request, the server root CA is client_auth and the client certificate and key are client_cert_path and client_secret_path.
because my server code is deployed and I cannot change much on that side, I am wondering if I can do the following on the client side:
enable server authentication only when the server root CA is provided by the client
enable client authentication only when the client cert and key are provided by the client.
I have found out that I am able to do the first thing, "enable server authentication only when the server root CA is provided by the client", by passing verify=False if the server root CA is not presented on the client side., referring to the Python requests docs. However, when I try to remove the cert=(client_cert_path, client_secret_path), the request will fail, throwing this error:
ssl.SSLError: [SSL] tlsv13 alert certificate required (_ssl.c:2570)
It seems that this error is suggesting I must pass in the client cert and key to make the server accept my request. but is there a way to disable client authentication on the client side, when the client cert and key are not present?

ssl.SSLError: [SSL] tlsv13 alert certificate required (_ssl.c:2570)
... is there a way to disable client authentication on the client side, when the client cert and key are not present?
The alert you get is because the server is requiring the client certificate. Since it is required by the server it is not sufficient to change some client side behavior only. This is similar to a server requesting authentication by password - this cannot be simply skipped in the client too.
With many servers there is a way to have client certificates optional, i.e. request a client certificate but don't insist that one is sent. How to do this with your specific server and if it is supported by your server at all is unknown though.

SSL: SSLV3_ALERT_HANDSHAKE_FAILURE sslv3 alert handshake failure (_ssl.c:833)

I have a simple TLS client in python running in Ubuntu 18.04 and openssl version 1.1.0g. The client supports a single ciphersuite. I get an error when trying to connect to a TLS 1.0 server. The cipher suite is not supported by the server. I know that the reason for the error is most likely due to lack of ciphersuite mismatch but I am looking for a more meaningful error for the user in this case. The error I am getting at the moment is pointing to SSLv3 which neither the client nor the server has anything to do with SSLv3. The client disables SSLv3 and the server as well. This is the error :
[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:833)
My question is: I need a better error message says for example (lack of ciphersuite mismatch or something like that is relevant to ciphersuite issue). Is there any? Of course I could write my own message but the socket connection can fail for various reasons and I can not make a general error that always says "ciphersuite mismatch".
This is the client script:
import socket,ssl
import itertools
context = ssl.SSLContext()
context.verify_mode = ssl.CERT_NONE
context.check_hostname = False
ciphers = "ECDHE-ECDSA-AES128-GCM-SHA256"
context.set_ciphers(ciphers)
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
domainName = "privatedomain.com"
sslSocket = context.wrap_socket(s, server_hostname = domainName)
try:
sslSocket.connect((domainName, 443))
except (ssl.SSLError, ssl.SSLEOFError, ssl.CertificateError,ssl.SSLSyscallError, ssl.SSLWantWriteError, ssl.SSLWantReadError,ssl.SSLZeroReturnError) as e:
print("Error: ",e)
sslSocket.close()

From the client's view, it is not possible to get another message than the one sent by the server, which is handshake failure in your case. The error message are, for example, documented in RFC 2246 7.2.
The reason why you see SSLv3 in your message, is that you probably send a SSLv3 Hello, which is something allowed to negotiate a TLS 1.0 or later protocol.

Late answer but hopefully helpful . . .
Both client and server must agree on the transport layer version for the connection to be successful. Consider meeting a person for the first time. The person (client) extends their hand to you (server) in a gesture of greeting. If you just saw the person come out of the latrine without washing hands and you see (and/or smell) something undesirable, you will not extend your hand in return.
It is similar with an SSL handshake. The client says "Hey I'd like to communicate via TLS v1.0". The savvy admin for the server knows TLS v1.0 is not secure and they have disabled it on the server--so the server responds to the client, "No, but how about version 1.3?" (ie: "Go wash your hands first"). If the client accepts (washes hands), the handshake is accepted and the connection is established. If the client refuses, the server keeps asking for lower versions ("How about a gallon of Purell then?") until the client accepts or the server has no other versions to offer (walks away).
Basically, the handshake is designed to use the highest version that both the client and server support.
This page has a nice table of versions for client & server (about half way down in the "SSL Contexts" section:
https://docs.python.org/3/library/ssl.html
Note that TLS v1.0 is no longer considered secure (Google "POODLE attack"). If your server supports it, disable it ASAP.

For me this:
urllib.error.URLError: <urlopen error [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:1123)>
meant I was doing this
cipherstr = 'MEDIUM:!aNULL:!eNULL'
context = ssl._create_unverified_context()
context.set_ciphers(cipherstr)
commenting out the set_ciphers and it works now.
Other thing to check: make sure your version of OpenSSL is new enough.

"The connection was reset" on web browsers when trying to connect to a localhost socket server

I am trying to make a server in python using sockets that I can connect to on any web browser. I am using the host as "localhost" and the port as 8888.
When I attempt to connect to it, the stuff I want to be shown shows up for a split-second, and then it goes away with the browser saying "The connection was reset".
I've made it do something very simple to test if it still does it, and it does.
Is there a way to stop this?
import time
import socket
HOST = "localhost"
PORT = 8888
def function(sck):
sck.send(bytes("test"),"UTF-8"))
sck.close()
ssck=socket.socket(socket.AF_INET,socket.SOCK_STREAM)
ssck.bind((HOST,PORT))
ssck.listen(1)
while True:
sck,addr=ssck.accept()
function(sck)

Probably the same problem as Perl: Connection reset with simple HTTP server, Ultra simple HTTP socket server, written in PHP, behaving unexpectedly, HTTP Server Not Sending Complete File To WGET, Firefox. Connection reset by peer?. That is you don't read the HTTP header from the browser but simply send your response and close the connection.

tl;dr
your function should be
def function(sck):
sck.send(bytes("HTTP/1.1 200 OK\n\n<header><title>test page</title></header><body><h1>test page!</h1></body>"),"UTF-8"))
sck.close()
With a server as simple as that, you're only creating a TCP socket.
HTTP protocols suggest that the client should ask for a page, something like:
HTTP/1.1 GET /somepath/somepage.html
Host: somehost.com
OtherHeader: look at the http spec
The response should then be:
HTTP/1.1 200 OK
some: headers
<header></header><body></body>

Sending SSL data over a TCP proxy connection in Python

I am facing the following scenario:
I am forced to use an HTTP proxy to connect to an HTTPS server. For several reasons I need access to the raw data (before encryption) so I am using the socket library instead of one of the HTTP specific libraries.
I thus first connect a TCP socket to the HTTP proxy and issue the connect command.
At this point, the HTTP proxy accepts the connection and seemingly forwards all further data to the target server.
However, if I now try to switch to SSL, I receive
error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol
indicating that the socket attempted the handshake with the HTTP proxy and not with the HTTPS target.
Here's the code I have so far:
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(('proxy',9502))
s.send("""CONNECT en.wikipedia.org:443 HTTP/1.1
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:15.0) Gecko/20100101 Firefox/15.0.1
Proxy-Connection: keep-alive
Host: en.wikipedia.org
""")
print s.recv(1000)
ssl = socket.ssl(s, None, None)
ssl.connect(("en.wikipedia.org",443))
What would be the correct way to open an SSL socket to the target server after connecting to the HTTP proxy?

(Note that in generally, it would be easier to use an existing HTTPS library such as PyCurl, instead of implementing it all by yourself.)
Firstly, don't call your variable ssl. This name is already used by the ssl module, so you don't want to hide it.
Secondly, don't use connect a second time. You're already connected, what you need is to wrap the socket. Since Python doesn't do any certificate verification by default, you'll need to verify the remote certificate and verify the host name too.
Here are the steps involved:
Establish your plain-text connection and use CONNECT like you're doing in the first few lines.
Read the HTTP response you get, and make sure you get a 200 status code. (You'll need to read the header line by line).
Use ssl_s = ssl.wrap_socket(s, cert_reqs=ssl.CERT_REQUIRED, ssl_version=ssl.PROTOCOL_TLS1, ca_certs='/path/to/cabundle.pem') to wrap the socket. Then, verify the host name. It's worth reading this answer: the connect method and what it does after wrapping the socket.
Then, use ssl_s as if it was your normal socket. Don't call connect again.

works with python 3
< proxy > is an ip or domain name
< port > 443 or 80 or whatever your proxy is listening to
< endpoint > your final server you want to connect to via the proxy
< cn > is an optional sni field your final server could be expecting
import socket,ssl
def getcert_sni_proxy(cn,endpoint,PROXY_ADDR=("<proxy>", <port>)):
#prepare the connect phrase
CONNECT = "CONNECT %s:%s HTTP/1.0\r\nConnection: close\r\n\r\n" % (endpoint, 443)
#connect to the actual proxy
conn = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
conn.connect(PROXY_ADDR)
conn.send(str.encode(CONNECT))
conn.recv(4096)
#set the cipher for the ssl layer
context = ssl.SSLContext(ssl.PROTOCOL_SSLv23)
#connect to the final endpoint via the proxy, sending an optional servername information [cn here]
sock = context.wrap_socket(conn, server_hostname=cn)
#retreive certificate from the server
certificate = ssl.DER_cert_to_PEM_cert(sock.getpeercert(True))
return certificate