I'm writing a simple OpenVPN client (with Python & Scapy & [scapy-ssl_tls]
) which should connect to OpenVPN server.
I open UDP socket in Python and with Scapy I define my own OpenVPN layer on top of UDP (according to OpenVPN specs) and send packets on it (just like original client would).
I am able to successfully send initial P_CONTROL_HARD_RESET_CLIENT_V2 message and receive response from server, which is P_CONTROL_HARD_RESET_SERVER_V2, then I send P_ACK_V1 message.
Keep in mind I generate all session ids correctly.
Now when I send first P_CONTROL_V1 message, which is essentially TLS ClientHello on top of OpenVPN layer, I get a P_ACK_V1 acknowledgement from server but that's it. Note that this ACK does only mean that server received OpenVPN message, not necessarily TLS data. I'm supposed to get ServerHello and all the remaining stuff but server does not send anything after ACK.
I compared the packet format and all network layers of my sent packet with communication of real client (image below) and pretty much all the fields are identical.
Wireshark combines and assembles packets automatically when it has the full handshake, so little tricky to compare it.
I also tried replaying complete ClientHello message from previous real client communication (I generated my own local time though) but results were the same - ACK and then nothing.
I also checked server logs and didn't find any errors or anything what could help me.
I create my TLS packet like this (with more options):
pack = openvpn(opcode=0x20, session_id=ses, message_packet_id_array_length=0, message_packet_id=0000)/TLSRecord()/TLSHandshake()/TLSClientHello()
openvpn is a layer I defined myself in Scapy.
Any ideas why I don't get ServerHello?
EDIT: considering that I don't get any alerts from server I'm pretty sure server does not even see my ClientHello for some reason.
Apparently Message Packet-ID must be 1 (or more). Now I get response from server.
Official specification only mentions that Packet-id is for replay protection though..
Related
Currently I am trying to reverse engineer how the key exchange and encryption works for a program. I mainly used wire shark to figure out how the packets where being encrypted. Of what I have found they are using ECDHE-RSA-AES128-GCM-SHA256 with TLS1.2
This is currently what I have.
import socket, ssl
tcpSocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
tcpSocket.bind(('', 443))
tcpSocket.listen(1)
while True:
newsocket, fromaddr = tcpSocket.accept()
newsocket = ssl.wrap_socket(newsocket,
server_side=True,
do_handshake_on_connect=True,
certfile="cert.pem",
ssl_version=ssl.PROTOCOL_TLSv1_2,
ciphers="ECDHE-RSA-AES128-GCM-SHA256")
try:
#Later add stuff
pass
finally:
newsocket.shutdown(socket.SHUT_RDWR)
newsocket.close()
I have tried to visualize how this works using a simple picture here.
I made this using the wireshark packets that I saw here
Currently with this code, the server never replies with a client key exchange and instead just crashes leading to https://hastebin.com/wiqojapule.sql
So the main question im trying to ask is why does the python program never aw acknowledge the client and then send a server hello. Currently it just sends a Client Hello back.
Also, here is the wire shark for my python socket and the program: here
Comparing the two pcap files (with real server and with test server) two things stand out:
In the working case (real server) the client is sending a server_name extension (SNI) with the target hostname and the server sends a certificate back which is signed by a public CA.
In the failing case (test server) the client is sending no server_name extension and the server is sending only a self-signed certificate back. The connection is then closed by the client, i.e. the handshake is not completed.
The missing SNI extension against the test server is probably due to given the destination as IP address and not a hostname. And this is probably also not the problem.
Very likely instead is that the client simply does not like the self-signed certificate send by the server and is therefore closing the connection. A nicer behaving client would probably send first a TLS alert unknown CA before the close but it is not uncommon that TLS stacks don't send these alerts and instead just close the connection.
... I am trying to reverse engineer how the key exchange and encryption works for a program
There does not seem anything special to reverse engineer here. The client seems to use TLS as specified in the standard, likely by using some of the many available TLS stacks and not by implementing its own. And contrary to the assumption of the OP that only a specific cipher is used the client is actually offering several ciphers (19 ciphers offered inside ClientHello) and the server can choose any of these.
The behavior one is seeing regarding self-signed certificates is actually expected: a proper client should not accept connections to a server with an untrusted certificate.
There're basically two issues I'd like to resolve:
Client side send query string when initializing the connection to server
Server side validate user token in handshake (not after the connection is established and then validate streaming message that contains the token) and set user session accordingly.
I read a article (https://auth0.com/blog/2014/01/15/auth-with-socket-io/) that talks about this process implemented in nodejs, just wonder if the same function can be achieved by using python. (Currently I'm doing some research on twisted but haven't found anything similar)
PS: guess it's helpful to demo the use case as well. A user may login to your server over normal http then server will issue him/her an valid accessToken. Then this user may need to establish a socket connection with the server (or some other server), then the server needs to figure out who the user is and validate before establishing the socket connection.
Query strings are part of HTTP URLs.
If you're building a TCP socket server instead of an HTTP server, you don't get URLs—or headers, or anything else out-of-band.* All you get is a stream of data. You need to come up with a protocol for your data that you can fit the token into.
This means the server can't "figure out who the user is and validate before establishing the socket connection". It has to establish the socket connection, read the first message, parse it, validate the token, and then drop or continue the connection. (You can, of course, put up a front-end server that accepts connections, validates them, and then migrates or proxies them to the real back-end server. But someone has to accept, read, and parse.)
Note that this is exactly what HTTP does—it can't see the query string until it accepts the connection and reads the first line of data.
Meanwhile, the example you're looking at appears to be using WebSockets. A WebSockets client can't talk to a socket server (well, unless you build a WebSockets server on top of your socket server, or a proxy in front of it) in the first place.
* This isn't quite true. You can cram 40 bytes of options into TCP header extensions. But then you have to go below the level people are usually talking about when they say "socket server"—and there's a good chance it won't make it through the internet. Also, TCP does have a concept of "out-of-band" data, but that isn't relevant here; you still have to accept the connection and read from it to get an OOB data.
I am learning network programing in python and I'm trying to write a Toy vpn forked from android sdk https://github.com/android/platform_development/tree/master/samples/ToyVpn.
My Toy vpn is https://github.com/325862401/ToyVPN.
It's only for Linux.
My home network is behind NAT.
I can use this vpn to surf the internet after connect to remote sever.
But about half an hour or some time later the client udp socket stops receiving any data but the server can receive and send normally.
At this point I must terminate my client and run ToyVpnClient again.
It works normal for some time until it stop receiving again.
Please help me check the client logs.
>2013-08-24 11:42:38 INFO receive data from the tunnel timeout`
you can see that when problem happens, the socket always sends, not receive.
> means send, < means receive
I want to know why the udp socket stops receiving data.
Is there any debug method to find the cause?
For now I've just used logging to debug my program.
Since you're trying your client on the Internet, there is the whole universe of possible causes represented by all the Internet newtwork.
There's not a simple way of debugging here. Possible causes could be of course a software error but also some intermediate network configurations between you and the remote server.
You should capture the udp traffic using the good wireshark or the commandline tcpdump between you and the server and check if you're stopping sending packets or if the server is stopping receiving them.
If you send packets but your server doesn't receive them ( tcpdump on the server ) then there is something on the network which decides to filter your packets. And if it's not on the server (firewall rules to rate limit packets for example or something like that) then there's nothing you can do to that without modifying the logic of your program. Like changing UDP port every X seconds or using a persistent tcp connection.
A udp socket is not stable and may become null once a scanning or other event occupy your network interface for a while (especially true on Android). Using tcp avoids this problem. If you wants to maintain a stable udp, keep monitoring the status of your udp socket; if it becomes null or any unusual things happens, delete this socket and create a new one. Put this reactivating staff in a loop so that your udp socket is always alive.
I'm trying to isolate a bug that exists either in Python's httplib2 HTTP client or an API. (First guess is the API.) While using httplib2 to POST data to a RESTful API, I'm getting a 401 response status (no authorization) and saving data to the API.
I'd like to examine the HTTP request and response to the client, the very strings put onto and received from the network. The httplib2 code seems too involved to easily capture the values from within it, and might possibly miss the bug.
It seems quicker to look at the network communications with the client. Is there some tool I can use to monitor the client's communications with the local network socket?
I use http://www.charlesproxy.com for all my network debugging.
http://www.wireshark.org/ enables you to monitor local sockets too.
I was able to monitor local loopback even on windows using trick whit adding route.
http://wiki.wireshark.org/CaptureSetup/Loopback check Other Alternatives
Or you can just write raw socket server that listen on client side on one port and send data to server on other port and vice versa and prints out all data. It should not take more than dozen of lines of code
As a learning exercise, I'm writing a Python program to connect to a channel on an IRC network, so I can output messages in the channel to stdout. I'm using asynchat and manually sending the protocol messages, rather than using something like Twisted or existing bot code from the net - again, it's a more useful learning experience that way.
I can send JOIN and USER commands quite happily, and can PING/PONG away as required. However, I've noticed when opening a socket to port 6667, I'll receive some messages:
NOTICE AUTH :*** Looking up your hostname...
NOTICE AUTH :*** Checking ident
NOTICE AUTH :*** Found your hostname
NOTICE AUTH :*** No identd (auth) response
even if I've not yet sent the JOIN/USER commands.
So, is this opening sequence of notifications specified anywhere? As far as I can see, the RFC doesn't specify for anything in particular to happen before the client sends the JOIN command, and I wasn't sure whether to wait for receipt of these notices before sending the JOIN command, and if so how do I detect that I've received all of the notices?
There's no RFC requirement to do this, it's just a common thing that servers in the wild do. Observe that they're plain old NOTICE commands (i.e. just messages). Just treat them as messages sent to a psuedo-user "AUTH" (since the server doesn't have a better name for you yet). You're not required to wait for them, and the server is not required to send them.