Having great difficulty getting a response, even a reassuring error response, after attempts to connect to Coinfloor's websocket API. Docs here: https://github.com/coinfloor/API/blob/master/WEBSOCKET-README.md
'Commands, replies, and notifications traverse the WebSocket in text
frames with JSON-formatted payloads.'
Here is my attempt:
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server = 'api.coinfloor.co.uk'
port = 443
server_ip = socket.gethostbyname('api.coinfloor.co.uk')
payload = '{"method": "WatchTicker","base": int("0xF800", 16),"counter":int("0xFA20",16),"watch":True}'
s.connect((server_ip, port))
s.sendall(payload.encode('utf-8'))
result = s.recv(4096)
print(result)
It just returns this:
b''
i.e. an empty byte string.
Because sockets and WebSocket are completely different things. AF_INET/SOCK_STREAM socket is a facility that uses TCP to communicate to the remote peer. On the other hand, WebSocket is a binary protocol that
Works on the top of TCP or TLS.
Has to perform HTTP handshake before data exchange.
Since WebSocket is a rather complex protocol (see the standard), your best course of action is to find a WebSocket library and use it instead of trying to implement the protocol starting from TCP.
Related
I'm relatively new to Python, so apologies for what me a simple question, I just cannot find the solution. First off, I am not looking for the client's hostname. My situation is that I have a simple socket server (basically this https://docs.python.org/3/library/socketserver.html#socketserver-tcpserver-example) which clients connect to. The exact server code is;
import socketserver
class MyTCPHandler(socketserver.BaseRequestHandler):
"""
The request handler class for our server.
It is instantiated once per connection to the server, and must
override the handle() method to implement communication to the
client.
"""
def handle(self):
# self.request is the TCP socket connected to the client
self.data = self.request.recv(1024).strip()
print("{} wrote:".format(self.client_address[0]))
print(self.data)
# just send back the same data, but upper-cased
self.request.sendall(self.data.upper())
if __name__ == "__main__":
HOST, PORT = "0.0.0.0", 8080
# Create the server, binding to localhost on port 9999
with socketserver.TCPServer((HOST, PORT), MyTCPHandler) as server:
# Activate the server; this will keep running until you
# interrupt the program with Ctrl-C
server.serve_forever()
The clients connect successfully and are sending data which the server is receiving. My issue is that need to know the hostname that the client used to connect. The architecture will be like;
client1 will connect to client1.mydomain.net:8080
client2 will connect to client2.mydomain.net:8080
client3 will connect to client3.mydomain.net:8080
The DNS entry for client1.mydomain.net, client2.mydomain.net and client3.mydomain.net will all map to 123.123.123.123 so behind the scenes there is only one server.
The 3 clients will connect to their respective server and send their data. I have no control over the payload and I cannot augment it with a string or parm like "client=1".
So my question is, is there a way in python sockets (on the server) to know the hostname that a client connected to, so for example I know when I'm process data from client1.
Thanks!
Nothing at the TCP level reveals which hostname the client is connected to. This means there is no way for a generic TCP server to get this information.
Various protocols on top of TCP contains such information though. For example HTTP has a Host header so that different domains with different contents can be server on the same IP and port. TLS has the server_name extension in the TLS handshake so that the expected certificate can be given matching the hostname used by the client.
Thus, if you need this information you need to define your application protocol so that the client will include this information.
Suppose we have an echo server and a file transfer app.
There is a sender (Client) to send files and a receiver (Server) to receive files. The echo server will echo whatever received from the Client and the Server.
However, Client and Server cannot communicate directly, i.e., all packets have to go through the echo server. For example, the Client sends a UDP packet to the Echo server, and the Echo server echo that packet to Server and Server send Acknowledgment to the Echo server, and the Echo server echo that ack packet to the Client.
The objective is to implement a reliable UDP for file transfer. And we have only one UDP socket.
This figure demonstrates what the setup is
Client, Server and Echo Server
I have tried to use multi-thread and select.select and both do not work perfectly
The issue with multi-thread is that since Client and Server cannot communicate internally, and we have only one socket, it is difficult to choose who should send or receive now.
The issue with select.select is that the return list always has writeable non-empty, which makes the Client continues to send a bunch of packets before the readable is ready.
Here is the implementation for both Client and Server inside one file (say transceiver.py) what I do not use select.select (instead using send bool variable) but it seems to work fine. But I do believe this is bad practice, so I wonder what can I do to improve my design.
def create_transceiver(ip, port):
address = (ip, port)
udp = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
udp.settimeout(1)
inout = [udp]
client = Client(udp, address)
server = Server(udp, address)
client_to_server = True
send = True
while True:
# infds, outfds, errfds = select.select(inout, inout, [])
if not send: # len(infds) != 0
if client_to_server:
server.start_recv()
client_to_server = False
else:
client.start_recv()
client_to_server = True
send = True
elif send: # len(outfds) != 0
if client_to_server:
if client.has_ack_all():
print(server.write_content())
break
client.start_send()
client_to_server = True
else:
server.start_send()
client_to_server = False
send = False
Here is the implementation of Echo Server:
import socket
ip = "10.10.1.100"
port = 8888
address = (ip, port)
udp_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
udp_socket.bind(address)
while True:
data, address = udp_socket.recvfrom(2048)
udp_socket.sendto(data, address)
And we have only one UDP socket.
As far as I understand this objective, you have a server process somewhere (you call it echo server) that is listening on a specific port. Also there is a client that wants to send data to some sort of server.
You provided code shows an implementation of what is called a state machine that (in your case) switches between receiving and sending.
However, Client and Server cannot communicate directly
The scenario you are describing makes your echo server a classic server that handles different types of clients. In your case this would be your "client" and your "server". I would like to call these just client-A and client-B. Most tutorials on the internet would call them Alice and Bob, I guess.
The objective is to implement a reliable UDP for file transfer.
So you want to transfer files between different clients using UDP as the base protocol.
UDP is not very well suited for this purpose. It does not garantee delivery of each packet transmitted. It is possible that packets arrive in different order than they were sent.
Usually you would use TCP for this kind of transmission. UDP is usually used for live streaming data like audio/video calls and stuff like that.
For more information on the differences between UDP and TCP you might check out the wikipedia pages for each:
https://en.wikipedia.org/wiki/User_Datagram_Protocol
https://en.wikipedia.org/wiki/Transmission_Control_Protocol
It is possible to use UDP for your transfers but you would have to implement all the safeties provided by TCP yourself.
I assume, your client and your server are actually different programs. Otherwise there would be a way they could communicate directly.
If that is the case, this tutorial might give you a starting point:
https://pythonprogramming.net/server-chatroom-sockets-tutorial-python-3/
I am trying to modify a socket server I wrote with the python socket library to use encryption using python's SSL library.
I am no able to successfully open a connection to the server, wrap it with an SSL context and send data to the server, but data sent back to the client is not what it should be.
My suspicion is that the server responses are not being decrypted on the client side, but I don't know why. I'm pretty new to SSL/TLS, and networking in general so... what am I missing?
The client is also written in python (for now, to facilitate testing)
Code:
Relevant Server stuff:
def sslServerLoop():
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((host, port))
s.listen(5)
context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
context.load_cert_chain('cert.pem')
while True:
conn, addr = s.accept()
sslConn = context.wrap_socket(conn, server_side=True)
data = sslConn.recv(1024)
sslConn.sendall(response)
sslConn.close()
Relevant Client stuff:
context = ssl.create_default_context(cafile='cert.pem')
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s = context.wrap_socket(s, server_hostname=server_addr)
s.connect((address, port))
s.sendall(msg)
s.shutdown(socket.SHUT_WR)
response = s.recv(1024)
Sending from client to server works fine, but data sent back to the client is wrong. For example if I set response = bytes([1]) on the server side, I receive b'\x17\x03\x03\x00\x19\xac\xb6\x7f#\xc0\xd3\xce%\x13G\x01\xbd\x88y\xf0\xda..\x02\xf9\xe4o\xdd\x1a\xdb' on the client side. Most of that changes every time I try to run it, but the first 5 bytes are always the same (which is partly why I suspect it isn't being decrypted).
cert.pem is a self signed certificate generated using openssl as described in the python 3 SSL module documentation
It is not legal to shutdown a socket that is being used for SSL. It is a protocol violation. You must close via the SSL/TLS API you are using.
I have read several different SO posts on using python as a TCP client and am not able to connect to a server sending data via TCP at 1hz which is hosted locally. The connection parameters I am using are:
import socket
client = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
ip=socket.gethostbyname("127.0.0.1")
port=1234
address=(ip,port)
client.connect(address)
while True:
print("test1")
data = client.recv(1024)
print("test2")
print(data)
I believe that it is failing on the second line of the while statement but do not know why because it hangs and I am not given an error. Below are links to the SO articles, I have read and I have attached a screenshot from a TCP client tool that I am able to connect to the data server with. I'm expecting the data to stream in my print statement, is this not how it works? Whats the best way to make a persistent connection to a TCP connection with python?
Researched:
(Very) basic Python client socket example,Python continuous TCP connection,Python stream data over TCP
Working with sockets: In order to communicate over a socket, you have to open a connection to an existing socket (a "client"), or create an open socket that waits for a connection (a "server"). In your code, you haven't done either, so recv() is waiting for data that will never arrive.
The simple case is connecting as a client to a server which is waiting/listening for connections. In your case, assuming that there is a server on your machine listening on port 1234, you simply need to add a connect() call.
import socket
client = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
ip=socket.gethostbyname("127.0.0.1")
port=1234
address=(ip,port)
client.connect(address) ## <--Add this line.
while True:
print("test1")
data = client.recv(1024)
print("test2")
print(data)
I am having trouble receiving UDP packets on an Android device, so I want to find out if I am sending them properly. Using Wireshark, everytime I try to send a UDP packet to a remote address, the following error message occurs:
232646 311.898009000 172.56.16.78 192.168.0.3 UDP 64 Source port: 31947 Destination port: 5001 [ETHERNET FRAME CHECK SEQUENCE INCORRECT]
Frame check sequence: 0xf5b6d06d [incorrect, should be 0xb0c869e3]
Does anyone know how to fix this? Would this be the cause of why I could not receive UDP packets on my Android device?
Server Code:
import http.server
import socket
import threading
import socketserver
class ThreadedUDPRequestHandler(socketserver.BaseRequestHandler):
def handle(self):
data = self.request[0].strip().decode("utf-8")
print("{} Recieved: ".format(self.client_address) + data)
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
response = data.upper()
sock.sendto(bytes(response, "utf-8"), self.client_address)
print("{} Sent: {}".format(self.client_address,response))
if __name__ == "__main__":
udpserver = ThreadedUDPServer((HOST,PORT+1), ThreadedUDPRequestHandler)
udp_thread = threading.Thread(target=udpserver.serve_forever)
udp_thread.daemon = True
udp_thread.start()
print("UDP serving at port", PORT+1)
while True:
pass
udpserver.shutdown()
It seems like you're sending packets using regular userspace sockets. In that case, there's very little chance that the packets are being sent malformed since the FCS is generated physically by the network interface card.
What you're probably seeing is an FCS error due to completely different reasons, which can be safely disregarded.
I'd look for other reasons for why the other device doesn't receive the packet, like firewalls or NAT. Start by using netcat or a similar tool for sending and receiving the UDP packets between the two machines.