I have written small programs to create both client and server on the host machine using python sockets. On the other side of the connection, I have my FPGA board which runs LwIP client and server also, and the connection is Gbit ethernet. My aim is to send and receive dummy data and monitor the throughput of both client and server.
client.py
from socket import socket, AF_INET, SOCK_STREAM, SOL_SOCKET, SO_SNDBUF
send_data = []
for i in range(1458):
send_data.append(i % 256)
send_data = bytearray(send_data)
CLIENT_SOCK = socket(AF_INET, SOCK_STREAM)
CLIENT_SOCK.setsockopt(SOL_SOCKET, SO_SNDBUF, 524288)
CLIENT_SOCK.connect(("192.168.1.10", 42000))
print("CLIENT CONNECTED")
i=0
while True:
#print(i)
#i+=1
CLIENT_SOCK.sendall(send_data)
server.py
from socket import socket, AF_INET, SOCK_STREAM, SOL_SOCKET, SO_RCVBUF
SERV_SOCK = socket(AF_INET, SOCK_STREAM)
SERV_SOCK.setsockopt(SOL_SOCKET, SO_RCVBUF, 524288)
SERV_SOCK.bind(("192.168.1.1", 42001))
SERV_SOCK.listen(1)
conn, b = SERV_SOCK.accept()
print("CLIENT ACCEPTED")
i=0
while True:
#print(i)
#i+=1
data = conn.recv(16384)
I run these codes in separate command lines. When I run only one of them, I get ~800 Mbit/s throughput for server.py, and ~950 Mbit/s for client.py. However, the problem arises when I run them at the same time. At this time, I still get the same speed for server.py but I only get a fluctuating throughput between ~100 Kbit/s and ~10 Mbit/s throughput for client.py.
When I uncomment the print statements in the while loops, I saw that in server.py the print statements are printing on the command line smoothly and very fast, but on the other hand client.py prints some and then stops for a while and then prints again but not in a fast manner. I tried to reduce the bandwidth of the server.py by decreasing the size of the receive bytes from 16384 to 256, and I saw that when I run it individually the throughput is around ~500 Mbit/s. However, again, when I run them at the same time, client.py throughput showed a similar pattern as previous. I tried to increase/decrease the send_data size, or adding "\r\n" at the end of the send_data but it didn't work.
Does anyone have any idea what might be the problem and the possible solution?
Thanks.
python socket client slows down when server is also running
In TCP a client will only send data as fast as the server can receive and ACK these (and vice versa). This means another perspective of looking at your problem is not a slow Python client but a slow LwIP server.
Since you run both LwIP client and server on the FPGA there might be a bad management of resources between these. If there is only one LwIP server/client running (and thus only one Python client/server), it will get all the resources on the FPGA and thus can run with max speed. If instead both Python client/server are running there will also be both LwIP both server/client running on the FPGA - and the system resources must be properly managed between these.
The results you get suggest that your LwIP client on the FPGA is getting most of the resources though, which means that the Python server will get max bandwidth. The LwIP server on the other hand will get far less resources which makes it slow - which thus causes the Python client to be slow.
Thus, don't look on the Python side but instead look on your resource managed between LwIP client and server on the FPGA.
Related
I'm sending data via sockets between godot and python like this:
godot:
var socket = PacketPeerUDP.new()
socket.set_dest_address("127.0.0.1", 6000)
var data={...}
socket.put_packet(JSON.print(data).to_ascii())
python server:
s= socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
s.bind(("127.0.0.1", 6000))
while True:
data = s.recvfrom(1024)
but the problem is even when the python server is not running the godot code sends the data instead of giving an error that the server is not available
I even tried var err=socket.set_dest_address("127.0.0.1", 6000) hopin this would print out the error
but it always prints 0 whether the python server is running or not
so how do I check if the server is available or not?
This is UDP we are talking about. So there isn't really a session or connection established. Also there isn't really an acknowledged package. So at the end the only solution is to implement your own reliability protocol on top of it (e.g. have the server respond and the client wait for the response). Try searching dor UDP reliability on the gamedev site.
The return values for set_dest_address are ERR_CANT_RESOLVE (the IP is not valid) or OK.
The returns values of put_packet. It can return ERR_BUSY (send buffers are full), FAILED (the socket is otherwise in use) or OK.
I am implementing a socket in Python to pass data back and forth between two scripts running on the same machine as part of a single Tkinter application.
This data, in many cases, will be highly sensitive (i.e. personal credit card numbers).
Does passing the data between scripts in this way open me up to any security concerns?
Server side:
import socket
serversocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
serversocket.bind(('localhost', 8089))
serversocket.listen(5) # become a server socket, maximum 5 connections
while True:
connection, address = serversocket.accept()
buf = connection.recv(64)
if len(buf) > 0:
print buf
break
Client side:
import socket
clientsocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
clientsocket.connect(('localhost', 8089))
clientsocket.send('hello')
Code source.
Additional considerations:
This will only ever function as part of a single Tkinter application, on a single machine. Localhost will always be specified.
I am unable to use multiprocessing or threading; please no suggestions for using one of those or an alternative, other than varieties of socket. For more info as to why, see this SO question, answers, and comments. It has to do with this needing to function on Windows 7 and *nix, as well as my desired set-up.
Yes, passing the data between scripts in this way may raise a security concerns. If the attacker has an access to the same machine - he can easily sniff the traffic using the tool like tcpdump for example.
To avoid this you should encrypted your traffic - I have posted a comment below your question with an example solution.
I'm currently working with python's socket library for the first time and i'm not very experienced with computer networking.
I'm able to connect to the server and the tcp handshake has happened as viewed by wireshark. After establishing a connection to the server(I have no control over the server), the connection stays open for a while, but after a small amount of time, the server sends a "FIN, ACK" and the connection is terminated. I'm trying to understand how I can keep this connection alive while the client is capable of reaching the server.
Looking at a tcp connection, it seems a packet can be sent every so often. Maybe a sort of keep alive message. I had thought using socket.send('hello') every 5 seconds in another thread would keep the connection with the server open, but I still get the "FIN, ACK" after some time.
In the documentation I found a setsockopt() but using this made no noticeable difference. I've tried client.setsockopt(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1) both before and after the connection is made. I don't completely understand how this method is supposed to work, so maybe I used it incorrectly. There isn't much mention of this. I read somewhere about it being broken on windows. I don't know the truth in that.
What am I missing? The documentation for sockets doesn't seem to have anything about this unless I may have missed something.
import socket
import time
import threading
SERVER_IP = 'THE SERVER'
SERVER_PORT = SERVER_PORT
SOURCE_IP = socket.gethostname()
SOURCE_PORT = 57004
KEEP_ALIVE_INTERVAL = 5
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
def keep_alive(interval):
data = 'hello'
while True:
client.send(data)
time.sleep(interval)
client.connect((SERVER_IP, SERVER_PORT))
t = threading.Thread(target=keep_alive, args = (KEEP_ALIVE_INTERVAL,))
t.start()
while True:
data = client.recv(1024)
if not data:
break
print data
client.close()
For enabling keep alive there is a duplicate question at How to change tcp keepalive timer using python script?
Keep in mind some servers and intermediate proxies forcibly close long lived connections regardless of keep alives being used or not, in which case you will see a FIN,ACK after X amount of time no matter what.
I would like to implement a TCP server using a Python script.
The server should basically do the following task:
It will be cyclically polled by a remote client, then reads certain data on its local workstation and sends it to the client polling.
One can assume the following:
There is always only one client connecting to the server (via Ethernet)
Client and server are running on a Windows platform
Python 2.5.1 will be used for implementation
Goal:
The server should be as efficient as possible with respect to reading/writing data from/to the client
The server shall stress the local workstation as less as possible
The server shall not cause system instability
Since I have only a little experience with the topic, I would like to discuss here how my current (very simple) code could be optimized to meet the requirements previously mentioned.
So far I have the following:
import socket
import sys
port_number = 12345
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_address = (socket.gethostname(), port_number)
sock.bind(server_address)
sock.listen(1)
while True:
connection, client_address = sock.accept()
try:
while True:
data = connection.recv(4096)
if data:
connection.sendall(getdesireddata(data))
else:
break
finally:
connection.close()
Thank you for all your responses.
First of all, there's minor bug in the code. The line...
data = connection.recv(4096)
...will attempt to read up to 4k from a single TCP packet, but if the client sends more than 4k, or decides to break up the data into several packets, you may not get all the data in a single call.
You'd typically continue to read data from the socket, appending to a buffer, until either you have a complete message as defined by your protocol specification, or until the remote host closes the outbound half of its TCP connection.
You'll also have issues with the server getting stuck if the client crashes, and doesn't shut down the socket properly. To avoid this, you'll need to look at either non-blocking IO, which would be something like...
connection.setblocking(0)
buffer = ''
while 1:
try:
data = connection.recv(4096)
if not data:
# Client has closed outbound connection
break
else:
# Append to buffer
buffer += data
except socket.error, e:
code = e.args[0]
if code == 11:
# Got EWOULDBLOCK/EAGAIN
time.sleep(0.1)
else:
# Got another error
raise
do_something_with(buffer)
...or look at using the Python select module to do the same thing.
With regards to the lesser concern of performance, I don't see any major improvements you can make with such a small code sample.
I currently have a client/server pair coded against PyBlueZ. Right now the server can connect to sequential clients - it will work until its completed with a client, then it will begin listening for another client.
However, what I really want is to run client communication in separate threads so I have multiple clients at the same time. When I try a 2nd client connection, however, PyBlueZ advertises the same port that the first client is currently using. I am setting up connections like this:
self.port = bluetooth.PORT_ANY
print "Accepting clients..."
self.server_sock=bluetooth.BluetoothSocket( bluetooth.RFCOMM )
self.server_sock.bind(("",self.port))
self.server_sock.listen(5)
print "listening on port %d" % self.port
bluetooth.advertise_service( self.server_sock, MY_SERVICE, MY_UUID )
client_sock,address = self.server_sock.accept()
print "Accepted connection from ",address
commThread = ServerThread(client_sock, self.bn_id, self.bn_name, self.bn_thumbnail)
Again, this code works fine for sequential connections, but when I try it in parallel my client gets a "busy" response from the server's bluetooth system. On the client side I output the port its trying to connect to and it always shows port "1".
Is there a limitation in PyBlueZ which only allows for a single connection? Or am I doing something wrong here for parallel connections?
I think your problem has nothing to do with the Bluetooth client part of the code.
You were right to show the Bluetooth server code. What you should try to change:
Only advertise a service once, and once only (No need to advertise it for every server thread)
Allocate a different server channel for each thread. (On RFCOMM connection, there usually is a RFCOMM manager which allocates a new server channel per each socket. I think in your case you might have to do that manually.) Change this code
self.port = bluetooth.PORT_ANY
Try channels 1, 2 and so on and see if it works! Then all you have to do is keep track of the allocated channels.
Please let me know if it worked!