I have a server written in Python that basically accepts incoming connection from clients and feeds the data received from them into a subprocess (one instance per connection) which then processes the data and returns the result to the server so that it can send it back to the client.
The problem is that the data is streaming in and I need to be able to execute multiple read/write operations with no EOF in sight. So far, I've been unable to come up with a solution that would enable me to do just that without getting my server program blocked on reading. Any suggestions? Thanks.
You could use select.select (available on Unix and Windows).
while True:
rlist, wlist, xlist = select.select([client, proc.stdout], [], [])
The call to select.select will block until either the client socket or proc.stdout is ready to be read.
rlist holds a subset of [client.stdin, proc.stdout] which is ready to be read.
An example of its usage (though for a different problem) can be found here.
It sounds like you have to establish a UDP server and client.
Related
I want to connect a Tun to a socket so that whatever data is stored in the Tun file will then end up being pushed out to a socket which will receive the data. I am struggling with the higher level conceptual understanding of how I am supposed to connect the socket and the Tun. Does the Tun get a dedicated socket that then communicates with another socket (the receive socket)? Or does the Tun directly communicate with the receive socket? Or am I way off all together? Thanks!
If I am understanding your problem, you should be able to write an application that connects to the tun device and also maintains another network socket. You will need some sort of multiplexing such as epoll or select. But, basically, whenever you see data on the tun interface, you can receive the data into a buffer and then provide this buffer (with the correct number of received octets) to the send call of the other socket. Typically you use such a setup when you insert some custom header or something to e.g., implement a custom VPN solution.
I am busy developing a Python system that uses web-sockets to send/received data from a serial port.
For this to work I need to react to data from the serial port as it is received. Problem is to detect incoming data the serial port needs to queried continuously looking for incoming data. Most likely a continuous loop. From previous experiences(Slow disk access + heavy traffic) using Flask this sounds like it could cause the web-sockets to be blocked. Will this be the case or is there a work around?
I have looked at how NodeJS interact with serial ports and it seems much nicer. It raises an event when there is incoming data instead of querying it all the time. Is this an option in Python?
Extra Details:
For now it will only be run on Linux.(Raspbian)
Flask was my first selection but I am open to other Python Frameworks.
pyserial for serial connection.(Is the only option I know of)
Python provides the select module in the stdlib which can do what you want. It DOES depend on what operating system you are using though. So since you haven't provided that information I can't be that helpful. However a simple example under Linux would be:
import select
epoll = select.epoll()
# Do stuff to create serial connection and websocket connection
epoll.register(websocket_file_descriptor, select.EPOLLIN)
epoll.register(serial_file_descriptor, select.EPOLLIN)
while True:
events = epoll.poll(1)
# Do stuff with the event,
for fileno, event in events:
if fileno == serial_file_descriptor:
data = os.read(serial_file_descriptor)
os.write(websocket_file_descriptor, data)
elif fileno == websocket_file_descriptor:
data = os.read(websocket_file_descriptor)
# Do something with the incoming data
That's a basic, incomplete, example. But it should give you an idea of the general process of using a system like epoll.
Simply start a subprocess that listens to the serial socket and raises an event when it has a message. Have a separate sub-process for each web port that does the same.
I have a client and a server communicating as follows:
# server
running = 1
while running:
data = self.client.recv(self.size)
self.client.send(str(self.vel))
# client
def runit(self):
self.s.send('test')
data = float(self.s.recv(self.size))
self.master_.after(5,self.runit)
So both are in infinity loops, although this does transfer data, it is inefficient for my application. I am making a game and I want the server to send data to the client at specific instances, and I also want the client to receive that data at that instance. Something similar to a callback would work. Unfortunately, I wasn't able to find anything suitable for my needs.
First, I don't see anything "inefficient", a while loop is very common in such case, and as the comment says, the loop will just blocks in recv until a client connects.
So your question is how to send data from server to client? If a connection is established, assuming you're using TCP, then just call send() method on that socket. Maybe you want to set socket SO_KEEPALIVE, see How to change tcp keepalive timer using python script?.
I'm working on a really basic "image streaming" server as a school subject, and I've done most of the work but I'm still stuck on the separation between data and control related sockets:
My structure is : TCPServer (my server, used as control socket) contains a dataSocket (only used to send images and initialized within my TCPServer object, when I receive a certain query)
When I'm sending data (images) through my dataSocket, I still need to see if the client sent a PAUSE or STOP request, but if I use python's self.request.recv(1024) the server awaits a response instead of continuing to send data (which is quite logical).
What should I do to prevent this behavior ? Should I launch my recv(1024) on a separate thread and run it at each loop (and check if I get any relevant data in between two iterations) ?
Twisted should do the trick! It handles asynchronous sockets in Python
I want a two way communication in Python :
I want to bind to a socket where one client can connect to, and then server and client can "chat" with eachother.
I already have the basic listener :
import socket
HOST='' #localhost
PORT=50008
s=socket.socket(socket.AF_INET, socket.SOCK_STREAM ) #create an INET, STREAMing socket
s.bind((HOST,PORT)) #bind to that port
s.listen(1) #listen for user input and accept 1 connection at a time.
conn, addr = s.accept()
print "The connection has been set up"
bool=1
while bool==1:
data=conn.recv(1024)
print data
if "#!END!#" in data:
print "closing the connection"
s.close()
bool=0
What I want to do now is implement something so this script also accepts user input and after the enter key is hit, send it back to the client.
But I can't figure out how I can do this ? Because if I would do it like this :
while bool==1:
data=conn.recv(1024)
print data
u_input = raw_input("input now")
if u_input != "":
conn.send(u_input)
u_input= ""
Problem is that it probably hangs at the user input prompt, so it does not allow my client to send data.
How do I solve this ?
I want to keep it in one window, can this be solved with threads ?
(I've never used threads in python)
Python's sockets have a makefile tool to make this sort of interaction much easier. After creating a socket s, then run f = s.makefile(). That will return an object with a file-like interface (so you can use readline, write, writelines and other convenient method calls). The Python standard library itself makes use of this approach (see the source for ftplib and poplib for example).
To get text from the client and display it on the server console, write a loop with print f.readline().
To get text from the server console and send it to the client, write a loop with f.write(raw_input('+ ') + '\n').
To be send and receive at the same time, do those two steps separate threads:
Thread(target=read_client_and_print_to_console).start()
Thread(target=read_server_console_and_send).start()
If you prefer async over threads, here are two examples to get you started:
Basic Async HTTP Client
Basic Async Echo Server
The basic problem is that you have two sources of input you're waiting for: the socket and the user. The three main approaches I can think of are to use asynchronous I/O, to use synchronous (blocking) I/O with multiple threads, or to use synchronous I/O with timeouts. The last approach is conceptually the simplest: wait for data on the socket for up to some timeout period, then switch to waiting for the user to enter data to send, then back to the socket, etc.
I know at a lower level, you could implement this relatively easily by treating both the socket and stdin as I/O handles and use select to wait on both of them simultaneously, but I can't recall if that functionality is mapped into Python, or if so, how. That's potentially a very good way of handling this if you can make it work. EDIT: I looked it up, and Python does have a select module, but it sounds like it only functions like this under Unix operating systems--in Windows, it can only accept sockets, not stdin or files.
have you checked twisted? twisted python event driven networking engine and library or
oidranot a python library especially for that based on torando web server