timing problems in lisp and python programs communicating over sockets - python

I am trying to get a python program and Allegro Common Lisp program to communicate over sockets. For now, I am trying to set up a Lisp server that listens for connections, get a python client to connect to the server, and then send a simple message from the client to the server. The Lisp server looks as follows:
(let ((socket (socket:make-socket :connect :passive
:type :stream
:address-family :internet
:local-port 45676)))
(format t "opened up socket ~A for connections~%" socket)
;; now wait for connections and accept
(let ((client (socket:accept-connection socket
:wait t)))
(when client
;; we've got a new connection from client
(format t "got a new connection from ~A~%" client)
;; now wait for data from client
(loop until (listen client) do
(format t "first character seen is ~A~%" (peek-char t client))
(let ((data (read-line client)))
(format t "data received: ~A~%" data))))))
The python client looks as follows:
import socket
import time
s = socket.socket (socket.AF_INET, socket.SOCK_STREAM)
s.connect (('', 45676))
time.sleep (1) # works if uncommented, does not work if commented
s.sendall ("(list A B)")
s.close ()
Since I want to have multiple messages pass over this stream, I listen for incoming data on
the server side and then echo it. However, I noticed a problem when I ran this with the sleep command commented. The output simply looked like this:
opened up socket #<MULTIVALENT stream socket waiting for connection
at */45676 # #x207b0cd2> for connections
got a new connection from #<MULTIVALENT stream socket connected from
localhost/45676 to localhost/60582 #
#x207b34f2>
In other words, it did not actually echo the data (in my case, "(list A B)"). If I uncommented the sleep command (to introduce some delay between the connection initiation and sending of the data), the output looked like this:
opened up socket #<MULTIVALENT stream socket waiting for connection
at */45676 # #x207b0bea> for connections
got a new connection from #<MULTIVALENT stream socket connected from
localhost/45676 to localhost/60572 #
#x207b340a>
data received: (list A B)
I'm not sure why this is the case. Does anyone have a solution for this? Is it a bad idea to reuse the same socket connection for multiple exchanges of data? If I remove the entire loop macro call (and thus make it a one-time exchange) the data is received without any problems and is echoed properly
EDIT 1: The last statement is true even with the sleep command commented.

Related

How to check if localhost python server is running?

I'm sending data via sockets between godot and python like this:
godot:
var socket = PacketPeerUDP.new()
socket.set_dest_address("127.0.0.1", 6000)
var data={...}
socket.put_packet(JSON.print(data).to_ascii())
python server:
s= socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
s.bind(("127.0.0.1", 6000))
while True:
data = s.recvfrom(1024)
but the problem is even when the python server is not running the godot code sends the data instead of giving an error that the server is not available
I even tried var err=socket.set_dest_address("127.0.0.1", 6000) hopin this would print out the error
but it always prints 0 whether the python server is running or not
so how do I check if the server is available or not?
This is UDP we are talking about. So there isn't really a session or connection established. Also there isn't really an acknowledged package. So at the end the only solution is to implement your own reliability protocol on top of it (e.g. have the server respond and the client wait for the response). Try searching dor UDP reliability on the gamedev site.
The return values for set_dest_address are ERR_CANT_RESOLVE (the IP is not valid) or OK.
The returns values of put_packet. It can return ERR_BUSY (send buffers are full), FAILED (the socket is otherwise in use) or OK.

Python Socket only sends after programm ends [duplicate]

Im trying to write perl TCP server / python TCP client, and i have the such code now:
import socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_address = ("127.0.0.1", 9000)
sock.connect(server_address)
try:
message = unicode('Test')
sock.sendall(message)
data = sock.recv(1024)
print data
finally:
sock.close()
And i have noticed, that my TCP server (written in perl) is getting message not after sendall(message), but after close(). Server is working like an echo server, and sends data to client after getting a message. And that causes deadlock, server never gets a message, client never gets a response. What could be a problem? What is going to happen during close(), that message comes to server?
I'm going to hazard a guess that this is due to the server's implementation. There are many ways of writing an echo server:
receieve bytes in a loop (or async callback) until EOF; as the bytes are recieved (each loop iteration), echo them without any processing or buffering; when an EOF is found (the inbound stream is closed), close the outbound stream
read lines at a time (assume it is a text protocol), i.e. looking for CR / LF / EOF; when a line is found, return the line - when an EOF is found (the inbound stream is closed), close the outbound stream
read to an EOF; then return everything and close the outbound stream
If the echo server uses the first approach, it will work as expected already - so we can discount that.
For the second approach, you are sending text but no CR / LF, and you haven't closed the stream from client to server (EOF), so the server will never reply to this request. So yes, it will deadlock.
If it is the third approach, then again - unless you close the outbound stream, it will deadlock.
From your answer, it looks like adding a \n "fixes" it. From that, I conclude that your echo-server is line-based. So two solutions, and a third that would work in any scenario:
make the echo-server respond to raw data, rather than lines
add an end-of-line marker
close the outbound stream at the client, i.e. the client-to-server stream (many network APIs allow you to close the outbound and inbound streams separately)
Additionally: ensure Nagle is disabled (often called NO_DELAY) - this will prevent the bytes sitting at the client for a while, waiting to be composed into a decent sized packet (this applies to 1 & 2, but not 3; having Nagle enabled would add a delay, but will not usually cause a deadlock).

Asyncio Socket Not Reading Data Sent on Socket after Working Initially

I'm running into a somewhat perplexing problem in my setup and I'm not sure if it's because I'm doing something wrong with the asyncio socket primitives or there's a deeper issue with my networking setup.
Basically, I have a server (call it sleep-server) that instantiates a long lived TCP socket connection with another server (call it beacon-server). This is done using asyncio.open_connection:
beacon_reader, beacon_writer = await asyncio.open_connection(
beacon_server_endpoint.ip, BEACON_PORT)
The long lived connection is kicked off with an initial request/response to the beacon server. Here's that code:
beacon_writer.write(request)
await beacon_writer.drain()
# Read the beacon server's response
try:
beacon_response = await asyncio.wait_for(beacon_reader.read(4096), timeout=10)
except asyncio.TimeoutError as e:
print('Timed out waiting for beacon server sleep response')
raise SleepRequestError(e)
# Do some stuff with the response data
This works completely fine and well. After that, the application enters a flow where it periodically writes TCP data to the Beacon server on the same socket:
while True:
data = method_that_gets_some_data()
beacon_writer.write(data)
await beacon_writer.drain()
This also works completely fine. The beacon server receives its data and does what it needs to do. During this stage, the beacon server does not write anything back on the socket.
It's only when I try to receive data from the beacon server socket again that things go wrong. Basically, via a separate mechanism the beacon server is told to fetch the existing socket (the one that is currently being written to by the sleep server) and send a message down to it. For business reasons, the beacon server manages its socket connections using gevent and Python's built in socket library, but it's the same stuff, and based on the above flow we already know it works. Here's the code for sending a message back down onto the socket:
# client.connection is just a python Socket object
client.connection.setblocking(0)
client.connection.send(client.wake_up_secret.decode('hex'))
print('sending client shutdown')
client.connection.shutdown(socket.SHUT_WR)
And here's the code on the other server that receives this message:
data = beacon_reader.read()
Based on some logging I can see that the Beacon server is properly performing the send() to the socket. I also see that the sleep-server is getting to the code where it runs beacon_reader.read(). However, the sleep server just blocks there even after the Beacon server has sent down its packet. I've tried playing around with the size paramater in the read() command, even setting it as low as 2 just to see if I could get anything off the socket, but there's nothing. I've also tried removing the connection.shutdown(), which doesn't seem to help either.
What's baffling is clearly it worked before since I was able to get an initial response, and clearly it's tied to the correct socket since the beacon_writer is functioning with no problems. So I'm curious if I'm just doing something stupid with the underlying asyncio library or something.

Why is `select()` returning a client as writable when the client has disconnected?

I am implementing this example for using select in a simple echo server. Everything works fine, the client sends the message, receives the echo and disconnects.
This is the code I used for the client:
import socket
ECHOER_PORT = 10000
if __name__ == '__main__':
sockfd = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sockfd.connect(('localhost', ECHOER_PORT))
msg = input('Message: ').encode()
sockfd.send(msg)
response = sockfd.recv(len(msg))
print('Response: {}'.format(response))
sockfd.close()
The issue is with the server (full gist code is here), after sending the echo, select is called again one more time and the current client (which received the echo and disconnected) is returned from select as both readable and writable.
I understand why it's returned as readable, since according to the article:
A readable socket without data available is from a client that has
disconnected, and the stream is ready to be closed.
But my question is why does return as writable also?
But my question is why does return as writable also?
The main thing you want to have select() do when a client has disconnected is return immediately, so that your code can detect the disconnection-event and handle it ASAP (by closing the socket).
One way to do this is the common way, by having the server select-for-read on every socket, and when a client disconnects, select() will return ready-for-read on that socket, then the program will call recv() on the socket to find out what the data is, recv() will return EOF, and the server will close the socket. That is all well and good.
Now imagine the less-common (but not unheard of) case where the server writes to its client sockets, but doesn't want to read from them. In this case, the server has no need (or desire) to select for ready-to-read on its sockets; it only needs to select for ready-to-write, to know when there is some outgoing-data-buffer-space available to send more data to a client. That server still needs to know when a client has disconnected, though -- which is why the disconnected socket selects as ready-for-write as well, so that a select() that is only watching for ready-for-write can also detect and react to a disconnected socket immediately.

Socket : 2 way communication in python

I want a two way communication in Python :
I want to bind to a socket where one client can connect to, and then server and client can "chat" with eachother.
I already have the basic listener :
import socket
HOST='' #localhost
PORT=50008
s=socket.socket(socket.AF_INET, socket.SOCK_STREAM ) #create an INET, STREAMing socket
s.bind((HOST,PORT)) #bind to that port
s.listen(1) #listen for user input and accept 1 connection at a time.
conn, addr = s.accept()
print "The connection has been set up"
bool=1
while bool==1:
data=conn.recv(1024)
print data
if "#!END!#" in data:
print "closing the connection"
s.close()
bool=0
What I want to do now is implement something so this script also accepts user input and after the enter key is hit, send it back to the client.
But I can't figure out how I can do this ? Because if I would do it like this :
while bool==1:
data=conn.recv(1024)
print data
u_input = raw_input("input now")
if u_input != "":
conn.send(u_input)
u_input= ""
Problem is that it probably hangs at the user input prompt, so it does not allow my client to send data.
How do I solve this ?
I want to keep it in one window, can this be solved with threads ?
(I've never used threads in python)
Python's sockets have a makefile tool to make this sort of interaction much easier. After creating a socket s, then run f = s.makefile(). That will return an object with a file-like interface (so you can use readline, write, writelines and other convenient method calls). The Python standard library itself makes use of this approach (see the source for ftplib and poplib for example).
To get text from the client and display it on the server console, write a loop with print f.readline().
To get text from the server console and send it to the client, write a loop with f.write(raw_input('+ ') + '\n').
To be send and receive at the same time, do those two steps separate threads:
Thread(target=read_client_and_print_to_console).start()
Thread(target=read_server_console_and_send).start()
If you prefer async over threads, here are two examples to get you started:
Basic Async HTTP Client
Basic Async Echo Server
The basic problem is that you have two sources of input you're waiting for: the socket and the user. The three main approaches I can think of are to use asynchronous I/O, to use synchronous (blocking) I/O with multiple threads, or to use synchronous I/O with timeouts. The last approach is conceptually the simplest: wait for data on the socket for up to some timeout period, then switch to waiting for the user to enter data to send, then back to the socket, etc.
I know at a lower level, you could implement this relatively easily by treating both the socket and stdin as I/O handles and use select to wait on both of them simultaneously, but I can't recall if that functionality is mapped into Python, or if so, how. That's potentially a very good way of handling this if you can make it work. EDIT: I looked it up, and Python does have a select module, but it sounds like it only functions like this under Unix operating systems--in Windows, it can only accept sockets, not stdin or files.
have you checked twisted? twisted python event driven networking engine and library or
oidranot a python library especially for that based on torando web server

Categories

Resources