I am doing a combination of Socket programming and pexpect in the same code. I have got it working but with a minor glitch. The select API waits for the specified 5 seconds in the first iteration. As soon as it received an input from the client it no longer waits for 5 seconds even though it is specified in the loop. In short after the first client server interaction takes place, select has no effect! I understand how to bypass this in C++ but I am comparatively new to Python and am unable to figure out the reason. I have attached the code below and is a pretty simple one.
#!/usr/bin/python
#Basic Functionailty: to create a process and to take inputs from client machines
#This input will be given to a background process which will display the user the parsed output
#using pexpect
import pexpect
import socket
import select
TCP_IP = '127.0.0.1'
TCP_PORT = 50050
BUFFER_SIZE = 1024
#spawning a pexpect process and printing everything before the prompt appears
child = pexpect.spawn('./pox.py')
child.expect ('POX>')
print child.before
#binding to a port number and acting as a server
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind((TCP_IP, TCP_PORT))
server.listen(1)
input = [server]
#loop infinitely and get input and serve it to the process
#in addition to which if the process shows some messages display
#it to the user
while 1:
print 'Before select'
inputready,outputready,exceptready = select.select(input,[],[],5)
for s in inputready:
if s == server:
#if the input is from the server
conn, addr = s.accept()
print 'Connection address:', addr
input.append(conn)
data = conn.recv(BUFFER_SIZE)
if not data: continue
print "received data:", data
child.sendline (data)
else:
#if a time out occurs check the pexpect if it has any debug messages
i = child.expect ([pexpect.TIMEOUT, 'POX>'], timeout=1)
print child.before
if i == 0:
print child.before
You've modified input:
input.append(conn)
(and then called conn.recv to get some data, which will block until there is some data or EOF, but presumably there was some and you got it).
Having done that, on the next trip through the loop, it's likely that there is receive data, or EOF, ready for input on conn. I assume you immediately call child.expect (because s == conn and hence s != server). I'd bet that at this point, conn is at EOF, so that returns immediately, having done nothing. conn is still open, and still in input, so every time you call select it returns right away telling you that you can read another EOF from conn.
Related
I have just started learning python network programming. I was reading Foundations of Python Network Programming and could not understand the use of s.shutdown(socket.SHUT_WR) where s is a socket object.
Here is the code(where sys.argv[2] is the number of bytes user wants to send, which is rounded off to a multiple of 16) in which it is used:
import socket, sys
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
HOST = '127.0.0.1'
PORT = 1060
if sys.argv[1:] == ['server']:
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind((HOST, PORT))
s.listen(1)
while True:
print 'Listening at', s.getsockname()
sc, sockname = s.accept()
print 'Processing up to 1024 bytes at a time from', sockname
n = 0
while True:
message = sc.recv(1024)
if not message:
break
sc.sendall(message.upper()) # send it back uppercase
n += len(message)
print '\r%d bytes processed so far' % (n,),
sys.stdout.flush()
print
sc.close()
print 'Completed processing'
elif len(sys.argv) == 3 and sys.argv[1] == 'client' and sys.argv[2].isdigit():
bytes = (int(sys.argv[2]) + 15) // 16 * 16 # round up to // 16
message = 'capitalize this!' # 16-byte message to repeat over and over
print 'Sending', bytes, 'bytes of data, in chunks of 16 bytes'
s.connect((HOST, PORT))
sent = 0
while sent < bytes:
s.sendall(message)
sent += len(message)
print '\r%d bytes sent' % (sent,),
sys.stdout.flush()
print
s.shutdown(socket.SHUT_WR)
print 'Receiving all the data the server sends back'
received = 0
while True:
data = s.recv(42)
if not received:
print 'The first data received says', repr(data)
received += len(data)
if not data:
break
print '\r%d bytes received' % (received,),
s.close()
else:
print >>sys.stderr, 'usage: tcp_deadlock.py server | client <bytes>'
And this is the explanation that the author provides which I am finding hard to understand:
Second, you will see that the client makes a shutdown() call on the socket after it finishes sending its transmission. This solves an important problem: if the server is going to read forever until it sees end-of-file, then how will the client avoid having to do a full close() on the socket and thus forbid itself from doing the many recv() calls that it still needs to make to receive the server’s response? The solution is to “half-close” the socket—that is, to permanently shut down communication in one direction but without destroying the socket itself—so that the server can no longer read any data, but can still send any remaining reply back in the other direction, which will still be open.
My understanding of what it will do is that it will prevent the client application from further sending the data and thus will also prevent the server side from further attempting to read any data.
What I cant understand is that why is it used in this program and in what situations should I consider using it in my programs?
My understanding of what it will do is that it will prevent the client
application from further sending the data and thus will also prevent
the server side from further attempting to read any data.
Your understanding is correct.
What I cant understand is that why is it used in this program …
As your own statement suggests, without the client's s.shutdown(socket.SHUT_WR) the server would not quit waiting for data, but instead stick in its sc.recv(1024) forever, because there would be no connection termination request sent to the server.
Since the server then would never get to its sc.close(), the client on his part also would not quit waiting for data, but instead stick in its s.recv(42) forever, because there would be no connection termination request sent from the server.
Reading this answer to "close vs shutdown socket?" might also be enlightening.
The explanation is half-baked, it applies only to this specific code and overall I would vote with all-fours that this is bad practice.
Now to understand why is it so, you need to look at a server code. This server works by blocking execution until it receives 1024 bytes. Upon reception it processes the data (makes it upper-case) and sends it back. Now the problem is with hardcoded value of 1024. What if your string is shorter than 1024 bytes?
To resolve this you need to tell the server that - hey there is no more data coming your way, so return from message = sc.recv(1024) and you do this by shutting down the socket in one direction.
You do not want to fully close the socket, because then the server would not be able to send you the reply.
import socket
sock = socket.socket()
sock.bind(('127.0.0.1', 1600))
sock.listen(1)
(client_sock,client_address) = sock.accept()
client_name = client_sock.recv(1024)
client_sock.send('hi its '+client_name)
client_sock.close()
sock.close();
So i run on cmd command prompt the file containing the code above, and all i get is this:
As it can be seen my cmd is as if waiting for something to happen, it doesnt print out "Hi its ..." and never ends from waiting.
Why doesn't my code work ?
It is simple: you have first to send the data and then receive. The receive command wait until the timeout (if it is set) and then go over. Remember that you have also to print the received data.
The best practice for doing what you want to do is a server-client approach having at least two different threads.
I'm trying to implement the most basic python TCP server. Windows 8, Python 2.7, firewall is turned off. Code is from here: https://wiki.python.org/moin/TcpCommunication
If I do the client stuff (socket(...), connect(...), send(...)) via python repl, things work fine, ie the server correctly blocks when calling recv.
However if I run the exact same code via python script (both with and without explicitly calling python.exe at windows command line), the recv returns immediately with no data. I read elsewhere on SO this means it's an invalid socket, but I'm not sure what that means or how to check for it. I'm using the socket returned by accept() not the one used to initiate the connection.
I'm trying to block on recv so I can take advantage of the timeout (I don't want to use select module, which BTW also returns immediately) and process some keyboard stuff between attempts to recv, ie user presses 'q' to quit.
In various experiments I've shown that once this occurs, recv will always return immediately (as will select.select(...)) if I put it in a loop, so it's not like the client is sending a single "bad" packet initially. If the client happens to have sent something, then the recv returns with that data, but it certainly doesn't block waiting for data when put in a tight loop.
Is this behavior expected?
Server code:
import sys
import socket
TCP_IP = '192.168.1.10'
TCP_PORT = 5005
BUFFER_SIZE = 20 # Normally 1024, but we want fast response
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((TCP_IP, TCP_PORT))
s.listen(1)
conn, addr = s.accept()
print 'Connection address:', addr
while 1:
data = conn.recv(BUFFER_SIZE) # This returns immediately with no data, when client connection is run from script and doesn't send() anything, just connects.
if not data:
print "broken"
break
print "received data:", data
conn.send(data) # echo
conn.close()
sys.exit()
Client code:
import sys
import socket
TCP_IP = '192.168.1.10'
TCP_PORT = 5005
BUFFER_SIZE = 1024
MESSAGE = "Hello, World!"
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((TCP_IP, TCP_PORT))
# Commenting out the following to prove the recv() call on the other
#end returns with nothing instead of blocking indefinitely. If I
#type the rest of this at the REPL the server behaves correctly,
#ie, the recv call blocks forever until socket.send("bla") from client.
#s.send(MESSAGE) data = s.recv(BUFFER_SIZE)
#s.close()
#print "received data:", data
sys.exit()
Yes, this is expected behavior.
The client does not send anything. And it exit as soon as it connect to the server; cause disconnection.
socket.recv returns an empty string if the peer performed shutdown (disconnect).
While, in the REPL, the socket is not closed until you issue sys.exit() or you quit the interactive shell.
I am working on a chat server that runs on my local network using socket, and then I have a client program running on all of the computers in my house, and this program allows all of the clients to talk to each other.
The problem is, you have to manually update the chat log by pressing enter.
The way I want it to work, maybe, is to check for a new message every few seconds, and if there is a new one, play a sound. Does anyone know how I can do this, I'll try to figure it out on my own, as I have done with most of this project, but any help is appreciated.
Here is the server:
import socket
import sys
# Create a TCP/IP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_address = ('192.168.1.80', 10000)
print >>sys.stderr, 'starting up on %s port %s' % server_address
sock.bind(server_address)
sock.listen(1)
print 'Waiting for user...'
convo='Welcome!'
while True:
# Find connections
connection, client_address = sock.accept()
try:
data = connection.recv(999)
if data=='EMPTY':
pass
else:
print data
convo=convo+'\n'+data
connection.sendall(convo)
except:
connection.close()
Here is the client:
import socket
import sys,os
name=raw_input("Enter name: ")
# Create a TCP/IP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_address = ('192.168.1.80', 10000)
print >>sys.stderr, 'connecting to %s port %s' % server_address
while True:
message=raw_input('Message: ')
try:
os.system('cls')
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect(server_address)
if message is not '':
sock.sendall(name+": "+message)
else:
sock.sendall('EMPTY')
if message=='quit':
break
x=sock.recv(999)
print x
except:
break
sock.close()
Thanks!
If you need two operations to happen at the same time (the client script needs to read input from the user and read new messages from the server), then you'd need to either use threads (one thread for reading user input, and one for reading messages from the server), or futures (since python3.2).
Here's question for playing audio in python: Play audio with Python
As for your client, why are you reconnecting to your server every single time? Anyway, if I understand the problem correctly you're blocking on user input, but also want to handle messages from the server.
Without getting complicated with threads, I would recommended using a recurring signal, which I believe could handle this. There's a function call setitimer(), which will break what you're doing and call a function every so often then return to where you were (user input). In your timer function, check for server messages, print any received, play your sound and return to user input. There's an setitimer() example enter link description here.
Might be a little ugly with the user typing, so you may need to reprint what they're currently typing, but haven't sent out (using something other than raw_input()).
For a slightly more complicated option, which may help you there's a function call select(), which can block while listening for socket input AND user input. Then you just distinguish which is which and keep it all in one loop.
while True:
# select on server socket and user input (blocks for either one)
# if here, either a message has been received or the user typed something
# if a message from server
...
playSound()
# else
# send to server
I have a Blender code which takes sets of data from a csv file and uses them to rotate a robot arm and a human model in the Game Engine. This code works fine, but now I want to send data across a wireless connection to Blender.
I have a server code set up in Blender (which runs on Python 3)
# Server Program
# Make sure the client is being run on the data generation computer
SERVER_LOOP = True
import socket
import sys
import json
import bge
cont = bge.logic.getCurrentController()
owner = cont.owner
print ('INFO: Starting up')
# Create a TCP/IP socket to listen on
print ('INFO: Creating TCP/IP Socket')
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# Prevent from 'ADDRESS ALREADY IN USE' upon restart
print ('INFO: Housekeeping...')
server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
# Bind the socket to port 8081 on all interfaces
server_address = ('localhost', 8081)
print ('INFO: Binding and starting up on %s port %s' % server_address)
server.bind(server_address)
print ('INFO: Server bound')
def send_welcome(cont):
cont.send('SERVER: Welcome'.encode('utf8'))
# Listen for connectons for 5 seconds
server.listen(5)
# Connection is the SOCKET OBJECT for the connection
# Client_address is the connected peer(the client)
connection, client_address = server.accept()
print ('INFO: Connection from', connection.getpeername())
print ('INFO: Sending welcome msg')
send_welcome(connection)
print ()
while SERVER_LOOP:
# Receive data
try:
data = connection.recv(10000)
# Unless there's an error
except OSError:
print (connection)
# Decode the data into usable lists
if type(data) != type(''):
data = data.decode()
# If we want to end the client stream but keep the server running
if data=='end' or data=='End' or data=='END':
print ('INFO: Closing connection with ',connection.getpeername())
connection.shutdown(socket.SHUT_RD | socket.SHUT_WR)
print ()
connection.close()
connection, client_address = server.accept()
print ('INFO: Connection from', connection.getpeername())
print ('INFO: Sending welcome msg')
send_welcome(connection)
print ()
# If we want to stop running the server
elif data=='end server' or data=='End server' or data=='End Server':
print ()
print ('SERVER SHUT DOWN')
SERVER_LOOP = False
# Display when data is loaded back on the client side
else:
# gives feedback in server command line
data = json.loads(data)
owner['test'] = data
print ('CLIENT: %s' % data)
message = 'ping'
connection.send(('SERVER: %s' % message).encode('utf-8'))
print ('SERVER: %s' % message)
And the client code to run with it (this one runs on Python 2.7)
# Client Program
# Make sure the server is being run in Blender
import socket
import time
import json
print 'INFO: Creating Socket'
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
ip_addr = raw_input('IP: ')
port_addr = raw_input('PORT: ')
# Type 'localhost' in the IP field
# Type '8081' in the PORT field
print 'INFO: Connecting to server'
s.settimeout(5) # Times out if 5 seconds without connecting to client
s.connect((ip_addr, int(port_addr)))
# Listen for welcome
data = s.recv(10000)
print data
print ''
while 1:
message = raw_input('CLIENT: ')
if message=='end' or message=='End' or message=='END':
print ''
print 'SHUTTING DOWN CLIENT, SERVER STILL RUNNING'
s.send(message)
break
elif message=='end server' or message=='End server' or message=='End Server':
print ''
print 'SHUTTING DOWN SERVER'
s.send(message)
break
else:
s.send(message)
data = s.recv(10000)
print data
print 'INFO: Closing socket'
s.close()
print 'INFO: Quitting'
Now, obviously this doesn't do the rotations; it's just a test script to make sure that the data transfer between the two works. And it does - in Blender's system console, the data is displayed just as I want it. However, I have a string debug property in Blender titled "test", which is supposed to display the current number just typed in the client, and it's not until I close the whole program down.
For example:
I run the server script in Blender
I run the client script in IDLE
I type in numbers on the client side
They appear in the system console on the server side, but they do NOT appear in the Game Engine
I close the server from the client side
Now, the last number I typed finally appears on the server side
So the problem is that Blender runs my script and then the Game Engine after it's done, but I want them to run concurrently.
Let me know if my explanation doesn't make sense; I can provide downloads to my stuff if need be.
I don't know if this is still a problem - you posted in February and it's now August, but I was just searching for the answer of a similar problem. Your problem is that Blender doesn't update its frames until a script has finished running. Your game is literally stuck on the first frame it plays because it starts a script as soon as that frame hits, and because of the nature of your script, never ends.
Currently, you use server.listen(5) to mean that it listens to five seconds, but the number 5 in that function refers to the backlog instead of the length of time [source]. socket.listen() will stall your game indefinitely (as far as I understand) just like an infinite loop would.
This may not be the answer you were looking for, but it's definitely an answer.