Hi i have an exercise to build with sockets select and msvcrt, server and clients of mltiplie chat(the server and the clients need to be built non-blocking) that every client will send message and the server will send the message to all the clients except the one who sent it, the server:
import socket
import select
IP = "192.168.1.154"
port = 123
default_buffer_size = 1024
open_client_sockets = []
messages_to_send = []
def send_waiting_messages(wlist):
for message in messages_to_send:
(client_sock, data) = message
if client_sock in wlist:
for sock in open_client_sockets:
if sock is not client_sock:
sock.send(data)
messages_to_send.remove(message)
def main():
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.bind((IP, port))
sock.listen(5)
print("The server is on and waiting for client...")
while True:
rlist, wlist, xlist = select.select([sock] + open_client_sockets, open_client_sockets, [])
for current_socket in rlist:
if current_socket is sock:
(new_socket, addr) = sock.accept()
open_client_sockets.append(new_socket)
else:
data = current_socket.recv(default_buffer_size)
if data == "":
open_client_sockets.remove(current_socket)
print("Connection with client closed")
else:
messages_to_send.append((current_socket, 'Hello ' + data))
send_waiting_messages(wlist)
if __name__ == '__main__':
main()
Building the server wasnt hard because it was guided(if it was not guided i would never got this code working) by the book but i have problem building the client and the main reason is that i dont understand how select.select works, couldn't find answer that will simplify enough this module for me.
this is what i did with the client:
import socket
import select
import msvcrt
IP = "192.168.1.154"
port = 123
sockets = []
def write():
pass
def main():
sock = socket.socket()
sock.connect((IP, port))
while True:
rlist, wlist, xlist = select.select(sockets, sockets, [])
for current_socket in rlist:
if current_socket is sock:
data = current_socket.recv(1024)
print(data)
else:
sockets.append(current_socket)
write()
if __name__ == '__main__':
main()
This probably shows you that I have low understanding of the module select and the exercise actually. I saw some threads that has similar question but I understand nothing from them so I realy need good explantion.
In conclusion I realy am lost...
select takes as parameters a list of sockets to wait for readablity, a list of sockets to wait for writability, and a list of sockets to wait for errors. It returns lists of ready to read, ready to write, and error sockets. From help:
>>> help(select.select)
Help on built-in function select in module select:
select(...)
select(rlist, wlist, xlist[, timeout]) -> (rlist, wlist, xlist)
Wait until one or more file descriptors are ready for some kind of I/O.
The first three arguments are sequences of file descriptors to be waited for:
rlist -- wait until ready for reading
wlist -- wait until ready for writing
xlist -- wait for an ``exceptional condition''
If only one kind of condition is required, pass [] for the other lists.
A file descriptor is either a socket or file object, or a small integer
gotten from a fileno() method call on one of those.
The optional 4th argument specifies a timeout in seconds; it may be
a floating point number to specify fractions of seconds. If it is absent
or None, the call will never time out.
The return value is a tuple of three lists corresponding to the first three
arguments; each contains the subset of the corresponding file descriptors
that are ready.
*** IMPORTANT NOTICE ***
On Windows, only sockets are supported; on Unix, all file
descriptors can be used.
So to fix your client, you need to add the socket you opened (sock) to the sockets list. Your write function can then be called if your socket is ready to be written.
In write, use msvcrt.kbhit() to test for characters typed. You can't just use input because it will block. Then read the character if one has been typed. Collect up the characters until you hit enter, then build a message and write it to the socket. Something like:
message = []
def write(sock):
if msvcrt.kbhit():
c = msvcrt.getche()
if c == '\r':
data = ''.join(message)
print 'sending:',data
sock.sendall(data)
message.clear()
else:
message.append(c)
Related
This question already has an answer here:
Python Bidirectional TCP Socket Hanging on socket.recv
(1 answer)
Closed 3 years ago.
I'm trying to make a client-server application in which:
the client sends a message (name of a function) to the server
the server receives the message, calls the corresponding function and returns the results to the client.
I'm able to do this only if the message from the client is one; if I want to call two or more functions, and therefore send two or more messages, I get some problems because I need to add a loop inside the server.
In particular, what is not clear to me is the socket function recv(). Usually, to receive data I write something like this (without setblocking()) and I never had any problems:
while True:
data = sock.recv(BUFFER)
results += data.decode()
if not data:
break
But if I add a while True: to the server (in order to wait for the other functions), then the client never exits from sock.rev(). Why? I expect that function to be always blocking or always non-blocking based on how it was set sock.setblocking().
I already tried with settimeout() and it worked, but performances are really important so I would like to avoid every possible delay.
Anyway, the main question is why recv() is behaving differently; then if you had any suggestions to solve my task it would be really appreciated.
Here's a simple simulation of the real application.
Client:
import socket
BUFFER = 1024
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_address= ('localhost', 14000)
sock.connect(bridge_address)
msg1 = "function 1".encode()
msg2 = "function 2".encode()
results1 = " "
results2 = " "
sock.sendall(msg1)
while True:
data = sock.recv(BUFFER)
results1 += data.decode()
if not data:
print('no more data')
break
sock.sendall(msg2)
while True:
data = sock.recv(BUFFER)
results2 += data.decode()
if not data:
print('no more data')
break
sock.close()
print('Results 1: ',results1)
print('Results 2: ',results2)
Server:
import socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_address = ('localhost', 14000)
sock.bind(server_address)
sock.listen(1)
msg = ""
for i in range(4096):
msg += str(i) + " "
while True:
print('waiting for a connection')
client, client_address = sock.accept()
try:
while True:
data = client.recv(128)
if not data:
print('no more data from')
break
client.sendall(msg.encode())
finally:
print('connection closed.\n')
client.close()
You are explicitly forcing the server side socket to recieve the message from the client, even when the client is not sending anything.
data = client.recv(128)
if not data:
print('no more data from')
break
if case in the code snippet above will only execute when the client socket goes down.
One method to solve this would be to first send the size of the message and then call recv accordingly for the specified number of bytes.
Im sure there are easier ways with particular python modules, but for an assignment I need to create a program that can act as a client/server. As of right now I have it working to the point of only being able to send a message if the reciever has responded. I need it to just send and appear on the respective client/server terminal when enter is pressed. Any help would be greatly appreciated!
These are pictures of what happens as of now
https://i.stack.imgur.com/T9CsJ.png
import sys
import socket
import getopt
def usage(script_name):
print('Usage: py' + script_name + '-l' +' <port number>' + '[<server>]')
def sockObj():
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
return sock
def serversockConn(serversocket,port):
serversocket.setsockopt(socket.SOL_SOCKET,socket.SO_REUSEADDR,1)
# bind the serversocket to address
serversocket.bind(('',int(port)))
# enable a server to accept connections
serversocket.listen(5)
# wait for a connection and accept it
sock,addr = serversocket.accept()
return sock
def connToServersock(sock,server, port):
# connect to a serversocket
if server:
sock.connect((server, int(port)))
else:
sock.connect(('localhost', int(port)))
return sock
if __name__ == '__main__':
## get the command line arguments
try:
options, non_option_args = getopt.getopt(sys.argv[1:],'l')
except getopt.GetoptError as err:
print(err)
sys.exit(2)
#check if '-l' is present in command line agrument
serverSide = False
for opt,arg in options:
if opt == "-l":
serverSide = True
# port number
port = non_option_args[0]
# server address
server = None
hostLen = len(non_option_args)
if hostLen == 2:
server = non_option_args[1]
# create a communicator object, and make a connection between server and client
# server
if serverSide:
serversocket = sockObj()
sock = serversockConn(serversocket,port)
# client
else:
sock = sockObj()
sock = connToServersock(sock,server,port)
while True:
## read a message from standard input
message = sys.stdin.readline().encode()
if len(message) != 0:
sock.send(message)
return_msg = sock.recv( 1024 )
if return_msg:
print("Message recieved: " + return_msg.decode())
else:
print("Other side shut down")
else:
try:
sock.shutdown(socket.SHUT_WR)
sock.close()
except:
pass
I think your issue is that there are two places in your event loop where you block:
message = sys.stdin.readline().encode()
Here, you block until the user has pressed return -- during this time, your program is unable to respond to any data received over the network, because it is blocked waiting for data from stdin.
... and:
return_msg = sock.recv( 1024 )
Here, you are waiting for data to be received from the network -- during this time, your program is unable to respond to any data received from stdin, because it is blocked waiting for data from the network.
The behavior you'd ideally like to have is for your program to wait for both stdin and network traffic simultaneously -- i.e. have it block until either the user has pressed return, or some network data has been received, whichever comes first.
The easiest way to achieve that behavior is to use select(); its purpose is to block until at least one of several file descriptors is ready to be acted on. (Note, however, that Windows does not support using select() on stdin, so if your program needs to run under Windows you will probably have to spawn a second thread instead).
To implement the event loop using select(), add import select to the top of your script, then replace your event loop with something like this instead:
while True:
## block here until either sock or sys.stdin has data ready for us
readable, writable, exceptional = select.select([sock, sys.stdin], [], [])
if sys.stdin in readable:
## read a message from standard input
message = sys.stdin.readline().encode()
if len(message) != 0:
sock.send(message)
if sock in readable:
## read a message from the network
try:
return_msg = sock.recv( 1024 )
if (return_msg):
print("Message received: " + return_msg.decode())
else:
print("Other side shut down")
break
except:
print("recv() threw an exception")
break
The server only listens for a message from the first socket to connect, even though it is set to nonblocking, it doesn't skip over it when it doesn't receive data. I'm new to networking and this is my first project, if anyone know of any others good for beginners please let me know. Thanks! Here is the code.
import socket
CONNECTED_SENDERS = []
CONNECTED_LISTENERS = []
def Main():
HOST = socket.gethostname()
PORT = 4444
SERVER_SOCKET = socket.socket()
SERVER_SOCKET.bind((HOST, PORT))
SERVER_SOCKET.listen(1)
for i in range(2):
CONNECTION, ADDRESS = SERVER_SOCKET.accept()
CONNECTED_LISTENERS.append(CONNECTION)
for i in range(2):
CONNECTION, ADDRESS = SERVER_SOCKET.accept()
CONNECTED_SENDERS.append(CONNECTION)
for DEVICE in CONNECTED_LISTENERS:
DEVICE.send(b'SERVER: You have succesfully connected.')
DEVICE.send(b'SERVER: Please wait for permission to talk.')
x = 0
for DEVICE in CONNECTED_LISTENERS:
DEVICE.send(b'SERVER: What is your name?')
Name = CONNECTED_SENDERS[x].recv(1024)
CONNECTED_LISTENERS[x] = (CONNECTED_LISTENERS[x], Name)
x += 1
del x, Name
for DEVICE, _ in CONNECTED_LISTENERS:
DEVICE.send(b'SERVER: You may now talk.')
SERVER_SOCKET.setblocking(0)
LEAVE = False
while LEAVE == False:
try:
MESSAGE = CONNECTED_SENDERS[0].recv(1024)
NAME = CONNECTED_LISTENERS[0][1]
for DEVICE, _ in CONNECTED_LISTENERS:
DEVICE.send(NAME + b': ' + MESSAGE)
if MESSAGE == 'QUIT':
LEAVE = True
except:
try:
MESSAGE = CONNECTED_SENDERS[1].recv(1024)
NAME = CONNECTED_LISTENERS[1][1]
for DEVICE, _ in CONNECTED_LISTENERS:
DEVICE.send(NAME + b': ' + MESSAGE)
if MESSAGE == 'QUIT':
LEAVE = True
except:
pass
for CONNECTION in CONNECTED_LISTENERS:
CONNECTION.close()
for CONNECTION in CONNECTED_SENDERS:
CONNECTION.close()
if __name__ == "__main__":
Main()
There are a number of issues with your code, some small and some big. But the main problem is that you're marking the server socket nonblocking, not any of the sockets on which communication takes place.
In standard TCP socket programming, you set up a server which listens for incoming connections. When that server accepts a new client, this returns a new socket, and it's on this new socket that all communication with the remote client happens. In other words, the server socket is just for accepting new connections, and nothing else. You never write data through the server socket.
So it doesn't matter that SERVER_SOCKET is marked nonblocking, you must do something like this:
conn, addr = server.accept()
conn.setblocking(False)
conn is the new socket through which you talk to the client, and can be used in a nonblocking fashion.
Smaller issues:
I should also point out that you call SERVER_SOCKET.listen(1). That argument of 1 means that the server will only have a backlog of waiting connections from one client. So if a second client connects before the first connection is made, the second client will receive an error, ECONNREFUSED. Given what it looks like you're trying to do, I'd guess SERVER_SOCKET.listen(4) is appropriate.
Next, nonblocking communication is much harder than blocking protocols. I'd suggest you improve your networking skills before tackling them, but when you're ready, look at the select or selectors modules for help. They provide tools to wait for communication from any of a number of clients, rather than looping over them all and checking if data is available, as you've done here. This looping is very inefficient.
Finally, in Python, it's good practice to name variables with lower case, underscore-separated names. UPPER_CASE_NAMES are usually reserved for constants. So change SERVER_SOCKET to server_socket, CONNECTED_LISTENERS to connected_listeners, etc.
In Python how do I fill a buffer with lines of data (strings) and consume it with a second process? There are ample of examples here adding and reading lines from a string, but I need to remove the consumed line from the string for the string to work as a buffer.
Example: read sporadic data from a serial port and send it via TCP/IP to a server. Line-by-line within one loop and no buffering = no problem, but in case the destination is unreachable the data should be stored in the buffer and then sent once connection is available.
#!/usr/bin/python
import serial
import socket
from multiprocessing import Process
ip = "someURL"
port = 12345
ser = serial.Serial("/dev/ttyUSB0", 57600, timeout=0)
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
def serial_reader():
while True:
for line in ser.read():
try:
response = ser.readlines(None)
response = str(response)
message = response[7:]
except:
print datetime.datetime.now(), " No data from serial connection."
##
def data_sender():
s.connect((ip, port))
while True:
for line in queue():
try:
s.send(message)
except:
try:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((ip, port))
continue
except:
s.close()
##
if __name__ == '__main__':
Process(target=serial_reader).start()
Process(target=data_sender).start()
I think the best way to achieve what you want is to use a queue:
from multiprocessing import Queue
specifically use queue.put() to put a string on the queue, queue.get() to retrieve it, and queue.task_done() to indicate that the task is complete.
https://docs.python.org/2/library/queue.html#Queue.Queue
if you need a bigger gun take a look at RabbitMQ and python libraries that implement the AMPQ protocol such as rabbitpy. This is the defacto standard for inter process/inter service communication and has a lot of usefyl stuff already baked in, such as persisting messages in case the processes shut down, load balancing tasks across multiple processes, etc.
I made a better chat client following help from people:
They told me that if I didn't want to be blocked on .recv when waiting for messages, I would need to use threads, classes, functions, and queues to do so.
So I followed some help a specific person gave me where I created a thread from a class and then defined a function that was supposed to read incoming messages and print them.
I also created a function that allows you to enter stuff to be sent off.
Thing is, when I run the program. Nothing happens.
Can somebody help point out what is wrong? (I've asked questions and researched for 3 days, without getting anywhere, so I did try)
from socket import *
import threading
import json
import select
print("Client Version 3")
HOST = input("Connect to: ")
PORT = int(input("On port: "))
# Create Socket
s = socket(AF_INET,SOCK_STREAM)
s.connect((HOST,PORT))
print("Connected to: ",HOST,)
#-------------------Need 2 threads for handling incoming and outgoing messages--
# 1: Create out_buffer:
Buffer = []
rlist,wlist,xlist = select.select([s],Buffer,[])
class Incoming(threading.Thread):
# made a function a thread
def Incoming_messages():
while True:
for i in rlist:
data = i.recv(1024)
if data:
print(data.decode())
# Now for outgoing data.
def Outgoing():
while True:
user_input=("Your message: ")
if user_input is True:
Buffer += [user_input.encode()]
for i in wlist:
s.sendall(Buffer)
Buffer = []
Thanks for taking a look, thanks also to Tony The Lion for suggesting this
Take a look at this revised version of your code: (in python3.3)
from socket import *
import threading
import json
import select
print("client")
HOST = input("connect to: ")
PORT = int(input("on port: "))
# create the socket
s = socket(AF_INET, SOCK_STREAM)
s.connect((HOST, PORT))
print("connected to:", HOST)
#------------------- need 2 threads for handling incoming and outgoing messages--
# 1: create out_buffer:
out_buffer = []
# for incoming data
def incoming():
rlist,wlist,xlist = select.select([s], out_buffer, [])
while 1:
for i in rlist:
data = i.recv(1024)
if data:
print("\nreceived:", data.decode())
# now for outgoing data
def outgoing():
global out_buffer
while 1:
user_input=input("your message: ")+"\n"
if user_input:
out_buffer += [user_input.encode()]
# for i in wlist:
s.send(out_buffer[0])
out_buffer = []
thread_in = threading.Thread(target=incoming, args=())
thread_out = threading.Thread(target=outgoing, args=())
thread_in.start() # this causes the thread to run
thread_out.start()
thread_in.join() # this waits until the thread has completed
thread_out.join()
in your program you had various problems, namely you need to call the threads; to just define them isn't enough.
you also had forgot the function input() in the line: user_input=input("your message: ")+"\n".
the "select()" function was blocking until you had something to read, so the program didn't arrive to the next sections of the code, so it's better to move it to the reading thread.
the send function in python doesn't accept a list; in python 3.3 it accepts a group of bytes, as returned by the encoded() function, so that part of the code had to be adapted.