Develop Reference

Is there any way to send a message to everyone except the sender?

This question is similar to this one, but that was for JavaScript whereas mine is for Python.
How do I send a message to every connected client from the server except a selected client in Python using the sockets library?
I am making a simple game, where I want to detect the first person to press a button among three clients, and then notify the other two clients that they lost while notifying the winner that they won.
Usually, to send information to a client you do (on a new thread):
connected_client.sendall(data)
To receive, you do:
data = socket.recv()
But from what I searched, I couldn't find a way to send data to every connected client except a certain client.
I thought I could get around this by creating an 'identifying name' for each thread which ran the receiving function, but I couldn't find a good way to do this due to which I decided to search for a better option.
How can I do this?

Inserting them into a list can help. For example...
For the server side:
import socket
import threading
# This is where you store all of your Client IP's
client_list = []
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_ip = "yourip"
server_port = 8888
server.bind((server_ip, server_port))
def check_client(client_ip):
while True:
data = client_ip.recv(1024).decode()
if "condition" in data:
for ip in client_list:
if ip != client_ip:
ip.send("something".encode())
def check_connection():
server.listen()
while True:
client_ip, client_address = server.accept()
client_list.append(client_ip)
threading.Thread(target=check_client, args=(client_ip,), daemon=True).start()
check_connection()
So what happens is you call the check_connection function to check for incoming connections. After it receives one, it appends the connection inside the client_list variable. At the same time, it creates a thread to the current connection, check_client, which checks for any info being sent. If there's an info being sent by one of your clients, it checks if the "condition" string is inside your sent data. If so, it sends "something" string into all of your clients with exception to itself. Take note that when you send data, it must be in bytes.
For the client side:
import socket
import threading
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_ip = "serverip"
server_port = 8888
server.connect((server_ip, server_port))
def receive_info():
while True:
data = server.recv(1024).decode()
if "something" in data:
print("Someone already sent something")
threading.Thread(target=receive_info, daemon=True).start()
while True:
user_input = input("Type 'condition': ")
server.send(user_input.encode())
What this only does is, it sends your input into the server. If you typed "condition" on your input, it will send "something" on the other clients except you. So you need to setup 2 more clients in order to see the results.
Don't forget to set server_ip and server_port's values!

How can I continually send data without shutdown socket connection in python

I wrote a python client to communicate with server side. Each time when I finished sanding out data, I have to call sock.shutdown(socket.SHUT_WR), otherwise the server would not do any response. But after calling sock.shutdown(socket.SHUT_WR), I have to reconnect the connection as sock.connect((HOST, PORT)), other wise I can not send data to server. So how can I keep the connection alive without close it.
My sample code as following:
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((HOST, PORT))
sock.sendall(data)
sock.shutdown(socket.SHUT_WR)
received = sock.recv(1024)
while len(received)>0:
received = sock.recv(1024)
sock.sendall(newdata) # this would throw exception
The Server Side code as following:
def handle(self):
cur_thread = threading.current_thread()
while True:
self.data = self.rfile.read(bufsiz=100)
if not self.data:
print 'receive none!'
break
try:
data = self.data
print 'Received data, length: %d' % len(data)
self.wfile.write('get received data\n')
except Exception:
print 'exception!!'

You didn't show any server side code but I suspect it simply reads bytes until it gets none anymore.
You can't do this as you found out, because then the only way to tell the server the message is complete is by killing the connection.
Instead you'll have to add some form of framing in your protocol. Possible approaches include a designated stop character that the server recognises (such as a single newline character, or perhaps a 0-byte), sending frames in fixed sizes that your client and server agree upon, or send the frame size first as a network encoded integer followed by exactly the specified number of bytes. The server then first reads the integer and then exactly that same number of bytes from the socket.
That way you can leave the connection open and send multiple messages.

Python sockets; sending from client receiving on server

I am trying to send messages on TCP/IP all on host machine. This is working, although for some reason the socket needs to be re-instantiated for every new message on the client side only. For example here is a basic client that sends three separate messages:
import socket
host = '127.0.0.1'
class Client:
def __init__(self):
self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
def connect(self):
self.sock.connect((host,12347))
def send(self,message):
self.sock.sendall(message)
def close(self):
self.sock.close()
if __name__ == "__main__":
message1 = "I am message 1"
message2 = "I am message 2"
message3 = "I am message 3"
#exp = Client()
#exp.connect()
for i in range(0,3):
try:
exp = Client()
exp.connect()
if i == 0:
txt = message1
elif i == 1:
txt = message2
elif i == 2:
txt = message3
exp.send(txt)
exp.close()
print i
exp.send(txt)
except:
pass
and the server that receives:
#!/usr/bin/env python
import socket
class communication:
def __init__(self):
try:
host = '127.0.0.1'
self.Server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.Server.bind((host,12347))
self.Server.listen(1)
finally:
print "setup finished"
def recieve(self):
(connection, client_address) = self.Server.accept()
data = connection.recv(128)
return data
def close(self):
self.server.close()
if __name__ == "__main__":
exp = communication()
while True:
try:
(connection,client_address) = exp.Server.accept()
message = connection.recv(128)
finally:
print message
if message == "I am message 3":
exp.close()
You see how I re-call the Client class in each iteration of the for loop. This seems to be necessary for sending messages 2 and 3. If the socket is instantiated only once at the start of the main code along with the connect() function, then the server hangs on the recv() after the first message has been sent.
I can't understand why this is happening and the socket only needs to be setup once on the server side. I am doing something wrong, or is this normal?
Thanks!

It's even worse than you think. Take a look at your server code. exp.Server.accept() accepts a connection from the client, but connection.receive() ignores that connection completely and does a second self.Server.accept(). You ignore half of your connections!
Next, your server only does a single receive.... Even if you tried to send more messages on the connection, the server would ignore them.
But you can't just add a recv loop. Your client and server need some way to mark message boundaries so the server knows how to pull them out. Some text based systems use a new line. Others send a message size or fixed size header that the server can read. HTTP for example uses a combination of new lines and data count.
If you want to learn sockets from the ground up just know that they are complicated and you'll need to study. There are lots of ways to build a server and you'll need to understand the trade-offs. Otherwise, there are many frameworks from XMLRPC to zeromq that do some of the heavy lifting for you.

Pickle EOFError: Ran out of input when recv from a socket

I am running a very simple python (3.x) client-server program (both locally on my PC) for a school project (not intended for the real world) which just sends messages back-and-forth (like view customers, add customer, delete customer, etc... real basic).
Sometimes the data can be multiple records which I had stored as namedTuples (just made sense) and then went down the path of using Pickle to transfer then.
So for example on the client I do something like this:
s.send(message.encode('utf-8'))
pickledResponse = s.recv(4096);
response = pickle.loads(pickledResponse)
Now ever so often I get the following error:
response = pickle.loads(pickledResponse)
EOFError: Ran out of input
My fear is that this has something to do with my socket (TCP) transfer and maybe somehow I am not getting all the data in time for my pickle.loads - make sense? If not I am really lost as to why this would be happening so inconsistently.
However, even if I am right I am not sure how to fix it (quickly), I was considering dropping pickle and just using strings (but couldn't this suffer from the same fate)? Does anyone have any suggestions?
Really my message are pretty basic - usually just a command and some small data like "1=John" which means command (1) which is FIND command and then "John" and it returns the record (name, age, etc...) of John (as a namedTuple - but honestly this isn't mandatory).
Any suggestions or help would be much appreciated, looking for a quick fix...

The problem with your code is that recv(4096), when used on a TCP socket, might return different amount of data from what you might have expected, as they are sliced at packet boundaries.
The easy solution is to prefix each message with length; for sending like
import struct
packet = pickle.dumps(foo)
length = struct.pack('!I', len(packet)
packet = length + packet
then for receiving
import struct
buf = b''
while len(buf) < 4:
buf += socket.recv(4 - len(buf))
length = struct.unpack('!I', buf)[0]
# now recv until at least length bytes are received,
# then slice length first bytes and decode.
However, Python standard library already has a support for message oriented pickling socket, namely multiprocessing.Connection, that supports sending and receiving pickles with ease using the Connection.send and Connection.recv respectively.
Thus you can code your server as
from multiprocessing.connection import Listener
PORT = 1234
server_sock = Listener(('localhost', PORT))
conn = server_sock.accept()
unpickled_data = conn.recv()
and client as
from multiprocessing.connection import Client
client = Client(('localhost', 1234))
client.send(['hello', 'world'])

For receiving everything the server sends until it closes its side of the connection try this:
import json
import socket
from functools import partial
def main():
message = 'Test'
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
sock.connect(('127.0.0.1', 9999))
sock.sendall(message.encode('utf-8'))
sock.shutdown(socket.SHUT_WR)
json_response = b''.join(iter(partial(sock.recv, 4096), b''))
response = json.loads(json_response.decode('utf-8'))
print(response)
if __name__ == '__main__':
main()
I've used sendall() because send() has the same ”problem” as recv(): It's not guaranteed everything is sent. send() returns the number of bytes actually sent, and the programmer has to make sure that matches the length of the argument and if not to send the rest until everything is out. After sending the writing side of the connection is closed (shutdown()) so the server knows there is no more data coming from the client. After that, all data from the server is received until the server closes its side of the connection, resulting in the empty bytes object returned from the recv() call.
Here is a suitable socketserver.TCPServer for the client:
import json
from socketserver import StreamRequestHandler, TCPServer
class Handler(StreamRequestHandler):
def handle(self):
print('Handle request...')
message = self.rfile.read().decode('utf-8')
print('Received message:', message)
self.wfile.write(
json.dumps(
{'name': 'John', 'age': 42, 'message': message}
).encode('utf-8')
)
print('Finished request.')
def main():
address = ('127.0.0.1', 9999)
try:
print('Start server at', address, '...')
server = TCPServer(address, Handler)
server.serve_forever()
except KeyboardInterrupt:
print('Stopping server...')
if __name__ == '__main__':
main()
It reads the complete data from the client and puts it into a JSON encoded response with some other, fixed items. Instead of the low level socket operations it makes use of the more convenient file like objects the TCPServer offers for reading and writing from/to the connection. The connection is closed by the TCPServer after the handle() method finished.

Client socket not receiving data correctly

I've tried looking about for an answer but I can't seem to find one that answers my specific problem.
Perhaps I don't know how to articulate the problem correctly.
I think I've pinpointed what it is, but the thing is I just don't know how to fix it.
EDIT: I was trying to use two clients on one TCP Socket. Can't do that. I'll have to think of another way. Solved, I guess.
So what I've got is are
1: Two Clients
2: One Server
The objective is this:
Have the server distribute new usernames to all the clients as they connect.
This is what happens when I run the program:
Server: Define Host, and Port, initialize it. Check
Client 1: Connects to the server. Check
Client 1: Once connected, sends a string to the server. Check
Server: Receives a string, checks if the string is in a list is created. If it is: Pass, if it's not, send to everyone the new string. Check
Client 1: [Now waiting to receive data] Recieves data, checks if the string received matches the one it sent. If it does, print("It's one of ours!"), else, make the new string = to Client 2 Username. Check
Client 2: Connects to server: Check
Server: [If it receives a string, prints it.] (Works) Checks if the new string is in the list. [It isn't] So It sends the new username to everyone, and then prints ("Sent to everyone") Check
But, when client 2 receives the string, it prints it. However, client 1 never recives the string.
And when running client one in IDLE, I noticed something went wrong as Client 1 tried to receive the data. (The while loop that the data = s.recv began looping real fast, instead of waiting)
I've asked around in chat, but it seems nobody's around right now. I've tried looking this up but I really can't find an answer. What I suspect is happening is that when my server sends to 'connection' the second time, it somehow overrides the original client connection.
Here's my server code:
from socket import *
import threading
import os
import csv
Username_List = []
host = input("Host: ")
port = input("Port: ")
ss = socket(AF_INET,SOCK_STREAM)
ss.bind((host,int(port)))
ss.listen(2)
while True:
try:
connection,address = ss.accept()
data = connection.recv(1024)
if data:
translated_data = data.decode()
print(translated_data)
if translated_data in Username_List:
pass
else:
Username_List.append(translated_data)
connection.sendall(translated_data.encode())
print("Sent new username to everyone")
except IOError:
connection.close()
print("An exception with a connected user occured")
break
And here is my client code: [The only difference between client 1 and 2 is I changed the username variable]
# Sample Username Client Service Handler.
from socket import *
import threading
import os
import csv
Username = ("Owatch")
host = input("Host: ")
port = input("Port: ")
try:
ss = socket(AF_INET,SOCK_STREAM)
ss.connect((host,int(port)))
except IOError:
print("Aw no man")
ss.send(Username.encode())
while True:
try:
print("Waiting to Recieve Data")
data = ss.recv(1024)
if data:
translated_data = data.decode()
print(translated_data)
if translated_data == Username:
print("It's one of ours!")
else:
Client_Username = translated_data
print (Client_Username)
except Exception as e:
print (vars(e))
If you could please help I'd be grateful.
If you know of an answer to my question that's already been asked, please tell me and I'll remove this post to avoid breaking rules. Thanks!

Right then I started with what you had then changed it till it worked what I've done is created a client class which starts a thread with each connection and adds it to a list of threads (please if I'm doing something horribly wrong smarter people correct me), the thread runs gets some data checks if that's in the list of user names if its not sends out a message to all the clients in the thread list with that name then the thread just chills out. Anyway on to the code.
SERVER!!!
import csv
class client(threading.Thread):
Username_List = []
def __init__(self, conn):
super(client, self).__init__()
self.conn = conn
def run(self):
print "Client thread started"
data = self.conn.recv(1024)
print "Received: {0}".format(data)
if data in client.Username_List:
self.send_msg("Welcome Back!")
else:
for cnt in threadz:
cnt.send_msg(data)
print("Sent new username to everyone")
client.Username_List.append(data)
while True:
# dont need nothing now
pass
def send_msg(self,msg):
self.conn.send(msg)
host = input("Host: ")
port = input("Port: ")
ss = socket() #AF_INET,SOCK_STREAM)
ss.bind((host,int(port)))
print "Server Opening on port: {0}".format(port)
ss.listen(2)
threadz = []
print "Begining Wait for connections"
while True:
try:
connection, address = ss.accept()
print "Got ONE!"
c = client(connection)
print "Recevied connection from:{0} On port:{1}".format(address[0],address[1])
c.start()
threadz.append(c)
print "Client appended to threadz, currently {0} threadz active".format(len(threadz))
except IOError,KeyboardInterrupt:
connection.close()
print("An exception with a connected user occured")
break
The CLIENT:
# Sample Username Client Service Handler.
from socket import *
import threading
import os
import csv
Username = ("ShyGuy")
host = input("Host: ")
port = input("Port: ")
try:
ss = socket() #AF_INET,SOCK_STREAM)
ss.connect((host,int(port))) #I was using ("localhost",1234) for testing
ss.send(Username)
except IOError:
print("Aw no man")
print("Waiting to Recieve Data")
while True:
try:
data = ss.recv(1024)
if data:
translated_data = data.decode()
print(translated_data)
if translated_data == Username:
print"Name: {0} has been registered on server!".format(translated_data)
else:
Client_Username = translated_data
print "New client name received: {0}".format(Client_Username)
except Exception as e:
print (vars(e))
That works on python 2.7 with two clients locally. Needs to use a semaphore to stop the threads printing at the same time as the main server loop prints: http://en.wikipedia.org/wiki/Semaphore_(programming)
This code does nothing graceful with client disconnects, but once you can work with the exceptions that a raised when that happens I'm sure you'll learn some more.