So I'm learning about socket programming and have wrote a nifty little chat server. The problem I am having is that my client cannot read and write at the same time. I'm not too sure how to set this up.
This is what I have so far, I want read() and write() to be running concurrently (It isn't so much about reading and writing at the same time - it's about being able to receive messages while input() hangs waiting for user input.):
import socket
import threading
class Client(threading.Thread):
def __init__(self):
self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.socket.connect(('127.0.0.1', 1234))
print('Client connected to server')
self.readThread = threading.Thread.__init__(self)
self.writeThread = threading.Thread.__init__(self)
def read(self):
data = self.socket.recv(1024)
if data:
print('Received:', data)
def write(self):
message = input()
self.socket.send(bytes(message, 'utf-8'))
client = Client()
while True:
#do both
You're really close. Try something like this:
import socket
import threading
class Client(threading.Thread):
def __init__(self):
self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.socket.connect(('127.0.0.1', 1234))
print('Client connected to server')
t = threading.Thread(target = self.read)
t.daemon = True # helpful if you want it to die automatically
t.start()
t2 = threading.thread(target = self.write)
t2.daemon = True
t2.start()
def read(self):
while True:
data = self.socket.recv(1024)
if data:
print('Received:', data)
def write(self):
while True:
message = input()
self.socket.send(bytes(message, 'utf-8'))
client = Client()
It's worth pointing out that if you're reading and writing from a single terminal this way your prompt could get a little out of hand. I imagine though that you're starting with print statements, but will eventually collect data into other containers in your app.
Related
I need to test a device update function. The function opens a socket on a host and sends a block of text.
The update can take up to 120 seconds. It returns a code for success/failure. To allow continued functioning of the program the update is launched in a thread.
I cannot control the response of the device. The simulation needs to be able to hold an open connection for at least 120 seconds.
It does not need to be safe or scalable since it will only be used for an integration test. The simplest solution is preferred. Pure python is best, but a docker is also acceptable.
I wrote this up based on rdas's pointer.
import json
import logging
import socket
import socketserver
import threading
import time
log = logging.getLogger(__name__)
log.setLevel(logging.INFO)
class LongRequestHandler(socketserver.BaseRequestHandler):
def handle(self):
# Echo the back to the client
data = json.loads(self.request.recv(1024).decode())
t = 0
while t < data['delay']:
time.sleep(1)
print(".", end='')
t += 1
if t % 80 == 0:
print("\n")
print("\n")
self.request.send(b"ok")
class Server():
def __init__(self, host='localhost', port=0):
self.host = host
self.port = port
self.ip = None
self.server = None
def run(self):
address = (self.host, self.port) # let the kernel assign port if port=0
self.server = socketserver.TCPServer(address, LongRequestHandler)
self.ip, self.port = self.server.server_address # what port was assigned?
t = threading.Thread(target=self.server.serve_forever)
t.setDaemon(True) # don't hang on exit
t.start()
return True
def send_request(self, data: dict ):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((self.ip, self.port))
message = json.dumps(data).encode()
s.send(message)
response = s.recv(1024)
s.close()
return response
def __exit__(self):
self.server.shutdown()
self.server.socket.close()
if __name__ == '__main__':
# For simple testing and config example...
server = Server()
server.run()
# Send the data
d = dict(delay=5) # set delay here to desired
out = server.send_request(d)
print('Received: {!r}'.format(out))
I want to implement a streaming server which sends and endless stream of data to all connected clients. Multiple clients should be able to connect and disconnect from the server in order to process the data in different ways.
Each client is served by a dedicated ClientThread, which sub-classes Thread and contains a queue of the data to be sent to the client (necessary, since clients might process data at different speeds and because bursts of data can occur which the clients might be unable to handle).
The program listens to incoming client connections via a seperate ClientHandlerThread. Whenever a client connects, the ClientHandlerThread spawns a ClientThread and adds it to a list.
As a dummy example, the main Thread increments an integer each second and pushes it to all ClientThread queues through ClientHandlerThread.push_item().
Every 10 increments, the number of items in the client queues is printed.
Now to my questions:
When a client disconnects, the Thread terminates and no more data is send, however, the ClientThread object remains in the ClientHandlerThreads list of clients and items are continuously pushed to its queue.
I'm therefore looking for either (1) a way to delete the ClientThread object from the list whenever a client disconnects, (2) a better way to monitor the ClientThreads than a list or (3) a different (better) architecture to archive my goal.
Many thanks!
Server
import socket
import time
from threading import Thread
from queue import Queue
class ClientThread(Thread):
def __init__(self, conn, client_addr):
Thread.__init__(self)
self.queue = Queue()
self.conn = conn
self.client_addr = client_addr
def run(self):
print('Client connected')
while True:
try:
self.conn.sendall(self.queue.get().encode('utf-8'))
time.sleep(1)
except BrokenPipeError:
print('Client disconnected')
break
class ClientHandlerThread(Thread):
def __init__(self):
Thread.__init__(self, daemon = True)
self.clients = list()
def push_item(self, item):
for client in self.clients:
client.queue.put(str(i))
def run(self):
with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as s:
s.bind('./socket')
s.listen()
i = 1
while True:
conn, client_addr = s.accept()
client = ClientThread(conn, client_addr)
client.start()
self.clients.append(client)
i += 1
if __name__ == '__main__':
client_handler = ClientHandlerThread()
client_handler.start()
i = 1
while True:
client_handler.push_item(str(i))
if i % 10 == 0:
print('[' + ', '.join([str(client.queue.qsize()) for client in client_handler.clients]) + ']')
i += 1
time.sleep(1)
Client:
import socket
if __name__ == '__main__':
with socket.socket(socket.AF_UNIX, socket.SOCK_STREAM) as s:
s.connect('./socket')
print('Connected to server')
while True:
data = s.recv(1024)
if not data:
print('Disconnected from server')
break
print(data.decode('utf-8'))
Note You should probably read up on things like aiohttp for much more scalable versions to your answer.
For your question, you can make a few changes to achieve this:
First, change ClientThread's constructor:
class ClientThread(Thread):
def __init__(self, client_handler, conn, client_addr):
self.client_handler = client_handler
self.running = True
...
When the handler creates the object, it should pass self to it as client_handler.
In the run method, use
def run(self):
while True:
...
self.running = False
self.client_handler.purge()
That is, it marks itself as not running, and calls the purge method of handler. This can be written as
class ClientHandlerThread(Thread):
...
def purge(self):
self.clients = [c for c in self.clients if c.running]
I am working on UDP chat which should be listening and being able to send message any time using only one socket. Example, I will have the chat program done, I will open it first time, then second time and I must be able to communicate over UDP from both programs, simply each program has only one opened socket.
My two threads are for listening, which is deamon thread, because I want it to listen to new messages nonstop, and my other is sending the messages, which is just like a normal thread.
First of all, my problem is that it looks like my threads are blocking each other, because if I run the program, I only get output from the first thread I start.
Second problem is that I am not sure if my sending function or the entire class is written properly, or if there is something missing or incorrect.
Thanks in advance. Btw, I am new into python and I am using python 3, just to make it clear.
import socket
import threading
import logging
import time
from sys import byteorder
class Sending():
def __init__(self, name, tHost, tPort):
self.name = name
self.host = tHost
self.port = tPort
def set_name(self, name):
self.name = name
def send(self, name, tHost, tPort, msgType, dgramSize):
logging.debug('Starting send run')
message = input('Enter message: ')
data = bytearray()
data.extend( (name.encode('utf-8'), message.encode('utf-8'), msgType.to_bytes(1, byteorder = 'little')) )
#data.extend(message.encode(encoding='utf_8'))
self.sock.sendto(bytearray(data), (tHost, tPort))
def run(self):
th2 = threading.Thread(name = 'send', target=self.send('username', 'localhost', 8001, 1, 1400))
th2.start()
class Receiving():
def __init__(self, host, port):
self.host = host
self.port = port
def create_socket(self, host, port):
logging.debug('Starting socket')
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind((host, port))
#print ('socket ready')
time.sleep(5)
while True:
data, addr = sock.recvfrom(1500)
print('Prijata:' + data + addr)
def run(self):
th1 = threading.Thread(name = 'rec', target=self.create_socket('localhost', 8000))
th1.setDaemon(True)
th1.start()
if __name__ == '__main__':
#print ('running')
rec = Receiving('localhost', 8000)
send = Sending('username', 'localhost', 8001)
send.run()
rec.run()
Congrats on your introduction to Python! It looks like you're using Python 3, and in future questions it's helpful if you are explicit about which version you're using because there are minor but program-breaking incompatibilities in some code (including this code!).
I found a few errors in your program:
The most major issue - as Trevor Barnwell says, you're not calling threading.Thread quite correctly. The target= argument needs to be a callable object (i.e. function), but in this case it should just be a reference to the function. If you add brackets to the function, self.create_socket(host, port) as you have above, it actually runs the function immediately. As Trevor explained, your Sending.send() method was called early, but additionally there was a similar bug in Receiving. Because Receiving.create_socket() creates an infinite loop, it never returns program execution. While the console output looks correct to the user, the actual program execution has never made it to running the listener in a separate thread.
bytearray.extend() takes an iterable of ints, what you're passing right now is a tuple of byte objects.
In Sending.send() you call self.sock, but you never assign self.sock a value, so it fails.
Sending.run() only runs Sending.send() one time. After completing input for the user, it immediately exits, because the program has finished.
If you're looking for an in-depth, project based introduction to Python appropriate for an experienced programmer (including an exercise very similar to this question on basic sockets, and another on threading), I highly recommend you check out Wesley Chun's "Core Python Applications Programming". The most recent edition (3rd) has a lot of Python 2 code, but it's easily portable to Python 3 with some minor work on the reader's part.
I tried to modify your code as little as possible to get it working, here it is:
import socket
import threading
import logging
import time
class Sending():
def __init__(self, name, tHost, tPort, target):
self.name = name
self.host = tHost
self.port = tPort
self.target_port = target
self.sock = self.create_socket()
def create_socket(self):
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind((self.host, self.port))
return sock
def set_name(self, name):
self.name = name
def send_loop(self):
while True:
logging.debug('Starting send run')
message = input('Enter message: ')
data = bytearray()
data.extend(message.encode('utf-8'))
self.sock.sendto(bytearray(data), (self.host, self.target_port))
def run(self):
th2 = threading.Thread(name='send', target=self.send_loop)
th2.start()
class Receiving():
def __init__(self, host, port):
self.host = host
self.port = port
def create_socket(self):
logging.debug('Starting socket')
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind((self.host, self.port))
print ('socket ready')
time.sleep(5)
while True:
data, addr = sock.recvfrom(1500)
print('\nPrijata:' + data.decode('utf-8') + str(addr))
def run(self):
th1 = threading.Thread(name='rec', target=self.create_socket)
print("Made it here")
th1.daemon = True
th1.start()
return
if __name__ == '__main__':
print('running')
rec = Receiving('localhost', 8000)
send = Sending('username', 'localhost', 8001, 8000)
rec.run()
send.run()
The threads are not blocking each other. send is called before a thread is even created.
th2 = threading.Thread(name = 'send', target=self.send('username', 'localhost', 8001, 1, 1400))
This line makes a call to send at:
self.send('username', 'localhost', 8001, 1, 1400)
I think you meant to do this:
th2 = threading.Thread(
target=self.send
args=('username', 'localhost', 8001, 1, 1400))
That way a thread will start that calls send on the next line.
Two other things:
You will want to loop in your functions because the thread terminates once the function does.
I think you mean raw_input instead of input
I've started Python a few times ago and now, I'm currently creating a socket server. I already have the server functioning with multiple threads with multiple clients (Hurray !) But I'm looking for functionality I can't call (i don't even know if it exists) I would like to create a kind of channel where client can send different type of message.
An example I create a channel INFO and if the server received this type of socket it just does a print
I create another channel DEBUG where I can send custom command which the server will execute
etc
In a non-programming language it will do this:
def socketDebug(command):
run command
def socketInfo(input):
print input
if socket == socketDebug:
socketDebug(socket.rcv)
else:
if socket == socketInfo:
socketInfo(socket.rcv)
I hope I'm clear.
Here is a quite simple implementation of a Channel class. It creates a socket, to accept
connections from clients and to send messages. It is also a client itself,
receiving messages from other Channel instances (in separate processes for example).
The communication is done in two threads, which is pretty bad (I would use async io). when
a message is received, it calls the registered function in the receiving thread which
can cause some threading issues.
Each Channel instance creates its own sockets, but it would be much more scalable to
have channel "topics" multiplexed by a single instance.
Some existing libraries provide a "channel" functionality, like nanomsg.
The code here is for educational purposes, if it can help...
import socket
import threading
class ChannelThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.clients = []
self.chan_sock = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
self.chan_sock.bind(('',0))
_, self.port = self.chan_sock.getsockname()
self.chan_sock.listen(5)
self.daemon=True
self.start()
def run(self):
while True:
new_client = self.chan_sock.accept()
if not new_client:
break
self.clients.append(new_client)
def sendall(self, msg):
for client in self.clients:
client[0].sendall(msg)
class Channel(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.daemon = True
self.channel_thread = ChannelThread()
def public_address(self):
return "tcp://%s:%d" % (socket.gethostname(), self.channel_thread.port)
def register(self, channel_address, update_callback):
host, s_port = channel_address.split("//")[-1].split(":")
port = int(s_port)
self.peer_chan_sock = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
self.peer_chan_sock.connect((host, port))
self._callback = update_callback
self.start()
def deal_with_message(self, msg):
self._callback(msg)
def run(self):
data = ""
while True:
new_data = self.peer_chan_sock.recv(1024)
if not new_data:
# connection reset by peer
break
data += new_data
msgs = data.split("\n\n")
if msgs[-1]:
data = msgs.pop()
for msg in msgs:
self.deal_with_message(msg)
def send_value(self, channel_value):
self.channel_thread.sendall("%s\n\n" % channel_value)
Usage:
In process A:
c = Channel()
c.public_address()
In process B:
def msg_received(msg):
print "received:", msg
c = Channel()
c.register("public_address_string_returned_in_process_A", msg_received)
In process A:
c.send_value("HELLO")
In process B:
received: HELLO
I have been trying to create a two player game in pygame. I did some research on sockets and have been trying to put them into the game. This is my setup sockets function in the server, I won't put all the game code in since its quite long:
def create_sockets(self):
self.ip = "192.168.1.68"
self.port = 8888
self.server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.server_socket.bind((self.ip, self.port))
self.server_socket.listen(5)
self.conn, self.addr = self.server_socket.accept()
But through trial and error, when server_socket.accept() is run, I get a black screen and a color wheel (I'm on a mac). Why is this happening? the same code works fine in my server test from before. Since I am very new to sockets please correct me on any mistakes/bad practice
Thanks in advance
+1 to svk. Whenever I use Socket in pygame, I use threading to make it asynchronous. I'm pretty sure both .listen() and .accept() will freeze your program in a loop as those methods are waiting for something to happen.
Here is the full code for a pong clone in pygame. This is a "dumb" server, meaning it is not handling game logic, just sharing data with clients.
This may not be your preferred approach as a whole, but it does show how to handle asynchronous connection/listening. As well as using pickle to encode/decode whatever data type you want. That way you can kick lists around and stuff.
# Server example:
from threading import Thread
import socket, pickle, logging
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(("0.0.0.0", 12354))
logging.basicConfig(format='%(asctime)s:%(levelname)s:%(lineno)s %(message)s', level=logging.DEBUG)
client_list = []
max_clients = 2
global started
started = 0
class Client():
def __init__(self, conn = ''):
self.conn = conn
# add to global clients list
client_list.append(self)
self.client_thread = Thread(target = self.process_messages)
self.client_thread.start()
def process_messages(self):
while True:
try:
data = self.conn.recv(1024)
# send to all in client_list except self
data = pickle.loads(data)
data.append(started)
logging.info("Sending Data: {0}".format(data))
data = pickle.dumps(data)
for client in client_list:
if client != self:
client.conn.sendall(data)
data = ""
except ConnectionResetError:
logging.debug("Client Disconnected")
break
def connection_manager():
while len(client_list) < max_clients:
logging.info('Listening for connections...')
s.listen(1)
conn, addr = s.accept()
logging.info("Client connected: {0}".format(addr))
x = Client(conn)
logging.debug(client_list)
logging.warning("Max clients reached")
logging.info("No longer listening..")
started = 0
accept_connections_thread = Thread(target = connection_manager)
accept_connections_thread.start()