How to implement multi-threading into a Python WSGI server - python

Just to clarify up front that this is just a learning project and I have no intention of using this in production. There are several very good Python application servers out there already. But I am trying to learn more about concurrency, so I set out to write one of the things (I thought) I knew.
Also, because I wanted to "closer to the metal" so I started out with just Socket and want to keep it that way.
Below is the important parts of what I have so far. self.iq is a Queue object (inbound_queue) which then does nothing really, but puts the request (which includes the socket object) into the outbound_queue and then a Consumer object takes the request from the outbound_queue and passes it to the ResponseHandler. This seems to work fine with just me hitting it but I am concerned that I am opening myself up to a race condition with a naive implementation. Specifically assigning things to the ServerClass object that are request specific.
So the question is: Is there a better way to do this, or does my Queue usage prevent two threads from picking up and operating on the same object? Should I be encapsulating things like the WSGI environment stuff into a separate object that can be also passed into the queue? Doing stuff like that gets tricky in trying to write a server that is WSGI compliant because of the need to pass in callback functions.
class Consumer(threading.Thread):
def __init__(self, out_queue, server):
threading.Thread.__init__(self)
self.out_queue = out_queue
self.server = server
def run(self):
while True:
item = self.out_queue.get()
self.server.ResponseHandler(self.server, item)
self.out_queue.task_done()
class QueueConsumerServer(object):
methods_allowed = ['get', 'post', 'put', 'patch', 'delete', 'options', 'upgrade']
def __init__(self, host, port, application):
self.host = host
self.port = port
self.application = application
self.iq = Queue.Queue()
self.oq = Queue.Queue()
self.socket = socket.socket()
self.socket.bind((self.host, self.port))
#<snip of lots of assigning stuff to environ>
self.environ = environ
headers_set = []
headers_sent = []
for i in xrange(3):
thr = Producer(self.iq, self.oq)
thr.daemon = True
thr.start()
for i in xrange(3):
thr = Consumer(self.oq, self)
thr.daemon = True
thr.start()
def handle_request(self):
self.socket.listen(1)
try:
while True:
cli, addr = self.socket.accept()
data = cli.recv(1024)
request_data = self.parse_request_data(data)
req.path = request_data[1]
req.cli = cli
self.iq.put(req)
return
except Exception, ex:
print 'e', ex,
sys.exit(1)
finally:
sys.stdout.flush()
self.socket.close()

Python queues are thread safe, so there is no race in your code as written.
Regarding a better way to approach this, your code will probably hit the GIL before too long. I would suggest looking to into multiprocessing.

Related

Audio chunk distribution issue in Python based multi-client voice chat with 2 simultaneous speakers

I'm working on a Python based multi-client voice chat application (learning project). While voice transfer between 2 clients works like a charm and the server is also able to handle multiple connections, problems start as soon as 2 persons talk simultaneously. Whenever a client sends audio data (e.g. "AAAA"), while another client is sending "BBBB" the third clients receives something like "ABABABAB".
I've already tried to spawn multiple threads for each user and let a Mutex take control over the clients[] array (the latter one was either implemented wrong or a stupid idea since it doesn't change anything)
I've also included RMS (only send data while somebody is speaking) in order to stop the constant data stream from every connected client, which took a bit of heat from the server and helped a little bit without solving the real problem.
from socket import *
from threading import Thread, Lock
class entry_thread (Thread):
def __init__(self, host, port):
Thread.__init__(self)
adress_voice = (host, port)
self.voice_socket = socket(AF_INET, SOCK_STREAM)
self.voice_socket.bind(adress_voice)
self.voice_socket.listen(10)
def run(self):
while True:
sock_v, data_v = self.voice_socket.accept()
thread = handle_client_thread(sock_v, data_v)
thread.start()
class handle_client_thread (Thread):
def __init__(self, vsock, vdata):
global clients
Thread.__init__(self)
self.chunk = 1024
self.vsock = vsock
self.vdata = vdata
self.name = self.vsock.recv(1024).decode()
clients[self.name] = [self.vdata, self.vsock]
def run(self):
global clients
while True:
try:
audio_data = self.vsock.recv(self.chunk)
for x, client in clients.items():
if thread_safe:
mutex.acquire()
try:
if (client[0][1] != self.vdata[1]): #if receipient
# != sender
client[1].send(audio_data)
finally:
if thread_safe:
mutex.release()
except:
break
mutex = Lock()
thread_safe = False
clients = {}
server = entry_thread('', 20003)
server.start()
server.join()
Solved the issue by putting the whole for-loop into the mutex, while giving each user an individual audio channel within the client.

Using one socket in UDP chat using threading

I am working on UDP chat which should be listening and being able to send message any time using only one socket. Example, I will have the chat program done, I will open it first time, then second time and I must be able to communicate over UDP from both programs, simply each program has only one opened socket.
My two threads are for listening, which is deamon thread, because I want it to listen to new messages nonstop, and my other is sending the messages, which is just like a normal thread.
First of all, my problem is that it looks like my threads are blocking each other, because if I run the program, I only get output from the first thread I start.
Second problem is that I am not sure if my sending function or the entire class is written properly, or if there is something missing or incorrect.
Thanks in advance. Btw, I am new into python and I am using python 3, just to make it clear.
import socket
import threading
import logging
import time
from sys import byteorder
class Sending():
def __init__(self, name, tHost, tPort):
self.name = name
self.host = tHost
self.port = tPort
def set_name(self, name):
self.name = name
def send(self, name, tHost, tPort, msgType, dgramSize):
logging.debug('Starting send run')
message = input('Enter message: ')
data = bytearray()
data.extend( (name.encode('utf-8'), message.encode('utf-8'), msgType.to_bytes(1, byteorder = 'little')) )
#data.extend(message.encode(encoding='utf_8'))
self.sock.sendto(bytearray(data), (tHost, tPort))
def run(self):
th2 = threading.Thread(name = 'send', target=self.send('username', 'localhost', 8001, 1, 1400))
th2.start()
class Receiving():
def __init__(self, host, port):
self.host = host
self.port = port
def create_socket(self, host, port):
logging.debug('Starting socket')
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind((host, port))
#print ('socket ready')
time.sleep(5)
while True:
data, addr = sock.recvfrom(1500)
print('Prijata:' + data + addr)
def run(self):
th1 = threading.Thread(name = 'rec', target=self.create_socket('localhost', 8000))
th1.setDaemon(True)
th1.start()
if __name__ == '__main__':
#print ('running')
rec = Receiving('localhost', 8000)
send = Sending('username', 'localhost', 8001)
send.run()
rec.run()
Congrats on your introduction to Python! It looks like you're using Python 3, and in future questions it's helpful if you are explicit about which version you're using because there are minor but program-breaking incompatibilities in some code (including this code!).
I found a few errors in your program:
The most major issue - as Trevor Barnwell says, you're not calling threading.Thread quite correctly. The target= argument needs to be a callable object (i.e. function), but in this case it should just be a reference to the function. If you add brackets to the function, self.create_socket(host, port) as you have above, it actually runs the function immediately. As Trevor explained, your Sending.send() method was called early, but additionally there was a similar bug in Receiving. Because Receiving.create_socket() creates an infinite loop, it never returns program execution. While the console output looks correct to the user, the actual program execution has never made it to running the listener in a separate thread.
bytearray.extend() takes an iterable of ints, what you're passing right now is a tuple of byte objects.
In Sending.send() you call self.sock, but you never assign self.sock a value, so it fails.
Sending.run() only runs Sending.send() one time. After completing input for the user, it immediately exits, because the program has finished.
If you're looking for an in-depth, project based introduction to Python appropriate for an experienced programmer (including an exercise very similar to this question on basic sockets, and another on threading), I highly recommend you check out Wesley Chun's "Core Python Applications Programming". The most recent edition (3rd) has a lot of Python 2 code, but it's easily portable to Python 3 with some minor work on the reader's part.
I tried to modify your code as little as possible to get it working, here it is:
import socket
import threading
import logging
import time
class Sending():
def __init__(self, name, tHost, tPort, target):
self.name = name
self.host = tHost
self.port = tPort
self.target_port = target
self.sock = self.create_socket()
def create_socket(self):
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind((self.host, self.port))
return sock
def set_name(self, name):
self.name = name
def send_loop(self):
while True:
logging.debug('Starting send run')
message = input('Enter message: ')
data = bytearray()
data.extend(message.encode('utf-8'))
self.sock.sendto(bytearray(data), (self.host, self.target_port))
def run(self):
th2 = threading.Thread(name='send', target=self.send_loop)
th2.start()
class Receiving():
def __init__(self, host, port):
self.host = host
self.port = port
def create_socket(self):
logging.debug('Starting socket')
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.bind((self.host, self.port))
print ('socket ready')
time.sleep(5)
while True:
data, addr = sock.recvfrom(1500)
print('\nPrijata:' + data.decode('utf-8') + str(addr))
def run(self):
th1 = threading.Thread(name='rec', target=self.create_socket)
print("Made it here")
th1.daemon = True
th1.start()
return
if __name__ == '__main__':
print('running')
rec = Receiving('localhost', 8000)
send = Sending('username', 'localhost', 8001, 8000)
rec.run()
send.run()
The threads are not blocking each other. send is called before a thread is even created.
th2 = threading.Thread(name = 'send', target=self.send('username', 'localhost', 8001, 1, 1400))
This line makes a call to send at:
self.send('username', 'localhost', 8001, 1, 1400)
I think you meant to do this:
th2 = threading.Thread(
target=self.send
args=('username', 'localhost', 8001, 1, 1400))
That way a thread will start that calls send on the next line.
Two other things:
You will want to loop in your functions because the thread terminates once the function does.
I think you mean raw_input instead of input

Random freezing / hanging in Python ZeroMQ

I am writing a broker-less, balanced, client-worker service written in python with ZeroMQ.
The clients acquire a worker's address, establish a connection ( zmq.REQ / zmq.REP ), send single request, receive a single response and then disconnect.
I have chosen a broker-less architecture because the amount of a data that needs to get transferred between the clients and workers is relatively large, despite there only being a single REQ/REP pair per connection, and using a broker as a 'middle man' would create a bottleneck.
While testing the system, I noticed that the communication between the clients and workers was halting randomly, only sometimes resuming after a couple of seconds (often several minutes).
I narrowed down the issue to the .connect() / .disconnect() of clients to workers.
I have written two small python scripts that reproduce the bug.
import zmq
class Site:
def __init__(self):
ctx = zmq.Context()
self.pair_socket = ctx.socket(zmq.REQ)
self.num = 0
def __del__(self):
print "closed"
def run_site(self):
print "running..."
while True:
self.pair_socket.connect('tcp://127.0.0.1:5555')
print 'connected'
self.pair_socket.send_pyobj(self.num)
print 'sent', self.num
print self.pair_socket.recv_pyobj()
self.pair_socket.disconnect('tcp://127.0.0.1:5555')
print 'disconnected'
self.num += 1
s = Site()
s.run_site()
and
import zmq
class Server:
def __init__(self):
ctx = zmq.Context()
self.pair_socket = ctx.socket(zmq.REP)
self.pair_socket.bind('tcp://127.0.0.1:5555')
def __del__(self):
print " closed"
def run_server(self):
print "running..."
while True:
x = self.pair_socket.recv_pyobj()
print x
self.pair_socket.send_pyobj(x)
s = Server()
s.run_server()
I don't think the issue is related to memory or gc as I have tried disabling gc - without much affect.
I have tried using zmq.LINGER as described here: Zeromq with python hangs if connecting to invalid socket
What could cause these randoms freezes?
The REP socket is synchronous by definition. So your server can only serve one request at a time, rest of them will just fill up the buffer and get lost at some point.
To fix the root cause, you need to use the ROUTER socket instead.
class Server:
def __init__(self):
ctx = zmq.Context()
self.pair_socket = ctx.socket(zmq.ROUTER)
self.pair_socket.bind('tcp://127.0.0.1:5555')
self.poller = zmq.Poller()
self.poller.register(self.pair_socket, zmq.POLLIN)
def __del__(self):
print " closed"
def run_server(self):
print "running..."
while True:
try:
items = dict(self.poller.poll())
except KeyboardInterrupt:
break
if self.pair_socket in items:
x = self.pair_socket.recv_multipart()
print x
self.pair_socket.send_multipart(x)

Can pygame events be handled in select.select input list?

The documentation for python's select.select says:
Note that on Windows, it only works for sockets; on other operating
systems, it also works for other file types (in particular, on Unix,
it works on pipes).
My group is developing a simplistic multiplayer game using pygame and sockets. (We are not using Twisted or zeromq or any similar libraries; this being the only constraint).
Now, for the game design; we want the player to send data to the server when a key event occurs in the pygame screen. The client/player side's socket will otherwise be hooked to server and listen for changes occurring on other players' side. For this task, I'd need the pygame and socket to work parallely. I was recommended to use select module from several users on #python.
Can I do something like:
inp = [self.sock, pygame.event.get]
out = [self.server]
i, o, x = select.select( inp, out, [] )
If not, what should be the way to go?
You could use threads for this task. Is it necessary to process server messages and pygame events in series (not concurrently)? If so, you could do this:
class SocketListener(threading.Thread):
def __init__(self, sock, queue):
threading.Thread.__init__(self)
self.daemon = True
self.socket = sock
self.queue = queue
def run(self):
while True:
msg = self.socket.recv()
self.queue.put(msg)
class PygameHandler(threading.Thread):
def __init__(self, queue):
threading.Thread.__init__(self)
self.queue = queue
self.daemon = True
def run(self):
while True:
self.queue.put(pygame.event.wait())
queue = Queue.Queue()
PygameHandler(queue).start()
SocketListener(queue).start()
while True:
event = queue.get()
"""Process the event()"""
If not, you could process the events inside the run methods of the PygameHandler and SocketListener classes.

Is there a way for BaseRequestHandler classes to be stateful?

Short Question
Using my examples below, is there a Pythonic way to share my_object's actual instance with with the BaseRequestHandler class?
Background
By definition, the BaseRequestHandler class creates a new instance for each request. Because of this, I am struggling to try find a solution on how to get data from the handle() function back to the ProtocolInterface instance. Note that this might be the wrong approach if I am needing to do something in handle() other than print to stdout.
At this point in time, I do not believe that global variables will work because my_object is passed in and is expected to change often (this is why handle() needs to see it. To see an example client (sending bogus data) see my other SO question. I think the biggest issue I am facing is the the socketservers are running in a background thread.
Example of what I would like to do
class ProtocolHandler(SocketServer.BaseRequestHandler):
def handle(self):
while(1):
self.data = self.request.recv(1024)
if self.data == '':
break
self.request.send("Success")
self.my_object.success = True# <--- How can I share my_object's instance?
class ProtocolInterface():
def __init__(self, obj, host='127.0.0.1', port=8000, single_connection=False):
self.my_object = obj # <--- This ideally is the same instance seen in ProtocolHandler
self.host = host
self.port = port
# Create the socket server to process in coming traffic
if(single_connection):
self.server = SocketServer.TCPServer((self.host, self.port), ProtocolHandler)
else:
self.server = SocketServer.ThreadingTCPServer((self.host, self.port), ProtocolHandler)
def start(self):
print "Server Starting on HOST: " + self.host
server_thread = threading.Thread(target=self.server.serve_forever)
server_thread.daemon = True
server_thread.start()
You could pass the object through the server instance:
self.server = SocketServer.TCPServer((self.host, self.port), ProtocolHandler)
self.server.my_object = self.my_object
The documentation indicates that you can have access to the server instance in handle() as self.server.

Categories

Resources