I have a multithreaded server and client programs for a simple game. Whenever a client force exits the game, I try to catch the exception with a try catch "except BrokenPipeError" and inform the other players.
I also want to end the exited client thread; however, I take input such as this:
while True:
client = serverSocket.accept()
t = ServerThread(client)
t.start()
I tried to use a threading event with stop() function; however, I believe I can not use a .join statement to exit the thread because of the way I take input. How should I end the force exited client. I know that multiprocessing library has a terminate function but I am also required to use the threading library. I tried os_exit(1) but I believe this command kills the entire process. What is the standard exit process for programs such as this?
First of all join() do nothing else but waits for thread to stop.
Thread stops when it reaches end of threaded subroutine. For example
class ServerThread(threading.Thread):
def __init__(self,client,name):
super().__init__(self)
self.client = client
self.name = name
def inform(self,msg):
print("{}: got message {}".format( self.name, msg ))
self.client[0].send(msg)
def run(self):
while True:
try:
self.client[0].recv(1024)
except BrokenPipeError: #client exits
# do stuff
break # -> ends loop
return # -> thread exits, join returns
If you want to inform other clients that someone leaves, i would make another monitoring thread
class Monitoring(threading.Thread):
def __init__(self):
super().__init__(self,daemon=True) # daemon means thread stops when main thread do
self.clients=[]
def add_client(self,client):
self.clients.append(client)
def inform_client_leaves(self,client_leaved):
for client in self.clients:
if client.is_alive():
client.inform("Client '{}' leaves".format(client_leaved.name))
def run(self):
while True:
for thread in list(self.threads):
if not thread.is_alive(): # client exited
self.threads.remove(thread)
self.inform_client_exits(thread)
time.sleep(1)
So the initial code would looks like
mon = Monitoring()
mon.start()
while True:
client = serverSocket.accept()
t = ServerThread(client,"Client1")
t.start()
mon.add_client(t)
Related
TL;DR: Calling future.set_result doesn't immediately resolve loop.run_until_complete. Instead it blocks for an additional 5 seconds.
Full context:
In my project, I'm using autobahn and asyncio to send and receive messages with a websocket server. For my use case, I need a 2nd thread for websocket communication, since I have arbitrary blocking code that will be running in the main thread. The main thread also needs to be able to schedule messages for the communication thread to send back and forth with the server. My current goal is to send a message originating from the main thread and block until the response comes back, using the communication thread for all message passing.
Here is a snippet of my code:
import asyncio
import threading
from autobahn.asyncio.websocket import WebSocketClientFactory, WebSocketClientProtocol
CLIENT = None
class MyWebSocketClientProtocol(WebSocketClientProtocol):
# -------------- Boilerplate --------------
is_connected = False
msg_queue = []
msg_listeners = []
def onOpen(self):
self.is_connected = True
for msg in self.msg_queue[::]:
self.publish(msg)
def onClose(self, wasClean, code, reason):
is_connected = False
def onMessage(self, payload, isBinary):
for listener in self.msg_listeners:
listener(payload)
def publish(self, msg):
if not self.is_connected:
self.msg_queue.append(msg)
else:
self.sendMessage(msg.encode('utf-8'))
# /----------------------------------------
def send_and_wait(self):
future = asyncio.get_event_loop().create_future()
def listener(msg):
print('set result')
future.set_result(123)
self.msg_listeners.append(listener)
self.publish('hello')
return future
def worker(loop, ready):
asyncio.set_event_loop(loop)
factory = WebSocketClientFactory('ws://127.0.0.1:9000')
factory.protocol = MyWebSocketClientProtocol
transport, protocol = loop.run_until_complete(loop.create_connection(factory, '127.0.0.1', 9000))
global CLIENT
CLIENT = protocol
ready.set()
loop.run_forever()
if __name__ == '__main__':
# Set up communication thread to talk to the server
threaded_loop = asyncio.new_event_loop()
thread_is_ready = threading.Event()
thread = threading.Thread(target=worker, args=(threaded_loop, thread_is_ready))
thread.start()
thread_is_ready.wait()
# Send a message and wait for response
print('starting')
loop = asyncio.get_event_loop()
result = loop.run_until_complete(CLIENT.send_and_wait())
print('done') # this line gets called 5 seconds after it should
I'm using the autobahn echo server example to respond to my messages.
Problem: The WebSocketClientProtocol receives the response to its outgoing message and calls set_result on its pending future, but loop.run_until_complete blocks an additional ~4.9 seconds until eventually resolving.
I understand that run_until_complete also processes other pending events on the event loop. Is it possible that the main thread has somehow queued up a bunch of events that have to now get processed once I start the loop? Also, if I move run_until_complete into the communications thread or move the create_connection into the main thread, then the event loop doesn't block me.
Lastly, I tried to recreate this problem without using autobahn, but I couldn't cause the extra delay. I'm curious if maybe this is an issue with the nature of autobahn's callback timing (onMessage for example).
I am using socket in this code to connect with other machine.I want to terminate thread when i get message from other machine but how to terminate Thread in Python ?
I refer Many SO Questions and I found that there is no method in python to Close thread.Can anyone tell me the alternate way to close the thread ?
code:
from threading import Thread
import time
import socket
def background(arg):
global thread
thread = Thread(target=arg)
thread.start()
def display():
for i in range(0,20):
print(i)
time.sleep(5)
background(display)
s = socket.socket()
s.bind((ip,6500))
s.listen(5)
print("listening")
val,addr = s.accept()
cmd = val.recv(1024)
if cmd == "Terminate Process":
print("Connected")
thread.close()
print("Process Closed")
Error:
AttributeError: 'Thread' object has no attribute 'close'
Short answer:
thread.join()
The rule of thumb is: don't kill threads (note that in some environments this may not even be possible, e.g. standard C++11 threads). Let the thread fetch the information and terminate itself. Controlling threads from other threads leads to hard to maintain and debug code.
E.g.
SHOULD_TERMINATE = False
def display():
for i in range(0,20):
print(i)
time.sleep(5)
if SHOULD_TERMINATE:
return
thread = Thread(target=display)
thread.start()
// some other code
if cmd == "Terminate Process":
SHOULD_TERMINATE = True
thread.join()
This is of course heavily simplified. Your code can be further refined with event objects (instead of .sleep) or thread pools.
I have a python script which I'd like to stop by sending a "SIGINT" signal using the command line:
kill -2 <PID>
The script spawns a child process and then enters an infinite loop where it tries to receive data from a socket. I have installed the signal handler, but it looks like my program is not catching the signal. Here is my code:
import signal
import multiprocessing
def newBreakHandler(signum, frame):
global progBreak
progBreak = True
def runMain():
client = multiprocessing.Client(<address>)
# ......
# Insert break handler
signal.signal(signal.SIGINT, newBreakHandler)
while not progBreak:
try:
# Some other stuff...
# ...
childProc = multiprocessing.Process(target=childProcName)
childProc.start()
# Main loop
while not progBreak:
command = client.recv_bytes()
except:
pass
print "Exiting..."
The problem is that whenever I send a:
kill -2 <PID>
I never see the "Exiting" text printed and the PID is not killed. I think this is because the inner "while" loop is busy waiting for new data from the client, but if the client is not sending new data.
Is there a way to solve the problem?
TIA!!!
Looks like your internal loop is blocking progBreak check, this code is working for me:
import signal
progBreak = False
def newBreakHandler(signum, frame):
global progBreak
progBreak = True
def runMain():
global progBreak
signal.signal(signal.SIGINT, newBreakHandler)
while not progBreak:
while not progBreak and client.poll():
command = client.recv_bytes()
print "Exiting..."
runMain()
I am developing a server (daemon).
The server has one "worker thread". The worker thread runs a queue of commands. When the queue is empty, the worker thread is paused (but does not exit, because it should preserve certain state in memory). To have exactly one copy of the state in memory, I need to run all time exactly one (not several and not zero) worker thread.
Requests are added to the end of this queue when a client connects to a Unix socket and sends a command.
After the command is issued, it is added to the queue of commands of the worker thread. After it is added to the queue, the server replies something like "OK". There should be not a long pause between server receiving a command and it "OK" reply. However, running commands in the queue may take some time.
The main "work" of the worker thread is split into small (taking relatively little time) chunks. Between chunks, the worker thread inspects ("eats" and empties) the queue and continues to work based on the data extracted from the queue.
How to implement this server/daemon in Python?
This is a sample code with internet sockets, easily replaced with unix domain sockets. It takes whatever you write to the socket, passes it as a "command" to worker, responds OK as soon as it has queued the command. The single worker simulates a lengthy task with sleep(30). You can queue as many tasks as you want, receive OK immediately and every 30 seconds, your worker prints a command from the queue.
import Queue, threading, socket
from time import sleep
class worker(threading.Thread):
def __init__(self,q):
super(worker,self).__init__()
self.qu = q
def run(self):
while True:
new_task=self.qu.get(True)
print new_task
i=0
while i < 10:
print "working ..."
sleep(1)
i += 1
try:
another_task=self.qu.get(False)
print another_task
except Queue.Empty:
pass
task_queue = Queue.Queue()
w = worker(task_queue)
w.daemon = True
w.start()
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.bind(('localhost', 4200))
sock.listen(1)
try:
while True:
conn, addr = sock.accept()
data = conn.recv(32)
task_queue.put(data)
conn.sendall("OK")
conn.close()
except:
sock.close()
I am developing a multi-threaded application in python. I have following scenario.
There are 2-3 producer threads which communicate with DB and get some data in large chunks and fill them up in a queue
There is an intermediate worker which breaks large chunks fetched by producer threads into smaller ones and fill them up in another queue.
There are 5 consumer threads which consume queue created by intermediate worker thread.
objects of data sources are accessed by producer threads through their API. these data sources are completely separate. So these producer understands only presence or absence of data which is supposed to be given out by data source object.
I create threads of these three types and i make main thread wait for completion of these threads by calling join() on them.
Now for such a setup I want a common error handler which senses failure of any thread, any exception and decides what to do. For e.g if I press ctrl+c after I start my application, main thread dies but producer, consumer threads continue to run. I would like that once ctrl+c is pressed entire application should shut down. Similarly if some DB error occurs in data source module, then producer thread should get notified of that.
This is what I have done so far:
I have created a class ThreadManager, it's object is passed to all threads. I have written an error handler method and passed it to sys.excepthook. This handler should catch exceptions, error and then it should call methods of ThreadManager class to control the running threads. Here is snippet:
class Producer(threading.Thread):
....
def produce():
data = dataSource.getData()
class DataSource:
....
def getData():
raise Exception("critical")
def customHandler(exceptionType, value, stackTrace):
print "In custom handler"
sys.excepthook = customHandler
Now when a thread of producer class calls getData() of DataSource class, exception is thrown. But this exception is never caught by my customHandler method.
What am I missing? Also in such scenario what other strategy can I apply? Please help. Thank you for having enough patience to read all this :)
What you need is a decorator. In essence you are modifying your original function and putting in inside a try-except:
def exception_decorator(func):
def _function(*args):
try:
result = func(*args)
except:
print('*** ESC default handler ***')
os._exit(1)
return result
return _function
If your thread function is called myfunc, then you add the following line above your function definition
#exception_decorator
def myfunc():
pass;
Can't you just catch "KeyboardInterrupt" when pressing Ctrl+C and do:
for thread in threading.enumerate():
thread._Thread__stop()
thread._Thread__delete()
while len(threading.enumerate()) > 1:
time.sleep(1)
os._exit(0)
and have a flag in each threaded class which is self.alive
you could theoretically call thread.alive = False and have it stop gracefully?
for thread in threading.enumerate():
thread.alive = False
time.sleep(5) # Grace period
thread._Thread__stop()
thread._Thread__delete()
while len(threading.enumerate()) > 1:
time.sleep(1)
os._exit(0)
example:
import os
from threading import *
from time import sleep
class worker(Thread):
def __init__(self):
self.alive = True
Thread.__init__(self)
self.start()
def run(self):
while self.alive:
sleep(0.1)
runner = worker()
try:
raw_input('Press ctrl+c!')
except:
pass
for thread in enumerate():
thread.alive = False
sleep(1)
try:
thread._Thread__stop()
thread._Thread__delete()
except:
pass
# There will always be 1 thread alive and that's the __main__ thread.
while len(enumerate()) > 1:
sleep(1)
os._exit(0)
Try going about it by changing the internal system exception handler?
import sys
origExcepthook = sys.excepthook
def uberexcept(exctype, value, traceback):
if exctype == KeyboardInterrupt:
print "Gracefully shutting down all the threads"
# enumerate() thingie here.
else:
origExcepthook(exctype, value, traceback)
sys.excepthook = uberexcept