I have python file with httpHandler class.
I use it with ThreadingMixIn as follows:
from SocketServer import ThreadingMixIn
from BaseHTTPServer import HTTPServer, BaseHTTPRequestHandler
Successful_Attempts = 0
Failed_Attempts = 0
class httpHandler(BaseHTTPRequestHandler):
def do_POST(self):
.....
class ThreadingHTTPServer(ThreadingMixIn, HTTPServer):
pass
and later on I initiate it as follows:
server = ThreadingHTTPServer(('localhost', PORT_NUMBER), httpHandler)
print 'Started httpserver on port ' , PORT_NUMBER
So as I understand, the httpHandler class, once a connection is made, is already in a different thread.
I want to keep track on my threads, some sort of statistics handling.
However I cant access my variables.
Also, I need to lock them, so that they'll represent real values, not something undefined correctly
I would use a thread safe singleton for that.
Edit: johnthexiii suggested that I should post a simple example, so here it is. It's not exactly a singleton, but mimics one, class variables are global per application, so instead instantiating a singleton class we're working directly with the class, calling its class methods.
from threading import Lock
class ThreadSafeGlobalDataContainer:
__container = {}
__lock = Lock()
#classmethod
def set(cls, name, value):
with cls.__lock:
cls.__container[name] = value
#classmethod
def get(cls, name):
with cls.__lock:
return cls.__container[name]
#in some thread
ThreadSafeGlobalDataContainer.set('someName', 'someValue')
#somewhere else
print(ThreadSafeGlobalDataContainer.get('someName'))
Related
Is it possible to detach a specific event after attaching it to a COM object?
For example, how to deregister the ClassOfHandlers in the following snippet:
from win32com.client import WithEvents
# ...
class ClassOfHandlers():
def OnStart(self):
print("Start observed")
class AnotherClassOfHandlers():
def OnStart(self):
print("Start observed from another")
WithEvents(client, ClassOfHandlers)
# ...
WithEvents(client, AnotherClassOfHandlers)
# ...
# Deregister `ClassOfHandlers`
As a variation on the OP's answer, which avoids a static member variable, it is worth remembering that WithEvents() returns an instance of the handler class.
from win32com.client import WithEvents
def MyOnStart():
print("Start observed")
def MySecondOnStart():
print("Start observed from another")
class ClassOfHandlers():
def __init__(self):
self._fn = MyOnStart
def setStartFunction(self,fn):
self._fn = fn
def OnStart(self):
self._fn()
handler = WithEvents(client, ClassOfHandlers)
#then later
handler.setStartFunction(MySecondOnStart)
Hence you can re-use the handler class for a different client.
Alternatively you could try opening an issue here and maybe the developers can advise on whether they expose the IConnectionPoint::Unadvise() function which would be needed behind the scenes to switch event handlers (I think).
Edit
Based on DS_London's answer we could benefit from WithEvents return, thus the combined solution would look like
from win32com.client import WithEvents
def MyOnStart():
print("Start observed")
def MySecondOnStart():
print("Start observed from another")
class ClassOfHandlers():
def __init__(self):
self._onStarts = []
# self._onStops = []
# ... add a list of functions for each event type
# the following 3 methods are implemented for each event type
def attachStart(self, fn):
self._onStarts.append(fn)
def detachStart(self, fn):
self._onStarts.remove(fn)
def OnStart(self):
for fn in self._onStarts:
fn()
# Always at the beginning
handler = WithEvents(client, ClassOfHandlers)
handler.attachStart(MyOnStart)
# ...
handler.attachStart(MySecondOnStart)
# ...
handler.detachStart(MyOnStart)
Limitation
If support for multiple clients is needed and thus threading is used, this edit won't work, and it would be needed to use the original answer's approach.
The cause: one needs to pass the ClassOfHandlers to the thread runnable*, however the thread runnable would PumpWaitingMessages() till interrupted, thus it won't be able to return the client handler back, preventing us from being able to detach/attach further functions while waiting for messages.
* PumpWaitingMessages() requires that it runs on the same thread that connected the ClassOfHandlers to the client, thus we can't create the client handler out of the thread then send it into the thread runnable.
Following is a snippet that shows this scenario:
def threadRunnable(client, eventsClass, controller):
pythoncom.CoInitializeEx(pythoncom.COINIT_MULTITHREADED)
# Connect the custom events
# The connection needs to be done inside the same thread for PumpWaitingMessages
handler = WithEvents(client, eventsClass)
if controller == None:
print("no control was provided")
controller = { "sleep_time": 1, "running_flag": True}
# With this while we won't be able to return the handler
while controller["running_flag"]:
pythoncom.PumpWaitingMessages()
time.sleep(controller["sleep_time"])
pythoncom.CoUninitialize()
def connectEvents(client, eventsClass, controller=None, runnable=threadRunnable):
flusher = Thread(target=runnable, args=(client,eventsClass,controller))
flusher.daemon = True
flusher.start()
def main():
controller = { "sleep_time": 1, "running_flag": True}
connectEvents(client, ClassOfHandlers, controller)
Original
I'm now able to achieve the desired behavior, by attaching a single permanent observer class and managing the events myself.
For example:
from win32com.client import WithEvents
# ...
class ClassOfHandlers():
OnStarts = []
def OnStart(self):
for handler in ClassOfHandlers.OnStarts:
handler()
def MyOnStart():
print("Start observed")
def MySecondOnStart():
print("Start observed from another")
# Always at the beginning
WithEvents(client, ClassOfHandlers)
ClassOfHandlers.OnStarts.append(MyOnStart)
# ...
ClassOfHandlers.OnStarts.append(MySecondOnStart)
# ...
ClassOfHandlers.OnStarts.remove(MyOnStart)
Hint:
The class variable OnStarts shall be changed to an instance variable if the class represents an instantiable COM object, to allow having an instance of the ClassOfHandlers (each instance having a different handler list) for each instantiated COM object.
One also needs to ensure that WithEvents is called only once for each COM object instance.
How does one pass semaphore locks into threading.Thread objects in Python?
I am using locks to allow different threading.Thread classes to modify shared resources in their run functions. However, I'm unsure how to pass in the lock which is declared in a "main" file while the two threading objects are defined in separate files. For example:
Let's say I have one class producing data in producer.py:
import threading
class producer(threading.Thread):
def __init__(self, lock, shared):
self.lock = lock # Must be linked with global_lock
self.shared = shared # Must also be linked with global_shared
self.sock = # Declare listening socket
def run(self):
self.lock.acquire()
self.shared = sock.recv_msg() # Get data from socket
self.lock.release()
And another class consuming data in consumer.py:
import threading
class consumer(threading.Thread):
def __init__(self, lock, shared):
self.lock = lock # Must be linked with global_lock
self.shared = shared # Must also be linked with global_shared
def run(self):
self.lock.acquire()
print self.shared
self.shared = None
self.lock.release()
Then, in main.py:
import threading
global_lock = threading.Lock()
shared_dictionary = {}
producer_thread = producer(global_lock, global_shared)
consumer_thread = consumer(global_lock, global_shared)
producer.start()
consumer.start()
I understand that I could use global variables and declare them at the beginning of the run functions in each class but this is undesired since the purpose of these classes are to be reusable.
How can I pass a shared lock between two threading.Thread objects such that their .run() functions use the shared lock and shared object. Please help. Thank you in advance!
I am trying to define a handler class for a socketserver. When the handler class had no __init__() method defined, my server worked. The message sent by the client was written to the output window. However, when I added an __init__() method to declare a class member, my program threw an exception because RequestHandlerClass required exactly one argument, and I was passing four arguments to it. After pounding my head into a brick wall for a while, I remembered that the BaseRequestHandler class has an override-able setup() method. I declared an override for it and declared my class member inside it, and it worked.
While I have a solution to my immediate problem, I'd like to understand this. Should I never declare my own __init__() method in a request handler class? Or if I should, how should it be declared?
Here's my code:
import socketserver
import logging
import logging.config
import json
from TWMSMessageHandler import TWMSMessageHandler
class SingleTCPHandler(socketserver.BaseRequestHandler):
# def __init__(self): ## causes an error
def setup(self):
self.messageHandler = TWMSMessageHandler()
# One instance per connection. Override handle(self) to customize action.
def handle(self):
# self.request is the client connection
data = self.request.recv(1024) # clip input at 1Kb
dataString = data.decode()
print ("Received data: " + dataString)
self.request.close()
class MyTCPServer(socketserver.TCPServer):
def __init__(self, serverAddress, handler):
super().__init__(serverAddress, handler)
def handle_timeout(self):
print ("No message received in {0} seconds".format(self.timeout))
if __name__ == "__main__":
with open('TWMSHandler_log_config.json', 'rt') as f:
config = json.load(f)
logging.config.dictConfig(config)
tcpServer = MyTCPServer(("127.0.0.1", 5006), SingleTCPHandler)
tcpServer.timeout = 30
loopCount = 0
while loopCount < 5:
try:
print ("About to wait for request")
tcpServer.handle_request()
print ("Back from handle_request")
loopCount = loopCount + 1
except Exception as Value:
print ("Oops! " + str(Value))
break
I'm assuming python 2.7 since you haven't specified otherwise, this should apply to python 3.x too, however.
If you take a look at the source code (https://hg.python.org/cpython/file/2.7/Lib/SocketServer.py#l631), the BaseRequestHandler class which you are overriding takes 3 arguments besides self: request, client_address, server. If you want to override __init__ you must be compatible with this signature, unless you also override the callsite that calls __init__ from within the TCPServer inheritance chain (You don't want to do this).
Since all that function does is to save state you would otherwise have to save yourself (Or call the base function through a super call), you may as well just use setup as you are.
I'd like to do some cleanup operations inside the object just before its destruction. In this case it would be close the connection to the database.
Here is what I'm already doing:
Worker class:
from PyQt5 import QtCore
from pymongo import MongoClient, ASCENDING
from time import sleep
class StatusWidgetWorker(QtCore.QObject):
ongoing_conversions_transmit = QtCore.pyqtSignal([list])
def __init__(self, mongo_settings):
super().__init__()
print("StatusWidget Worker init")
mongo_client = MongoClient([mongo_settings["server_address"]])
self.log_database = mongo_client[mongo_settings["database"]]
self.ongoing_conversions = mongo_settings["ongoing_conversions"]
def status_retriever(self):
print("mongo bridge()")
while True:
ongoing_conversions_list = []
for doc in self.log_database[self.ongoing_conversions].find({}, {'_id': False}).sort([("start_date", ASCENDING)]):
ongoing_conversions_list.append(doc)
self.ongoing_conversions_transmit.emit(ongoing_conversions_list)
sleep(2)
And the function that call the worker from an other class :
def status_worker(self):
mongo_settings = "dict parameter"
self.worker_thread_status = QtCore.QThread()
self.worker_object_status = StatusWidgetWorker(mongo_settings)
self.worker_object_status.moveToThread(self.worker_thread_status)
self.worker_thread_status.started.connect(self.worker_object_status.status_retriever)
self.worker_object_status.ongoing_conversions_transmit.connect(self.status_table_auto_updater)
self.worker_thread_status.start()
Here is what I already tried:
Define a __del__ function in the Worker class, this function is never called.
Define a function in the Worker class and then connect it to the destroyed signal with self.destroyed.connect(self.function). This function is again never called. I think this happen because the signal is emitted when the object is already destroyed, not before its destruction.
I'm really wondering on how to this, here are some parts of answer:
http://www.riverbankcomputing.com/pipermail/pyqt/2014-November/035049.html
His approach seems a bit hacky to me (no offense to the author, there is probably no simple answer) and I have signals & parameters to pass to the worker witch would make the ThreadController class messier.
I find this solution a bit hacky because you have to set up a Controller class to do the Worker class job
If nobody has an answer, I'll probably use the ThreadController class and post the result here.
thank you for reading :-)
The usual rule in python apply:
there is a module for that
the solution is to use the atexit module and register the cleanup function in the __init__ function.
Example:
import atexit
class StatusWidgetWorker(QObject):
def __init__(self):
super().__init__()
# code here
atexit.register(self.cleanup)
def cleanup(self):
print("Doing some long cleanup")
sleep(2)
self.bla = "Done !"
print(self.bla)
I'm trying to dynamically add signals to a D-Bus service using dbus-python. It provides a decorator for this that works fine if the signal names are known at module load time; however, I don't know what name to export to D-Bus until runtime.
To illustrate the problem, what I'd like to do is the moral equivalent of:
import dbus
import dbus.service
import gobject
from dbus.mainloop.glib import DBusGMainLoop
class Event(dbus.service.Object):
def __init__(self, name):
self.name = name
self.busName = dbus.service.BusName('com.acme.EventManager',
bus=dbus.SessionBus())
dbus.service.Object.__init__(self,
self.busName,
'/com/acme/EventManager/' +
self.name)
self.signame = 'com.acme.EventManager.' + self.name
# THIS DOES NOT WORK: this decorator is parsed before the Event
# class, and 'self' wouldn't exist here, anyway...
#dbus.service.signal(dbus_interface=self.signame, signature='v')
def emit(self, data):
print "In %s event, got: %s " % (self.name, data)
if __name__ == "__main__":
DBusGMainLoop(set_as_default=True)
bus = dbus.SessionBus()
loop = gobject.MainLoop()
connect = Event('Connect')
disconnect = Event('Disconnect')
loop.run()
Not surprisingly, this generates:
#dbus.service.signal(dbus_interface=self.signame, signature='v')
NameError: name 'self' is not defined
I thought I could just dispense with the syntactic sugar provided by the # decoration operator and patch Event manually after the class has been defined, something like this:
import dbus
import dbus.service
import gobject
from dbus.mainloop.glib import DBusGMainLoop
class Event(dbus.service.Object):
def __init__(self, name):
self.name = name
self.busName = dbus.service.BusName('com.acme.EventManager',
bus=dbus.SessionBus())
dbus.service.Object.__init__(self,
self.busName,
'/com/acme/EventManager/' +
self.name)
self.signame = 'com.acme.EventManager.' + self.name
def emit(self, data):
print "In %s event, got: %s " % (self.name, data)
if __name__ == "__main__":
DBusGMainLoop(set_as_default=True)
bus = dbus.SessionBus()
loop = gobject.MainLoop()
e1 = Event('Connect')
e1.emit = dbus.service.signal(dbus_interface=e1.signame,
signature='v')(e1.emit)
loop.run()
This runs without errors, but it fails to export the signals to D-Bus. When I run D-Feet, I see the object path /com/acme/EventManager/Connect but it has no interface methods aside from Introspect().
To see if I could learn something about what was going on, I examined the value of the function passed to the dbus.service.signal decorator in a debugger. For this typical use case:
#dbus.service.signal(dbus_interface='com.acme.foo', signature='v')
def emit(self, data):
pass
the function passed in to the decorator (in the variable func) looks like this:
>>> func
>>> <function emit at 0x99fbed4>
But when I manually invoke the decorator function (as in the the e1.emit = assignment in the second example above), I see:
>>> func
>>> <bound method Event.emit of <__main__.Event at /com/acme/EventManager/Connect at 0x9b3348c>>
So... it seems that, for the normal use case, dbus.service.signal expects to receive a free function---not an unbound function, but a function that, for all intents and purposes, looks like it was defined using:
def emit():
pass
This behavior is utterly mystifying to me. I've read lots of tutorials on decorators, and thought I understood them pretty well, but I have been stumped on this for hours. If this decorator expects to be called with 'raw' function, how can I convert an object method such that I can invoke it manually?
From this question, I see that types.MethodType() can be used to convert a free function to a bound method. But it seems I need to do the opposite(?)
I think one way to do what you want is by using a factory function — however the following is untested because I don't have the D-Bus module installed.
The root of the problem is you're attempting to use a decorator at class definition time that requires data which isn't being provided until instances of that class are created. One workaround for that is to define the class inside a function and use closures so the data is available when needed. Note that the factory function returns instances of the class created, not the class itself, although it could if desired.
import dbus
import dbus.service
import gobject
from dbus.mainloop.glib import DBusGMainLoop
def event_factory(event_name):
class Event(dbus.service.Object):
def __init__(self):
self.busName = dbus.service.BusName('com.acme.EventManager',
bus=dbus.SessionBus())
dbus.service.Object.__init__(self,
self.busName,
'/com/acme/EventManager/'+event_name)
#dbus.service.signal(dbus_interface='com.acme.EventManager.'+event_name,
signature='v')
def emit(self, data):
print "In %s event, got: %s " % (event_name, data)
return Event() # return an instance of the class
if __name__ == "__main__":
DBusGMainLoop(set_as_default=True)
bus = dbus.SessionBus()
loop = gobject.MainLoop()
connect = event_factory('Connect')
disconnect = event_factory('Disconnect')
loop.run()