I need to allow other Python applications to register callback functions for events in my application. These need to have priorities associated with them (so a callback with a priority of 10 runs before a callback with a priority of 1) and callback functions need to be able to signal errors to the dispatcher.
These are all lightweight callbacks running in the same process, so I don't need to send signals across process boundaries.
Is there a good Python library to handle this, or do I need to write my own?
Are these other applications running in another address space? If so, you'll need to use an interprocess communication library like D-BUS.
If you're just sending signals in the same process, try PyDispatcher
What platform are you running under? GObject is the basis of the GTK GUI that's widely-used under Linux, and it supports event loops with prioritizable events like this.
Try Twisted for anything network-related. Its perspective broker is quite nice to use.
Try python-callbacks - http://code.google.com/p/python-callbacks/.
Related
Anyone who has worked with Multithreaded PyQt4 apps? I was just wondering if the inbuilt signal/slot mechanism coupled with QtThread of PyQt4 framework has any benefit over the standard Python threads (which are designed in my code to handle the UI components in a thread safe way offcourse) using event driven async callback.
I am looking for any major speed or security concerns, any specific run-time exceptions or edge cases. (The UI is quite complex hence a re-write at a later stage would be very counter-productive).
Thanks.
Edit: I realize this might mean replicating some of the already present PyQt core functionality but it is ok if it allows more flexibility within the app.
There's no point really using Qt/PyQt if your're not using the signal-and-slot mechanism and rolling your own event loop. Basically, you'd be reimplementing the core of the framework itself. But I'm guessing this is not what you're asking about.
It would be nice if you could clarify your question a bit, (because of which I've had to make a few assumptions)
but here's the deal:
I think you're a bit confused about what the signal and slot mechanism does. (or maybe not, forgive me for reiterating some stuff that might probably be obvious to you).
The signals-and-slots do not implement threading for you (so the question of using signal/slots having any benefit over standard Python threads is moot)
You're probably assuming that the signal-slot mechanism is multithreaded, and that a slot when called by a signal, executes in a new thread. Well, this is not the case.
The signal and slot mechanism in Qt runs in a single event loop in Qt (implemented by QApplication), which itself runs in a single thread.
So signals and slots are not replacements for multi-threading in any way.
If there's a slot that blocks, then it will block your entire application.
So any blocking I/O or time intensive functions should ideally be in a separate thread from the UI, and your slots should start execution of these threads.
Now whether to use QThread or standard Python threads to implement your own threads is another issue, and it's been asked on StackOverflow before, but I tend to use QThreads for Qt apps.
So if you have a button, and you want to start a file download with the Requests library when its clicked, you'll connect the clicked signal of the QPushButton to a slot say for example downloadButtonClicked, and that slot would start a new QThread which would take care of downloading the file using Requests. You can further connect the finished() signal from the QThread to know when the download is complete and to update your UI
(I'll add a code example if this is indeed what you're asking about. So please clarify your question)
Based on your comment to another reply:
Sorry for the ambiguity, I was talking about QtThread Slot/Signal
Mechanism vs. callbacks using inbuilt Python Threads. I intend on
creating separate threads from UI on event arrival (clicks, etc) and
then use callbacks to the main UI thread from the new threads to
update the UI (all UI logic in the main thread with locks to keep it
thread safe.) I know this might mean replicating some of the already
present PyQt functionality but I feel this way I would have a lot more
control over my app. (The extra work isn't a concern if it allows more
flexibility in the app. Plus it isn't so much of work)
I would say that what you are after is to use QApplication.postEvent() from your threads. With a bit of extra code, you can use this to execute arbitrary methods in the main thread either synchronously or asynchronously.
I'm not sure there are really any advantages or disadvantages to either option (Qt or Python threads). As far as I'm aware, they both still hold the GIL, meaning your program is never truly multithreaded. QThreads come with an event loop, but as you say that isn't hard to write yourself in your own Python thread.
Have you considered using multiple processes instead of multiple threads? While slower to start, you get the advantage of actually having your code run in parallel.
At the end of the day, I think the answer to your question is simply personal preference. Mine would be to avoid using a QThread because it makes it easier to port your application to another widget toolkit in the future if PyQt\PySide\Qt ever die (not that it is very likely, but I've had a bad experience with PyGTK so now I'm wary)
EDIT: Please also look at this, as it has people far better answers than I've given: Threading in a PyQt application: Use Qt threads or Python threads?
i want to create a python socket server to send and get data to html5 ,
what is the best way to do it , a python socket lib ? or a simple code ?
thanks
#srgerg's socket documentation is useful, but if you want to handle multiple sockets simultaneously, you'll also need other mechanisms, such as select, epoll, or kqueue (depending upon your platform). (You could also spawn multiple processes using fork, or threads if the python threading implementation meets your needs, but both these approaches have enough complications that I'm reluctant to suggest them.)
Another approach is to use Twisted to manage your networking via an event loop, similar to using libevent, but I always found Twisted documentation difficult to follow. Maybe you will have better luck than I did.
Is it possible to integrate asyncore with dbus through the same main loop?
Usually, DBus integration is done through glib main loop: is it possible to have either asyncore integrate this main loop or have dbus use asyncore's ?
asyncore sucks. glib already provides async stuff, so just use glib's mainloop to do everything.
I wrote a trivial GSource wrapper for one of my own projects called AsyncoreGSource
Just attach it to an appropriate MainContext:
source = AsyncoreGSource([socket_map])
source.attach([main_context])
Naturally the defaults are asyncore.socket_map and the default MainContext respectively.
You can also try monkey-patching asyncore.socket_map, which would have been my solution had I not poked through the GLib python bindings source code for GSource.
Although you got what is probably a perfectly reasonable answer, there is another approach - you don't need to use asyncore's loop per se. Just call asyncore.loop with a zero timeout and a count of 1, which stops it iterating (and thus makes the function name completely misleading) and polls the sockets just once. Call this as often as you need.
I don't know anything about glib's async support but if it requires threads you might still get better performance by using asyncore in this way since it will use select or poll and won't need to spawn additional threads.
I'm trying to write a Python lib that will implement the client side of a certain chat protocol.
After I connect to the server,
I start the main loop where I read from the server and handle received commands and here I need to call a callback function (like on_message or on file_received, etc).
How should I go about implementing this?
Should a start a new thread for each callback function? As maybe some callbacks will take some time to return and I will timeout.
Also,
If the main loop where I read from the server is in a thread can I write to the socket from another thread(send messages to the server)?
Or is there a better approach?
Thanks.
For a python app doing this, I wouldn't use threads. I would use a framework like Twisted.
The docs have examples; here's a chat example.
I would use the select module, or alternately twisted, however select is a bit more portable, and to my mind somewhat more pythonic.
Threads are just an unnecessary complication here and will lead to obscure bugs if you're not familiar with how to use them correctly. asyncore or asynchat are simple routes to the same goal, however.
I'm working on a small utility application in Python.
The networking is gonna send and receive messages.
The GUI is gonna display the messages from the GUI and provide user input for entering messages to be sent.
There's also a storage part as I call it, which is gonna get all the network messages and save them in some kind of database to provide some kind of search later.
What my question is, what would be the best way to design this? For sure all them have to happen in separate threads, but what I'm wondering is how to pass information between them the best way? What kind of signal passing / thread locking available in Python should I use to avoid possible problems?
One, probably the best, solution for this problem is to use Twisted. It supports all the GUI toolkits.
I think that you could use a Queue for passing messages between the GUI and the network threads.
As for GUI and threads in general, you might find the PyGTK and Threading article interesting.