Is declaring function within function for threading good programming style? - python

I have following code for click handler in my PyQT4 program:
def click_btn_get_info(self):
task = self.window.le_task.text()
self.statusBar().showMessage('Getting task info...')
def thread_routine(task_id):
order = self.ae.get_task_info(task_id)
if order:
info_str = "Customer: {email}\nTitle: {title}".format(**order)
self.window.lbl_order_info.setText(info_str)
self.statusBar().showMessage('Done')
else:
self.statusBar().showMessage('Authentication: failed!')
thread = threading.Thread(target=thread_routine, args=(task,))
thread.start()
Is it a good practice to declare function in function for using with threads?

In general, yes, this is perfectly reasonable. However, the alternative of creating a separate method (or, for top-level code, a separate function) is also perfectly reasonable. And so is creating a Thread subclass. So, there's no rule saying to always do one of the three; there are different cases where each one seems more reasonable than the others, but there's overlap between those cases, so it's usually a judgment call.
As Maxime pointed out, you probably want to use Qt's threading, not native Python threading. Especially since you want to call methods on your GUI objects. The Qt docs article Threads, Events and QObjects in the Qt documentation gives you an overview (although from a C++, not Python, viewpoint). And if you're using a QThread rather than a threading.Thread, it is much more common to use the OO method—define a subclass of QThread and override its run method than to define a function, which makes your question moot.
But if you do stick with Python threading, here's how I'd decide.
Pro separate method:
You're doing this in a class method, rather than a function, and that the only state you want to share with the new thread is self.
Non-trivial code, longer than the function it's embedded in.
Pro local function:
Pretty specific to the info button callback; no one else will ever want to call it.
I'd probably make it a method, but I wouldn't complain about someone else's code that made it a local function.
In a different case—e.g., if the thread needed access to a local variable that had no business being part of the object, or if it were a trivial function I could write as an inline lambda, or if this were a top-level function sharing globals rather than a method sharing self, I'd go the other direction.

Related

Selecting executed method of class at runtime in python?

This question is very generic but I don't think it is opinion based. It is about software design and the example prototype is in python:
I am writing a program which goal it is to simulate some behaviour (doesn't matter). The data on which the simulation works is fixed, but the simulated behaviour I want to change at every startup time. The simulation behaviour can't be changed at runtime.
Example:
Simulation behaviour is defined like:
usedMethod = static
The program than looks something like this:
while(true)
result = static(object) # static is the method specified in the behaviour
# do something with result
The question is, how is the best way to deal with exchangeable defined functions? So another run of the simulation could look like this
while(true)
result = dynamic(object)
if dynamic is specified as usedMethod. The first thing that came in my mind was an if-else block, where I ask, which is the used method and then execute this on. This solution would not be very good, because every time I add new behaviour I have to change the if-else block and the if-else block itself would maybe cost performance, which is important, too. The simulations should be fast.
So a solution I could think of was using a function pointer (output and input of all usedMethods should be well defined and so it should not be a problem). Then I initalize the function pointer at startup, where the used method is defined.
The problem I currently have, that the used method is not a function per-se, but is a method of a class, which depends heavily on the intern members of this class, so the code is more looking like this:
balance = BalancerClass()
while(true)
result = balance.static(object)
...
balance.doSomething(input)
So my question is, what is a good solution to deal with this problem?
I thought about inheriting from the balancerClass (this would then be an abstract class, I don't know if this conecpt exists in python) and add a derived class for every used method. Then I create the correct derived object which is specified in the simulation behaviour an run-time.
In my eyes, this is a good solution, because it encapsulates the methods from the base class itself. And every used method is managed by its own class, so it can add new internal behaviour if needed.
Furthermore the doSomething method shouldn't change, so therefore it is implemented the base class, but depends on the intern changed members of the derived class.
I don't know in general if this software design is good to solve my problem or if I am missing a very basic and easy concept.
If you have a another/better solution please tell me and it would be good, if you provide the advantages/disadvantages. Also could you tell me advantages/disadvantages of my solution, which I didn't think of?
Hey I can be wrong but what you are looking for boils down to either dependency injection or strategy design pattern both of which solve the problem of executing dynamic code at runtime via a common interface without worrying about the actual implementations. There are also much simpler ways just like u desrcibed creating an abstract class(Interface) and having all the classes implement this interface.
I am giving brief examples fo which here for your reference:
Dependecy Injection(From wikipedia):
In software engineering, dependency injection is a technique whereby one object supplies the dependencies of another object. A "dependency" is an object that can be used, for example as a service. Instead of a client specifying which service it will use, something tells the client what service to use. The "injection" refers to the passing of a dependency (a service) into the object (a client) that would use it. The service is made part of the client's state.
Passing the service to the client, rather than allowing a client to build or find the service, is the fundamental requirement of the pattern.
Python does not have such a conecpt inbuilt in the language itself but there are packages out there that implements this pattern.
Here is a nice article about this in python(All credits to the original author):
Dependency Injection in Python
Strategy Pattern: This is an anti-pattern to inheritance and is an example of composition which basically means instead of inheriting from a base class we pass the required class's object to the constructor of classes we want to have the functionality in. For example:
Suppose you want to have a common add() operation but it can be implemented in different ways(add two numbers or add two strings)
Class XYZ():
def __constructor__(adder):
self.adder = adder
The only condition being all adders passed to the XYZ class should have a common Interface.
Here is a more detailed example:
Strategy Pattern in Python
Interfaces:
Interfaces are the simplest, they define a set of common attributes and methods(with or without a default implementation). Any class then can implement an interface with its own functionality or some shared common functionality. In python Interfaces are implemented via abc package.

How can I learn how to implement a custom Python asyncio event loop?

I’m looking into implementing a new event loop to plug into asyncio based on existing run loop implementations, such as Cocoa’s NSRunLoop and Qt’s QEventLoop. but find it difficult to to pick a place to start.
The documentation says that the system is designed to be pluggable, but nowhere does it say exactly how this can be done. Should I start with AbstractEventLoop, or BaseEventLoop? What method does what, and what components do I need to provide? The only alternative implementation I find useful is uvloop, but find it difficult to understand because it relies heavily on Cython and libuv, which I am not familiar with.
Is there some kind of a write-up on how the event loop implementation is done, and how a custom one can be made? Or a less involved implementation I can wrap my head around more quickly? Thanks for any pointers.
The documentation says to inherit from AbstractEventLoop.
For the rest of your question, I didn't find the documentation very clear, but the source code for the concrete event loop in asyncio was helpful. I've written up a
pretty minimal example of inheriting from AbstractEventLoop to create an event driven simulator.
The main things that I'd have liked to be told are
Implement create_task. The end-user schedules a coroutine using asyncio.ensure_future(coro()), but that just calls your loop's create_task method. It doesn't need to be anything more than
def create_task(self, coro): return asyncio.Task(coro, loop=self).
Implement call_soon, call_at and call_later. These are invoked by the end-user to schedule a plain callback function. They are also invoked by the async/await system automatically, whenever the end-user schedules a coroutine.
If a regular callback raises an exception, it goes to your loop's call_exception_handler method. If a coroutine raises an exception, the exception lives in some asynchronous never-never land, and you have to catch it there.
Look up the source code for AbstractEventLoop to see all the other methods that you should be overriding. Bonus: somewhat helpful comments.

Are classmethods thread safe?

I am working on a class which operates in a multithreaded environment, and looks something like this (with excess noise removed):
class B:
#classmethod
def apply(cls, item):
cls.do_thing(item)
#classmethod
def do_thing(cls, item)
'do something to item'
def run(self):
pool = multiprocessing.Pool()
for list_of_items in self.data_groups:
pool.map(list_of_items, self.apply)
My concern is that two threads might call apply or do_thing at the same time, or that a subclass might try to do something stupid with cls in one of these functions. I could use staticmethod instead of classmethod, but calling do_thing would become a lot more complicated, especially if a subclass reimplements one of these but not the other. So my question is this: Is the above class thread-safe, or is there a potential problem with using classmethods like that?
Whether a method is thread safe or not depends on what the method does.
Working with local variables only is thread safe. But when you change the same non local variable from different threads, it becomes unsafe.
‘do something to item’ seems to modify only the given object, which is independent from any other object in the list, so it should be thread safe.
If the same object is in the list several times, you may have to think about making the object thread safe. That can be done by using with self.object_scope_lock: in every method which modifies the object.
Anyway, what you are doing here is using processes instead of threads. In this case the objects are pickled and send through a pipe to the other process, where they are modified and send back. In contrast to threads processes do not share memory. So I don’t think using a lock in the class-method would have an effect.
http://docs.python.org/3/library/threading.html?highlight=threading#module-threading
There's no difference between classmethods and regular functions (and instance methods) in this regard. Neither is automagically thread-safe.
If one or more classmethods/methods/functions can manipulate data structures simultaneously from different threads, you'd need to add synchronization protection, typically using threading.Locks.
Both other answers are technically correct in that the safety of do_thing() depends on what happens inside the function.
But the more precise answer is that the call itself is safe. In other words if apply()and do_thing()are a pure functions, then your code is safe. Any unsafe-ness would be due to them not being pure functions (e.g. relying on or affecting a shared variable during execution)
As shx2 mentioned, classmethods are only "in" a class visually, for grouping. They have no inherent attachment to any instance of the class. Therefore this code is roughly equivalent in functioning:
def apply(item):
do_thing(item)
def do_thing(item)
'do something to item'
class B:
def run(self):
pool = multiprocessing.Pool()
for list_of_items in self.data_groups:
pool.map(list_of_items, apply)
A further note on concurrency given the other answers:
threading.Lock is easy to understand, but should be your last resort. In naive implementations it is often slower than completely linear processing. Your code will usually be faster if you can use things like threading.Event, queue.Queue, or multiprocessing.Pipe to transfer information instead.
asyncio is the new hotness in python3. It's a bit more difficult to get right but is generally the fastest method.
If you want a great walkthrough modern concurrency techniques in python check out core developer Raymond Hettinger's Keynote on Concurrency. The whole thing is great, but the downside of lockis highlighted starting at t=57:59.

Two threads using same function

Basic using threads question here.
I'm modifying a program with 2 thread classes and I'd like to use a function defined in one class in both classes now.
As a thread newbie (only been playing with them for a few months) is it OK to move the function out of the thread class into the main program and just call it from both classes or do I need to duplicate the function in the other class that doesn't have it?
regards
Simon
You can call the same function from both threads. The issue to be aware of is modifying shared data from two threads at once. If the function attempts to modify the same data from both threads, you will end up with an unpredictable program.
So the answer to your question is, "it depends what the function does."
It certainly won't help to copy the function into both thread classes. What matters is what the function does, not how many copies of the code there are.
might wanna checkout thread locking. threads operating on 1 function/method can 'lock' that function in many languages so other threads can't access it at the same time. http://en.wikipedia.org/wiki/Lock_(computer_science)

Implementing a macro recorder for a python gui?

I'm wondering how to go about implementing a macro recorder for a python gui (probably PyQt, but ideally agnostic). Something much like in Excel but instead of getting VB macros, it would create python code. Previously I made something for Tkinter where all callbacks pass through a single class that logged actions. Unfortunately my class doing the logging was a bit ugly and I'm looking for a nicer one. While this did make a nice separation of the gui from the rest of the code, it seems to be unusual in terms of the usual signals/slots wiring. Is there a better way?
The intention is that a user can work their way through a data analysis procedure in a graphical interface, seeing the effect of their decisions. Later the recorded procedure could be applied to other data with minor modification and without needing the start up the gui.
You could apply the command design pattern: when your user executes an action, generate a command that represents the changes required. You then implement some sort of command pipeline that executes the commands themselves, most likely just calling the methods you already have. Once the commands are executed, you can serialize them or take note of them the way you want and load the series of commands when you need to re-execute the procedure.
Thinking in high level, this is what I'd do:
Develop a decorator function, with which I'd decorate every event-handling functions.
This decorator functions would take note of thee function called, and its parameters (and possibly returning values) in a unified data-structure - taking care, on this data structure, to mark Widget and Control instances as a special type of object. That is because in other runs these widgets won't be the same instances - ah, you can't even serialize a toolkit widget instances, be it Qt or otherwise.
When the time comes to play a macro, you fill-in the gaps replacing the widget-representating object with the instances of the actually running objects, and simply call the original functions with the remaining parameters.
In toolkits that have an specialized "event" parameter that is passed down to event-handling functions, you will have to take care of serializing and de-serializing this event as well.
I hope this can help. I could come up with some proof of concept code for that (although I am in a mood to use tkinter today - would have to read a lot to come up with a Qt4 example).
An example of what you're looking for is in mayavi2. For your purposes, mayavi2's "script record" functionality will generate a Python script that can then be trivially modified for other cases. I hear that it works pretty well.

Categories

Resources