I have a python function that turns on some LEDs, then pauses (time.sleep), and then turns off the LEDs via the Raspberry Pi. It's a bit more complicated than that - it's actually a bunch of LEDs in various patterns so several hundred lines of code. This function does everything in an infinite loop. Originally, I called the function in a thread because I have some other code that runs continuously as well.
Now, I need to be able to terminate the function. This could be required after 10 seconds or 100 seconds. Each time will just depend. In looking through the site and researching threading, it doesn't sound wise to just terminate the thread and I can't really use a flag because there are so many lines of code in the function.
Is there an alternative to using threads?
If you don't need much explicit data sharing between threads, you could use multiprocessing, which is very similar to the threading module, but uses processes (which can be terminated safely).
Related
I have a program that fetches data via an API. I created a function that only takes the target data as an argument and with a for-loop I run this method 10 times.
The programm takes quite some time to display the data because the next function call only happens when the function before has done its work.
I want to use Threads to make it all happen quicker. However, I'm confused. On realpython.org I read this:
A thread is a separate flow of execution. This means that your program will have two things happening at once. But for most Python 3 implementations the different threads do not actually execute at the same time: they merely appear to. It’s tempting to think of threading as having two (or more) different processors running on your program, each one doing an independent task at the same time. That’s almost right. The threads may be running on different processors, but they will only be running one at a time.
First they say: "This means that your program will have two things happening at once" and then they say "but they will only be running one at a time". So my threads are not done simultaneously?
I want to make a decision on whether to use Threads or Multiprocessing but I can't figure it out.
Can somebody help?
With both Threads or Multiprocessing you must assume that execution of your program could jump from one thread/process to another randomly. The difference is that with Threads, code is never really executed at the same time. That means there is always only one CPU core doing your work. With Multiprocessing, your code runs on multiple cores at the same time. So only Multiprocessing will solve your computation N times faster with N processes. (There will be some overhead of course.) If you are not doing any heavy computation, but need to create the illusion of things running in parallel, use threads. This is especially useful for GUIs.
The confusing part is that IO (copying files or loading something from the web for example) is not CPU bound, as it does not require a lot of CPU instructions to happen. So always use threads for this. To understand it a bit more, you should realise that when a thread is waiting for an IO operation to finish, it is actually in a blocked state. This allows other threads to run. So if you use threads to fetch data the first thread will start loading it and then block. This makes room for the the second thread to do the same and so on. When one of the threads has the data ready, it will unblock, run the rest of its code and finish.
(Note that when multiple threads are running they can pause randomly and give room for other threads to run for a while and then carry on. (See first sentence of this answer.))
Generally always use threads unless you need to do something CPU heavy in parallel. Multiprocessing has a lot of limitations when it comes to how it works internally and using it is more complicated and heavy.
This only applies to some implementations of Python tough, for example the most commonly used "official" implementation, CPython. In other languages or less common Python implementations threads are often able to execute instructions on multiple cores at the same time.
I'm writing a program to control GPIO's on my raspberry pi. I would like my program to ask me how long I would like to keep a GPIO on before it turns off.
Is it possible to have it stay on for 1 hour and then turn off. The problem I'm having is that when its on for an hour I cant issue any other commands to turn on other GPIO's because sleep.time is still being processed. I'd like to set multiple GPIO's for different times at the same time.
There are many ways to solve the problem. Conceptually, instead of sleeping 30 seconds and then doing something, you can sleep one second, do a bunch of stuff, check the time, lather, rinse, repeat. And by "sleep one second" it could just as easily be a tenth of a second or five seconds or whatever seems reasonable to me.
Another solution is to create a second thread (or process) for this sleep command so that your main thread (or process) runs unabated.
The choice depends on what all you need to do, how accurate you need the delay to be, what other things are running on the system, and so on.
Your current script only have one thread running, the sleep() will put the current running thread into sleep mode which blocks further commands.
time.sleep(secs) Python Doc
time.sleep(secs)
Suspend execution of the current thread for the given number of seconds.
You will need one more thread on the background which keeps the timer for you. In the mean time, the thread on the foreground can still takes other commands.
I recommend you read this page
threading – Manage concurrent threads
I am using Python with the Rasbian OS (based on Linux) on the Raspberry Pi board. My Python script uses GPIOs (hardware inputs). I have noticed when a GPIO activates, its callback will interrupt the current thread.
This has forced me to use locks to prevent issues when the threads access common resources. However it is getting a bit complicated. It struck me that if the GPIO was 'queued up' until the main thread went to sleep (e.g. hits a time.sleep) it would simplify things considerably (i.e. like the way that javascript deals with things).
Is there a way to implement this in Python?
Are you using RPi.GPIO library? Or you call your Python code from C when a callback fires?
In case of RPi.GPIO, it runs a valid Python thread, and you do not need extra synchronization if you organize the threads interaction properly.
The most common pattern is to put your event in a queue (in case of Python 3 this library will do the job, Python 2 has this one). Then, when your main thread is ready to process the event, process all the events in your queue. The only problem is how you find a moment for processing them. The simplest solution is to implement a function that does that and call it from time to time. If you use a long sleep call, you may have to split it into many smaller sleeps to make sure the external events are processed often enough. You may even implement your own wrapper for sleep that splits one large delay into several smaller ones and processes the queue between them. The other solution is to use Queue.get with timeout parameter instead of sleep (it returns immediately after an event arrives into the queue), however, if you need to sleep exactly for a period you specified, you may have to do some extra magic such as measuring the time yourself and calling get again if you need to wait more after processing the events.
Use a Queue from the multithreading module to store the tasks you want to execute. The main loop periodically checks for entries in the queue and executes them one by one when it finds something.
You GPIO monitoring threads put their tasks into the queue (only one is required to collect from many threads).
You can model your tasks as callable objects or function objects.
What is the best way to continuously repeat the execution of a given function at a fixed interval while being able to terminate the executor (thread or process) immediately?
Basically I know two approaches:
use multiprocessing and function with infinite cycle and time.sleep at the end. Processing is terminated with process.terminate() in any state.
use threading and constantly recreate timers at the end of the thread function. Processing is terminated by timer.cancel() while sleeping.
(both “in any state” and “while sleeping” are fine, even though the latter may be not immediate). The problem is that I have to use both multiprocessing and threading as the latter appears not to work on ARM (some fuzzy interaction of python interpreter and vim, outside of vim everything is fine) (I was using the second approach there, have not tried threading+cycle; no code is currently left) and the former spawns way too many processes which I would like not to see unless really required. This leads to a problem of having to code two different approaches while threading with cycle is just a few more imports for drop-in replacements of all multiprocessing stuff wrapped in if/else (except that there is no thread.terminate()). Is there some better way to do the job?
Currently used code is here (currently with cycle for both jobs), but I do not think it will be much useful to answer the question.
Update: The reason why I am using this solution are functions that display file status (and some other things like branch) in version control systems in vim statusline. These statuses must be updated, but updating them immediately cannot be done without using hooks and I have no idea how to set hooks temporary and remove on vim quit without possibly spoiling user configuration. Thus standard solution is cache expiring after N seconds. But when cache expired I need to do an expensive shell call and the delay appears to be noticeable, the more noticeable the heavier IO load is. What I am implementing now is updating values for viewed buffers each N seconds in a separate process thus delays are bothering that process and not me. Threads are likely to also work because GIL does not affect calls to external programs.
I'm not clear on why a single long-lived thread that loops infinitely over the tasks wouldn't work for you? Or why you end up with many processes in the multiprocess option?
My immediate reaction would have been a single thread with a queue to feed it things to do. But I may be misunderstanding the problem.
I do not know how do it simply and/or cleanly in Python, but I was wondering if maybe you couldn't take avantage of an existing system scheduler, e.g. crontab for *nix system.
There is an API in python and it might satisfied your needs.
I'm working with Python and Jython to deploy applications to WebSphere. However, we are running into an issue with the WAS libraries where calls to these actions will sometimes take up to 30 minutes to execute. In order to troubleshoot this issue, I need to be able to keep tabs on if a process is still executing after so many minutes and, if so, send an alert.
I'm assuming I will need to put the call in a separate thread and watch it, but I have no experience with multithreading. What is the best way to do so and are there any gotchas?
Python has a built-in Timer class which will handle the threading stuff for you:
import threading
def after_2_minutes():
if process_still_running():
print "ALERT!!"
# start a timer in the background that waits 2 minutes
threading.Timer(2 * 60, after_2_minutes).start()
# start the process
foo.bar()
Gotchas: after_30_minutes will run in a separate thread, so as with any thread, you have to make sure that its actions don't interfere with your other threads (although you can manipulate simple data structures without problems due to the Global interpreter lock in CPython).