This question already has answers here:
How to repeatedly execute a function every x seconds?
(22 answers)
Closed 6 years ago.
I've been reading up on threading and tried implementing it into my code however I'm not sure if the way I'm doing it, is the best practise.
My code simply imports a self scripted package which pulls weather data and runs the package every 60 seconds thereafter.
I plan on running multiple packages which gather data at once, when I have worked out a good code technique.
from package.weather import weatherapi
import threading
def update():
weatherapi()
threading.Timer(60, update).start()
update()
Firstly it just seems messy and if I wanted more packages running in a thread, I'd need to create another update function
Secondly I'm not able to kill my process
If anyone has any suggestions, it would be greatly appreciated.
This is a really bad use of Threading.timer. You're constantly starting new threads, when you just want one thread to do something regularly. This code is equivalent:
from package.weather import weatherapi
import threading
import time
def update():
while True:
weatherapi()
time.sleep(60)
WHEATHER_THREAD=threading.Thread(target=update)
WHEATHER_THREAD.daemon = True # main can exit while thread is still running
WHEATHER_THREAD.start()
Since threads all use the same namespace, you can also make do with just one function.
UPDATE_CALLABLES = [weatherapi] # add new functions to have them called by update
def update():
while True:
for func in UPDATE_CALLABLES:
func()
time.sleep(60)
Note that UPDATE_CALLABLES can also be appended while the Thread is already running.
A class like this does what you want:
import threading
class Interval:
def __init__(self):
self.api=[]
self.interval=60
self.timer=self
def set_api(self,api):
self.api=api
def set_interval(self,interval):
self.interval=interval
def cancel(self):
pass
def stop(self):
self.timer.cancel()
def update(self):
for api in self.api:
api()
self.timer = threading.Timer(self.interval,self.update).start()
# Create instance and start with default parameters
interval=Interval()
interval.update()
# Later on change the list of items to call
interval.set_api([thisApi,thatApi])
# Later on still change the interval between calls
interval.set_interval(30)
# When you have had enough, cancel the timer
interval.stop()
Note that it still creates a new thread for each interval timed, but you can change the list of calls made at any time and stop it repeating at any time.
Related
This question already has an answer here:
Python Threading not processed parallel
(1 answer)
Closed 2 years ago.
I'm working with threading for the first time and somehow it doesn't work. When I start the first thread it starts but the second one doesn't. After a bit of debugging I noticed that the problem is that because thread 1 has an infinite loop, the other one waits until the first one stops. However, I want them to run in parallel.
Code:
Main Script
com = Communication("RPI", "192.168.2.156")
water = Water(com)
t_com = threading.Thread(target= com.server())
t_water = threading.Thread(target=water.main())
if __name__ == '__main__':
print("Starting")
t_com.start()
t_water.start()
Communication: Here the program is getting stuck
class Communication:
def __init__(self, clientName, serverAddress):
print("Com")
self.mqttClient = mqtt.Client(clientName)
self.mqttClient.connect(serverAddress, 1883)
self.stunde = None
self.minute = None
self.active = False
self.time_to_water = None
self.dauer = None
def server(self): <!-- a necessary infinity loop -->
print("Thread1 activated")
self.mqttClient.on_connect = self.connectionStatus
self.mqttClient.on_message = self.messageDecoder
self.mqttClient.loop_forever()
Water: This Thread doesn't start but should
class Water:
def __init__(self, com):
print("Water")
self.sensoren = Sensoren(pinRain=22, pinVent1=17, pinVent2=27)
self.com = com
def main(self):
print("Thread2 activated")
while True:
In the first Version of my programm Water and Communication weren't classes, but a mix of many functions. There the above code in main.py worked.
When creating your threads there shouldn't be parentheses on the parameter to target. You are calling the function and then (which in this case is never) passing the result of that function as an argument, instead you should pass the function to be called (only the name, without parentheses).
To answer the question on the title, the Python interpreter only allows execution of one single thread at a time due to the Global Interpreter Lock, as far as I know it's used primarily for memory safety but it won't allow true parallelism from pure Python.
You can still do parallel tasks using multiproccess or a compiled C function (at least on CPython) since those can bypass the GIL, but you'll have to manage your own locks in case more than one thread access the same data.
https://en.wikipedia.org/wiki/Global_interpreter_lock
https://wiki.python.org/moin/GlobalInterpreterLock
https://realpython.com/python-gil/
https://www.youtube.com/watch?v=m2yeB94CxVQ
This question already has answers here:
Python spawn off a child subprocess, detach, and exit
(2 answers)
Closed 3 years ago.
For example, I've got a function foo, and the caller function. I need foo to go to background, set lock file, set everything up, remove lock file. The caller must call foo and exit. I was thinking about Subprocess module, but as I see it can't do what I need from it to do. python-daemon seems to be promising, but I don't need it to run forever as a daemon.
Might want to look at threading
import threading
def foo(my_args):
# do something here
def caller(some_args):
# do some stuff
foo_thread = threading.Thread(target=foo, args=some_args)
foo_thread.start()
# continue doing stuff
caller()
You can daemonize your function in a thread: e.g.
import threading
import time
def worker(snooze):
print(f'snoozing {snooze} seconds')
time.sleep(snooze)
if __name__ == '__main___':
task = threading.Thread(name='daemonize_worker', target=worker, args=(5, ))
task.setDaemon(True)
task.start()
This throws worker with snoozing of 5 sec in the background in a daemonized way.
I have a function that is used by multiple threads. Because of its nature, this function should only ever called once at a time. Multiple threads calling the function at the same time could be bad.
If the function is in use by a thread, other threads should have to wait for it to be free.
My background isn't coding so I'm not sure, but I believe this is called "locking" in the jargon? I tried Googling it up but did not find a simple example for Python3.
A simplified case:
def critical_function():
# How do I "lock" this function?
print('critical operation that should only be run once at a time')
def threaded_function():
while True:
# doing stuff and then
critical_function()
for i in range(0, 10):
threading.Thread(target=threaded_function).start()
from threading import Lock
critical_function_lock = Lock()
def critical_function():
with critical_function_lock:
# How do I "lock" this function?
print('critical operation that should only be run once at a time')
For example i have function do_something() and I want it to run for exactly 1 second (and not .923 seconds. It won't do. However 0.999 is acceptable.)
However it is very very important that the do_something must exactly run for 1 second. I was thinking of using UNIX time stamp and calculate the seconds. But I am really wondering if Python has a way to do this in a more aesthetic way...
The function do_something() is long-running, and must be interrupted after exactly one second.
I gather from comments that there's a while loop in here somewhere. Here's a class that subclasses Thread, based on the source code for _Timer in the threading module. I know you said you decided against threading, but this is just a timer control thread; do_something executes in the main thread. So this should be clean. (Someone correct me if I'm wrong!):
from threading import Thread, Event
class BoolTimer(Thread):
"""A boolean value that toggles after a specified number of seconds:
bt = BoolTimer(30.0, False)
bt.start()
bt.cancel() # prevent the booltimer from toggling if it is still waiting
"""
def __init__(self, interval, initial_state=True):
Thread.__init__(self)
self.interval = interval
self.state = initial_state
self.finished = Event()
def __nonzero__(self):
return bool(self.state)
def cancel(self):
"""Stop BoolTimer if it hasn't toggled yet"""
self.finished.set()
def run(self):
self.finished.wait(self.interval)
if not self.finished.is_set():
self.state = not self.state
self.finished.set()
You could use it like this.
import time
def do_something():
running = BoolTimer(1.0)
running.start()
while running:
print "running" # Do something more useful here.
time.sleep(0.05) # Do it more or less often.
if not running: # If you want to interrupt the loop,
print "broke!" # add breakpoints.
break # You could even put this in a
time.sleep(0.05) # try, finally block.
do_something()
The 'sched' module of Python appears suitable:
http://docs.python.org/library/sched.html
Apart from that: Python is not a real-time language nor does it usually run on a real-time OS. So your requirement is kind of questionable.
This bit of code might work for you. The description sounds like what you want:
http://programming-guides.com/python/timeout-a-function
It relies on the python signal module:
http://docs.python.org/library/signal.html
I want to execute a function every 60 seconds on Python but I don't want to be blocked meanwhile.
How can I do it asynchronously?
import threading
import time
def f():
print("hello world")
threading.Timer(3, f).start()
if __name__ == '__main__':
f()
time.sleep(20)
With this code, the function f is executed every 3 seconds within the 20 seconds time.time.
At the end it gives an error and I think that it is because the threading.timer has not been canceled.
How can I cancel it?
You could try the threading.Timer class: http://docs.python.org/library/threading.html#timer-objects.
import threading
def f(f_stop):
# do something here ...
if not f_stop.is_set():
# call f() again in 60 seconds
threading.Timer(60, f, [f_stop]).start()
f_stop = threading.Event()
# start calling f now and every 60 sec thereafter
f(f_stop)
# stop the thread when needed
#f_stop.set()
The simplest way is to create a background thread that runs something every 60 seconds. A trivial implementation is:
import time
from threading import Thread
class BackgroundTimer(Thread):
def run(self):
while 1:
time.sleep(60)
# do something
# ... SNIP ...
# Inside your main thread
# ... SNIP ...
timer = BackgroundTimer()
timer.start()
Obviously, if the "do something" takes a long time, then you'll need to accommodate for it in your sleep statement. But, 60 seconds serves as a good approximation.
I googled around and found the Python circuits Framework, which makes it possible to wait
for a particular event.
The .callEvent(self, event, *channels) method of circuits contains a fire and suspend-until-response functionality, the documentation says:
Fire the given event to the specified channels and suspend execution
until it has been dispatched. This method may only be invoked as
argument to a yield on the top execution level of a handler (e.g.
"yield self.callEvent(event)"). It effectively creates and returns
a generator that will be invoked by the main loop until the event has
been dispatched (see :func:circuits.core.handlers.handler).
I hope you find it as useful as I do :)
./regards
It depends on what you actually want to do in the mean time. Threads are the most general and least preferred way of doing it; you should be aware of the issues with threading when you use it: not all (non-Python) code allows access from multiple threads simultaneously, communication between threads should be done using thread-safe datastructures like Queue.Queue, you won't be able to interrupt the thread from outside it, and terminating the program while the thread is still running can lead to a hung interpreter or spurious tracebacks.
Often there's an easier way. If you're doing this in a GUI program, use the GUI library's timer or event functionality. All GUIs have this. Likewise, if you're using another event system, like Twisted or another server-process model, you should be able to hook into the main event loop to cause it to call your function regularly. The non-threading approaches do cause your program to be blocked while the function is pending, but not between functioncalls.
Why dont you create a dedicated thread, in which you put a simple sleeping loop:
#!/usr/bin/env python
import time
while True:
# Your code here
time.sleep(60)
I think the right way to run a thread repeatedly is the next:
import threading
import time
def f():
print("hello world") # your code here
myThread.run()
if __name__ == '__main__':
myThread = threading.Timer(3, f) # timer is set to 3 seconds
myThread.start()
time.sleep(10) # it can be loop or other time consuming code here
if myThread.is_alive():
myThread.cancel()
With this code, the function f is executed every 3 seconds within the 10 seconds time.sleep(10). At the end running of thread is canceled.
If you want to invoke the method "on the clock" (e.g. every hour on the hour), you can integrate the following idea with whichever threading mechanism you choose:
import time
def wait(n):
'''Wait until the next increment of n seconds'''
x = time.time()
time.sleep(n-(x%n))
print(time.asctime())
[snip. removed non async version]
To use asyncing you would use trio. I recommend trio to everyone who asks about async python. It is much easier to work with especially sockets. With sockets I have a nursery with 1 read and 1 write function and the write function writes data from an deque where it is placed by the read function; and waiting to be sent. The following app works by using trio.run(function,parameters) and then opening an nursery where the program functions in loops with an await trio.sleep(60) between each loop to give the rest of the app a chance to run. This will run the program in a single processes but your machine can handle 1500 TCP connections insead of just 255 with the non async method.
I have not yet mastered the cancellation statements but I put at move_on_after(70) which is means the code will wait 10 seconds longer than to execute a 60 second sleep before moving on to the next loop.
import trio
async def execTimer():
'''This function gets executed in a nursery simultaneously with the rest of the program'''
while True:
trio.move_on_after(70):
await trio.sleep(60)
print('60 Second Loop')
async def OneTime_OneMinute():
'''This functions gets run by trio.run to start the entire program'''
with trio.open_nursery() as nursery:
nursery.start_soon(execTimer)
nursery.start_soon(print,'do the rest of the program simultaneously')
def start():
'''You many have only one trio.run in the entire application'''
trio.run(OneTime_OneMinute)
if __name__ == '__main__':
start()
This will run any number of functions simultaneously in the nursery. You can use any of the cancellable statements for checkpoints where the rest of the program gets to continue running. All trio statements are checkpoints so use them a lot. I did not test this app; so if there are any questions just ask.
As you can see trio is the champion of easy-to-use functionality. It is based on using functions instead of objects but you can use objects if you wish.
Read more at:
[1]: https://trio.readthedocs.io/en/stable/reference-core.html