I am running the below code as an example where the function gets data and cleans it and shows the result every five minutes.
import schedule
import time
def job():
print("I'm working...")
schedule.every(5).minutes.do(job)
while True:
schedule.run_pending()
time.sleep(1)
The problem I have now is when the function runs, it takes a few seconds to do everything. For example, if the code runs at 9:00 am, it takes 2-5 seconds to complete the task. Due to this, the next time code runs at 9:05:05 seconds.
Is there a solution that can help me run the function every 5 minutes even after taking some time to complete the tasks in the function? I want the function to run exactly at 9:00 am, 9:05 am, and 9:10 am respectively.
Run another thread as mentioned in schedule docs: https://schedule.readthedocs.io/en/stable/
Schedule does not account for the time it takes for the job function to execute. To guarantee a stable execution schedule you need to move long-running jobs off the main-thread (where the scheduler runs). See Parallel execution for a sample implementation.
and once again I copy from documentation:
import threading
import time
import schedule
def job():
print("I'm running on thread %s" % threading.current_thread())
def run_threaded(job_func):
job_thread = threading.Thread(target=job_func)
job_thread.start()
schedule.every(10).seconds.do(run_threaded, job)
schedule.every(10).seconds.do(run_threaded, job)
schedule.every(10).seconds.do(run_threaded, job)
schedule.every(10).seconds.do(run_threaded, job)
schedule.every(10).seconds.do(run_threaded, job)
while 1:
schedule.run_pending()
time.sleep(1)
I already searched and see many people are with problems because the schedule doesn't control the execution time, and the workaround for this is the Parallel execution: here
Even after trying this, I still have the problem and even missed one call on date: 2023-02-15 21:03:11
Current Time = 2023-02-15 21:03:09.996591 <Thread(Thread-196 (test), started 123145329221632)>
Current Time = 2023-02-15 21:03:10.999913 <Thread(Thread-197 (test), started 123145329221632)>
Current Time = 2023-02-15 21:03:12.000702 <Thread(Thread-198 (test), started 123145329221632)>
Current Time = 2023-02-15 21:03:13.002731 <Thread(Thread-199 (test), started 123145329221632)>
Can someone help me with this? I would appreciate it so much.
my code:
import time
from datetime import datetime, timedelta
import threading
def test():
current_time = datetime.now()
print("Current Time =", current_time, threading.current_thread())
def run_threaded(job_func):
job_thread = threading.Thread(target=job_func)
job_thread.start()
schedule.every(1).seconds.do(run_threaded, test)
if __name__ == "__main__":
while True:
schedule.run_pending()
time.sleep(1)
I made the following ,but it doesn't print the time.
from apscheduler.schedulers.background import BackgroundScheduler
from datetime import datetime
def tick():
print('Tick! The time is: %s' % datetime.now())
scheduler = BackgroundScheduler()
scheduler.add_job(tick,'interval',seconds=3)
print('starting')
scheduler.start()
print('stopped')
This is because your program is exiting before the interval has elapsed and needs to be kept alive at least until the first interval, consider using the following example:
while True:
#Thread activity here (time.sleep(2) for example)
or using other forms of activity to keep your main thread alive. Or just print out the time without this scheduling if that's what you really need.
I am using background scheduler to schedule my jobs. When I am executing python script in the console the print statements are not executed. Is the scheduler being terminated? Below is my sample code
from apscheduler.schedulers.background import BackgroundScheduler
def my_task1():
print("Task 1")
def ny_task2():
print("Task 2")
if __name__=='__main__':
scheduler = BackgroundScheduler()
scheduler.add_job(my_task1, 'cron', id='my_task1', seconds=5)
scheduler.add_job(my_task1, 'cron', id='my_task1', seconds=10)
scheduler.start()
When I run the following script in the command line. I am not able to see the print statements in the console. Am I missing something?
You have selected a scheduler that runs in a background thread. Then you let the script exit. This is why nothing happens. The jobs have not had any time to be executed. Use BlockingScheduler instead if you want to keep the script running.
You can use while loop to keep it alive
from apscheduler.schedulers.background import BackgroundScheduler
def my_task1():
print("Task 1")
def ny_task2():
print("Task 2")
if __name__=='__main__':
scheduler = BackgroundScheduler()
scheduler.add_job(my_task1, 'cron', id='my_task1', seconds=5)
scheduler.add_job(my_task1, 'cron', id='my_task1', seconds=10)
scheduler.start()
while True:
time.sleep(1)
The job_function isnt getting executed even once, even when i waited for more than 10 mins.
from apscheduler.schedulers.background import BackgroundScheduler
import send_mail
def job_function():
print("Hello World")
send_mail('abc#test.com')
sched = BackgroundScheduler()
sched.add_job(job_function, 'interval', minutes=1)
sched.start()
Based on this code, and this code only, the problem looks like that your program terminates before the time limit is reached.
Try adding this infinite loop at the end of your program which will prevent it to quit:
while True:
time.sleep(1000)
Then terminate your program with CTRL+C.
I'm working on a Django app. I have an API endpoint, which if requested, must carry out a function that must be repeated a few times (until a certain condition is true). How I'm dealing with it right now is -
def shut_down(request):
# Do some stuff
while True:
result = some_fn()
if result:
break
time.sleep(2)
return True
While I know that this is a terrible approach and that I shouldn't be blocking for 2 seconds, I can't figure out how to get around it.
This works, after say a wait of 4 seconds. But I'd like something that keeps the loop running in the background, and stop once some_fn returns True. (Also, it is certain that some_fn will return True)
EDIT -
Reading Oz123's response gave me an idea which seems to work. Here's what I did -
def shut_down(params):
# Do some stuff
# Offload the blocking job to a new thread
t = threading.Thread(target=some_fn, args=(id, ), kwargs={})
t.setDaemon(True)
t.start()
return True
def some_fn(id):
while True:
# Do the job, get result in res
# If the job is done, return. Or sleep the thread for 2 seconds before trying again.
if res:
return
else:
time.sleep(2)
This does the job for me. It's simple but I don't know how efficient multithreading is in conjunction with Django.
If anyone can point out pitfalls of this, criticism is appreciated.
For many small projects celery is overkill. For those projects you can use schedule, it's very easy to use.
With this library you can make any function execute a task periodically:
import schedule
import time
def job():
print("I'm working...")
schedule.every(10).minutes.do(job)
schedule.every().hour.do(job)
schedule.every().day.at("10:30").do(job)
schedule.every().monday.do(job)
schedule.every().wednesday.at("13:15").do(job)
while True:
schedule.run_pending()
time.sleep(1)
The example runs in a blocking manner, but if you look in the FAQ, you will find that you can also run tasks in a parallel thread, such that you are not blocking, and remove the task once not needed anymore:
import threading
import time
from schedule import Scheduler
def run_continuously(self, interval=1):
"""Continuously run, while executing pending jobs at each elapsed
time interval.
#return cease_continuous_run: threading.Event which can be set to
cease continuous run.
Please note that it is *intended behavior that run_continuously()
does not run missed jobs*. For example, if you've registered a job
that should run every minute and you set a continuous run interval
of one hour then your job won't be run 60 times at each interval but
only once.
"""
cease_continuous_run = threading.Event()
class ScheduleThread(threading.Thread):
#classmethod
def run(cls):
while not cease_continuous_run.is_set():
self.run_pending()
time.sleep(interval)
continuous_thread = ScheduleThread()
continuous_thread.setDaemon(True)
continuous_thread.start()
return cease_continuous_run
Scheduler.run_continuously = run_continuously
Here is an example for usage in a class method:
def foo(self):
...
if some_condition():
return schedule.CancelJob # a job can dequeue it
# can be put in __enter__ or __init__
self._job_stop = self.scheduler.run_continuously()
logger.debug("doing foo"...)
self.foo() # call foo
self.scheduler.every(5).seconds.do(
self.foo) # schedule foo for running every 5 seconds
...
# later on foo is not needed any more:
self._job_stop.set()
...
def __exit__(self, exec_type, exc_value, traceback):
# if the jobs are not stop, you can stop them
self._job_stop.set()
This answer expands on Oz123's answer a little bit.
In order to get things working, I created a file called mainapp/jobs.py to contain my scheduled jobs. Then, in my apps.py module, I put from . import jobs in the ready method. Here's my entire apps.py file:
from django.apps import AppConfig
import os
class MainappConfig(AppConfig):
name = 'mainapp'
def ready(self):
from . import jobs
if os.environ.get('RUN_MAIN', None) != 'true':
jobs.start_scheduler()
(The RUN_MAIN check is because python manage.py runserver runs the ready method twice—once in each of two processes—but we only want to run it once.)
Now, here's what I put in my jobs.py file. First, the imports. You'll need to import Scheduler, threading and time as below. The F and UserHolding imports are just for what my job does; you won't import these.
from django.db.models import F
from schedule import Scheduler
import threading
import time
from .models import UserHolding
Next, write the function you want to schedule. The following is purely an example; your function won't look anything like this.
def give_admin_gold():
admin_gold_holding = (UserHolding.objects
.filter(inventory__user__username='admin', commodity__name='gold'))
admin_gold_holding.update(amount=F('amount') + 1)
Next, monkey-patch the schedule module by adding a run_continuously method to its Scheduler class. Do this by using the below code, which is copied verbatim from Oz123's answer.
def run_continuously(self, interval=1):
"""Continuously run, while executing pending jobs at each elapsed
time interval.
#return cease_continuous_run: threading.Event which can be set to
cease continuous run.
Please note that it is *intended behavior that run_continuously()
does not run missed jobs*. For example, if you've registered a job
that should run every minute and you set a continuous run interval
of one hour then your job won't be run 60 times at each interval but
only once.
"""
cease_continuous_run = threading.Event()
class ScheduleThread(threading.Thread):
#classmethod
def run(cls):
while not cease_continuous_run.is_set():
self.run_pending()
time.sleep(interval)
continuous_thread = ScheduleThread()
continuous_thread.setDaemon(True)
continuous_thread.start()
return cease_continuous_run
Scheduler.run_continuously = run_continuously
Finally, define a function to create a Scheduler object, wire up your job, and call the scheduler's run_continuously method.
def start_scheduler():
scheduler = Scheduler()
scheduler.every().second.do(give_admin_gold)
scheduler.run_continuously()
I recommend you use Celery's task management. You can refer this to set up this app (package if you're from javaScript background).
Once set, you can alter the code to:
#app.task
def check_shut_down():
if not some_fun():
# add task that'll run again after 2 secs
check_shut_down.delay((), countdown=3)
else:
# task completed; do something to notify yourself
return True
I can't comment on oz123's (https://stackoverflow.com/a/44897678/1108505) and Tanner Swett's (https://stackoverflow.com/a/60244694/5378866) excellent post, but as a final note I wanted to add that if you use Gunicorn and you have X number of workers, the section:
from django.apps import AppConfig
import os
class MainappConfig(AppConfig):
name = 'mainapp'
def ready(self):
from . import jobs
if os.environ.get('RUN_MAIN', None) != 'true':
jobs.start_scheduler()
will be executed that same number of times, launching X schedulers at the same time.
If we only want it to run only one instance (for example if you're going to create objects in the database), we would have to add in our gunicorn.conf.py file something like this:
def on_starting(server):
from app_project import jobs
jobs.start_scheduler()
And finally in the gunicorn call add the argument --preload
Here is my solution, with sources noted. This function will allow you to create a scheduler that you can start with your app, then add and subtract jobs at will. The check_interval variable allows you to trade-off between system resources and job execution timing.
from schedule import Scheduler
import threading
import warnings
import time
class RepeatTimer(threading.Timer):
"""Add repeated run of target to timer functionality. Source: https://stackoverflow.com/a/48741004/16466191"""
running: bool = False
def __init__(self, *args, **kwargs):
threading.Timer.__init__(self, *args, **kwargs)
def start(self) -> None:
"""Protect from running start method multiple times"""
if not self.running:
super(RepeatTimer, self).start()
self.running = True
else:
warnings.warn('Timer is already running, cannot be started again.')
def cancel(self) -> None:
"""Protect from running stop method multiple times"""
if self.running:
super(RepeatTimer, self).cancel()
self.running = False
else:
warnings.warn('Timer is already canceled, cannot be canceled again.')
def run(self):
"""Replace run method of timer to run continuously"""
while not self.finished.wait(self.interval):
self.function(*self.args, **self.kwargs)
class ThreadedScheduler(Scheduler, RepeatTimer):
"""Non-blocking scheduler. Advice taken from: https://stackoverflow.com/a/50465583/16466191"""
def __init__(
self,
run_pending_interval: float,
):
"""Initialize parent classes"""
Scheduler.__init__(self)
super(RepeatTimer, self).__init__(
interval=run_pending_interval,
function=self.run_pending,
)
def print_work(what_to_say: str):
print(what_to_say)
if __name__ == '__main__':
my_schedule = ThreadedScheduler(run_pending_interval=1)
job1 = my_schedule.every(1).seconds.do(print_work, what_to_say='Did_job1')
job2 = my_schedule.every(2).seconds.do(print_work, what_to_say='Did_job2')
my_schedule.cancel()
my_schedule.start()
time.sleep(7)
my_schedule.cancel_job(job1)
my_schedule.start()
time.sleep(7)
my_schedule.cancel()