Develop Reference

python 3.7 sched running sleep to neg value

Revised question
Is there a way by using treading and sched to create a for loop where you run a script exactly every second jumping between 3 treads constantly for as long as you need it Like
for thread01 in schead:
run at 00:00:01.0000
else(thread02):
run at 00:00:02.0000
else(thread03):
run at 00:00:03.0000
and have it constantly loop does mitigating the drift and neg sleep value as each time the code runs on a tread it has 1000 milliseconds to execute before the next second
I have tested and the session_create runs between 480 and 530 milliseconds constantly to execute
Old Question old Code to help mitigate time drift
from main import session_create
import sched, datetime, threading
def timestamp():
next_call = time.time()
while True:
session_create()
print (datetime.datetime.now())
nextcall = next_call+1;
time.sleep(nextcall - time.time())
timerThread = threading.Thread(target=timestamp)
timerThread.daemon = True
timerThread.start()

This was the closest I could get to what i wanted as I knew that the code that needed tobe called and run every second, and I knew it takes between 500 to 600 milliseconds to run.
I used threading.Timer to run the code every 900 milliseconds and this causes the code to run at least once every second and bypassing the time drift i was experiencing.
from Sessions import session
import sched, time, threading
Event_scheduler = sched.scheduler(time.time, time.sleep)
##
start_time = time.time()
def run():
threading.Timer(0.9, run).start()
session()
run()
The down side of this is simple math every Nth second ill have a point where the code ran twice (example 23:59:01:0050 and again at 23:59:01:0950). But the extra data is easily removed by finding the duplicates 23:59:01 == 23:59:01 and removing one of the two

Round robin scheduler - call functions periodically at appropriate times

I have a list of functions (1..N) and function i needs to be called every X_i seconds (X_i would be large such as 1000+ s). Each X_i doesn't have to be unique, i.e. it is possible that X_i == X_j.
Provided, I generate a list of (function_i, X_i) how can I simply execute these functions at their appropriate times in the future and sleep between calls? I have used ApScheduler before but it runs tasks in parallel and I need functions to be run one after the other.
I can write my own iterator which returns the current function that needs to be executed and blocks until the next one but I'd rather use a library if one exists?
EDIT: N is about 200 at the moment.

threading module
The threading module lets you start a new thread, which will not be affected by other threads' sleep statements. This requires N threads, so if N is extremely huge, let me know and I will try to think of an alternative solution.
You can create N threads and set each one on a timed loop, like so:
import threading, time
def looper(function, delay): # Creates a function that will loop that function
def inner(): # Will keep looping once invoked
while True:
function() # Call the function; you can optionally add args
time.sleep(delay) # Swap this line and the one before it to wait before running rather than after
return inner # The function that should be called to start the loop is returned
def start(functions, delays): # Call this with the two lists to start the loops
for function, delay in zip(functions, delays): # Goes through the respective pairs
thread = threading.Thread(target = looper(function, delay)) # This thread will start the looper
thread.start()
start([lambda: print("hi"), lambda: print("bye")], [0.2, 0.3])
You can try it online here; just hit run and then hit run again when you want to kill it (Thanks to #DennisMitchell for the online interpreter)

Python Scheduler vs loop + sleep

the following is:
python sched:
from time import time, sleep
from sched import scheduler
def daemon(local_handler):
print 'hi'
local_handler.enter(3, 1, daemon, (local_handler,))
if __name__ == '__main__':
handler = scheduler(time, sleep)
handler.enter(0, 1, daemon, (handler,))
handler.run()
python loop + sleep:
from time import sleep
while True:
print 'hello'
sleep(3)
What is the difference between sched and loop+sleep, and sched will stop when the system time is changed?

A big difference is that the delay between multiple tasks is calculated as necessary. That means your loop will take:
time it needs to print("hello") or do the task that you need to do
time it takes to sleep(3)
while if you change the order in your scheduler to:
local_handler.enter(3, 1, daemon, (local_handler,))
do_the_task
your next task will be run either after 3 seconds, or immediately after do_the_task if it took longer than 3 seconds.
So the decision really comes down to: do you want your task executed every X time units, or with X time units space between executions.
Assuming you're using the typical (time, sleep) parameters, if the system time is changed, you'll get the next task run after the expected amount of time (sleep takes care of this, unless some signals were received in the meantime), but your next scheduled task time will be shifted. I believe that the next execution time will not be what you'd normally expect.

The difference between the two is that scheduler is more pythonic than loop + sleep for two reasons: elegance and modularity.
Long loops easily become difficult to read and require a lot more code to be written within. However, with a scheduler, a specific function can be called on a delay, containing all of the code within. This makes code much more readable and allows for moving code into classes and modules to be called within the main loop.
Python knows what the current time is by checking the local system. If the local system's time is changed, then that will affect a currently running program or script.

Becaused the python sched is use system time for next iteration.
The sleep is use cpu time clock for next iteration.

Python/Django: Schedule task in realtime after every X duration (secs/mins/hours?)

I want to execute some task (function) within my Django application at a specified duration from when a call is made to that. Something like:
... some code
async_run_func(time_interval=15_mins) # Async call. Code within the function
# should be executed after 15 mins.
... some more code
async_run_func is to be executed after some custom interval.
What is the correct approach to achieve this? One way is to create a separate thread and sleep it for time_duration period. But that will result into too many threads on the server. Also, in case the gunicorn process is restarted, the state will be lost. I want the information to persistent. So, I do not want to go with this approach. Currently I am using celery for executing long async and periodic tasks. But celery do not allow option to run a function single time after the specified duration.
It will be great if there is anyway to do it on distributed system. For example, function will be call from one system but the code to be executed on other system (use of queue like RabbitMQ is fine with me). Else, I can also go for executing it on the same machine. Any suggestion?

Celery has the option of enqueuing at a specific time:
your_async_function.apply_async(args=(your, args, tuple),
kwargs={your: kwargs},
countdown=15 * 60)
Or use the subtask syntax, to curry all args and then delay
your_async_function.s(your, args, tuple, your: kwargs).delay(countdown=15 * 60)
If the function has no args, you can skip them and do directly
your_async_function.delay(countdown=15 * 60)

What about using sched module? Simple and efficient.
import sched, time
sc = sched.scheduler(time.time, time.sleep)
sc.enter(15, 1, async_run_func, ())
sc.run

ETA and Countdown are options to perform this using django-celery.
From the document:
The ETA (estimated time of arrival) lets you set a specific date and time that is the earliest time at which your task will be executed. countdown is a shortcut to set ETA by seconds into the future.
For example:
>>> result = add.apply_async((2, 2), countdown=3)
>>> result.get() # this takes at least 3 seconds to return
20
The task is guaranteed to be executed at some time after the specified date and time, but not necessarily at that exact time. Possible reasons for broken deadlines may include many items waiting in the queue, or heavy network latency. To make sure your tasks are executed in a timely manner you should monitor the queue for congestion.
While countdown is an integer, eta must be a datetime object, specifying an exact date and time (including millisecond precision, and timezone information):
>>> from datetime import datetime, timedelta
>>> tomorrow = datetime.utcnow() + timedelta(days=1)
>>> add.apply_async((2, 2), eta=tomorrow)

Use sched module to run at a given time

I'm working on a python script that needs to run between two given times. I'm required to use the build in sched module as this script needs to be able to run directly on any machine that has python 2.7 as to reduce configuration time. (SO CRON IS NOT AN OPTION)
A few variables define the settings for the time to run, here set_timer_start=0600 and set_timer_end=0900 are written in HHMM. I'm able to stop the script at the right time.
I don't know exactly how sched works (the python doc page doesn't make to much sense to me), but as far as I understand It runs at a date/time (epoch) while I only want it to run at a given time (HHMM).
Can anyone give me an example (or link) on how to use the scheduler and perhaps calculate the next run date/time?

If I got your requirements right, what you need is probably a loop, that will re-enter a task in the queue every time it will be executed. Something along the lines of:
# This code assumes you have created a function called "func"
# that returns the time at which the next execution should happen.
s = sched.scheduler(time.time, time.sleep)
while True:
if not s.queue(): # Return True if there are no events scheduled
time_next_run = func()
s.enterabs(time_next_run, 1, <task_to_schedule_here>, <args_for_the_task>)
else:
time.sleep(1800) # Minimum interval between task executions
However, using the scheduler is - IMO - overkilling. Using datetime objects could suffice, for example a basic implementation would look like:
from datetime import datetime as dt
while True:
if dt.now().hour in range(start, stop): #start, stop are integers (eg: 6, 9)
# call to your scheduled task goes here
time.sleep(60) # Minimum interval between task executions
else:
time.sleep(10) # The else clause is not necessary but would prevent the program to keep the CPU busy.
HTH!