I am trying to create a while loop which will iterate between 2 time objects, while datetime.datetime.now().time() <= datetime.datetime.now() +relativedelta(hour=1): but on every n minutes or second interval. So if the starting time was 1:00 AM, the next iteration should begin at 1:05 AM with n being 5 mins. So the iteration should begin after 5 mins of the start time and not for example, from the end of an iteration, which is the case when using sleep. Could you please advise how this could be accomplished?
A possible solution to this was from here:
write python script that is executed every 5 minutes
import schedule
import time
def func():
print("this is python")
schedule.every(5).minutes.do(func)
while True:
schedule.run_pending()
time.sleep(1)
With this, the start time has to be 1 am. Secondly, what if the program needs to run say 5 min + 1. In that case a 6 min interval wont work.
Although schedule library has lots of capabilities, I think the following code will help you get what you want. you can simply change start_time, relativedelta and iteration_time
import time
import datetime
start_time = datetime.datetime(year=2022, month=4, day=5, hour=1, minute=00, second=00)
relativedelta = datetime.timedelta(hours=1)
iteration_time = datetime.timedelta(minutes=5)
end_time = start_time + relativedelta
last_run = None
def func():
print("this is python")
while True:
current_time = datetime.datetime.now()
if start_time <= current_time <= end_time:
if last_run:
if current_time >= last_run + iteration_time:
func()
last_run = current_time
else:
last_run = current_time
elif current_time > end_time:
break
time.sleep(1)
this code prints (this is python) each 5 minutes (iteration_time) from 4/5/2022 1:00:00AM (start_time) for 1 hour (relativedelta)
This can be achieved with a manual implementation.
Essentially, you would need to loop continuously until you reach the "active" time window. Once there, you basically execute your function and refuse to run again until the specified execution interval has passed. The main loop does not need to be executed as often as possible, but it is enough to run it once in a while as long as this is reasonably smaller than the execution interval. This effectively a way of limiting execution rate (throttling). Also, the execution time of the function func should be smaller than the interval, otherwise one or more executions are skipped.
import datetime
import time
def repeat_between(
start_dt,
stop_dt,
interval_td,
func,
func_args=None,
func_kws=None,
collect_results=True,
throttling_s=1):
# ensure valid `func_args` and `func_kws`
func_args = () if func_args is None else tuple(func_args)
func_kws = {} if func_kws is None else dict(func_kws)
# initialize current datetime and last run
curr_dt = datetime.datetime.now()
last_run = None
# ensure the start datetime is:
# - before the stop datetime
# - after the current datetime
if stop_dt < start_dt < curr_dt:
return
else:
# collect results here
result = []
# wait until reaching the start datetime
wait_td = (start_dt - curr_dt)
time.sleep(wait_td.total_seconds())
# loop until current datetime exceeds the stop datetime
while curr_dt <= stop_dt:
# if current time is
# - past the start datetime
# - near an interval timedelta
if curr_dt >= start_dt and \
(not last_run or curr_dt >= last_run + interval_td):
curr_result = func(*func_args, **func_kws)
if collect_results:
result.append(curr.result)
last_run = curr_dt
# wait some time before checking again
if throttling_s > 0:
time.sleep(throttling_s)
# update current time
curr_dt = datetime.datetime.now()
To test this, one could use for example:
r = repeat_between(
datetime.datetime.now() + datetime.timedelta(seconds=3),
datetime.datetime.now() + datetime.timedelta(seconds=10),
datetime.timedelta(seconds=2),
func=lambda: (datetime.datetime.now(), 'Hello!'),
throttling_s=0.1
)
print(r)
# [(datetime.datetime(2022, 4, 8, 15, 38, 21, 525347), 'Hello!'),
# (datetime.datetime(2022, 4, 8, 15, 38, 23, 530025), 'Hello!'),
# (datetime.datetime(2022, 4, 8, 15, 38, 25, 534628), 'Hello!'),
# (datetime.datetime(2022, 4, 8, 15, 38, 27, 539120), 'Hello!')]
I believe this could be considered an object-oriented "canonical" solution which creates a Thread subclass instance that will call a specified function repeatedly every datetime.timedelta units until canceled. The starting and how long it's left running are not details the class concerns itself with, and are left to the code making use of the class to determine.
Since most of the action occurs in a separate thread, the main thread could be doing other things concurrently, if desired.
import datetime
from threading import Thread, Event
import time
from typing import Callable
class TimedCalls(Thread):
"""Call function again every `interval` time duration after it's first run."""
def __init__(self, func: Callable, interval: datetime.timedelta) -> None:
super().__init__()
self.func = func
self.interval = interval
self.stopped = Event()
def cancel(self):
self.stopped.set()
def run(self):
next_call = time.time()
while not self.stopped.is_set():
self.func() # Target activity.
next_call = next_call + self.interval
# Block until beginning of next interval (unless canceled).
self.stopped.wait(next_call - time.time())
def my_function():
print(f"this is python: {time.strftime('%H:%M:%S', time.localtime())}")
# Start test a few secs from now.
start_time = datetime.datetime.now() + datetime.timedelta(seconds=5)
run_time = datetime.timedelta(minutes=2) # How long to iterate function.
end_time = start_time + run_time
assert start_time > datetime.datetime.now(), 'Start time must be in future'
timed_calls = TimedCalls(my_function, 10) # Thread to call function every 10 secs.
print(f'waiting until {start_time.strftime("%H:%M:%S")} to begin...')
wait_time = start_time - datetime.datetime.now()
time.sleep(wait_time.total_seconds())
print('starting')
timed_calls.start() # Start thread.
while datetime.datetime.now() < end_time:
time.sleep(1) # Twiddle thumbs while waiting.
print('done')
timed_calls.cancel()
Sample run:
waiting until 11:58:30 to begin...
starting
this is python: 11:58:30
this is python: 11:58:40
this is python: 11:58:50
this is python: 11:59:00
this is python: 11:59:10
this is python: 11:59:20
this is python: 11:59:30
this is python: 11:59:40
this is python: 11:59:50
this is python: 12:00:00
this is python: 12:00:10
this is python: 12:00:20
done
Did you try time.sleep(300)
where 300 is seconds.
if you want your program to run every 5 minuets you can use time.sleep
import time
while true:
#program
time.sleep(300)
if you want to iterate between dates use this template:
from datetime import timedelta
start_date = date_utils.parse('2021-01-01')
end_date = datetime.datetime.now()
while start_date <= end_date:
one_hour = timedelta(hours=1)
one_minute = timedelta(minutes=1)
start_date = start_date + datetime.timedelta(days=1)
Import threading module
import threading
Import time module to implement sleep functionality
import time
Import datetime to get start time, end time and elapsed time
from datetime import datetime
#Function to loop 100000 times
def loop_fun(name):
for i in range(0, 10):
time.sleep(0.0001)
print(name)
print("Loop executed!")
#Function to execute loop_fun without multithreading
num =0
def without_thread(num):
starting_time = datetime.now()
for i in range(0, num):
loop_fun("Called from without_thread")
ending_time = datetime.now()
elapsed_time = (ending_time - starting_time).total_seconds()
print("\n\nTime Elapsed without_thread: "+ str(elapsed_time)+"\n\n\n\n\n")
#Function to execute loop_fun with multithreading"
def with_thread(num):
start_time = datetime.now()
threads_lst = []
# Creating threads to call loop_fun
for i in range(0, num):
threads_lst.append(threading.Thread(target=loop_fun, args=("Called from with_thread",)))
# Running threads
for threads_ in threads_lst:
threads_.start()
# Waiting for completion of all threads
for threads_ in threads_lst:
threads_.join()
end_time = datetime.now()
elapsed_time = (end_time - start_time).total_seconds()
print("\n\nTime Elapsed with_thread: "+ str(elapsed_time)+"\n\n\n\n\n")
if __name__ == "__main__":
without_thread(10)
with_thread(10)
Your indentation is messed up. Copy/paste issue or does your code look like that ?
Missing Indentation in Python makes it hard to guess what you are trying to achieve.
Nevertheless I tried and came up with this code working w/o errors. Does it do what you want ? I frankly don't know.
I added these lines, which got messed up by the forum I think
from datetime import datetime
import time
import threading
I "fixed" indentation
#Function to loop 100000 times
def loop_fun(name):
for i in range(0, 10):
time.sleep(0.0001)
print(name)
print("Loop executed!")
#Function to execute loop_fun without multithreading
num =0
def without_thread(num):
starting_time = datetime.now()
for i in range(0, num):
loop_fun("Called from without_thread")
ending_time = datetime.now()
elapsed_time = (ending_time - starting_time).total_seconds()
print("\n\nTime Elapsed without_thread: "+ str(elapsed_time)+"\n\n\n\n\n")
#Function to execute loop_fun with multithreading"
def with_thread(num):
start_time = datetime.now()
threads_lst = []
# Creating threads to call loop_fun
for i in range(0, num):
threads_lst.append(threading.Thread(target=loop_fun, args=("Called from with_thread",)))
# Running threads
for threads_ in threads_lst:
threads_.start()
# Waiting for completion of all threads
for threads_ in threads_lst:
threads_.join()
end_time = datetime.now()
elapsed_time = (end_time - start_time).total_seconds()
print("\n\nTime Elapsed with_thread: "+ str(elapsed_time)+"\n\n\n\n\n")
if __name__ == "__main__":
without_thread(10)
with_thread(10)
It's printing stuff like that
Loop executed!
Called from without_thread
Loop executed!
Called from without_thread
Loop executed!
Time Elapsed without_thread: 6.378763
I have a program that logs some messages about data that I download. Besides that, I would like to display some stats about the requests with every k-requests that I make to a site (k is 10 in my case) + some overall stats at the end of the execution.
At the moment I have an implementation that I am not happy with, as it uses global variables. I am looking for a cleaner alternative. It looks like this (Note: please ignore the fact that I am using print instead of logging and that I am measuring the passing of time using time.time instead of time.perf_counter (read here that the latter would be a better option):
import time
import pprint
def f2(*args, **kwargs):
global START_TIME
global NO_REQUESTS
global TOTAL_TIME_FOR_REQUESTS
global MAX_TIME_FOR_REQUEST
global AVERAGE_TIME_FOR_REQUESTS
global TOTAL_TIME_FOR_DECODING
global TOTAL_TIME_FOR_INTERSECT
# ... logic that changes values of most of these global variables
if NO_REQUESTS % 10 == 0:
AVERAGE_TIME_FOR_REQUESTS = TOTAL_TIME_FOR_REQUESTS / NO_REQUESTS
print()
print('no requests so far: ' + str(NO_REQUESTS))
print('average request time: {:.2f}s'.format(AVERAGE_TIME_FOR_REQUESTS))
print('max request time: {:.2f}s'.format(MAX_TIME_FOR_REQUEST))
elapsed = time.time() - START_TIME
hours_elapsed = elapsed // 3600
minutes_elapsed = (elapsed % 3600) // 60
seconds_elapsed = ((elapsed % 3600) % 60)
print('time elapsed so far: {}h {}m {:.2f}s'.format(hours_elapsed, minutes_elapsed, seconds_elapsed))
print()
time5 = time.time()
decoded = some_module.decode(res.content)
time6 = time.time()
elapsed2 = time6 - time5
TOTAL_TIME_FOR_DECODING += elapsed2
return something
def f1(*args, **kwargs):
global START_TIME
global TOTAL_TIME_FOR_REQUESTS
TOTAL_TIME_FOR_REQUESTS = 0
global MAX_TIME_FOR_REQUEST
MAX_TIME_FOR_REQUEST = 0
global NO_REQUESTS
NO_REQUESTS = 0
global AVERAGE_TIME_FOR_REQUESTS
AVERAGE_TIME_FOR_REQUESTS = 0
global TOTAL_TIME_FOR_DECODING
TOTAL_TIME_FOR_DECODING = 0
global TOTAL_TIME_FOR_INTERSECT
TOTAL_TIME_FOR_INTERSECT = 0
f2() # notice call to other function!
# ... some logic
return some_results
def output_final_stats(elapsed, results, precision='{:.3f}'):
print()
print('=============================')
hours_elapsed = elapsed // 3600
minutes_elapsed = (elapsed % 3600) // 60
seconds_elapsed = ((elapsed % 3600) % 60)
print("TIME ELAPSED: {:.3f}s OR {}h {}m {:.3f}s".format(
elapsed, hours_elapsed, minutes_elapsed, seconds_elapsed))
print("out of which:")
# print((precision+'s for requests)'.format(TOTAL_TIME_FOR_REQUESTS)))
print('{:.3f}s for requests'.format(TOTAL_TIME_FOR_REQUESTS))
print('{:.3f}s for decoding'.format(TOTAL_TIME_FOR_DECODING))
print('{:.3f}s for intersect'.format(TOTAL_TIME_FOR_INTERSECT))
total = TOTAL_TIME_FOR_REQUESTS + TOTAL_TIME_FOR_DECODING + TOTAL_TIME_FOR_INTERSECT
print('EXPECTED: {:.3f}s'.format(total))
print('DIFF: {:.3f}s'.format(elapsed - total))
print()
print('AVERAGE REQUEST TIME: {:.3f}s'.format(AVERAGE_TIME_FOR_REQUESTS))
print('TOTAL NO. REQUESTS: ' + str(NO_REQUESTS))
print('MAX REQUEST TIME: {:.3f}s'.format(MAX_TIME_FOR_REQUEST))
print('TOTAL NO. RESULTS: ' + str(len(results)))
pprint('RESULTS: {}'.format(results), indent=4)
if __name__ == '__main__':
START_TIME = time.time()
results = f1(some_params)
final_time = time.time()
elapsed = final_time - START_TIME
output_final_stats(elapsed, results)
The way I thought of it (not sure if the best option, open to alternatives) is to somehow have a listener on the NO_REQUESTS variable and whenever that number reaches a multiple of 10 trigger the logging of the variables that I am interested in. Nonetheless, where would I store those variables, what would be their namespace?
Another alternative would be to maybe have a parametrised decorator for one of my functions, but in this case I am not sure how easy it would be to pass the values that I am interested in from one function to another.
I think the cleanest way is to use a parametrized class decorator.
class LogEveryN:
def __init__(self, n=10):
self.n = n
self.number_of_requests = 0
self.total_time_for_requests = 0
self.max_time_for_request = 0
self.average_time_for_request = 0
def __call__(self, func, *args, **kwargs):
def wrapper(*args, **kwargs):
self.number_of_request += 1
if self.number_of_request % self.n:
# Do your computation and logging
return func(*args, **kwargs)
return wrapper
#LogEveryN(n=5)
def request_function():
pass
I have a function adding subscription for my users. At the end of this function,
an instruction check the subscriptions and do some stuff with patron if necessary.
For my testing, I need to skip this last instruction ; so I tried to use mock.patch
but despite all my tries, the code is always executed (and my tests failed)
mymodule/classes.py
class User:
def add_subscription(self, start_date, end_date):
subscriptions = self.get('subscriptions', [])
subscriptions.append({'start': start_date, 'end': end_date})
self['subscriptions'] = subscriptions
check_subscriptions(self) # <-- skip this instruction for unitesting
def check_subscriptions(user):
print("Not need to print for unitesting")
# next doing more stuff...
...
tests/user_unitest.py
import mock
from mymodule.classes import User
from datetime import datetime, timedelta
...
def test_subscriptions(user_with_no_subscription_fixture):
user = user_with_no_subscription_fixture
start = datetime.now()
end = start + timedelta(days=10)
user.add_subscription(start, end)
...
How can I use the #mock.patch (or other mock function) to don't enter into the
User.check_subscriptions methods ?
What I tried (and didn't work)
#mock.patch('mymodule.classes.check_subscriptions')
def test_subscriptions(user_with_no_subscription_fixture):
user = user_with_no_subscription_fixture
start = datetime.now()
end = start + timedelta(days=10)
user.add_subscription(start, end)
def test_subscriptions(user_with_no_subscription_fixture):
with mock.patch('mymodule.classes.check_subscriptions'):
user = user_with_no_subscription_fixture
start = datetime.now()
end = start + timedelta(days=10)
user.add_subscription(start, end)
#mock.patch('mymodule.classes.check_subscriptions', MagicMock())
def test_subscriptions(user_with_no_subscription_fixture):
user = user_with_no_subscription_fixture
start = datetime.now()
end = start + timedelta(days=10)
user.add_subscription(start, end)
#mock.patch('mymodule.classes.check_subscriptions', side_effect=...)
def test_subscriptions(user_with_no_subscription_fixture):
user = user_with_no_subscription_fixture
start = datetime.now()
end = start + timedelta(days=10)
user.add_subscription(start, end)
thanks for your always usefull help
import time #useful for measuring code execution
class StopWatch:
def __init__(self, startTime = 0, endTime = 0, elapsedTime = 0):
self.__startTime = startTime
self.__endTime = endTime
self.__elapsedTime = elapsedTime
def start(self):
self.__startTime = time.clock()
def stop(self):
return self.getElapsedTime()
def reset(self):
self.__startTime = 0
self.__elapsedTime = 0
def getstarttime(self):
return self.__startTime
def getendtime(self):
return self.__endTime
def getElapsedTime(self):
elapsedTime = self.__elapsedTime
elapsedTime +=((time.clock() - self.__startTime) * 1000)
return elapsedTime
def main():
x = StopWatch()
x.start
a = time.clock() #code only works with this line of code in place (I don't understand why?)
sum = 0
for i in range(1 , 10000000):
sum += i
x.stop
print("Elapsed execution time is", x.getElapsedTime())
print(sum)
x.reset
main()
The code fails to produce the correct result if I remove the
a = time.clock()
portion. With that in place it produces the correct result but I am not really sure why it does this?
I realize there may be better ways to do this, but Im kind of a beginner at Python so I'd appreciate the help. Thanks! I am using a Windows system.
You wouldn't happen to be a rubyist, would you? x.start works to call methods in Ruby, but not in Python. You need x.start() - notice the parentheses. You have the same problem with x.stop and x.reset.
a = time.clock() is helping because time.clock() will sometimes (platform-dependent) return the time since the first call to clock(), instead of from process start. The actual assignment to a isn;t doing anything, it's simply creating a start point for clock to reference later. Don't rely on this - the Python docs state "Return the CPU time or real time since the start of the process or since the first call to clock()."