Complete newbie here so bare with me. I've got a number of devices that report status updates to a singular location, and as more sites have been added, drift with time.sleep(x) is becoming more noticeable, and with as many sites connected now it has completely doubles the sleep time between iterations.
import time
...
def client_list():
sites=pandas.read_csv('sites')
return sites['Site']
def logs(site):
time.sleep(x)
if os.path.isfile(os.path.join(f'{site}/target/', 'hit')):
stamp = time.strftime('%Y-%m-%d,%H:%M:%S')
log = open(f"{site}/log", 'a')
log.write(f",{stamp},{site},hit\n")
log.close()
os.remove(f"{site}/target/hit")
else:
stamp = time.strftime('%Y-%m-%d,%H:%M:%S')
log = open(f"{site}/log", 'a')
log.write(f",{stamp},{site},miss\n")
log.close()
...
if __name__ == '__main__':
while True:
try:
client_list()
with concurrent.futures.ThreadPoolExecutor() as executor:
executor.map(logs, client_list())
...
I did try adding calculations for drift with this:
from datetime import datetime, timedelta
def logs(site):
first_called=datetime.now()
num_calls=1
drift=timedelta()
time_period=timedelta(seconds=5)
while 1:
time.sleep(n-drift.microseconds/1000000.0)
current_time = datetime.now()
num_calls += 1
difference = current_time - first_called
drift = difference - time_period* num_calls
if os.path.isfile(os.path.join(f'{site}/target/', 'hit')):
...
It ends up with a duplicate entries in the log, and the process still drifts.
Is there a better way to schedule the function to run every x seconds and account for the drift in start times?
Create a variable equal to the desired system time at the next interval. Increment that variable by 5 seconds each time through the loop. Calculate the sleep time so that the sleep will end at the desired time. The timings will not be perfect because sleep intervals are not super precise, but errors will not accumulate. Your logs function will look something like this:
def logs(site):
next_time = time.time() + 5.0
while 1:
time.sleep(time.time() - next_time)
next_time += 5.0
if os.path.isfile(os.path.join(f'{site}/target/', 'hit')):
# do something that takes a while
So I managed to find another route that doesn't drift. The other method still drifted over time. By capturing the current time and seeing if it is divisible by x (5 in the example below) I was able to keep the time from deviating.
def timer(t1,t2)
return True if t1 % t2 == 0 else False
def logs(site):
while 1:
try:
if timer(round(time.time(), 0), 5.0):
if os.path.isfile(os.path.join(f'{site}/target/', 'hit')):
# do something that takes a while
time.sleep(1) ''' this kept it from running again immediately if the process was shorter than 1 second. '''
...
Related
I have a question on how I am able to set the timer so that every time it exits the loop it sets the time back to 2 seconds. The problem is that the first time the sound works after 2 seconds, the next times it is executed immediately. Thank you very much in advance for any advice.
This is my code:
time = 2
while time > 0:
timer = datetime.timedelta(seconds=time)
time -= 1
duration = 1000
freq = 440
winsound.Beep(freq, duration)
I am not sure if you meant that, but for me it seems like you just want to wait 2 seconds before executing the next steps. You can do that like so:
import time
while True:
time.sleep(2) # waits 2 seconds
winsound.Beep(440, 1000)
Anyways I don't recommend you to use a plain infinite loop, without a break statement. Therefore I recommend you to add one, like down below.
import time
while True:
time.sleep(2) # waits 2 seconds
winsound.Beep(440, 1000)
if True: # break on a specific statment
break
Edit: As CrazyChucky mentioned in the comments, this approach should work fine in most of the cases, but it can end up being more than two seconds sometimes. Therefore you should work with timedeltas or take a look at scheduler.
To be more accurate as possible use:
import time
timer = 0
step = 2
t0 = time.time()
while True:
timer = time.time() - t0
wait = step - timer
time.sleep(wait)
print(time.time())
winsound.Beep(freq, duration)
t0 = time.time()
This script take in count the execution time of script lines for your computer.
You just have to reinitialize the time at the end of the loop
time = 2
while True:
timer = datetime.timedelta(seconds=time)
time -= 1
duration = 1000
freq = 440
if time == 0:
time = 2
break
winsound.Beep(freq, duration)
I'm making a program that runs something for the amount of minutes the user alots (it's an idle game in beta). I put on a timer for one minute and noticed that the program ran over the minute by a couple of seconds-- Not very noticable, but I was wondering if this is because of how long a loop takes to execute? This is my code:
import time
foreverloop = True
automodeOn = False
idleSec = 0
idleMin = 0
pages = 0
pps = 0
while foreverloop:
if automodeOn == False:
msg = input("BTCG Command >> ")
if msg == 'auto':
autotime = input("How long would you like to go idle for? Answer in minutes.")
automodeOn = True
elif msg == 'autoMORE':
pps += .5
else:
pages += pps
print("You have auto-read",pps,"pages.")
idleSec += 1
if idleSec == 60:
idleSec = 0
idleMin += 1
if idleMin == int(autotime):
print("Idle mode turning off.")
automodeOn = False
time.sleep(1)
You could measure the time it takes for a number of lines of code to execute by measuring the start time:
start = time.time()
before any number of lines you'd like to measure the time, then at the end adding:
end = time.time()
the time elapse is then calculated as their subtraction:
elapsed_time = end-start
I suggest that you read about code complexity, the most popular of which is the Big O notation.
edit: as denoted in a comment, timeit is the better option if you're looking to precisely measure the time it takes for a certain line or function to execute, the main difference between the 2 approaches is that timeit is made specifically for this purpose and as part of this takes as a parameter a variable number indicating the number of times the specified code is run before determining how long it takes on average to run.
Instead of making the program wait in adittion to the time it takes to execute, I would use time.time() to get the system's current UNIX time in seconds as a float and only continue if a certain time has passed:
import time
time_begin = time.time()
wait_time = 60 # seconds to wait
while time.time() < time_begin + wait_time:
# do logic
print("Time passed:", time.time() - time_begin)
time.sleep(1) # can be whatever
print(wait_time, "seconds has passed!")
I am trying to make a python script that works in a loop mode with iteration through a text file to run for periods of one hour and make 30minute pauses between each hour loop .
After some searching I found this piece of code :
import datetime
import time
delta_hour = 0
while:
now_hour = datetime.datetime.now().hour
if delta_hour != now_hour:
# run your code
delta_hour = now_hour
time.sleep(1800) # 1800 seconds sleep
# add some way to exit the infinite loop
This code has a few issues though :
It does not consider one hour periods since the script starts running
It does not seem to work continuously for periods over one hour
Considering what I am trying to achieve (running script 1hour before each time it pauses for 30mins) what is the best approach to this ? Cron is not an option here .
For clarification :
1hour run -- 30min pause -- repeat
Thanks
Here is a so simple code, I have written for teaching purposes, which is very clear
from datetime import datetime
class control_process():
def __init__(self, woking_period, sleeping_period):
self.woking_period = woking_period # working period in minutes
self.sleeping_period = sleeping_period # sleeping period in minutes
self.reset()
def reset(self):
self.start_time = datetime.utcnow() # set starting point
def manage(self):
m = (datetime.utcnow() - self.start_time).seconds / 60 # how long since starting point
if m >= self.woking_period: # if exceeded the working period
time.sleep(self.sleeping_period * 60) # time to sleep in seconds
self.reset() # then reset time again
return # go to continue working
cp = control_process(60, 30) # release for 60 minutes and sleep for 30 minutes
while True: # you code loop
cp.manage()
'''
your code
'''
in which 'control_processobject - I calledcp- callscp.manage()` inside your executing loop.
you reset time via cp.reset() before going in the loop or whenever you want
Based on Comments
The simplicity I mean is to add this class to your general library so you can use it whenever you want by instantiation of cp then one or two controlling functions 'cp.manage()` which control the working cycles, and cp.reset() if you want to use it in another location of the code. I believe that use a function is better than a long condition statement.
Using the default library you could do something like call the script itself using subprocess. By checking whether conditions are met the process could do a task and call itself. Extending the logic with a kill pill would make it stop (I leave that up to you).
import argparse, time
from subprocess import call
DELAY = 60 * 30 # minutes
WORK_TIME = 60 * 60 # minutes
parser = argparse.ArgumentParser()
parser.add_argument("-s",
help = "interval start time",
type = float,
default = time.time())
parser.add_argument("-t",
help = "interval stop time",
type = float,
default = time.time() + WORK_TIME)
def do_task():
# implement task
print("working..")
return
if __name__ == "__main__":
args = parser.parse_args()
start = args.s
stop = args.t
# work
if start < time.time() < stop:
do_task()
# shift target
else:
start = time.time() + DELAY
stop = start + WORK_TIME
call(f"python test.py -t {stop} -s {start}".split())
The simplest solution I could come up with was the following piece of code, which I added inside my main thread :
start_time = int(time())
... #main thread code
#main thread code end
if int(time() - start_time >= 60 * 60):
print("pausing time")
sleep(30 * 60)
start_time = int(time())
From the moment the script starts this will pause every hour for 30mins and resume afterwards .
Simple yet effective !
I'm reading serial data with a while loop. However, I have no control over the sample rate.
The code itself seems to take 0.2s to run, so I know I won't be able to go any faster than that. But I would like to be able to control precisely how much slower I sample.
I feel like I could do it using 'sleep', but the problem is that there is potential that at different points the loop itself will take longer to read(depending on precisely what is being transmitted over serial data), so the code would have to make up the balance.
For example, let's say I want to sample every 1s, and the loop takes anywhere from 0.2s to 0.3s to run. My code needs to be smart enough to sleep for 0.8s (if the loop takes 0.2s) or 0.7s (if the loop takes 0.3s).
import serial
import csv
import time
#open serial stream
while True:
#read and print a line
sample_value=ser.readline()
sample_time=time.time()-zero
sample_line=str(sample_time)+','+str(sample_value)
outfile.write(sample_line)
print 'time: ',sample_time,', value: ',sample_value
Just measure the time running your code takes every iteration of the loop, and sleep accordingly:
import time
while True:
now = time.time() # get the time
do_something() # do your stuff
elapsed = time.time() - now # how long was it running?
time.sleep(1.-elapsed) # sleep accordingly so the full iteration takes 1 second
Of course not 100% perfect (maybe off one millisecond or another from time to time), but I guess it's good enough.
Another nice approach is using twisted's LoopingCall:
from twisted.internet import task
from twisted.internet import reactor
def do_something():
pass # do your work here
task.LoopingCall(do_something).start(1.0)
reactor.run()
An rather elegant method is you're working on UNIX : use the signal library
The code :
import signal
def _handle_timeout():
print "timeout hit" # Do nothing here
def second(count):
signal.signal(signal.SIGALRM, _handle_timeout)
signal.alarm(1)
try:
count += 1 # put your function here
signal.pause()
finally:
signal.alarm(0)
return count
if __name__ == '__main__':
count = 0
count = second(count)
count = second(count)
count = second(count)
count = second(count)
count = second(count)
print count
And the timing :
georgesl#cleese:~/Bureau$ time python timer.py
5
real 0m5.081s
user 0m0.068s
sys 0m0.004s
Two caveats though : it only works on *nix, and it is not multithread-safe.
At the beginning of the loop check if the appropriate amount of time has passed. If it has not, sleep.
# Set up initial conditions for sample_time outside the loop
sample_period = ???
next_min_time = 0
while True:
sample_time = time.time() - zero
if sample_time < next_min_time:
time.sleep(next_min_time - sample_time)
continue
# read and print a line
sample_value = ser.readline()
sample_line = str(sample_time)+','+str(sample_value)
outfile.write(sample_line)
print 'time: {}, value: {}'.format(sample_time, sample_value)
next_min_time = sample_time + sample_period
I have a program that I need to execute at certain intervals. For instance, I may want it to execute it every five minutes. I have several coordinators that communicate with several end nodes devices. The code below is on the coordinators. I need it so that if the interval is set to 5 then it runs and records information at for example: 9:05, 9:10, 9:15, 9:20, 9:25 and etc. The code I have so far is as follows:
if __name__ == '__main__':
while True:
try:
r = json.read(rqst_command())
interval = r.get('intvl')
collect_time = r.get('c_time')
command = r.get('cmd')
send_stats(cmd_nodes(command, collect_time))
time.sleep(interval)
except Exception, e:
print e
print "**Top Level Exception"
pass
The problem is that if I set the interval to 5 minutes it does not record exactly every 5 minutes. The execution time seems to slowly increase that time. So for example the above code may record as 9:05:09, 9:10:19, 9:15:29, 9:20:41, 9:25:50. The time it takes for the program to run depends on how fast the nodes communicate back.
Does anybody have any ideas on how I can change my code so that the program will execute exactly every 5 minutes?
EDIT/UPDATE
I think I have figured out a way to handle my problem. I grab the current datetime and then check to see if it is on the 5 minute mark. If it is then record the datetime and send it to the send_stats function. That way the datetime will always be exactly what I want it to be. If it is not on the 5 minute mark then sleep for awhile and then check again. I have the code mostly completed. However, I am getting the following error when I run the program: 'builtin_function_or_method' object has no attribute 'year'.
What am I doing incorrectly?
Here is my new code:
import os
import json
import datetime
from datetime import datetime
import urllib2
from urllib import urlencode
from socket import *
import time
import zigbee
import select
if __name__ == '__main__':
while True:
try:
r = json.read(rqst_command())
interval = r.get('intvl')
collect_time = r.get('c_time')
command = r.get('cmd')
tempTest = True
while tempTest == True:
start_time = datetime.now
compare_time = datetime(start_time.year, start_time.month, start_time.day, start_time.hour, 0, 0)
difff = int((start_time - compare_time).total_seconds() / 60)
if((difff % interval) == 0):
c_t = datetime(start_time.year, start_time.month, start_time.day, start_time.hour, start_time.minute, 0)
send_stats(cmd_nodes(command, collect_time), c_t)
tempTest = False
else:
time.sleep(30)
except Exception, e:
print e
print "**Top Level Exception"
pass
The below code is what ended up solving my problem. Now no matter what the interval is, it will always use the datetime starting at the beginning of each hour and incrementing by the interval. So, if the interval is 5 then the datetime shows up as for instance: 9:00:00, 9:05:00, 9:10:00, 9:15:00, 9:20:00, 9:25:00, and etc. If the interval is 3 then the datetime shows up as for instance: 5:00:00, 5:03:00, 5:06:00, 5:09:00, 5:12:00, 5:15:00, and etc. The coordinator gets the data from the end nodes and then sends the data to a remote server along with the datetime.
if __name__ == '__main__':
last_minute = -1
while True:
try:
r = json.read(rqst_command())
print r
command = r.get('cmd')
interval = r.get('intvl')
collect_time = r.get('c_time')
tempTest = True
while tempTest == True:
start_time = datetime.now()
s_minute = start_time.minute
if(((s_minute % interval) == 0) and (s_minute != last_minute)):
tempTest = False
c_t = datetime(start_time.year, start_time.month, start_time.day, start_time.hour, start_time.minute, 0)
last_minute = c_t.minute
send_stats(cmd_nodes(command, collect_time), c_t)
time.sleep(1)
else:
time.sleep(1)
except Exception, e:
print e
print "**Top Level Exception"
pass
As others have pointed out, there already are ready-to-use job/task schedulers like cron. You can just use them. But you could also implement your own simple solution in Python, which would be fine. You just have to do it the right way. The fundamental problem in your approach is that you sleep for a certain interval between actions and do not regularly check the system time. In your method, the duration of the action is the error of your time measurement. And this error sums up with each action. You need to have a time reference that is free of this error, which is the system time.
Implementation example:
Consider, for instance, one second precision is good enough for you. Then check the system time each second within a loop. This you can safely realize with a time.sleep(1). If the system time is e.g. 5 minutes later than the last_action_execution_time (which you have obviously stored somewhere), store the current time as last_action_execution_time and execute the action. As long as this action for sure lasts less then 5 minutes, the next execution will happen at last_action_execution_time + 5 min with only a very small error. Most importantly, this error does not grow during runtime with the number of executions.
For a rock-solid Python-based solution you should also look at http://docs.python.org/library/sched.html.
how about:
while true:
try:
start=time.time() # save the beginning time before execution starts
r = json.read(rqst_command())
interval = r.get('intvl')
collect_time = r.get('c_time')
command = r.get('cmd')
start_time = datetime.now()
send_stats(cmd_nodes(command, collect_time))
end_time = datetime.now()
sleepy_time = interval - (end_time - start_time)
while time.time() <= start + sleepy_time*60: #To wait until interval has ended note: I'm assuming sleepy_time is in minutes.
pass
except Exception, e:
print e
print "**Top Level Exception"
pass
If you use Linux, you probably want to set up a cronjob which runs your script in certain intervals.
There are two ways to do this.
The first and best would be to use your OS's task scheduler (Task Scheduler in Windows, cron in Linux). The developers of these tools probably anticipated more issues than you can imagine and code you don't have to do yourself is time and probably bugs saved.
Otherwise, you need to take into account the execution time of your script. The simplest way to do that is instead of sleeping for your interval (which as you saw slowly slides forward), you would compute when is the next time you should execute based on when you last woke up and after execution, sleep only for the interval between now and then.
I'm assuming you want to do this in Python and not rely on any other systems.
You just need to account for when your process starts and when it ends and set your interval accordingly. The code will look something like this.
if __name__ == '__main__':
while True:
try:
r = json.read(rqst_command())
interval = r.get('intvl')
collect_time = r.get('c_time')
command = r.get('cmd')
start_time = datetime.now()
send_stats(cmd_nodes(command, collect_time))
end_time = datetime.now()
sleepy_time = interval - (end_time - start_time)
time.sleep(sleepy_time)
except Exception, e:
print e
print "**Top Level Exception"
pass