I have a workflow in which I need to call a python method when either of:
1. specified timeout occurs, or
2. size of input data (list) reaches a threshold, like 10 data points
What is the best way to support the workflow?
[Edit] - The methods would be called in a serverless API so it needs to be stateless. Does it make sense to use some sort of queues to store and retrieve the data and how?
You could do it like this:
while True: #Keep checking condition
if timeoutCondition or len([list]) > 10: #check conditions
class.method() #execute method
You can pull the status at a certain interval, if timeout or data reaches threshold, do the thing you want.
import time
max_count = 10
timeout = 60 #1min
count_increase = 0
next_timeout = time.time() + timeout
while True:
current_time = time.time
count_increase = get_count() - count
if count >= max_count or current_time >= next_timeout:
do_what_you_want()
count_increase = 0
next_timeout = time.time() + timeout
time.sleep(1)
Related
This question already has answers here:
How to create a python loop that allows other code to run as well
(3 answers)
Closed 1 year ago.
I'm trying to do something simple here, i want to increase my cost variable which is (100.00) by .01 every 5 minutes, so after every 5 minutes my new value/variable should be, 100.01, then 100.02, then 100.03 and so on..
but i can't figure out how to add it to my existing value.
Here is what i've tried so far, i've reduced the 300 seconds to 10 seconds to speed things up.
import time
import datetime
tracking = time.time()
def values():
global tracking
now = datetime.datetime.now()
cost = 100.00
increase = .01
newvalue = []
for x in range(1,1000):
print(x)
time.sleep(2)
if time.time() - 10 > tracking:
newvalue.append(float(increase))
print(newvalue)
print(now)
tracking = time.time()
values()
any help appreciated.
It seems like you want the += operator. This adds the right-hand value to the value to the value stored in the left-hand variable.
import time
import datetime
tracking = time.time()
def values():
global tracking
now = datetime.datetime.now()
cost = 100.00
increase = .01
for x in range(1,1000):
print(x)
time.sleep(2)
if time.time() - 10 > tracking:
cost += increase
print('Cost: {}'.format(cost))
print(now)
tracking = time.time()
values()
There are other issues to consider:
Floats are not a good way to store currency values. Because they store binary fractions, they cannot represent all decimal fractions. Consider using decimal.Decimal instead.
To track elapsed time, it's best to use time.monotonic. datetime.now() can be adjusted externally (e.g. by ntpdate), so you cannot assume a change in datetime.now represents elapsed time.
There doesn't seem to be a need to store tracking as a global.
In a single-threaded program, values will not exit until 1000 is reached, which means you can't have other program logic running while the cost is incrementing. Calculating the cost on-demand (as others have suggested) will allow you to run other program logic instead of just looping.
Try this
import time
def increase_value_evrey_t_sec(initail_value, interval, increase_by,stop_after = -1):
counter = 0
values = []
while counter < stop_after or stop_after == -1:
time.sleep(interval)
initail_value += increase_by
print(initail_value)
values.append(initail_value)
counter += 1
increase_value_evrey_t_sec(2,2,3,4)
increase_value_evrey_t_sec(2,2,3,4)
I have a function that runs an iterative deepening search and would like to return the value from the deepest search after a certain amount of time has passed. The code skeleton would look something like
import time
answers = []
START = time.clock()
current_depth = 1
while time.clock() - START < DESIRED_RUN_TIME:
answers.append(IDS(depth=current_depth))
current_depth += 1
return answers[-1]
The problem with this code is it will not return until after the time limit has passed. What is the best way to solve this? If I should just add time checks in the IDS function, how can I make sure to return the last value found? Any help would be greatly appreciated.
Your code should work unless IDS is blocking and takes a very long time. Then you have to wait until IDS is finished and the time limit may not be all that precise.
I'm not sure exactly what you mean by
would like to return the value from the deepest search after a certain amount of time has passed.
and
The problem with this code is it will not return until after the time limit has passed.
If you have time limits and you have update times then you can use this code as a generator.
import time
answers = []
START = time.clock()
current_depth = 1
def get_ids(update_time, limit_time):
last_update = time.clock()
while time.clock() - START < DESIRED_RUN_TIME:
answers.append(IDS(depth=current_depth))
current_depth += 1
if time.clock() - last_update < update_time:
last_update = time.clock()
yield answers[-1]
yield answers[-1]
for i in get_ids(1, 10): # get an ids every second and stop after 10 seconds
print(i)
I'm making an alarm that adjusts according to traffic. After messing around a bit, I decided it would be best for the program to get data from the API every 15 minutes or so. However, I noticed that if the while loop condition was met while time.sleep() was active, it wouldn't break till the full duration of time.sleep() ended. Is there any way to have it break when the condition is met, even during time.sleep()? Thanks in advance!
while datetime.datetime.now() < self.wakeuptime: #refers to alarm time
resp = requests.get(url=url, params=param)
data = json.loads(resp.content)
simplifiedtime = datetime.datetime.strptime(data["routes"][0]["legs"][0]["departure_time"]["text"], "%I:%M%p").time()
#gets suggested departure time from api
combinedsimplifiedtime=datetime.datetime.combine(today, simplifiedtime)
self.wakeuptime = combinedsimplifiedtime - datetime.timedelta(minutes=15)
#resets wakeuptime value depending on api
self.timetogo = combinedsimplifiedtime
print self.wakeuptime
time.sleep(900)
#waits 15 minutes and checks again until wakeuptime
You can use another while loop to check the current time much more frequently than the one that updates from the API. Instead of time.sleep(900) you can do:
start_pause = datetime.datetime.now()
while (datetime.datetime.now()-start_pause).seconds < 900 \
and datetime.datetime.now() < self.wakeuptime:
time.sleep(1)
This will still only run the outer loop after 900 seconds.
I am working on a small proof of concept and using python to illustrate the idea. The idea is the program will run in a loop and will check for input. Now if the input falls under a threshold then it sends a notification. But I am trying to restrict the notification at an interval of 4 sec. And thats where I am loosing either with the logic or with some syntax. Either way It is doing some unexpected things
1: keep on entering 0 and it will display the below threshold message until it reaches a 4 sec mark and then it just prints out the message 4 times in a single line. I want them to show after every 4 seconds. The idea is (A)the input might change in that 4 sec and the notification switches. (B)I want the notification to play out as a reminder with a recurrence of 4 sec every time the script hits the condition if weightIn < 0.5..if it is true then the notification goes out after 4 sec from the first time it was sent
Sorry if I tried over explaining it. I am pretty new to python
import threading
def main():
while True:
weightIn = float(input("Get value: "))
threshold = .5
def operation():
if weightIn < 0.5:
#send notification at an interval of 4 sec
threading.Timer(4.0, operation).start()
print("Below weight threshhold...send notification")
else:
print("You are good")
if threshold is not None:
operation()
main()
First avoid declaring functions in a loop. Then ask yourself, if an object would not be appropriate, because it properly encloses state attributes.
But for the algorithmic part, it is simple (if I have correctly understood the problem ...). Store the timestamp of last notification and send a new one if more the 4 seconds have elapsed. In pseudo-code :
last_notification_time = 0
threshold = 0.5
loop:
weighIn = get_new_value()
if weightIn < threshold:
time = get_time_in_seconds()
if (time > last_notification_time + 4):
last_notification_time = time
send_notification()
# actual processing
In Python, it could look like :
#import time
def main():
last_notification_time = 0
threshold = 0.5
while True:
weighIn = float(input("Get value: "))
if weightIn < threshold:
cur_time = time.time()
if (cur_time > last_notification_time + 4):
last_notification_time = time
print("Below weight threshhold...send notification")
# actual processing
main()
I have a function which resembles:
def long_running_with_more_values(start, stop):
headers = get_headers.delay(start, stop)
insert_to_db.delay(headers)
This function is batch processing data that is requested from the net in parallel.
get_headers + insert_to_db is firing off to the message stack and is processed in the end by celery workers, so is not blocking execution.
It has to process every number between start and stop, but can split this up into sections (ranges).
I've found that the operation get_headers is optimal when the range is ~20000 where range = (stop - start)
I want to know how I can split an arbitrary range into groups of 20000 and run each group through the function so I end up with the function being called multiple times with different start and stop values, but still covering the previous range in total.
so for starting values for start and stop of 1 and 100000 respectively i'd expect get_headers to be called 5 times with the following:
[1,20000][20001,40000][40001,60000][60001,80000][80001,100000]
def long_running_with_more_values(start, stop):
while start < stop:
if stop - start < 20000:
headers = get_headers.delay(start, stop)
break
else:
headers = get_headers.delay(start, start + 20000)
start += 20000
insert_to_db.delay(headers)
Notice that headers will only store the return value of the last call to get_headers.delay(). You might need to change the code to headers += get_headers.delay(start, stop). I can't really tell without knowing what the return value of the get_headers.delay() method is.