I would like to implement a cron-like behaviour with my twisted application.
I want to trigger a periodic call (let's say every week) but running at a precise time only, not when i start my application.
My use case is the following:
my python application is started at any time in the week. I want the calls to be performed every monday at 8am.
But I don't want to perorm active waiting (using a time.sleep()), i would like to use callLater to trigger the call next monday and then start a looping call from that date.
any idea/advice?
thanks,
J.
If you are absolutely in love with cron-style specifiers, you could also consider using parse-crontab
Then your code looks basically like:
from crontab import CronTab
monday_morning = CronTab("0 8 * * 1")
def do_something():
reactor.callLater(monday_morning.next(), do_something)
# do whatever you want!
reactor.callLater(monday_morning.next(), do_something)
reactor.run()
If I understood your question correctly you are thinking of first time execution of a scheduled task and how to supply initial start time for the app. If this is a case, you just need to calculate timedelta value in seconds to be passed to callLater.
import datetime
from twisted.internet import reactor
def cron_entry():
full_weekseconds = 7*24*60*60
print "I was called at a specified time, now you can add looping task with a full weekseconds frequency"
def get_seconds_till_next_event(isoweekday,hour,minute,second):
now = datetime.datetime.now()
full_weekseconds = 7*24*60*60
schedule_weekseconds = ((((isoweekday*24)+hour)*60+minute)*60+second)
now_weekseconds=((((now.isoweekday()*24)+now.hour)*60+now.minute)*60+now.second)
if schedule_weekseconds > now_weekseconds:
return schedule_weekseconds - now_weekseconds
else:
return now_weekseconds - schedule_weekseconds + full_weekseconds
initial_execution_timedelta = get_seconds_till_next_event(3,2,25,1)
"""
This gets a delta in seconds between now and next Wednesday -3, 02 hours, 25 minutes and 01 second
"""
reactor.callLater(initial_execution_timedelta,cron_entry)
reactor.run()
Related
I want to write a program that keeps running in the background and performs a certain task at each hour of the day. How do I achieve this?
for production i would add cron or schedule
# Schedule Library imported
import schedule
import time
# Functions setup
def sudo_placement():
print("Get ready for Sudo Placement at Geeksforgeeks")
def good_luck():
print("Good Luck for Test")
def work():
print("Study and work hard")
def bedtime():
print("It is bed time go rest")
def geeks():
print("Shaurya says Geeksforgeeks")
# Task scheduling
# After every 10mins geeks() is called.
schedule.every(10).minutes.do(geeks)
# After every hour geeks() is called.
schedule.every().hour.do(geeks)
# Every day at 12am or 00:00 time bedtime() is called.
schedule.every().day.at("00:00").do(bedtime)
# After every 5 to 10mins in between run work()
schedule.every(5).to(10).minutes.do(work)
# Every monday good_luck() is called
schedule.every().monday.do(good_luck)
# Every tuesday at 18:00 sudo_placement() is called
schedule.every().tuesday.at("18:00").do(sudo_placement)
# Loop so that the scheduling task
# keeps on running all time.
while True:
# Checks whether a scheduled task
# is pending to run or not
schedule.run_pending()
time.sleep(1)
You can write a if condition in a infinite while loop to check if current time is equals to your time say (12:00:00pm, 04:00:00am) or you can make use of the sleep method, it stops the exexution of your code for the specified amount of time, you must find that by calculating the difference between your time and the current time and this method does not consume much memory and cpu cycles like the previous method.
I'd advise setting up a cron job to run your python program at specific time
Try this:
from datetime import datetime # Import datetime
def schedule(time, function): # Syntax:
cur_time = datetime.strftime("%T") # time: 24 hour time hh:mm:ss (09:00:00 or 21:00:00)
if cur_time == time: # function: lampda: to_execute()
function()
def scheduled_function():
print("TEST")
while True:
schedule("15:00:00", lampda:scheduled_function()) # Schedule scheduled_function() to execute at 3:00 pm
Is it possible make this .py script time out every 20 minutes and auto run again by it self?
Currently I'm using crontab to rerun it every 20 minutes but the thing is it's running multiple .py sometime and not actually rerunning the program. I just want it to rerun every 20 minutes not rerun another instance of it every 20 minutes.
#!/usr/bin/env python
from TwitterFollowBot import TwitterBot
my_bot = TwitterBot("/home/TwitterFollowBot/config.txt")
my_bot.sync_follows()
my_bot.auto_unfollow_nonfollowers()
my_bot.auto_rt("#RtwtKing", count=2000)
my_bot.auto_rt("#ShoutGamers", count=2000)
You have several ways to do this.
If you want to do it only with Python you can:
Use threads, it will work pretty well, but it's not really what threads are designed for.
Use Daemon (good example here)
Do a Python wrapper which will loop forever and call you script when needed. It's less clean, but less overkill too.
Example of the wrapper solution:
The goal is to create a new python script which will handle timer and so execute your Twitter code when needed.
1. Update your current code to encapsulate it in a method
let's say your current file is named core.py
core.py:
from datetime import datetime
from TwitterFollowBot import TwitterBot
def get_feed():
print "({}) LOG: get_feed starting".format(datetime.now())
my_bot = TwitterBot("/home/TwitterFollowBot/config.txt")
my_bot.sync_follows()
my_bot.auto_unfollow_nonfollowers()
my_bot.auto_rt("#RtwtKing", count=2000)
my_bot.auto_rt("#ShoutGamers", count=2000)
This just make your code in a function and add a logging line which print current time when function is executed.
2. Make a wrapper which handle timer and call your twitter code
wrapper.py:
import time
from datetime import datetime
# Import your twitter code, so you can use it by calling 'get_feed()'
from core import get_feed
# Define constants here because it's easier to find it on top of file
# Delta between the last call and the next one, bascially time between two calls: 20 minutes
DELTA = 20 * 60
# Time your wrapper will take between each verification: 5 minutes
SLEEP_TIME = 5 * 60
# Initialize timer and call for the first time your method
# last_run will store timestamp of the last time you called get_feed()
last_run = time.time()
get_feed()
# Start an inifinite loop
while True:
# Compute delta since last call
# last_run_delta will store the time in seconds between last call and now
last_run_delta = time.time() - last_run
# If last_run_delta is upper than DELTA so the number of seconds you want to separate two calls is reached.
if last_run_delta >= DELTA:
# Because time is reached you want to run again your code and reset timer to can handle next call
last_run = time.time()
get_feed()
# If you have not reach delta time yet, you want to sleep to avoid stack overflow and because you don't need to check each microseconds
else:
time.sleep(SLEEP_TIME)
Ouput with DELTA = 10 and SLEEP_TIME = 5 (core.py is called every 10 seconds and check is done each 5 seconds):
(2016-11-29 10:43:07.405750) LOG: get_feed starting
(2016-11-29 10:43:17.414629) LOG: get_feed starting
(2016-11-29 10:43:27.422033) LOG: get_feed starting
(2016-11-29 10:43:37.430698) LOG: get_feed starting
(2016-11-29 10:43:47.436595) LOG: get_feed starting
The only real good point with this method is you can't launch same process two times at once. Because it's not asynchronous, get_feed can't be called twice, but if get_feed take more time than SLEEP_TIME or DELTA you will miss some calls and so do not run it each 20min.
Last thing, because you are importing core.py in wrapper.py you have to create a __init__.py file in the same folder than the two others files. (/path/to/project/ should contains __init__.py (empty), core.py and wrapper.py).
The real good way will is to create a daemon, but it require more skills.
I want to execute some task (function) within my Django application at a specified duration from when a call is made to that. Something like:
... some code
async_run_func(time_interval=15_mins) # Async call. Code within the function
# should be executed after 15 mins.
... some more code
async_run_func is to be executed after some custom interval.
What is the correct approach to achieve this? One way is to create a separate thread and sleep it for time_duration period. But that will result into too many threads on the server. Also, in case the gunicorn process is restarted, the state will be lost. I want the information to persistent. So, I do not want to go with this approach. Currently I am using celery for executing long async and periodic tasks. But celery do not allow option to run a function single time after the specified duration.
It will be great if there is anyway to do it on distributed system. For example, function will be call from one system but the code to be executed on other system (use of queue like RabbitMQ is fine with me). Else, I can also go for executing it on the same machine. Any suggestion?
Celery has the option of enqueuing at a specific time:
your_async_function.apply_async(args=(your, args, tuple),
kwargs={your: kwargs},
countdown=15 * 60)
Or use the subtask syntax, to curry all args and then delay
your_async_function.s(your, args, tuple, your: kwargs).delay(countdown=15 * 60)
If the function has no args, you can skip them and do directly
your_async_function.delay(countdown=15 * 60)
What about using sched module? Simple and efficient.
import sched, time
sc = sched.scheduler(time.time, time.sleep)
sc.enter(15, 1, async_run_func, ())
sc.run
ETA and Countdown are options to perform this using django-celery.
From the document:
The ETA (estimated time of arrival) lets you set a specific date and time that is the earliest time at which your task will be executed. countdown is a shortcut to set ETA by seconds into the future.
For example:
>>> result = add.apply_async((2, 2), countdown=3)
>>> result.get() # this takes at least 3 seconds to return
20
The task is guaranteed to be executed at some time after the specified date and time, but not necessarily at that exact time. Possible reasons for broken deadlines may include many items waiting in the queue, or heavy network latency. To make sure your tasks are executed in a timely manner you should monitor the queue for congestion.
While countdown is an integer, eta must be a datetime object, specifying an exact date and time (including millisecond precision, and timezone information):
>>> from datetime import datetime, timedelta
>>> tomorrow = datetime.utcnow() + timedelta(days=1)
>>> add.apply_async((2, 2), eta=tomorrow)
I have a Django web app which is used by embedded systems to upload regular data, currently every 2 minutes, to the server where Django just pops it into a database.
I'd like to create an alert system where by if there's no data uploaded from the remote system in a time period, say 10 minutes for example, I raise an alarm on the server, via email or something.
In other programming languages/environments I'd create a 10 minute timer to execute a function in 10 minutes, but every time data is uploaded I'd restart the timer. Thus hopefully the timer would never expire and the expiry function would never get called.
I might well have missed something obvious but if there is something I have missed it. This just does not seem possible in Python. Have I missed something?
At present looks like I need an external daemon monitoring the database :-(
You could use the time module for this:
import time
def didEventHappen():
# insert appropriate logic here to check
# for what you want to check for every 10 minutes
value = True # this is just a placeholder so the code runs
return value
def notifyServer():
print("Hello server, the event happened")
start = time.clock()
delay = 10 * 60 # 10 minutes, converted to seconds
while True:
interval = time.clock() - start
eventHappened = False
if interval >= delay:
eventHappened = didEventHappen()
start = time.clock() # reset the timer
if eventHappened:
notifyServer()
else:
print("event did not happen")
Alternatively, you could use the sched module.
I'm working on a python script that needs to run between two given times. I'm required to use the build in sched module as this script needs to be able to run directly on any machine that has python 2.7 as to reduce configuration time. (SO CRON IS NOT AN OPTION)
A few variables define the settings for the time to run, here set_timer_start=0600 and set_timer_end=0900 are written in HHMM. I'm able to stop the script at the right time.
I don't know exactly how sched works (the python doc page doesn't make to much sense to me), but as far as I understand It runs at a date/time (epoch) while I only want it to run at a given time (HHMM).
Can anyone give me an example (or link) on how to use the scheduler and perhaps calculate the next run date/time?
If I got your requirements right, what you need is probably a loop, that will re-enter a task in the queue every time it will be executed. Something along the lines of:
# This code assumes you have created a function called "func"
# that returns the time at which the next execution should happen.
s = sched.scheduler(time.time, time.sleep)
while True:
if not s.queue(): # Return True if there are no events scheduled
time_next_run = func()
s.enterabs(time_next_run, 1, <task_to_schedule_here>, <args_for_the_task>)
else:
time.sleep(1800) # Minimum interval between task executions
However, using the scheduler is - IMO - overkilling. Using datetime objects could suffice, for example a basic implementation would look like:
from datetime import datetime as dt
while True:
if dt.now().hour in range(start, stop): #start, stop are integers (eg: 6, 9)
# call to your scheduled task goes here
time.sleep(60) # Minimum interval between task executions
else:
time.sleep(10) # The else clause is not necessary but would prevent the program to keep the CPU busy.
HTH!