I need a function to execute every time the date changes. Currently I'm checking in a loop to see if the date changed, but I'm looking for a more effective method....in Python
Any help appreciated
What you really want to do is schedule a function to be run at a certain time. You need to do this with a scheduling mechanism. You could, of course, write one yourself, but probably the best way to go would be to use a library that does this for you.
APScheduler is a very mature good library for just this sort of thing.
Docs: http://apscheduler.readthedocs.org/en/latest/
Pypi: https://pypi.python.org/pypi/APScheduler/3.0.0
Example
Here is a quick little example
from apscheduler.schedulers.background import BlockingScheduler
scheduler = BlockingScheduler()
#scheduler.scheduled_job('interval', seconds=5, timezone='UTC')
def hello():
print('Hello!')
scheduler.start()
This will run the function hello every five seconds. You can change seconds=5 to days=1 to have it run once a day. There is much more configuration you can do, so you'll probably want to read the documentation. It is able to express just about any date time format you could want, including cron.
It also supports different types of schedulers, for instance I chose a BlockingScheduler because wanted the entire program to run as a function of the scheduling mechanism (so you could try this out easily on your own system). You can also use, for instance, a BackgroundScheduler which will allow you to schedule tasks from within your program in an efficient manner that will not block the main thread (fixes your going in a loop forever problem).
Related
I wanted to ask if there is any way in Dask to know which output is from which worker ID. I especially need to know this for analyzing my benchmark results. I want to know what percentage of my tasks is assigned to each process.
I know that it is possible to monitor this using the web interface. But the problem is that when the number of processors is too much or there are lots of tasks per worker it is really hard to get this information only from web-interface. For multiprocessing I know that we can get this info using Profiler; but is there a way to get the results being shown using the web-interface into a dictionary or list?
Does anyone has any experience with this?
Short Answer
No, but you can achieve this with scheduler plugins
Long Answer
You can create a plugin that performs arbitrary computations like logging whenever a task changes state. Here is a simple plugin that prints lots of information about a task whenever it finishes computing:
class MyPlugin(SchedulerPlugin):
def transition(self, key, start, finish, *args, **kwargs):
if start == 'processing' and finish == 'memory':
print(kwargs)
plugin = MyPlugin()
scheduler.add_plugin(plugin)
You'll need some way to run this code on your scheduler. A simple way is to use the client.run_on_scheduler method:
def f(dask_scheduler):
plugin = MyPlugin()
dask_scheduler.add_plugin(plugin)
client.run_on_scheduler(f)
Of course, you probably want to do something more complex than printing, but hopefully this gets you on the right track.
Using APScheduler version 3.0.3. Services in my application internally use APScheduler to schedule & run jobs. Also I did create a wrapper class around the actual APScheduler(just a façade, helps in unit tests). For unit testing these services, I can mock the this wrapper class. But I have a situation where I would really like the APScheduler to run the job (during test). Is there any way by which one can force run the job?
There is no default trigger to launch a job immediately, to achieve this
you can get the current time and set a DateTrigger to the job like this:
my_job.modify_job(trigger=DateTrigger(run_date=datetime.datetime.now()))
this way you will "force" the job to run, but you have to make sure to insert the job again in the scheduler, another option is just creating a new job to run the same function with the add_job function
sched.add_job(func=your_function(),
trigger=DateTrigger(run_date=datetime.datetime.now()))
this way you don't have to do any additional step.
Another approach: you can write logic of your job in separate function. So, you will be able to call this function in your scheduled job as well as somewhere else. I guess that this is a more explicit way to do what you want.
Two alternatives.
change the jobs next_run_time(not verified personally, but here says it works: APScheduler how to trigger job now)
job = self.scheduler.get_job(job_id=job_id)
job.modify(next_run_time=datetime.now() + timedelta(seconds=5))
Add a run-once job to trigger it(verified):
self.scheduler.add_job(func=job.run,
executor=executor,
trigger="date",
max_instances=1,
run_date=datetime.now() + timedelta(seconds=5))
Im currently making a program that would send random text messages at randomly generated times during the day. I first made my program in python and then realized that if I would like other people to sign up to receive messages, I would have to use some sort of online framework. (If anyone knowns a way to use my code in python without having to change it that would be amazing, but for now I have been trying to use web2py) I looked into scheduler but it does not seem to do what I have in mind. If anyone knows if there is a way to pass a time value into a function and have it run at that time, that would be great. Thanks!
Check out the Apscheduler module for cron-like scheduling of events in python - In their example it shows how to schedule some python code to run in a cron'ish way.
Still not sure about the random part though..
As for a web framework that may appeal to you (seeing you are familiar with Python already) you should really look into Django (or to keep things simple just use WSGI).
Best.
I think that actually you can use Scheduler and Tasks of web2py. I've never used it ;) but the documentation describes creation of a task to which you can pass parameters from your code - so something you need - and it should work fine for your needs:
scheduler.queue_task('mytask', start_time=myrandomtime)
So you need web2py's cron job, running every day and firing code similar to the above for each message to be sent (passing parameters you need, possibly message content and phone number, see examples in web2py book). This would be a daily creation of tasks which would be processed later by the scheduler.
You can also have a simpler solution, one daily cron job which prepares the queue of messages with random times for the next day and the second one which runs every, like, ten minutes, checks what awaits to be processed and sends messages. So, no Tasks. This way is a bit ugly though (consider a single processing which takes more then 10 minutes). You may also want to have and check some statuses of the messages to be processed (like pending, ongoing, done) to prevent a situation in which two jobs are working on the same message and to allow tracking progress of the processing. Anyway, you could use the cron method it in an early version of your software and later replace it by a better method :)
In any case, you should check expected number of messages to process and average processing time on your target platform - to make sure that the chosen method is quick enough for your needs.
This is an old question but in case someone is interested, the answer is APScheduler blocking scheduler with jobs set to run in regular intervals with some jitter
See: https://apscheduler.readthedocs.io/en/3.x/modules/triggers/interval.html
got a simple question, I believe, but it got me stuck anyways.
Say I have a simple model:
class myModel(models.Model):
expires = models.DateTimeField(...)
and I want, say on the specified time do something: send an email, delete model, change some of the models fields... Something. Is there a tool in django core, allowing me to do so?
Or, if not, I think some task queuing tool might be in order. I have djcelery working in my project, though I'm a completely newbie in it, and all I was able to perform so far, is to run django-celery-email package, in order to send my mail asynchronically. Though I can't say I'm fully capable of defining task and workers to work in background and be reliable.
If any ideas, on how to solve such problem, please, do not hesitate =)
Write a custom management command to do the task that you desire. When you are done, you should be able to run your task with python manage.py yourtaskname.
Use cron, at, periodic tasks in celery, django-cron, djangotaskscheduler or django-future to schedule your tasks.
I think the best is a background-task the reads the datime and executes a task if a datetime is or has been reached.
See the solution given here for a scheduled task
So the workflow would be:
Create the task you want to apply on objects whose date has been reached
Create a managment command that checks the datetimes in your DB, and execute the above task for every object the datetime has been reached
Use cron (Linux) or at(Windows) to schedule the command call
If you're on a UNIX-like machine, it's possible that you have access to cronjobs. If you're on Windows, I hear there's a program called at that can do similar things. If this doesn't suit your needs, there are a number of ways to do things every X hours using the time library (time.sleep(SOME_NUMBER_OF_SECONDS) in a loop with whatever else you want to do will do it if you want something done regularly, otherwise you'll need to look at time.localtime() and check for conditions).
What is the best way to schedule a periodic task starting at specific datetime?
(I'm not using cron for this considering I've the need to schedule about a hundred remote rsyncs,
where I compute the remote vs local offset and would need to rsync each path the second the logs are generated in each host.)
By my understanding the celery.task.schedules crontab class only allows specifying hour, minute, day of week.
The most useful tip I've found so far was this answer by nosklo.
Is this the best solution?
Am I using the wrong tool for the job?
Celery seems like a good solution for your scheduling problem: Celery's PeriodicTasks have run time resolution in seconds.
You're using an appropriate tool here, but the crontab entry is not what you want. You want to use python's datetime.timedelta object; the crontab scheduler in celery.schedules has only minute resolution, but using timedelta's to configure the PeriodicTask interval provides strictly more functionality, in this case, per second resolution.
e.g. from the Celery docs
>>> from celery.task import tasks, PeriodicTask
>>> from datetime import timedelta
>>> class EveryThirtySecondsTask(PeriodicTask):
... run_every = timedelta(seconds=30)
...
... def run(self, **kwargs):
... logger = self.get_logger(**kwargs)
... logger.info("Execute every 30 seconds")
http://ask.github.com/celery/reference/celery.task.base.html#celery.task.base.PeriodicTask
class datetime.timedelta(days=0, seconds=0, microseconds=0, milliseconds=0, minutes=0, hours=0, weeks=0)
The only challenge here is that you have to describe the frequency with which you want this task to run rather than at what clock time you want it to run; however, I would suggest you check out the Advanced Python Scheduler http://packages.python.org/APScheduler/
It looks like Advanced Python Scheduler could easily be used to launch normal (i.e. non Periodic) Celery tasks at any schedule of your choosing using it's own scheduling functionality.
I've recently worked on a task that involved Celery, and I had to use it for asynchronous operation as well as scheduled tasks. Suffice to say I resorted back to the old crontab for the scheduled task, although it calls a python script that spawns a separate asynchronous task. This way I have less to maintain for the crontab (to make the Celery scheduler run there needs some further setup), but I am making full use of Celery's asynchronous capabilities.