I have written up a python script that allows a user to input a message, his email and the time and they would like the email sent. This is all stored in a mysql database.
However, how do I get the script to execute on the said time and date? will it require a cron job? I mean say at 2:15 on april 20th, the script will search the database for all times of 2:15, and send out those emails. But what about for emails at 2:16?
I am using a shared hosting provided, so cant have a continously running script.
Thanks
If you cannot have a continuously running script, something must trigger it, so that would have to rely on your OS internals. In a unix environment a cron job, as you self state, would do the trick.
Set cron to run the script, and make the script wait for a given time and then continue running and sending until the next email is more than this given time away. Then make your script add a new cron job for a new wakeup time.
Looks like this django application was made just for people in your situation...
http://code.google.com/p/django-cron/
Also, your design seems a little flawed. If its running at 2:15 you wouldn't want to send out just emails that should be sent at 2:15, but all ones that should have been sent in the past that have not been sent.
Your database should either:
A. Delete the entries once they send
or
B. Have a column defined on your database table to store whether it was sent or not. Then your logic should make use of that column.
A cronjob every minute or so would do it. If you're considering this, you might like to mind two things:
1 - How many e-mails are expected to be sent per minute? If it takes you 1 second to send an e-mail and you have 100 e-mails per minute, you won't finish your queue.
2 - What will happen if one job starts before the last one finishes? Be careful not to send e-mails twice. You need either to make sure first process ends (risk: you can drop an e-mail eventually), avoid next process to start (risk: first process hangs whole queue) or make them work in parallel (risk: synchronization problems).
If you take daramarak's suggestion - make you script add a new cron job at end - you have the risk of whole system colapsing if one error occurs.
Related
What are the best methods to set a .py file to run at one specific time in the future? Ideally, its like to do everything within a single script.
Details: I often travel for business so I built a program to automatically check me in to my flights 24 hours prior to takeoff so I can board earlier. I currently am editing my script to input my confirmation number and then setting up cron jobs to run said script at the specified time. Is there a better way to do this?
Options I know of:
• current method
• put code in the script to delay until x time. Run the script immediately after booking the flight and it would stay open until the specified time, then check me in and close. This would prevent me from shutting down my computer, though, and my machine is prone to overheating.
Ideal method: input my confirmation number & flight date, run the script, have it set up whatever cron automatically, be done with it. I want to make sure whatever method I use doesn't include keeping a script open and running in the background.
cron is best for jobs that you want to repeat periodically. For one-time jobs, use at or batch.
Summary: I have a python script which collects tweets using Twitter API and i have postgreSQL database in the backend which collects all the streamed tweets. I have custom code which overcomes the ratelimit issue and i made it to run 24/7 for months.
Issue: Sometimes streaming breaks and sleeps for given secs but it is not helpful. I do not want to check it manually.
def on_error(self,status)://tweepy method
self.mailMeIfError(['me <me#localhost'],'listen.py <root#localhost>','Error Occured on_error method',str(error))
time.sleep(300)
return True
Assume mailMeIfError is a method which takes care of sending me a mail.
I want a simple cron script which always checks the process and restart the python script if not running/error/breaks. I have gone through some answers from stackoverflow where they have used Process ID. In my case process ID still exists because this script sleeps if Error.
Thanks in advance.
Using Process ID is much easier and safer. Try using watchdog.
This can all be done in your one script. Cron would need to be configured to start your script periodically, say every minute. The start of your script then just needs to determine if it is the only copy of itself running on the machine. If it spots that another copy is running, it just silently terminates. Else it continues to run.
This behaviour is called a Singleton pattern. There are a number of ways to achieve this for example Python: single instance of program
Im currently making a program that would send random text messages at randomly generated times during the day. I first made my program in python and then realized that if I would like other people to sign up to receive messages, I would have to use some sort of online framework. (If anyone knowns a way to use my code in python without having to change it that would be amazing, but for now I have been trying to use web2py) I looked into scheduler but it does not seem to do what I have in mind. If anyone knows if there is a way to pass a time value into a function and have it run at that time, that would be great. Thanks!
Check out the Apscheduler module for cron-like scheduling of events in python - In their example it shows how to schedule some python code to run in a cron'ish way.
Still not sure about the random part though..
As for a web framework that may appeal to you (seeing you are familiar with Python already) you should really look into Django (or to keep things simple just use WSGI).
Best.
I think that actually you can use Scheduler and Tasks of web2py. I've never used it ;) but the documentation describes creation of a task to which you can pass parameters from your code - so something you need - and it should work fine for your needs:
scheduler.queue_task('mytask', start_time=myrandomtime)
So you need web2py's cron job, running every day and firing code similar to the above for each message to be sent (passing parameters you need, possibly message content and phone number, see examples in web2py book). This would be a daily creation of tasks which would be processed later by the scheduler.
You can also have a simpler solution, one daily cron job which prepares the queue of messages with random times for the next day and the second one which runs every, like, ten minutes, checks what awaits to be processed and sends messages. So, no Tasks. This way is a bit ugly though (consider a single processing which takes more then 10 minutes). You may also want to have and check some statuses of the messages to be processed (like pending, ongoing, done) to prevent a situation in which two jobs are working on the same message and to allow tracking progress of the processing. Anyway, you could use the cron method it in an early version of your software and later replace it by a better method :)
In any case, you should check expected number of messages to process and average processing time on your target platform - to make sure that the chosen method is quick enough for your needs.
This is an old question but in case someone is interested, the answer is APScheduler blocking scheduler with jobs set to run in regular intervals with some jitter
See: https://apscheduler.readthedocs.io/en/3.x/modules/triggers/interval.html
I'm fairly competent with Python but I've never 'uploaded code' to a server before and have it run automatically.
I'm working on a project that would require some code to be running 24/7. At certain points of the day, if a criteria is met, a process is started. For example: a database may contain records of what time each user wants to receive a daily newsletter (for some subjective reason) - the code would at the right time of day send the newsletter to the correct person. But of course, all of this is running out on a Cloud server.
Any help would be appreciated - even correcting my entire formulation of the problem! If you know how to do this in any other language - please reply with your solutions!
Thanks!
Here are two approaches to this problem, both of which require shell access to the cloud server.
Write the program to handle the scheduling itself. For example, sleep and wake up every few miliseconds to perform the necessary checks. You would then transfer this file to the server using a tool like scp, login, and start it in the background using something like python myscript.py &.
Write the program to do a single run only, and use the scheduling tool cron to start it up every minute of the day.
Took a few days but I finally got a way to work this out. The most practical way to get this working is to use a VPS that runs the script. The confusing part of my code was that each user would activate the script at a different time for themselves. To do this, say at midnight, the VPS runs the python script (using scheduled tasking or something similar) and runs the script. the script would then pull times from a database and process the code at those times outlined.
Thanks for your time anyways!
I use a python script (on a linux web server) to redirect a user on request. The redirection is based on a database (a python dictionary) and the database itself is builded from a remote CSV file.
For now, I have to manually update the database but the CSV file can change at any time.
I'm looking for a way to update the database after each user request (after 10 sec). In this way, the database is always up to date and the user do not suffer from the update.
I'm trying with the shed module but it doesn't work.
import sched, time
s = sched.scheduler(time.time, time.sleep)
s.enter(0, 1, app.redirect, ())
s.enter(10, 1, app.data_base_update, ())
s.run()
The goal is to keep the url redirection fast for the user and delay an update later. Is there a good solution to do it with a unique script file?
You would be better served by updating a COPY in the background, and instantly switching them and making the updated copy into the live copy. Thus there would be no wait for the user, and you could do so at any time. You are probably best not doing so 10sec after each user request (imagine a flood of requests... it will bring your server to its knees). You can schedule a cron script or other automated task to do so, every minute or half hour etc.; depending on the size of the task you can also limit the CPU utilization.
Note that your solution still doesn't ensure the database is always up to date, since you are working with remote data. But that is unfortunately the price of working with remote data. Make sure not to hammer the remote servers if they do not belong to you. =)