Question for Python 2.6
I would like to create an simple web application which in specified time interval will run a script that modifies the data (in database). My problem is code for infinity loop or some other method to achieve this goal. The script should be run only once by the user. Next iterations should run automatically, even when the user leaves the application. If someone have idea for method detecting apps breaks it would be great to show it too. I think that threads can be the best way to achive that. Unfortunately, I just started my adventure with Python and don't know yet how to use them.
The application will have also views showing database and for control of loop script.
Any ideas?
You mentioned that you're using Google App Engine. You can schedule recurring tasks by placing a cron.yaml file in your application folder. The details are here.
Update: It sounds like you're not looking for GAE-specific solutions, so the more general advice I'd give is to use the native scheduling abilities of whatever platform you're using. Cron jobs on a *nix host, scheduled tasks on Windows, cron.yaml on GAE, etc.
In your other comments you've suggested wanting something in Python that doesn't leave your script executing, and I don't think there's any way to do this. Some process has to be responsible for kicking off whatever it is you need done, so either you do it in Python and keep a process executing (even if it's just sleeping), or you use the platform's scheduling tools. The OS is almost guaranteed to do a better job of this than your code.
i think you'd want to use cron. write your script, and have cron run it every X minutes / hours.
if you really want to do this in Python, you can do something like this:
while(True):
<your app logic here>
sleep(TIME_INTERVAL)
Can you use cron to schedule the job to run at certain intervals? It's usually considered better than infinite loops, and was designed to help solve this sort of problem.
There's a very primitive cron in the Python standard library: import sched. There's also threading.Timer.
But as others say, you probably should just use the real cron.
Related
Okay, so basically I am creating a website. The data I need to display on this website is delivered twice daily, where I need to read the delivered data from a file and store this new data in the database (instead of the old data).
I have created the python functions to do this. However, I would like to know, what would be the best way to run this script, while my flask application is running? This may be a very simple answer, but I have seen some answers saying to incorporate the script into the website design (however these answers didn't explain how), and others saying to run it separately. The script needs to run automatically throughout the day with no monitoring or input from me.
TIA
Generally it's a really bad idea to put a webserver to handle such tasks, that is the flask application in your case. There are many reasons for it so just to name a few:
Python's Achilles heel - GIL.
Sharing system resources of the application between users and other operations.
Crashes - it happens, it could be unlikely but it does. And if you are not careful, the web application goes down along with it.
So with that in mind I'd advise you to ditch this idea and use crontabs. Basically write a script that does whatever transformations or operations it needs to do and create a cron job at a desired time.
I have a django app running as a microservice. Which has a function which does the following:
Checks for filenames matching *.json in a directory.
for each file:
processes json data and converts to xml.
saves xml in target directory.
Exits function.
Is there any reason i shouldn't do this to keep it running on a cycle:
While True:
main_function()
The previous developer was using threading which i think is making things far too complicated given there's no need to optimise performance.
You might be looking for this:
http://www.celeryproject.org/
Celery is easy to understand and implement and should not take a lot of time.
Let me know if it helps you.
I saw this post on Medium, and wondered how one might go about managing multiple python scripts.
How I Hacked Amazon's Wifi Button
This describes a system where you need to run one or more scripts continuously to catch and react to events in your network.
My question: Let's say I had multiple python scripts that I wanted to do run while I work on other things. What approaches are available to manage these scripts? I have to imagine there is a better way than having a large number of terminal windows running each script individually.
I am coming back to python, and have no formal training in computer programming, so any guidance you can provide will be greatly appreciated.
Let's say I had multiple python scripts that I wanted to do run. What
approaches are available to manage these scripts? I have to imagine
there is a better way than having a large number of terminal windows
running each script individually.
If you have several .py files in a directory that you want to run, without having a specific order, you can do:
import glob
pyFiles = glob.glob('path/*.py')
for pyFile in pyFiles:
execfile(pyFile)
Your system already runs a large number of background processes, with output to the system log or occasionally to a service-specific log file.
A common arrangement for quick and dirty deployments -- where you don't necessarily want to invest in making the scripts robust and well-behaved enough to run as proper services -- is to start the script inside screen or tmux. You can detach when you don't need to be looking at it, and can reattach at any time -- even from a remote login -- to view the output, or to troubleshoot.
Take a look at luigi (I've not used it).
https://github.com/spotify/luigi
These days (five years after the question was asked) a lot of people use docker compose. But that's a little heavy weight depending on what you want to do.
I just saw today the script server of bugy. Maybe it might be a solution for you or somebody else.
(I am just trying to find a tampermonkey script structure for python..)
I would like to recreate some data in my project every 30 minutes (prices that change). also I got another job that needs to refresh every minute.
Now I heard I should use a daemon. but I'm not sure how that works.
Can someone put me into the right direction.
Also should i make an extra model to save that temporary data or is that part of the daemon?
PS: not sure if stack overflow can be used for this sort of questions, but i don't know where to search for this sort of information
You don't want a daemon. You just want cron jobs.
The best thing to do is to write your scripts as custom Django management commands and use cron to trigger them to run at the specified intervals.
I would like to write a tiny calendar-like application for someone as a birthday present (to be run on Ubuntu). All it should do is display a separate picture each day, so whenever it's invoked it should check the date and select the appropriate picture from the collection I would provide, but also, in case it just keeps running, it should switch to the next picture when the next day begins.
The date-checking on invocation isn't the problem; my question pertains to the second case: how can I have the program notice the beginning of the next day? My clumsy approach would be to make it check the current date at regular intervals and let it change the displayed picture once there was a change in date, but that strikes me as very roundabout and not particularly elegant.
In case any of you have got some idea of how I could accomplish this, please don't hesitate to reply. I would aim to write the application in either Perl or Python, so suggestions concerning those two languages would be most welcome, but any other suggestions would be appreciated as well.
Thanks a lot for your time!
The answer to this could be very system dependant. Controlling the time at which your program is executed is likely to be system dependant. On all *nix type systems, I would use cron. Assuming for a moment that you are using a *nix system, the answer then depends on what the program actually does.
If it only needs to select an image, then I would suggest that it not be run continuously, but terminates itself after selecting it, and is then run again the next day (there are a lot of tutorials on how to setup cron).
If, however, it has some form of UI and it is likely (read possible) to keep running for several days, then you can follow two approaches:
Create your program as it is, to poll periodically for the current time, and do a date delta comparison. Python timedelta objects could help here. This is pretty much your inelegant approach.
The other solution would be to send it a signal from cron when you do wish it to update. This process would mean that you would have to make it signal aware, and respond to something like USR1. The Python docs talk to this, but you can find many tutorials on the web. This approach also works quite nicely for daemonised apps.
I'm sure there are many other approaches too, but those are the ones that come to mind for a quickish and nastyish app.
Did you think about scheduling the invoke of your script?
For me, the best approach is this:
1.Have two options to run the script:
run_script
run_script --update
2.Schedule the update run in some task scheduler (for example Cron) to be executed daily.
3.When you would want to check the image for current day, simply run the script without update option.
If you would like me to extend any part of these, simply ask about it.