I am trying to run this particular shell script only one time, daily. Here's my code for runLucene.py:
#!/usr/bin/env python
import os
from extras.download_datos_desambiguar import news_Lucene
x=datetime.today()
y=x.replace(day=x.day, hour=09, minute=00, second=0, microsecond=0)
delta_t=y-x
secs=delta_t.seconds+1
def fourdlife():
print "checking function"
os.system("~/code/4dlife_repo/4dbatch/src/engines/extras/download_datos_desambiguar/news_Lucene.py")
t = Timer(secs, fourdlife)
t.start()
print "timer started"
I am running this code in my ProcesosContinuous.py file like this:
while True:
os.system("./runl.sh")
#some other processes
where runl.sh is
python ~/code/4dlife_repo/4dbatch/src/engines/extras/download_datos_desambiguar/news_Lucene.py
This python code is always running on my apache2 server.
However, this is working at any given hour and not just the specified hour. What am I doing wrong?
Also, I feel there is a much better way to do this. I looked at the cron task but that's not what I am looking for. I don't want my program to go to sleep() because I need the other processes to run after the runl.sh process. What is the best way to do this?
The command crontab -e will open up an interactive editor within which you can define a cron job.
Within that editor's buffer, you can then enter a line such as the following (and then save it):
00 00 * * * /path/to/script.sh
This command will run script.sh every midnight.
cron is always doing its thing, I think it's quite a bit better to use that rather than having a python script dedicated to constantly checking if it's time to run the shell script.
With that said, when you create the crontab, it'll go under the users crontab, so just watch the user and/or -u it.
Related
Background
I'm struggling to find a example of WDT in the way I want to use it. Wondering if I misunderstanding its use.
my python writing is pure hobby, honestly Classes intimidate me.
in short my program reads a number of sensors connected to a raspberry pi and writes the data to a cloud hosted object database.
i have an intermittent error that while I try to figure out I want to implement a based watchdog timer.
This is what I'd like to implement so in the very least I continue to collect and store data.
I've read about the builtin watchdog timer the raspberry pi has built in here: https://diode.io/raspberry%20pi/running-forever-with-the-raspberry-pi-hardware-watchdog-20202/
The problem I want the raspberry pi to reboot if my program hangs, but when that happens the OS is still fine, so the solution in the link above is not effective.
What I'd like to implement:
set the builtin watchdog timer to reboot the raspberry pi after 200 seconds without restarting (patting?) the timer. I think the instructions for this are in the link above.
Within my python script, after I iterate through each sensor, restart (or pat?) the watchdog timer and if 200 seconds elapse between pattings (meaning my program hangs) then RPi reboots.
is this possible?
can someone help me with some simple code? I was hoping to keep this simple and avoid classes and/or threads...
thank you in advance
The WDT is probably not the right solution for the problem you are describing. Based on your description, it sounds like what you have is a program that is intended to run periodically (either on a fixed schedule or in response to some event), and that program currently has a bug that is causing it to hang intermittently and never complete it's task or terminate.
The first and best way to solve that, I'm sure you can guess, is to fix the bug. But your thinking is not unreasonable either, and doing what you describe is very common. There are many valid approaches that will do what you want without the complexity of trying to deal with a hardware timer. One of the easiest is probably to just wrap the program in a shell script and use the timeout command to limit how long it is allowed to execute before it is terminated.
Assuming the script is located at /home/user/my_script.py, and you want to run it every 10 minutes, allowing it 2 minutes before it is killed, this would work:
create a wrapper shell script:
#!/usr/bin/env bash
if ! timeout 120s python /home/user/my_script.py; then
msg="$(date) - script exceeded timeout and was killed"
else
msg="$(date) - script ran successfully"
fi
echo "${msg}" >> /home/user/my_script.log
put the script in a file, say at /home/user/wrapper_script.sh and run chmod 755 /home/user/wrapper_script.sh to make it executable.
schedule the script to run every 10 minutes using cron. At a shell, use crontab -e to edit the users crontab file and add a new line like this:
*/10 * * * * /home/user/wrapper_script.sh
now, every 10 minutes the wrapper will start automatically, and it will kick off the python script. It will wait 2 minutes for the script to stop normally, after which it will reach the timeout and terminate it.
Note: depending on how your python program is written, you might have to use some other options to the timeout command to specify which signal it should use to stop the program. If it is a very basic python script, it should be fine using the default.
Edit: based on the comments, you might be able to just change the command you're using to this:
xterm -T "HMS" -geometry 100x70+10+35 -hold -e sudo timeout 120s /usr/bin/python3 /home/pi/h$
Doing that won't actually schedule the script to run at any fixed interval, so it assumes you already have something in place to handle that. All this will do is make srue that the script is restricted to 120 seconds of run time before it is killed.
I want to be able to run my python script which scans for something related to cryptocurrencies 1 minute after the same script has just been complete. Is this possible? Or could I maybe set a limit before the loop repeats itself or something?
The code I have isn't something I am willing to share due to its sensitive nature. It is a trading bot.
You have a few solutions:
Use a cron job if you are on a unix like platform (you can use this for the syntax, or man cron in the terminal to learn more about it)
Create a long running python script that sleeps for one minute before executing your logic again. Something like this:
import time
while True:
# execute code here
time.sleep(60)
If you are running it on Windows platform ,
You can create a batch file to run your script in cmd using the following command:
start "" "path_to_python.exe" "path_to_python_script"
then create a task in windows task scheduler .
You can refer
https://dev.to/tharindadilshan/running-a-python-script-every-x-minutes-in-windows-10-3nm9 . It might helps.
I have a script of a bot deployed on azure that has to be always running. it's a python bot that tracks Twitter mentions in real time by opening a stream listener.
The script fails every once in a while for reasons not directly related to the script (timeouts, connection errors, etc). After searching for answers around here I found this piece of code as the best workaround for restarting the script every time it fails.
#!/usr/bin/env python3.7
import os
def run_bot():
while True:
try:
os.system("test_bot.py start")
except:
pass
if __name__ == "__main__":
run_bot()
I am logging all error messages to learn the reasons why it fails but I think there just be a better way to achieve the same, I would very much appreciate some hints.
So this is wrong way to run a script, you are running it in while loop forever.
A better way is either to schedule your main script in a cron job: Execute Python script via crontab
You can schedule this job to run every min, or hour or a specific time of the day, up to you.
If you wish to run something always, like a system monitor. Then you can run that part inside a while True loop that is fine. Like a loop which checks temperature every 5 secs writes to a file and sleep for 5 secs.
sample sudo code for the script: prog.py
while True:
log_temp()
time.sleep(5secs)
But if the script fails then schedule something to restart the script. Dont start the script inside another while loop.
Something like this: https://unix.stackexchange.com/questions/107939/how-to-restart-the-python-script-automatically-if-it-is-killed-or-dies
I am working on a Python program. It needs to run every 15 minutes. It currently waits 870 seconds (14.5 mins) before running again, but as the time it takes to complete the action varies, sometimes it runs before it has been 15 minutes since it last run, sometimes after 15 minutes.
My code for this part currently looks like this:
print(colour.BOLD, colour.PURPLE, "Finished", colour.END)
print(colour.BOLD, colour.BLUE, 'WAITING 15 MINUTES (900 SECONDS)', colour.END)
time.sleep(870)
Is there a way I can get it to run at xx:15, xx:30, xx:45, xx:00 where xx is every hour from 00 to 23?
Sorry if I'm being confusing here. Thanks for any help in advance.
I would use the schedule module: https://pypi.org/project/schedule/
you would run:
schedule.every().minute.at(":00").do(job)
schedule.every().minute.at(":15").do(job)
schedule.every().minute.at(":30").do(job)
schedule.every().minute.at(":45").do(job)
Use your OS tools to achieve similar results.
They are very reliable and, in case of your script failure, it will run anyway next time.
Linux
Use crontab.
How to set it will slightly change depending on your Distribution.
As a general idea:
sudo crontab -e
Inside the crontab write (be sure to customize the python executable and script path):
*/15 * * * * /usr/bin/python /path/to/your/script.py
This will make sure that your script is executed every 15 minutes.
Windows
How to schedule a task on Windows is more dependent on the Windows version you are using and it is a very visual task.
Googling "How to schedule a task in Windows" will return way better / more specific / updated results than the one I could clumsily explain here.
Here's a nice one I have found for you.
Mac
Read the amazing answer by Meki here on StackOverflow.
Having a script that does a thing at discrete intervals taking control of its own destiny like that makes me squirrely. I would use an external scheduling framework to run this job at discrete intervals. In Linux, this can be done with cronjobs; in Windows, it can be done with Task Scheduler.
Linux:
In terminal, type
crontab -e
to edit the cron schedule for the current user context. Docs on editing cron can be found all over the internet - here's one: https://www.raspberrypi.org/documentation/linux/usage/cron.md
Windows:
You can schedule a python script to run on that schedule in Windows Task Scheduler. here's a link walking through that: https://www.esri.com/arcgis-blog/products/product/analytics/scheduling-a-python-script-or-model-to-run-at-a-prescribed-time/
be certain to utilize the "if the task is already running" and "run task as soon as possible after a scheduled start is missed" options if you do this method to control appropriate behavior:
I have this bash script on the server that runs every hour, via cron. I was perfectly happy, but now the user wants to be able to configure the frequency through the web interface.
I don't feel comfortable manipulating the cron configuration programmatically, but I'm not sure if the other options are any better.
The way I see it, I can either:
Schedule a script to run once a minute and check if it should really run "now"
Forgo cron altogether and use a deamon that is its own scheduler. This probably means rewriting the script in python
...or suck it up and manipulate the cron configuration from the web interface (written in python BTW)
What should I do?
EDIT: to clarify, the main reason I'm apprehensive about manipulating cron is because it's basically text manipulation with no validation, and if I mess it up, none of my other cron jobs will run.
Here's what I ended up doing:
Taking stefanw's advice, I added the following line at the top of my bash script:
if [ ! `cat /home/username/settings/run.freq` = $1 ]; then
exit 0
fi
I set up the following cron jobs:
0 */2 * * * /home/username/scripts/publish.sh 2_hours
#hourly /home/username/scripts/publish.sh 1_hour
*/30 * * * * /home/username/scripts/publish.sh 30_minutes
*/10 * * * * /home/username/scripts/publish.sh 10_minutes
From the web interface, I let the user choose between those four options, and based on what the user chose, I write the string 2_hours/1_hour/30_minutes/10_minutes into the file at /home/username/settings/run.freq.
I don't love it, but it seems like the best alternative.
Give your users some reasonable choices like every minute, every 5 minutes, every half an hour, ... and translate these values to a cron job string. This is user friendly and forbids users to tamper directly with the cron job string.
You could use a python scheduler library that does most of the work already:
pycron
scheduler-py
What about Webmin? I have never used it myself but it seems you could configure the cron module and give permissions to the user who wants to configure the job.
Well something I use is a main script started every minute by cron. this script check touched files. If the files are there the main cron script start a function/subscript. You just have to touch a defined file and "rm -f"ed it when done. It has the side benefits to be more concurrent proof if you want other way to start jobs. Then you can use your favourite web programming language to handle your users scheduling ...
The main script looks like :
[...]
if [ -e "${tag_archive_log_files}" ]; then
archive_log_files ${params}
rm -f ${tag_archive_files}
fi
if [ -e "${tag_purge_log_files}" ]; then
purge_log_files ${params}
rm -f ${tag_purge_log_files}
fi
[...]
I found a module that can manipulate the cron info for me. It's called python-crontab, and it's available with easy_install. From the source:
Example Use:
from crontab import CronTab
tab = CronTab()
cron = tab.new(command='/usr/bin/echo')
cron.minute().during(5,50).every(5)
cron.hour().every(4)
cron2 = tab.new(command='/foo/bar',comment='SomeID')
cron2.every_reboot()
list = tab.find('bar')
cron3 = list[0]
cron3.clear()
cron3.minute().every(1)
print unicode(tab.render())
for cron4 in tab.find('echo'):
print cron4
for cron5 in tab:
print cron5
tab.remove_all('echo')
t.write()
(I kept googling for "cron" and couldn't find anything. The keyword I was missing was "crontab").
I'm now deciding if I'll use this or just replace cron entirely with a python based scheduler.