I have a python program that has to be running all the time. If for some reason it was stopped I want to restart it automatically. I thought of having a cron that will run every n number of seconds and check the program is running. My shell script is looks like this:
#!/usr/bin/env bash
CM_COMMAND=`ps aux| grep abc| grep def| grep sudo`
LEN_COMMAND=${#CM_COMMAND}
if[["$LEN_COMMAND" -le "5"]]
then
echo "start the python program"
fi
exit
When I run this script I am getting the error: my_prog.sh: line 4: $'if[[118\r -le 5]]\r': command not found'
What is the alternative of doing this and what is the problem with my script?
Maybe this would be more robust?
1) save the PID of your process when you start it with:
{your_python_command} & echo $! >>/{some_folder}/your_app.pid
2) This script will check and restart if it can't find the PID..
#!/usr/bin/env bash
PID=`cat /{some_folder}/your_app.pid`
if ! ps -p $PID > /dev/null
then
rm /{some_folder}/your_app.pid
{your_python_command} & echo $! >>/{some_folder}/your_app.pid
fi
3) To add it to a cronjob:
crontab -e
choose your text editor and add this row at the end of the file:
*/1 * * * * /{your_path}/{your_script_name}
exit and save
(this will run the script every minute, check crontab manual to set your exact interval)
How about making it a service? A very clean solution, in my opinion.
For more information on how to do it, you can read this article.
Related
I'm a brand new noob in python universe, so don't judge me too fast :-)
I'm trying to force a python script to reload or restart at the beggining of a bash script.
I've tried :
pkill -f myscript.py
and
killall myscript.py
and others...
Actually, I would like to make run the same script that call .wav files after having changed those .wav files... If I don't reload the script or restart it, it keeps playing the old files.
Maybe, there is other solutions.
Here is the script I want to reload (it's a button script playing music for my daughter)
#!/usr/bin/env python3
import pygame
from gpiozero import LED, Button
from signal import pause
pygame.init()
button_sounds = {Button(2): pygame.mixer.Sound("/home/pi/gpio-music-box/samples/1.wav"),
Button(3): pygame.mixer.Sound("/home/pi/gpio-music-box/samples/2.wav"),
Button(4): pygame.mixer.Sound("/home/pi/gpio-music-box/samples/3.wav"),
Button(17): pygame.mixer.Sound("/home/pi/gpio-music-box/samples/4.wav")}
for button, sound in button_sounds.items():
button.when_pressed = sound.play
pause()
And here is my bash script :
#!/bin/bash
***HERE THE COMMAND I NEED !***
rm -r /home/pi/gpio-music-box/samples/*
cp -r /home/pi/gpio-music-box/comptines/* /home/pi/gpio-music-box/samples/
/home/pi/gpio-music-box/music.py
Thank you very much, and scuze my english, I'm french :-)
Andy
try this
#!/bin/bash
pid=$(ps auxwww | grep nameOfScript.py | grep -v grep | awk '{print $2}')
kill -9 $pid
rm -r /home/pi/gpio-music-box/samples/*
cp -r /home/pi/gpio-music-box/comptines/* /home/pi/gpio-music-box/samples/
nohup /home/pi/gpio-music-box/music.py &
Have a nice day
Firstly, you can reduce a lot of the "noise" from ps by using output formatting. You can then stop the need for using both grep and awk by using awk to do the searching also.
ps -eo "%p %a" | awk '/nameOfScript.py/ && $1 != PROCINFO["pid"] { print "kill -9 "$1 }'
This forces ps to only print the pid (%p) and the full command (%a). The output is then piped to awk where is searches for lines with the name of the script contained. It discounts any entries with the current process id of awk and then uses this to print a kill command with the relevant process id.
Once you have verified that the kill command displays as expected, you can use awk's system function to actually run the command through:
ps -eo "%p %a" | awk '/prometheous-things.py/ && $1 != PROCINFO["pid"] { system("kill -9 "$1) }'
I have a Python script that I'd like to run from a cronjob and then check every minute to see if it is still running and if not then start it again.
Cronjob is:
/usr/bin/python2.7 /home/mydir/public_html/myotherdir/script.py
there is some info on this but most answers don't really detail the full process clearly, e.g.:
Using cron job to check if python script is running
e.g. in that case, it doesn't state how to run the initial process and record the PID. It leaves me with a lot of questions unfortunately.
Therefore, could anyone give me a simple guide to how to do this?
e.g. full shell script required, what command to start the script, and so on.
There is now a better way to do this via utility called flock, directly from cron task within a single line. flock will acquire a lock before it runs your app and release it after it is run. The format is as follows:
* * * * * /usr/bin/flock -n /tmp/fcj.lockfile <your cron job>
For your case, this'd be:
* * * * * /usr/bin/flock -n /tmp/fcj.lockfile /usr/bin/python2.7 /home/mydir/public_html/myotherdir/script.py
-n tells it to fail immediately if it cannot acquire the lock. There is also -w <seconds> option, it will fail if it cannot acquire the lock within given time frame.
* * * * /usr/bin/flock -w <seconds> /tmp/fcj.lockfile <your cron job>
It's not that hard. First set up the crontab to run a checker every minute:
* * * * * /home/mydir/check_and_start_script.sh
Now in /home/mydir/check_and_start_script.sh,
#!/bin/bash
pid_file='/home/mydir/script.pid'
if [ ! -s "$pid_file" ] || ! kill -0 $(cat $pid_file) > /dev/null 2>&1; then
echo $$ > "$pid_file"
exec /usr/bin/python2.7 /home/mydir/public_html/myotherdir/script.py
fi
This checks to see if there's a file with the process id of the last run of the script. If it's there, it reads the pid and checks if the process is still running. If not, then it puts the pid of the currently running shell in the file and executes the python script in the same process, terminating the shell. Otherwise it does nothing.
Don't forget to make the script executable
chmod 755 /home/mydir/check_and_start_script.sh
You should actually take a step back and think what and why you are doing it
Do you need to actually start your long_running program from cron? The reason i ask is that you then write another program(watcher) that starts your long_running program. I hope that you see that yo actually do not need cron to start your long_running program.
So what you have is a watcher firing every minute and starting long_running program if it is not running.
Now you need to actually understand what you are trying to do because there are many ways this can be done. If this is an exercise watcher can call ps and work out from there if it runs and start it.
in bash you can do
[ -z "$(ps -ef | grep -v grep | grep myprogname)" ] && { echo runit; }
you just replace myprogname with the name and run it with command that starts your long_running code on the back ground - perhaps
however you should use another tool. systemd, daemontools or others
In your crontab file, have this code:
command='/usr/bin/python2.7 /home/mydir/public_html/myotherdir/script.py'
*/1 * * * * pgrep -af "$command" || $command
Explanation:
Every minute, pgrep checks whether the python script is already running. If it is running, the 2nd statement will not be called since we used ||. If pgrep fails to find the python script in the existing processes, the 2nd statement will be executed and the python script will be back on work
I run a bash script with which start a python script to run in background
#!/bin/bash
python test.py &
So how i can i kill the script with bash script also?
I used the following command to kill but output no process found
killall $(ps aux | grep test.py | grep -v grep | awk '{ print $1 }')
I try to check the running processes by ps aux | less and found that the running script having command of python test.py
Please assist, thank you!
Use pkill command as
pkill -f test.py
(or) a more fool-proof way using pgrep to search for the actual process-id
kill $(pgrep -f 'python test.py')
Or if more than one instance of the running program is identified and all of them needs to be killed, use killall(1) on Linux and BSD
killall test.py
You can use the ! to get the PID of the last command.
I would suggest something similar to the following, that also check if the process you want to run is already running:
#!/bin/bash
if [[ ! -e /tmp/test.py.pid ]]; then # Check if the file already exists
python test.py & #+and if so do not run another process.
echo $! > /tmp/test.py.pid
else
echo -n "ERROR: The process is already running with pid "
cat /tmp/test.py.pid
echo
fi
Then, when you want to kill it:
#!/bin/bash
if [[ -e /tmp/test.py.pid ]]; then # If the file do not exists, then the
kill `cat /tmp/test.py.pid` #+the process is not running. Useless
rm /tmp/test.py.pid #+trying to kill it.
else
echo "test.py is not running"
fi
Of course if the killing must take place some time after the command has been launched, you can put everything in the same script:
#!/bin/bash
python test.py & # This does not check if the command
echo $! > /tmp/test.py.pid #+has already been executed. But,
#+would have problems if more than 1
sleep(<number_of_seconds_to_wait>) #+have been started since the pid file would.
#+be overwritten.
if [[ -e /tmp/test.py.pid ]]; then
kill `cat /tmp/test.py.pid`
else
echo "test.py is not running"
fi
If you want to be able to run more command with the same name simultaneously and be able to kill them selectively, a small edit of the script is needed. Tell me, I will try to help you!
With something like this you are sure you are killing what you want to kill. Commands like pkill or grepping the ps aux can be risky.
ps -ef | grep python
it will return the "pid" then kill the process by
sudo kill -9 pid
eg output of ps command:
user 13035 4729 0 13:44 pts/10 00:00:00 python (here 13035 is pid)
With the use of bashisms.
#!/bin/bash
python test.py &
kill $!
$! is the PID of the last process started in background. You can also save it in another variable if you start multiple scripts in the background.
killall python3
will interrupt any and all python3 scripts running.
I usually use:
nohup python -u myscript.py &> ./mylog.log & # or should I use nohup 2>&1 ? I never remember
to start a background Python process that I'd like to continue running even if I log out, and:
ps aux |grep python
# check for the relevant PID
kill <relevantPID>
It works but it's a annoying to do all these steps.
I've read some methods in which you need to save the PID in some file, but that's even more hassle.
Is there a clean method to easily start / stop a Python script? like:
startpy myscript.py # will automatically continue running in
# background even if I log out
# two days later, even if I logged out / logged in again the meantime
stoppy myscript.py
Or could this long part nohup python -u myscript.py &> ./mylog.log & be written in the shebang of the script, such that I could start the script easily with ./myscript.py instead of writing the long nohup line?
Note : I'm looking for a one or two line solution, I don't want to have to write a dedicated systemd service for this operation.
As far as I know, there are just two (or maybe three or maybe four?) solutions to the problem of running background scripts on remote systems.
1) nohup
nohup python -u myscript.py > ./mylog.log 2>&1 &
1 bis) disown
Same as above, slightly different because it actually remove the program to the shell job lists, preventing the SIGHUP to be sent.
2) screen (or tmux as suggested by neared)
Here you will find a starting point for screen.
See this post for a great explanation of how background processes works. Another related post.
3) Bash
Another solution is to write two bash functions that do the job:
mynohup () {
[[ "$1" = "" ]] && echo "usage: mynohup python_script" && return 0
nohup python -u "$1" > "${1%.*}.log" 2>&1 < /dev/null &
}
mykill() {
ps -ef | grep "$1" | grep -v grep | awk '{print $2}' | xargs kill
echo "process "$1" killed"
}
Just put the above functions in your ~/.bashrc or ~/.bash_profile and use them as normal bash commands.
Now you can do exactly what you told:
mynohup myscript.py # will automatically continue running in
# background even if I log out
# two days later, even if I logged out / logged in again the meantime
mykill myscript.py
4) Daemon
This daemon module is very useful:
python myscript.py start
python myscript.py stop
Do you mean log in and out remotely (e.g. via SSH)? If so, a simple solution is to install tmux (terminal multiplexer). It creates a server for terminals that run underneath it as clients. You open up tmux with tmux, type in your command, type in CONTROL+B+D to 'detach' from tmux, and then type exit at the main terminal to log out. When you log back in, tmux and the processes running in it will still be running.
I've created a simple init script for an application I'm building. The start part of the script looks like this:
user="ec2-user"
name=`basename $0`
pid_file="/var/run/python_worker.pid"
stdout_log="/var/log/worker/worker.log"
stderr_log="/var/log/worker/worker.err"
get_pid() {
cat "$pid_file"
}
is_running() {
[ -f "$pid_file" ] && ps `get_pid` > /dev/null 2>&1
}
case "$1" in
start)
if is_running; then
echo "Already started"
else
echo "Starting $name"
cd /var/lib/worker
. venv/bin/activate
. /etc/profile.d/worker.sh
python run.py >> "$stdout_log" 2>> "$stderr_log" &
echo $! > "$pid_file"
if ! is_running; then
echo "Unable to start, see $stdout_log and $stderr_log"
exit 1
fi
echo "$name running"
fi
I'm having trouble with this line:
python run.py >> "$stdout_log" 2>> "$stderr_log" &
I want to start my application with this code and redirect the outputs to the files specified above. However, when I include the & to make it run in the background, nothing appears in the two log files. BUT, when I remove the & from this line, the log files get data. Why is this happening?
Obviously I need to run the command to make it run as a background process in order to stop the shell waiting.
I am also sure that the process is running when I use the &. I can find it with a ps -aux :
root 11357 7.0 3.1 474832 31828 pts/1 Sl 21:22 0:00 python run.py
Anyone know what I'm doing wrong? :)
Anyone know what I'm doing wrong? :)
Short Answer:
Yes. add -u to the python command and it should work.
python -u run.py >> "$stdout_log" 2>> "$stderr_log" &
Long Answer:
It's a buffering issue (from man python):
-u Force stdin, stdout and stderr to be totally unbuffered. On systems where it matters, also put stdin, stdout
and stderr in binary mode. Note that there is internal buffering in xreadlines(), readlines() and file-
object iterators ("for line in sys.stdin") which is not influenced by this option. To work around this, you
will want to use "sys.stdin.readline()" inside a "while 1:" loop.