I have a shell script calling Python inside it.
#! /bin/bash
shopt -s extglob
echo "====test===="
~/.conda/envs/my_env/bin/python <<'EOF'
import sys
import os
try:
print("inside python")
x = 2/0
except Exception as e:
print("Exception: %s" % e)
sys.exit(2)
print("at the end of python")
EOF
echo "end of script"
If I execute this, the lines below still get printed.
"end of script"
I want to exit the shell in the exception block of the python script and let the script not reach EOF
Is there a way to create and kill a subprocess in the except block above, that will kill the entire shell script?
Can I spawn a dummy subprocess and kill it inside the exception block there by killing the entire shell script?
Any examples would be helpful.
Thanks in advance.
The whole EOF ... EOF block gets executed within the Python runtime so exiting from it doesn't affect the bash script. You'll need to collect the exit status and check it after the Python execution if you want to stop the further bash script progress, i.e.:
#!/bin/bash
~/.conda/envs/my_env/bin/python <<'EOF'
import sys
sys.exit(0x01) # use any exit code from 0-0xFF range, comment out for a clean exit
print("End of the Python script that will not execute without commenting out the above.")
EOF
exit_status=$? # store the exit status for later use
# now lets check the exit status and see if python returned a non-zero exit status
if [ $exit_status -ne 0 ]; then
echo "Python exited with a non-zero exit status, abort!"
exit $exit_status # exit the bash script with the same status
fi
# continue as usual...
echo "All is good, end of script"
From the shell script you have 2 options:
set -e: all errors quit the script
check python subcommand return code, abort if non-zero
(maybe more details here: Aborting a shell script if any command returns a non-zero value?)
Now, if you don't want to change the handling from your shell script, you could get the parent process of the python script and kill it:
except Exception as e:
import os,signal,sys
print("Exception: %s" % e)
os.kill(os.getppid(),signal.SIGTERM)
sys.exit(2)
if you need this on windows, this doesn't work (os.kill doesn't exist), you have to adapt it to invoke taskkill:
subprocess.call(["taskkill","/F","/PID",str(os.getppid())])
Now I would say that killing the parent process is bad practice. Unless you don't control the code of this parent process, you should try to handle the exit gracefully.
One way to kill the entire script could be to save the PID and then using Python's system commands to execute a kill command on the PID when the exception happens. If we imported 'os' it would be something along the lines of:
# In a shell
PID=$$
...
// Some Python Exception happens
os.system('kill -9' + $PID)
Related
I have a parent shell script that calls a python script. To notify in case of python script failure, have added a TRAP in shell script. But somehow, python script is getting killed/stopped for some reason without going through the TRAP function.
Request to help with the scenarios when a script can behave in such manner
Shell Script (Parent Process):
parent.sh
set -e
on_exit(){
if [ "$?" -eq 0 ]
then
echo "Success"
else
echo "Failure"
fi
}
trap on_exit EXIT
py_script=$(python child.py)
Python Script (Child Process): child.py
def func():
isDone = "false"
while isDone == "false":
print("Waiting")
try:
## GET request which sets isDone="true" on specific value
except Exception as e:
print("Something went wrong")
sys.exit(1)
time.sleep(10)
print("Completed")
Python script never prints "Something went wrong".
Is it possible that Linux is killing the process in background if it runs for around 12 hours?
EDIT:
Investigated it further and got to know that the python process was still running in the background without performing anything. When killed it manually it threw the notification.
But the question remains, in which scenario can a process go into a state without executing further lines without being killed. I am not aware of any pending state.
Is there a way to have Python print a statement when a script finishes successfully?
Example code would be something like:
if 'code variable' == 0:
print "Script ran successfully"
else:
print "There was an error"
How could I pass the value of the exit code to a variable (e.g. 'code variable')?
I feel like this would be a nice thing to include in a script for other users.
Thanks.
You can do this from the shell -- e.g. in Bash:
python python_code.py && echo "script exited successfully" || echo "there was an error."
You can't have a program write something like this for itself because it doesn't know it's exit code until it has exited -- at which time it isn't running any longer to report the error :-).
There are other things you can do to proxy this behavior from within the process itself:
try:
main()
except SystemExit as ext:
if ext.code:
print ("Error")
else:
print ("Success")
raise SystemExit(ext.code)
else:
print ("Success")
However, this doesn't help if somebody uses os._exit -- and we're only catching sys.exit here, no other exceptions that could be causing a non-zero exit status.
Just write print at the end of a script if it's in form of executing straight from top to bottom. If there's an error, python stops the script and your print won't be executed. The different case is when you use try for managing exceptions.
Or make yourself a script for running python script.py with try and your except will give you an exception for example to a file or wherever you'd like it to store/show.
I have a simple perl script that calls another python script to do the deployment of a server in cloud .
I capture the exit status of the deployment inside perl to take any further action after success/failure setup.
It's like:
$cmdret = system("python script.py ARG1 ARG2");
Here the python script runs for 3hrs to 7 hrs.
The problem here is that, irrespective of the success or failure return status, the system receive a Signal HUP at this step randomly even if the process is running in backened and breaks the steps further.
So does anyone know, if there is any time limit for holding the return status from the system which leads to sending Hangup Signal?
Inside the python script script.py, pexpect is used execute scripts remotely:
doSsh(User,Passwd,Name,'cd '+OutputDir+';python host-bringup.py setup')
doSsh(User,Passwd,Name,'cd '+OpsHome+'/ops/hlevel;python dshost.py start')
....
And doSsh is a pexpect subroutine:
def doSsh(user,password,host,command):
try:
child = pexpect.spawn("ssh -o ServerAliveInterval=100 -n %s#%s '%s'" % (user,host,command),logfile=sys.stdout,timeout=None)
i = child.expect(['password:', r'\(yes\/no\)',r'.*password for paasusr: ',r'.*[$#] ',pexpect.EOF])
if i == 0:
child.sendline(password)
elif i == 1:
child.sendline("yes")
child.expect("password:")
child.sendline(password)
data = child.read()
print data
child.close()
return True
except Exception as error:
print error
return False
This first doSsh execution takes ~6 hours and this session is killed after few hours of execution with the message : Signal HUP caught; exitingbut
the execution python host-bringup.py setup still runs in the remote host.
So in the local system, the next doSsh never runs and also the rest steps inside the perl script never continue.
SIGHUP is sent when the terminal disconnects. When you want to create a process that's not tied to the terminal, you daemonize it.
Note that nohup doesn't deamonize.
$ nohup perl -e'system "ps", "-o", "pid,ppid,sid,cmd"'
nohup: ignoring input and appending output to `nohup.out'
$ cat nohup.out
PID PPID SID CMD
21300 21299 21300 -bash
21504 21300 21300 perl -esystem "ps", "-o", "pid,ppid,sid,cmd"
21505 21504 21300 ps -o pid,ppid,sid,cmd
As you can see,
perl's PPID is that of the program that launched it.
perl's SID is that of the program that launched it.
Since the session hasn't changed, the terminal will send SIGHUP to perl when it disconnects as normal.
That said, nohup changes how perl's handles SIGHUP by causing it to be ignored.
$ perl -e'system "kill", "-HUP", "$$"; print "SIGHUP was ignored\n"'
Hangup
$ echo $?
129
$ nohup perl -e'system "kill", "-HUP", "$$"; print "SIGHUP was ignored\n"'
nohup: ignoring input and appending output to `nohup.out'
$ echo $?
0
$ tail -n 1 nohup.out
SIGHUP was ignored
If perl is killed by the signal, it's because something changed how perl handles SIGHUP.
So, either daemonize the process, or have perl ignore use SIGHUP (e.g. by using nohup). But if you use nohup, don't re-enable the default SIGHUP behaviour!
If your goal is to make your perl program ignore the HUP signal, you likely just need to set the HUP entry of the $SIG global signal handler hash:
$SIG{ 'HUP' } = 'IGNORE';
for gory details, see
perldoc perlipc
I am using the following code to execute a remote script. The execution might take a few seconds to complete and I don't want to block the code while the script is being executed. How can I check to see if the script has been executed and finished successfully?
import pxssh
s = pxssh.pxssh()
s.login (ip, username, password):
s.sendline('./script &')
If sendline is not the best way to go, please suggest an alternative.
You could have your script write its exit status to a file when it finishes, then wait for that file to appear (by periodically looking for it, most probably):
(./script ; echo $? > exit_status) &
This assumes your script signals failure by exiting with a non-zero status code.
I want to launch a background Python job from a bash script and then gracefully kill it with SIGINT. This works fine from the shell, but I can't seem to get it to work in a script.
loop.py:
#! /usr/bin/env python
if __name__ == "__main__":
try:
print 'starting loop'
while True:
pass
except KeyboardInterrupt:
print 'quitting loop'
From the shell I can interrupt it:
$ python loop.py &
[1] 15420
starting loop
$ kill -SIGINT 15420
quitting loop
[1]+ Done python loop.py
kill.sh:
#! /bin/bash
python loop.py &
PID=$!
echo "sending SIGINT to process $PID"
kill -SIGINT $PID
But from a script I can't:
$ ./kill.sh
starting loop
sending SIGINT to process 15452
$ ps ax | grep loop.py | grep -v grep
15452 pts/3 R 0:08 python loop.py
And, if it's been launched from a script I can no longer kill it from the shell:
$ kill -SIGINT 15452
$ ps ax | grep loop.py | grep -v grep
15452 pts/3 R 0:34 python loop.py
I'm assuming I'm missing some fine point of bash job control.
You're not registering a signal handler. Try the below. It seems to work fairly reliably. I think the rare exception is when it catches the signal before Python registers the script's handler. Note that KeyboardInterrupt is only supposed to be raised, "when the user hits the interrupt key". I think the fact that it works for a explicit (e.g. via kill) SIGINT at all is an accident of implementation.
import signal
def quit_gracefully(*args):
print 'quitting loop'
exit(0);
if __name__ == "__main__":
signal.signal(signal.SIGINT, quit_gracefully)
try:
print 'starting loop'
while True:
pass
except KeyboardInterrupt:
quit_gracefully()
In addition to #matthew-flaschen's answer, you can use exec in the bash script to effectively replace the scope to the process being opened:
#!/bin/bash
exec python loop.py &
PID=$!
sleep 5 # waiting for the python process to come up
echo "sending SIGINT to process $PID"
kill -SIGINT $PID
I agree with Matthew Flaschen; the problem is with python, which apparently doesn't register the KeyboardInterrupt exception with SIGINT when it's not called from an interactive shell.
Of course, nothing prevents you from registering your signal handler like this:
def signal_handler(signum, frame):
raise KeyboardInterrupt, "Signal handler"
When you run command in background with &, SIGINT will be ignored.
Here's the relevant section of man bash:
Non-builtin commands run by bash have signal handlers set to the values inherited by the shell from
its parent. When job control is not in effect, asynchronous commands ignore SIGINT and SIGQUIT in
addition to these inherited handlers. Commands run as a result of command substitution ignore the
keyboard-generated job control signals SIGTTIN, SIGTTOU, and SIGTSTP.
I think you need to set signal handler explicitly as Matthew commented.
The script kill.sh also have a problem. Since loop.py is sent to background, there's no guarantee that kill runs after python loop.py.
#! /bin/bash
python loop.py &
PID=$!
#
# NEED TO WAIT ON EXISTENCE OF python loop.py PROCESS HERE.
#
echo "sending SIGINT to process $PID"
kill -SIGINT $PID
Tried #Steen's approach, but alas, it does not apparently hold on Mac.
Another solution, pretty much the same as the above but a little more general, is to just re-install the default handler if SIGINT is being ignored:
def _ensure_sigint_handler():
# On Mac, even using `exec <cmd>` in `bash` still yields an ignored SIGINT.
sig = signal.getsignal(signal.SIGINT)
if signal.getsignal(signal.SIGINT) == signal.SIG_IGN:
signal.signal(signal.SIGINT, signal.default_int_handler)
# ...
_ensure_sigint_handler()