python: subprocess calling a script which runs a background process hanging - python

I have an issue using subprocess.call when calling scripts which in turn run background processes.
I am calling a bash script from a python script.
python 2.7.3.
#!/bin/python
from subprocess import call
.
.
call(["run_exp",file_name])
print "exp complete!"
.
.
run_exp is a bash script which runs a process in the background.
#!/bin/bash
.
.
run_task auto_output 2>/dev/null &
.
.
echo "run_exp finished!"
The run task command is another bash script. This is always completed by the time run_exp has finished.
Running run_exp from command line I see expected behaviour and all processes finish completed.
An issue arises when i call the run_exp command using python call. When using call I see the output "run_exp finished!" but never "exp complete!". If I remove the run_task operation (and associated code with its operation in run_exp) from run_exp, the call command runs to completion as expected. This leads me to believe there is an issue using call when the script called runs processes in the background.
Can anyone shed any light on why this might occur.
Thanks!

The output of the background scripts is still going to the same file descriptor as the child script. That's why the parent script is still waiting for it to finish.
You should close all file descriptors in your background scripts if you want to demonize them:
(run_task auto_output >/dev/null 2>&1) &
(The parentheses do this in a subshell which I sometimes found was needed.)
Also it can help to wait explicitly at the end of your child script for the background process:
run_task auto_output 2>/dev/null & backgroundPid=$!
...
echo "run_exp finished!"
wait "$backgroundPid"
And maybe combining both strategies should also tried if both fail alone.

Related

continue program even after logout [duplicate]

I have Python script bgservice.py and I want it to run all the time, because it is part of the web service I build. How can I make it run continuously even after I logout SSH?
Run nohup python bgservice.py & to get the script to ignore the hangup signal and keep running. Output will be put in nohup.out.
Ideally, you'd run your script with something like supervise so that it can be restarted if (when) it dies.
If you've already started the process, and don't want to kill it and restart under nohup, you can send it to the background, then disown it.
Ctrl+Z (suspend the process)
bg (restart the process in the background
disown %1 (assuming this is job #1, use jobs to determine)
Running a Python Script in the Background
First, you need to add a shebang line in the Python script which looks like the following:
#!/usr/bin/env python3
This path is necessary if you have multiple versions of Python installed and /usr/bin/env will ensure that the first Python interpreter in your $$PATH environment variable is taken. You can also hardcode the path of your Python interpreter (e.g. #!/usr/bin/python3), but this is not flexible and not portable on other machines. Next, you’ll need to set the permissions of the file to allow execution:
chmod +x test.py
Now you can run the script with nohup which ignores the hangup signal. This means that you can close the terminal without stopping the execution. Also, don’t forget to add & so the script runs in the background:
nohup /path/to/test.py &
If you did not add a shebang to the file you can instead run the script with this command:
nohup python /path/to/test.py &
The output will be saved in the nohup.out file, unless you specify the output file like here:
nohup /path/to/test.py > output.log &
nohup python /path/to/test.py > output.log &
If you have redirected the output of the command somewhere else - including /dev/null - that's where it goes instead.
# doesn't create nohup.out
nohup command >/dev/null 2>&1
If you're using nohup, that probably means you want to run the command in the background by putting another & on the end of the whole thing:
# runs in background, still doesn't create nohup.out
nohup command >/dev/null 2>&1 &
You can find the process and its process ID with this command:
ps ax | grep test.py
# or
# list of running processes Python
ps -fA | grep python
ps stands for process status
If you want to stop the execution, you can kill it with the kill command:
kill PID
You could also use GNU screen which just about every Linux/Unix system should have.
If you are on Ubuntu/Debian, its enhanced variant byobu is rather nice too.
You might consider turning your python script into a proper python daemon, as described here.
python-daemon is a good tool that can be used to run python scripts as a background daemon process rather than a forever running script. You will need to modify existing code a bit but its plain and simple.
If you are facing problems with python-daemon, there is another utility supervisor that will do the same for you, but in this case you wont have to write any code (or modify existing) as this is a out of the box solution for daemonizing processes.
Alternate answer: tmux
ssh into the remote machine
type tmux into cmd
start the process you want inside the tmux e.g. python3 main.py
leaving the tmux session by Ctrl+b then d
It is now safe to exit the remote machine. When you come back use tmux attach to re-enter tmux session.
If you want to start multiple sessions, name each session using Ctrl+b then $. then type your session name.
to list all session use tmux list-sessions
to attach a running session use tmux attach-session -t <session-name>.
You can nohup it, but I prefer screen.
Here is a simple solution inside python using a decorator:
import os, time
def daemon(func):
def wrapper(*args, **kwargs):
if os.fork(): return
func(*args, **kwargs)
os._exit(os.EX_OK)
return wrapper
#daemon
def my_func(count=10):
for i in range(0,count):
print('parent pid: %d' % os.getppid())
time.sleep(1)
my_func(count=10)
#still in parent thread
time.sleep(2)
#after 2 seconds the function my_func lives on is own
You can of course replace the content of your bgservice.py file in place of my_func.
Try this:
nohup python -u <your file name>.py >> <your log file>.log &
You can run above command in screen and come out of screen.
Now you can tail logs of your python script by: tail -f <your log file>.log
To kill you script, you can use ps -aux and kill commands.
The zsh shell has an option to make all background processes run with nohup.
In ~/.zshrc add the lines:
setopt nocheckjobs #don't warn about bg processes on exit
setopt nohup #don't kill bg processes on exit
Then you just need to run a process like so: python bgservice.py &, and you no longer need to use the nohup command.
I know not many people use zsh, but it's a really cool shell which I would recommend.
If what you need is that the process should run forever no matter whether you are logged in or not, consider running the process as a daemon.
supervisord is a great out of the box solution that can be used to daemonize any process. It has another controlling utility supervisorctl that can be used to monitor processes that are being run by supervisor.
You don't have to write any extra code or modify existing scripts to make this work. Moreover, verbose documentation makes this process much simpler.
After scratching my head for hours around python-daemon, supervisor is the solution that worked for me in minutes.
Hope this helps someone trying to make python-daemon work
You can also use Yapdi:
Basic usage:
import yapdi
daemon = yapdi.Daemon()
retcode = daemon.daemonize()
# This would run in daemon mode; output is not visible
if retcode == yapdi.OPERATION_SUCCESSFUL:
print('Hello Daemon')

How to run python script in linux terminal console and continue using command line?

My question seems to be quite easy, but for some reason I did not find a quick answer to it. I have a python script that I want to run on the terminal command line (Ubuntu linux server), which works for me. But then I can't use the command line until the script ends. The script takes a long time to run, and I would like to continue using the command line to perform other tasks. How can you do the work of a script when its progress is not shown on the command line, but keep its work? And how can I see the active processes that are running on the server to see if a process is running?
Run script command:
python script.py
Add next with & echo "123":
The script takes a long time to run, and I would like to continue
using the command line to perform other tasks.
It seems that you want to run said process in background, please try pasting following
python script.py &
echo "123"
it should start your script.py and then output 123 (without waiting until script.py ends)
how can I see the active processes that are running on the server to
see if a process is running?
Using ps command
ps -ef
will list all processes which you would probably want to filter to get interested one to you.

cron invokes bash script which invokes "script" to run python program. Next line of bash script runs immediately, before script/python completes

When I run the following bash script via cron, the last line runs before the prior line has completed. Why? And how can I enforce the order I want?
I'm convinced that cron is the culprit here somehow, but for the sake of argument, here's a dummy bash script (Obviously, this is just an illustration. In the real world I'm doing some work in the python program and then trying to copy its work product to another place after it's done.):
#!/usr/bin/env bash
cd /tmp/kier/script
script output -c "./sleeper.py; echo '...and we are done'"
echo "This is the next line after invoking script..."
...and for completeness, here's the python script, sleeper.py:
#!/usr/bin/env python3
import time
print("python program starting")
time.sleep(5)
print("python program done")
When I run the bash script from the command line all is well. Specifically, the "This is the next line..." text prints at the very end, after the 5-second sleep.
But when I run it from cron, the output comes in the wrong order (this is the email that comes to me after cron runs the job):
Script started, file is output
Script done, file is output
This is the next line after invoking script...
python program starting
python program done
...and we are done
Script started, file is output
So you can see that "This is the next line..." prints before the python script has even really started. As though it's running the python script in background or something.
I'm stumped. Why is this happening and how can I make the echo command wait until script has finished running the python program?
(Finally, Yes, I could include my extra command inside the commands I send to script, and I am actually considering that. But come on! This is nuts!)
I should follow up and share the solution I came up with. In the end I never got a good answer to WHY it is behaving this way in my (RedHat) environment, so I settled on a workaround. I...
created a sentinel file before invoking "script",
included an extra command deleting the sentinel file in the script's command text, and then
waited for the sentinel file to go away before continuing.
Like this:
sentinel=`mktemp`
script output -c "./sleeper.py; rm $sentinel"
while [ -f $sentinel ]
do
sleep 3
done
Yes, it's a hack, but I needed to move on.

Running one process in parallel linux

I need to make sure to run two processes (python scripts) almost at the same time. But I want the program to continue until one of them is finished. I am running these processes from a C++ program using system.
Is this the right way to run script1 and script2 at the same time and continue just after script2 is finished?
python ./script1.py & python ./script2.py
Thank you!
Your snippet won't work because it will continue as soon as script2 finishes. script1 may still be working at the background.
If you are using bash shell you can do the following:
python ./script1.py &
PID1=$!
python ./script2.py
wait $PID1
$! has the process id of the previously background command. So we run script1 in the background, then we run script2 until completion, and then we wait for script1 to finish (if not already finished).

Shell script to run task and shutdown after task is done

I want to run a shell script that runs a python program and shutdowns after the program is done. Here is what I wrote
#!/bin/bash
python program
sudo shutdown -h now
This just shutdowns the system without waiting for the program to complete. Is there a different command to use that waits for the program to complete?
What you have in your example should actually only shutdown once the python command has completed, unless the python program forks or backgrounds early.
Another way to run it would be to make the shutdown conditional upon the success of the first command
python command && sudo shutdown -h now
Of course this still will not help you if the python program does anything like forking or daemonizing. Simply try running the python script alone and take note if control returns immediately to the console or not.
You could run the command halt to stop your system:
#!/bin/sh
python program
sudo halt
The python program is running first, and halt would run after its completion (you might test the exit code of your python program). If it does not behave like expected, try to understand why. You could add a logger command before the halt to write something in the system logs.
Alternatively, you can use command substitution like this:
$(command to run your program)
The script waits until the wrapped command finishes before moving onto the next one!
#!/bin/sh
$(python program.py)
sudo shutdown -P 0

Categories

Resources