I have Python script bgservice.py and I want it to run all the time, because it is part of the web service I build. How can I make it run continuously even after I logout SSH?
Run nohup python bgservice.py & to get the script to ignore the hangup signal and keep running. Output will be put in nohup.out.
Ideally, you'd run your script with something like supervise so that it can be restarted if (when) it dies.
If you've already started the process, and don't want to kill it and restart under nohup, you can send it to the background, then disown it.
Ctrl+Z (suspend the process)
bg (restart the process in the background
disown %1 (assuming this is job #1, use jobs to determine)
Running a Python Script in the Background
First, you need to add a shebang line in the Python script which looks like the following:
#!/usr/bin/env python3
This path is necessary if you have multiple versions of Python installed and /usr/bin/env will ensure that the first Python interpreter in your $$PATH environment variable is taken. You can also hardcode the path of your Python interpreter (e.g. #!/usr/bin/python3), but this is not flexible and not portable on other machines. Next, you’ll need to set the permissions of the file to allow execution:
chmod +x test.py
Now you can run the script with nohup which ignores the hangup signal. This means that you can close the terminal without stopping the execution. Also, don’t forget to add & so the script runs in the background:
nohup /path/to/test.py &
If you did not add a shebang to the file you can instead run the script with this command:
nohup python /path/to/test.py &
The output will be saved in the nohup.out file, unless you specify the output file like here:
nohup /path/to/test.py > output.log &
nohup python /path/to/test.py > output.log &
If you have redirected the output of the command somewhere else - including /dev/null - that's where it goes instead.
# doesn't create nohup.out
nohup command >/dev/null 2>&1
If you're using nohup, that probably means you want to run the command in the background by putting another & on the end of the whole thing:
# runs in background, still doesn't create nohup.out
nohup command >/dev/null 2>&1 &
You can find the process and its process ID with this command:
ps ax | grep test.py
# or
# list of running processes Python
ps -fA | grep python
ps stands for process status
If you want to stop the execution, you can kill it with the kill command:
kill PID
You could also use GNU screen which just about every Linux/Unix system should have.
If you are on Ubuntu/Debian, its enhanced variant byobu is rather nice too.
You might consider turning your python script into a proper python daemon, as described here.
python-daemon is a good tool that can be used to run python scripts as a background daemon process rather than a forever running script. You will need to modify existing code a bit but its plain and simple.
If you are facing problems with python-daemon, there is another utility supervisor that will do the same for you, but in this case you wont have to write any code (or modify existing) as this is a out of the box solution for daemonizing processes.
Alternate answer: tmux
ssh into the remote machine
type tmux into cmd
start the process you want inside the tmux e.g. python3 main.py
leaving the tmux session by Ctrl+b then d
It is now safe to exit the remote machine. When you come back use tmux attach to re-enter tmux session.
If you want to start multiple sessions, name each session using Ctrl+b then $. then type your session name.
to list all session use tmux list-sessions
to attach a running session use tmux attach-session -t <session-name>.
You can nohup it, but I prefer screen.
Here is a simple solution inside python using a decorator:
import os, time
def daemon(func):
def wrapper(*args, **kwargs):
if os.fork(): return
func(*args, **kwargs)
os._exit(os.EX_OK)
return wrapper
#daemon
def my_func(count=10):
for i in range(0,count):
print('parent pid: %d' % os.getppid())
time.sleep(1)
my_func(count=10)
#still in parent thread
time.sleep(2)
#after 2 seconds the function my_func lives on is own
You can of course replace the content of your bgservice.py file in place of my_func.
Try this:
nohup python -u <your file name>.py >> <your log file>.log &
You can run above command in screen and come out of screen.
Now you can tail logs of your python script by: tail -f <your log file>.log
To kill you script, you can use ps -aux and kill commands.
The zsh shell has an option to make all background processes run with nohup.
In ~/.zshrc add the lines:
setopt nocheckjobs #don't warn about bg processes on exit
setopt nohup #don't kill bg processes on exit
Then you just need to run a process like so: python bgservice.py &, and you no longer need to use the nohup command.
I know not many people use zsh, but it's a really cool shell which I would recommend.
If what you need is that the process should run forever no matter whether you are logged in or not, consider running the process as a daemon.
supervisord is a great out of the box solution that can be used to daemonize any process. It has another controlling utility supervisorctl that can be used to monitor processes that are being run by supervisor.
You don't have to write any extra code or modify existing scripts to make this work. Moreover, verbose documentation makes this process much simpler.
After scratching my head for hours around python-daemon, supervisor is the solution that worked for me in minutes.
Hope this helps someone trying to make python-daemon work
You can also use Yapdi:
Basic usage:
import yapdi
daemon = yapdi.Daemon()
retcode = daemon.daemonize()
# This would run in daemon mode; output is not visible
if retcode == yapdi.OPERATION_SUCCESSFUL:
print('Hello Daemon')
I would like to know whether it possible to launch bash script out of python in another process, whereas in the bash script the is a kill command for the parent python code which gave birth to the bash script?
When the bash script will kill its parent script, will the bash child still be alive?
Thanks
I have setup a run configuration in Eclipse and need to send SIGINT (Ctrl+C) to the program. There is cleanup code in the program that runs after SIGINT, so pressing Eclipse's "Terminate" buttons won't work (they send SIGKILL I think). Typing CTRL+C into the Console also doesn't work.
How do I send SIGINT to a process running inside an Eclipse Console?
(FWIW I am running a Twisted daemon and need Twisted to shutdown correctly, which only occurs on SIGINT)
If you can determine the process with a utility such as ps, you can use kill to send it a SIGINT. The program will likely be a child process of eclipse.
kill -s INT <pid>
You can send the command via one line:
kill -SIGINT $(ps aux | grep ProgrammName | grep -v grep | awk '{print $2}')
Get the process id and than send the sigint signal
That still seems to be an open issue:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=38016
Just for the sake of completeness: If you came here to find a way to terminate a read line from System.in,
Ctrl + Z worked for me (on Windows).
in some versions, you can do the following.
In the Debug perspective, you can open a view called "Signals"
(Window/Show View/Signals" or Left-Bottom Icon).
You will get a list of all supported signals. Right-click and "Resume
with Signal" will give you the result you need.
I'm making an answer out of a modification of Artur Czajka's comment.
You can use pkill -SIGINT -f ProgramName. Explanation: pkill is similar to killall, -SIGINT states the signal to be used, -f makes it work better in this case (it will look through arguments and stuff instead of just the command name), and ProgramName is the target for pkill.
My friend is in a macOS environment and he wanted to call os.system('exit') at the end of his python script to make the terminal close. It doesn't. This doesn't surprise me but I would like to know what exactly is going on between the python script and the terminal when this call is made.
In my mental simulation the terminal should have to tell you that there are still running jobs, but that doesn't happen either.
As a side question : will some less common terminals close when a process calls this?
read the help:
Execute the command (a string) in a subshell.
A subshell is launched, and exit is run in that subshell.
To exit the enclosing terminal, you have to kill the parent. One way to do it is:
os.system("kill -9 %d"%(os.getppid())
The system function starts another shell to execute a command. So in this case your Python scripts starts a shell and runs "exit" command in there, which makes that process exit. However, the Python script itself, including a terminal where it is running, continues to run. If the intent is to kill the terminal, you have to get the parent process ID and send a signal requesting it to stop. That will kill both Python script and a terminal.
Remember that system first spawns/forks a sub-shell to execute its commands. In effect, you are asking only the sub-shell to exit.
i want to run and control PSFTP from a Python script in order to get log files from a UNIX box onto my Windows machine.
I can start up PSFTP and log in but when i try to run a command remotely such as 'cd' it isn't recognised by PSFTP and is just run in the terminal when i close PSFTP.
The code which i am trying to run is as follows:
import os
os.system("<directory> -l <username> -pw <password>" )
os.system("cd <anotherDirectory>")
i was just wondering if this is actually possible. Or if there is a better way to do this in Python.
Thanks.
You'll need to run PSFTP as a subprocess and speak directly with the process. os.system spawns a separate subshell each time it's invoked so it doesn't work like typing commands sequentially into a command prompt window. Take a look at the documentation for the standard Python subprocess module. You should be able to accomplish your goal from there. Alternatively, there are a few Python SSH packages available such as paramiko and Twisted. If you're already happy with PSFTP, I'd definitely stick with trying to make it work first though.
Subprocess module hint:
# The following line spawns the psftp process and binds its standard input
# to p.stdin and its standard output to p.stdout
p = subprocess.Popen('psftp -l testuser -pw testpass'.split(),
stdin=subprocess.PIPE, stdout=subprocess.PIPE)
# Send the 'cd some_directory' command to the process as if a user were
# typing it at the command line
p.stdin.write('cd some_directory\n')
This has sort of been answered in: SFTP in Python? (platform independent)
http://www.lag.net/paramiko/
The advantage to the pure python approach is that you don't always need psftp installed.