I've setup an Amazon EC2 server. I have a Python script that is supposed to download large amounts of data from the web onto the server. I can run the script from the terminal through ssh, however very often I loose the ssh connection. When I loose the connection, the script stops.
Is there a method where I tell the script to run from terminal and when I disconnect, the script is still running on the server?
You have a few options.
You can add your script to cron to be run regularly.
You can run your script manually, and detach+background it using nohup.
You can run a tool such as GNU Screen, and detach your terminal and log out, only to continue where you left off later. I use this a lot.
For example:
Log in to your machine, run: screen.
Start your script and either just close your terminal or properly detach your session with: Ctrl+A, D, D.
Disconnect from your terminal.
Reconnect at some later time, and run screen -rD. You should see your stuff just as you left it.
You can also add your script to /etc/rc.d/ to be invoked on book and always be running.
You can also use nohup to make your script run in the background or when you have disconnected from your session:
nohup script.py &
The & at the end of the command explicitly tells nohup to run your script in the background.
If it just a utility you run ad-hoc, not a service daemon of some kind, i would just run it in screen. Than you can disconnect if you want and open the terminal back up later... Or reconnect the terminal if you get disconnected. It should be in your linux distros package manager. Just search for screen
http://www.gnu.org/software/screen/
nohup runs the given command with hangup signals ignored, so that the command can continue running in the background after you log out.
Syntax:
nohup Command [Arg]...
Example:
nohup example.py
nohup rasa run
Also, you can run scripts continuously using the cron command.
For more:
https://ss64.com/bash/nohup.html
https://opensource.com/article/17/11/how-use-cron-linux
Related
I have Python code that I want to run on a GPU server. Running it is time-consuming and sometimes, I get disconnected from the Internet or server and I have to re-run it. So, I need to let it run and shut down my computer. Is there any way?
You can use screen to maintain a session and achieve the goal that allows you to run the code on the server.
Refer to this: https://www.gnu.org/software/screen/
Some basic commands:
Create session named RunWork
screen -S RunWork
List all sessions:
screen -ls
Open a session
screen -r SessionID
...
If it is a windows server create a bat file.
Run the python script and the shutdown command. You will have to be an admin to shutdown the computer from a script.
bat file
c:\python27\python.exe c:\somescript.py %*
shutdown /s /f /t 0
If you are Linux-based OS then Tmux can help you.
Tmux is a terminal multiplexer. It lets you switch easily between several programs in one terminal, detach them (they keep running in the background) and reattach them to a different terminal.
https://github.com/tmux/tmux/wiki
I've coded a stock trading bot in Python3. I have it hosted on a server (Ubuntu 18.10) that I use iTerm to SSH into. Wondering how to keep the script actively running so that when I exit out of my session it won't kill the active process.
Basically, I want to SSH into my server, start the script then close out and come back into it when the days over to stop the process.
You could use nohup and add & at the end of your command to safely exit you session without killing original process. For example if your script name is script.py:
nohup python3 script.py &
Normally, when running a command using & and exiting the shell afterwards, the shell will terminate the sub-command with the hangup signal (kill -SIGHUP <pid>). This can be prevented using nohup, as it catches the signal and ignores it so that it never reaches the actual application.
You can use screen
sudo apt-get install screen
screen
./run-my-script
Ctrl-A then D to get out of your screen
From there you will be able to close out your ssh terminal. Come back later and run
screen -ls
screen -r $screen_running
The screen running is usually the first 5 digits you see after you've listed all the screens. You can see if you're script is still running or if you've added logging you can see where in the process you are.
Using tmux is a good option. Alternatively you could run the command with an & at the end which lets it run in the background.
https://tmuxcheatsheet.com/
I came here for finding nohup python3 script.py &
Right solution for this thread is screen OR tmux. :)
I have a python application which I uploaded to an EC2 instance as a file named "example.py". I want this code to be running continuously, as it has a while loop that calls a 3rd party Twitter API every minute. If I run "python3 example.py", and close the terminal, the SSH session terminates. Will the python application continue running in that EC2 instance? How can I make that sure? Thanks.
Quick Solution - You run in SSH terminal:
nohup python3 example.py &
(best if you don't wanna see what is going on later)
or
Best Solution - You create a Screen in SSH terminal:
screen
Then you press ctrl+a+d to detach from it. Then you can close SSH safely. If you want to see what is going on later, you can just:
screen -r
You can use the nohup command to disconnect from the current session and & to run in background:
nohup python3 example.py &
Okay here's my situation:
On a server I have a screen session (which was initiated using screen -S python-script) running a python script inside of it.
I make changes to this project by pushing to a git repo. When I want to have the changes reflected on the project, I ssh into the server, screen -x python-script and then Ctrl-c to send keyboard interrupt, then git pull origin master and then ./run_script.py
I want to do something clever and make my project able to update and restart itself. Another small python server would live on this remote server and listen for a request from my project, this python server would then use subprocess to run a bash script which would update and restart my project.
It's a bit convoluted I know, and I'd really like some suggestions if there's a better way to do what I'm wanting.
Right now if I could formulate a bash script to do what I'm looking for, then I know this would work (probably..).
1) Use nohup instead of screen, redirect stdout to a rotative log if you're interested in the outputs. Store the PID somewhere.
2) Set a 'deployment' branch on your repo.
3) Set a cronjob on the server to pull 'deployment' branch each x minutes.
4) Use a githook to trigger an start/stop each time there's a new commit on your 'deployment' branch. Use the stored pid to kill it.
EDIT:
If you need to kill via Ctrl + C, just send SIGINT with kill then wait for the pid to die.
I need to execute python script on remote server (access through puTTY), but I don't have a stable Internet connection, and every time I execute the script I get problems after several minutes due to my Internet getting disconnected.
How do I remotely execute the script without being connected to server?
(e.g. I connect to server, run script, and can logout while executing)
You can use a Linux Screen, it opens a background terminal and keeps a shell active even through network disruptions.
Open the screen typing in your terminal $ screen and execute there your script, even if you lose connection it won't kill the process.
Here you will find a well explained How to for this program. I use it for my regular day working on remote.
try this
nohup your_script >/dev/null 2>&1 &
program will be running in background