I am using Jupyter Notebook for a project. Since I ssh into a linux cluster at work I use
ssh -Y -L 8000:localhost:8888 user#host
Then I start the notebook with jupyter notebook --no-browser & so that I can continue using the terminal. Then on my local machine I open to localhost:8000 and go about my work.
My problem is that I forgot several times to close the server by foregrounding the process and killing it with Ctrl-C. Instead I just logged out of the ssh session. Now when I run jupyter notebook list I get
Currently running servers:
http://localhost:8934/ :: /export/home/jbalsells
http://localhost:8870/ :: /export/home/jbalsells
http://localhost:8892/ :: /export/home/jbalsells
http://localhost:8891/ :: /export/home/jbalsells
http://localhost:8890/ :: /export/home/jbalsells
http://localhost:8889/ :: /export/home/jbalsells
http://localhost:8888/ :: /export/home/jbalsells
I obviously do not want all of these servers running on my work's machine, but I do not know how to close them!
When I run ps I get nothing:
PID TTY TIME CMD
12678 pts/13 00:00:00 bash
22584 pts/13 00:00:00 ps
I have Jupyter 4.1.0 installed.
So I found a solution.
Since jupyter notebook list tells you which ports the notebook servers are running on I looked for the PIDs using netstat -tulpn I got the information from http://www.cyberciti.biz/faq/what-process-has-open-linux-port/
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State
PID/Program name
tcp 0 0 0.0.0.0:8649 0.0.0.0:* LISTEN
-
tcp 0 0 0.0.0.0:139 0.0.0.0:* LISTEN
-
tcp 0 0 0.0.0.0:33483 0.0.0.0:* LISTEN
-
tcp 0 0 0.0.0.0:5901 0.0.0.0:* LISTEN
39125/Xvnc
Without looking too hard I was able to find the ports I knew to look for from jupyter notebook list and the processes running them (you could use grep if it were too hard to find them). Then I killed them with
kill 8337 (or whatever number was associated).
Windows Systems commands on Command Prompt
Be careful to save all the changes made in your notebooks prior to kill the jupyter notebook server process.
i) find the port number used by jupyter notebook server
jupyter notebook list
ex.)
jupyter notebook list
Currently running servers:
http://127.0.0.1:8888/ :: D:\kimkk\Documents\JupyterNotebook
ii) find process ids that use the found port number of jupyter notebook
netstat -ano | find "found portnumber"
ex.)
netstat -ano | find "8888"
TCP 127.0.0.1:8888 0.0.0.0:0 LISTENING 24140
TCP 127.0.0.1:8888 127.0.0.1:55588 ESTABLISHED 24140
TCP 127.0.0.1:8888 127.0.0.1:55612 ESTABLISHED 24140
TCP 127.0.0.1:55588 127.0.0.1:8888 ESTABLISHED 6492
TCP 127.0.0.1:55612 127.0.0.1:8888 ESTABLISHED 6492
find rows with second column value equals to "8888". In above example first, second, and third rows are target rows. In those rows, you can find PID in the last column (ex. 24140).
iii) kill jupyter notebook process with found PID
taskkill /PID found_PID /F
ex.)
taskkill /PID 24140 /F
/F means forcely kill the process.
FYI, Jupyter notebook from version 5.1 supports stop command as follows:
jupyter notebook stop 8888
refer to https://github.com/jupyter/notebook/issues/1950
Use the following command to stop Jupyter notebook running on port 8888:
fuser -k 8888/tcp
This might help:
run jupyter notebook list to get the port number jupyter uses.
run lsof -n -i4TCP:[port-number] to get PID.
The PID is the second field in the output.
run kill -9 [PID] to kill this process.
I ran into the same issue and followed the solution posted above. Just wanted to clearify the solution a little bit.
netstat -tulpn
will list all the active connections.
tcp 0 0 0.0.0.0:8888 0.0.0.0:* LISTEN 19524/python
you will need the PID "19524" in this case. you can even use the following to get the PID of the port you are trying to shut down
fuser 8888/tcp
this will give you 19524 as well.
kill 19524
will shut down the port
Section 3.3 should be applicable to this.
http://jupyter-notebook-beginner-guide.readthedocs.io/en/latest/execute.html
When a notebook is opened, its “computational engine” (called the kernel) is automatically started. Closing the notebook browser tab, will not shut down the kernel, instead the kernel will keep running until is explicitly shut down.
To shut down a kernel, go to the associated notebook and click on menu File -> Close and Halt. Alternatively, the Notebook Dashboard has a tab named Running that shows all the running notebooks (i.e. kernels) and allows shutting them down (by clicking on a Shutdown button).
Here's a bash script that will kill ALL active Jupyter notebook servers at one go, based on the answers given by #Joalito and #Hongsoog:
#!/bin/bash
jupyter notebook list | {
while IFS= read -r line
do
port=`echo "$line" | grep -o -P '(?<=localhost:).*(?=/ :)'`
echo "killing jn in port $port"
if [ -z "$port" ]
then
netstat -tulpn | grep "$port" | grep -o -P '(?<=LISTEN ).*(?=/py)' | xargs kill -15
fi
done
}
On your notebook the welcoming page is named "Files" and you can see "Running" next to it. There is where you would want to shutdown and see the running notebooks
What worked for me was
jupyter notebook list, which in my case returned:
http://localhost:8889/?token=77d01d687da830b74eba946060660d :: /gpfs/blah/
http://localhost:8889/?token=1243162854ee3648e3154b26643794 :: /ifs/hello/world/
netstat -tulpn | grep "8888", which in my case returned:
tcp 7 0 127.0.0.1:8888 0.0.0.0:* LISTEN 17602/python3.9
And I found the PID in the last column: 17602.
kill -9 17602, which freed up the port.
Related
I can ssh into a remote machine.
I then try to connect to a jupyter notebook job that I started on one of the nodes of the remote machine:
ssh -L 8069:localhost:8069 me#remote.machine ssh -L 8069:localhost:8069 me#node14
This has always worked fine in the past.
When I execute this lately, nothing happens until I eventually get a time out message. If I cancel it and then try to simply ssh into the remote machine again, it again does nothing until I get the error message:
ssh: connect to host remote.machine port 22: Connection timed out
I am trying to figure out if this is a problem at my end or at the remote machine. If it's the latter I can't understand why I am able to ssh to the remote machine fine until I try the
ssh -L 8069:localhost:8069 me#remote.machine ssh -L 8069:localhost:8069 me#node14
connection.
You are trying to do a double ssh connection: one to remote.machine and then another one to node14.
The problem seems to be the ssh process in the node14 machine. So, you can connect to the first machine but no to the second one. Ask your administrator to enable the sshd process in node14
You can test this case by logging into remote.machine via:
ssh -L 8069:localhost:8069 me#remote.machine.
Once you get shell access you can try the connection to node14 via:
ssh -L 8069:localhost:8069 me#node14.
According to the description, this last try should fail with the timeout.
I am trying to run server3.py, but whenever I try to do this, I am not able to because I keep getting an error that the server is already in use. So naturally, I tried to find if any other process is using that port and then killing it. However, either I am doing that wrong, or there is some hidden process because no other process is using python.
I am new to servers and local hosts, so any help would be appreciated.
Try using netstat -vanp tcp | grep 8000 (mac)
or
netstat -tulpn | grep 8000 (linux)
or
netstat -abno | findstr :8000 (windows)
I am presuming you are using flask kind of framework, In any server instance only one port will be listening, in your case port is already been used by other service
Change the port number and try
If really want to check which port is listening use "netstat commond"
In Linux:
netstat -tulnp
In windows:
netstat -abno
In Mac:
netstat -pant | grep LISTEN
if output is too lengthy you can grep with port number
I am using anaconda navigator to launch jupyter notebook server, But when I want to stop the server, I could not find a way to do it.
Because there is no terminal when I'm using anaconda navigator, I can't use Ctrl +c to stop the server. And I am running on Windows.
Any help will be appreciated ^_^
You can click on the check box which appears when you close anaconda navigator.
If you have already closed the navigator, open cmd and type jupyter-notebook list.
Then you can kill the port using following commands:
netstat -o -n -a | findstr :3000
TCP 0.0.0.0:3000 0.0.0.0:0 LISTENING 3116
taskkill /F /PID 3116
Substitute findstr parameter with which ever port is running.
More info on how to kill a process in windows.
I got the following issue when I try to run my edX LMS (port 8000):
Error: That port is already in use
So in my vagrant account I found and did kill -9 on process which was using 8000. But as soon as I killed them, the process was automatically restarting and using port 8000 and I am unable to run LMS.
When that happens, I just do:
vagrant reload
(You will have to logout from SSH before by typing logout)
It is equivalent to:
vagrant halt
vagrant up
I've had times on OS/X with Vagrant where I've had to kill not only the vagrant process, but also virtualbox, when vagrant reload hasn't worked.
On your machine (not the guest VM):
ps -eaf | fgrep -i vagrant
ps -eaf | fgrep -i virtualbox
Then kill all those processes and "vagrant up."
vagrant halt is enough to kill all the processes related to the used port.
I have a Windows machine where I start an IPython kernel (to do some stuff not possible on another machine).
I have a Linux machine from which I would like to connect to the IPython kernel running on the Windows machine.
I can SSH from the Linux machine to the Windows machine (using this solution: https://superuser.com/a/172299).
I have tried following: https://github.com/ipython/ipython/wiki/Cookbook:-Connecting-to-a-remote-kernel-via-ssh. Both the automatic and the manual solution gives the following:
"ERROR: Kernel did not respond"
Using the same solution, I can connect from my Linux machine to an IPython kernel running on a Linux server. Any solution to get this to work with Linux to Windows?
You don’t need SSH to connect to a remote ipython kernel, regardless of whether it’s a ipython kernel running on Windows or Linux or Mac. What you do need is to have the Ip of the remote kernel visible to the terminal from which you are trying to connect from. Here are the steps:
Find out the ip address of the server (the machine on which the ipython kernel is running i.e. where you want the computation to happen) and client (the machine from which you are trying to connect to):
1.1. If you are on Windows, open up the command prompt and do a ipconfig to find out the ip addresses. If the Windows server has a direct Internet connection/lan connection, you should see a couple of ips like 192.168.57.1 and 10.2.3.64 and 127.0.0.1.
1.2. If you are on linux, open up a terminal and type ifconfig or ip addr show. You should again see a couple of ips like 192.168.57.1 and 10.2.3.64 and 127.0.0.1.
1.3. Test that atleast one of your server ip addresses is visible from the client: Ping your server from your client, using the command ping. ping will work on either Windows or Linux terminals. If you are running the windows/Linux as a VM or is behind a firewall, it is very much possible that your client or server is not visible from the other side. You don’t have to ping the ip address 127.0.0.1. This is a loop back address, and is only visible from the same machine where you got this ip address from. For example if you ping 127.0.0.1 from the Windows machine, it will ping the same Windows machine. If your client and server instances are running on the same machine, then its fine to use this address. However, if your client or server is running on a VM or a different machine altogher, then 127.0.0.1 wont work.
Start the remote kernel:
2.1. Once you have figured out which ip address on the server is visible from the client, start a kernel on the machine using ipython kernel. The ipython kernel will startup and show that `To connect another client to this kernel, use:
--existing kernel-1234.json
2.2. Locate the kernel-1234.json file on your server by importing (https://stackoverflow.com/a/35094772/4752883)
In [1]: from jupyter_client import find_connection_file
In [2]: find_connection_file()
Out[2]: 'C:\\Users\\me\\AppData\\Roaming\\jupyter\\runtime\\kernel-1234.json'
This will work either for Linux or Windows.
Start the remote client:
3.1. Once you locate the file, copy it over to your server machine using scp in linux or pscp or winscp in windows SCP w/ ssh: copying a local file from windows to a remote server using scp
3.2. Make sure that you are the same directory as the kernel-1234.json file.
3.3. Open up the kernel-1234.json file using vim or your favorite text editor. You will notice a line saying "ip": "127.0.0.1". Change 127.0.0.1 the ip address from your server that is visible from the client, that you found in step 1.3 and save the json file.
3.4. Start up the remote kernel using jupyter console –existing=kernel-1234.json, while located in the same drive where kernel-1234.json is located.
If you have followed the steps above, you should now be able to connect to the remote ipython kernel regardless of whether the ipython kernel is running on Windows/Linux/Mac.
I tried the manual way on https://github.com/ipython/ipython/wiki/Cookbook%3a-Connecting-to-a-remote-kernel-via-ssh once again and it worked. In detail:
windows-machine$ ipython kernel -f kernel-1234.json
linux-machine$ scp windows-machine:path/to/kernel-1234.json .
linux-machine$ cat kernel-1234.json
{
"stdin_port": 55534,
"ip": "127.0.0.1",
"control_port": 58391,
"hb_port": 60540,
"signature_scheme": "hmac-sha256",
"key": "fa461cf7-f078-4c22-909f-cfa7d2a30596",
"shell_port": 60159,
"transport": "tcp",
"iopub_port": 59207
}
linux-machine$ ssh -f -N -L 55534:127.0.0.1:55534
linux-machine$ ssh -f -N -L 58391:127.0.0.1:58391
linux-machine$ ssh -f -N -L 60540:127.0.0.1:60540
linux-machine$ ssh -f -N -L 60159:127.0.0.1:60159
linux-machine$ ssh -f -N -L 59207:127.0.0.1:59207
linux-machine$ ipython console --existing kernel-1234.json