Can you tell me what is the use of jupyter cluster. I created jupyter cluster,and established its connection.But still I'm confused,how to use this cluster effectively?
Thank you
With Jupyter Notebook cluster, you can run notebook on the local machine and connect to the notebook on the cluster by setting the appropriate port number. Example code:
Go to Server using ssh username#ip_address to server.
Set up the port number for running notebook. On remote terminal run jupyter notebook --no-browser --port=7800
On your local terminal run ssh -N -f -L localhost:8001:localhost:7800 username#ip_address of server.
Open web browser on local machine and go to http://localhost:8001/
Related
When I start a jupyter notebook, I am able to start a terminal prompt running on the host machine with "New > Terminal".
Is it possible to connect to this terminal with, for instance, iTerm, instead of using the web interface?
How does jupyter connect to the remote terminal?
Note: I am using a remote Jupyter notebook with several port forwarding. I am not able to directly open a terminal onto this machine.
Would it possible to run Jupyter lab or notebook which is installed on a different machine on local network? Something like 192.168.1.8:port_address
Yes of course you can.
Start the the jupyter notebook with this:
jupyter notebook --ip <your_LAN_ip> --port 8888
Then visit:
http://your_LAN_ip:8888
from another computer.
You can easily change the port number 8888 to any number you want, for example 6006.
It's recommended that you use a password when accessing your notebook server. To set up a password, just run jupyter notebook password. Otherwise, someone can access to your notebook.
Here is the link for setting notebook's password: https://jupyter-notebook.readthedocs.io/en/latest/public_server.html#notebook-server-security
Remotely accessed Jupyter notebook from a linux machine has working too slowly and ambari server hosts shows the components are down ?
Working jupyter notebook can connected via putty from that machine using no-browser commnd.
I am using the Ubuntu sub-system on Windows 10.
I opened a terminal and entered the command:
jupyter notebook
and then a notebook is started at port 8888. Without closing this one, I opened another terminal and typed in jupyter notebook again, the new terminal says that:
[I 15:09:00.576 NotebookApp] The Jupyter Notebook is running at:
[I 15:09:00.577 NotebookApp] http://localhost:8888/?token=
with a different token number from the notebook running in the first terminal. If I open localhost:8888 with my browser and enter the token from the second terminal, jupyter rejects me, while the token from the first terminal passes.
I was wondering why can there be two notebooks running on the same 8888 port? If I kill the notebook in the second terminal, will it affect the notebook running in the first terminal?
Short answer - you can't have 2 notebooks server running on the same port.
From Jupyter documentation:
You can start more than one notebook server at the same time, if you
want to work on notebooks in different directories. By default the
first notebook server starts on port 8888, and later notebook servers
search for ports near that one. You can also manually specify the port
with the --port option.
This is probably a networking issue or caused by your environment configuration. Basically, For TCP/IP you can only have one application listening on the same port at one time. Now if you had 2 network cards, you could have one application listen on the first IP and the second one on the second IP using the same port number.
For UDP (Multicasts), multiple applications can subscribe to the same port.
You can always run the second server with --port option and port of your choosing.
I have launched an AWS EMR cluster following the steps on the EMR page. After connecting through SSH (putty in Windows 7) and enabling foxyproxy (Chrome), it launched fine and can be accessed in my laptop browser.
Pyspark and sparkR come with the EMR Spark 1.6.0 installation and work perfectly in the terminal.
The ports for Hue etc. work fine in the following format:
ec2-xx-xxx-xxx-xxx.us-west-2.compute.amazonaws.com:/
I installed Jupyter by following the steps on http://jupyter.readthedocs.org/en/latest/install.html#using-pip
sudo pip install jupyter
I opened a new notebook with
jupyter notebook
It opened a browser in the terminal that I shut down. It gave the following output:
[I 14:32:12.001 NotebookApp] Writing notebook server cookie secret to /home/hadoop/.local/share/jupyter/runtime/notebook_cookie_secret
[I 14:32:12.033 NotebookApp] The port 8888 is already in use, trying another random port.
[I 14:32:12.037 NotebookApp] Serving notebooks from local directory: /home/hadoop
[I 14:32:12.037 NotebookApp] 0 active kernels
[I 14:32:12.038 NotebookApp] The Jupyter Notebook is running at: http://localhost:8889/
[I 14:32:12.038 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
I tried accessing it in my browser by:
localhost:8889/
(didn't work of course)
then by:
ec2-xx-xxx-xxx-xxx.us-west-2.compute.amazonaws.com:8889/
(replacing the x's), but this gave an error as well.
This webpage is not available
ERR_CONNECTION_RESET
So how can I access Jupyter in my local browser when it has been installed on the head node of an EMR cluster?
I haven't actually used Jupyter yet, but I tried installing and running it like you did, and I noticed that Jupyter is configured by default to listen only on localhost, which is why you can't access it from your browser.
I then found that running "jupyter notebook --generate-config" would generate a config file ~/.jupyter/jupyter_notebook_config.py, which you can edit in order to make it listen on 0.0.0.0 instead of localhost. Just change c.NotebookApp.ip to '0.0.0.0' and uncomment the line.
After doing this, I was able to access Jupyter from my browser using a URL like http://ip-10-168-157-117.ec2.internal:8888/. (Mine is listening on port 8888 by default, but I'm assuming yours started on port 8889 due to having Hue installed and listening on port 8888 already.)