I have launched an AWS EMR cluster following the steps on the EMR page. After connecting through SSH (putty in Windows 7) and enabling foxyproxy (Chrome), it launched fine and can be accessed in my laptop browser.
Pyspark and sparkR come with the EMR Spark 1.6.0 installation and work perfectly in the terminal.
The ports for Hue etc. work fine in the following format:
ec2-xx-xxx-xxx-xxx.us-west-2.compute.amazonaws.com:/
I installed Jupyter by following the steps on http://jupyter.readthedocs.org/en/latest/install.html#using-pip
sudo pip install jupyter
I opened a new notebook with
jupyter notebook
It opened a browser in the terminal that I shut down. It gave the following output:
[I 14:32:12.001 NotebookApp] Writing notebook server cookie secret to /home/hadoop/.local/share/jupyter/runtime/notebook_cookie_secret
[I 14:32:12.033 NotebookApp] The port 8888 is already in use, trying another random port.
[I 14:32:12.037 NotebookApp] Serving notebooks from local directory: /home/hadoop
[I 14:32:12.037 NotebookApp] 0 active kernels
[I 14:32:12.038 NotebookApp] The Jupyter Notebook is running at: http://localhost:8889/
[I 14:32:12.038 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
I tried accessing it in my browser by:
localhost:8889/
(didn't work of course)
then by:
ec2-xx-xxx-xxx-xxx.us-west-2.compute.amazonaws.com:8889/
(replacing the x's), but this gave an error as well.
This webpage is not available
ERR_CONNECTION_RESET
So how can I access Jupyter in my local browser when it has been installed on the head node of an EMR cluster?
I haven't actually used Jupyter yet, but I tried installing and running it like you did, and I noticed that Jupyter is configured by default to listen only on localhost, which is why you can't access it from your browser.
I then found that running "jupyter notebook --generate-config" would generate a config file ~/.jupyter/jupyter_notebook_config.py, which you can edit in order to make it listen on 0.0.0.0 instead of localhost. Just change c.NotebookApp.ip to '0.0.0.0' and uncomment the line.
After doing this, I was able to access Jupyter from my browser using a URL like http://ip-10-168-157-117.ec2.internal:8888/. (Mine is listening on port 8888 by default, but I'm assuming yours started on port 8889 due to having Hue installed and listening on port 8888 already.)
Related
I have configure Azure VM ubuntu 18.04
I have installed Anaconda in VM
I need to start jupyter-notebook on the VM and has to enable across the network. Below is the procedure I have done, but jupyter-notebook is not opening
jupyter notebook --generate-config
2.vim /home/Nons/.jupyter/jupyter_notebook_config.py
added 2 lines
c.NotebookApp.allow_origin = '' #allow all origins
*c.NotebookApp.ip = '0.0.0.0' # listen on all IPs
3.sudo ufw allow 8888
4.jupyter-notebook password(optional)
When I started the jupyter-notebook --ip 40.xx.xx.xx Error thrown "OSError: [Errno 99] Cannot assign requested address
"
When I started jupyter-notebook [W 11:43:33.495 NotebookApp] No web browser found: could not locate runnable browser.
If you want to connect to the notebook from another computer, or over the internet, you need to configure the notebook server to listen on all network interfaces. It's unnecessary to set the IP address with the public IP of the VM, just set it with *:
jupyter notebook --ip=*
And then create the NSG unbound rule to allow the port 8888.
I am using the Ubuntu sub-system on Windows 10.
I opened a terminal and entered the command:
jupyter notebook
and then a notebook is started at port 8888. Without closing this one, I opened another terminal and typed in jupyter notebook again, the new terminal says that:
[I 15:09:00.576 NotebookApp] The Jupyter Notebook is running at:
[I 15:09:00.577 NotebookApp] http://localhost:8888/?token=
with a different token number from the notebook running in the first terminal. If I open localhost:8888 with my browser and enter the token from the second terminal, jupyter rejects me, while the token from the first terminal passes.
I was wondering why can there be two notebooks running on the same 8888 port? If I kill the notebook in the second terminal, will it affect the notebook running in the first terminal?
Short answer - you can't have 2 notebooks server running on the same port.
From Jupyter documentation:
You can start more than one notebook server at the same time, if you
want to work on notebooks in different directories. By default the
first notebook server starts on port 8888, and later notebook servers
search for ports near that one. You can also manually specify the port
with the --port option.
This is probably a networking issue or caused by your environment configuration. Basically, For TCP/IP you can only have one application listening on the same port at one time. Now if you had 2 network cards, you could have one application listen on the first IP and the second one on the second IP using the same port number.
For UDP (Multicasts), multiple applications can subscribe to the same port.
You can always run the second server with --port option and port of your choosing.
I am creating a local development environtment using Pycharm Professional Edition, Vagrant and Jupyter notebook.
I am able to connect to Jupyter notebook with the browser from the host to the vagrant machine using http://localhost:8888.
I tried to connect to the Jupyter notebook from PyCharm with token but it did not work, so I changed the login option to password, setting it as vagrant.
Also I have change a remote interpreter, which is python3 on the vagrant machine
Despite I have already changed the Jupyter Notebook settings on Pycharm to password login, I receive on the vagrant the following response:
[W 16:47:03.263 NotebookApp] Forbidden
[W 16:47:03.264 NotebookApp] 403 POST /api/kernels (10.0.2.2) 1.35ms referer=None
Is it possible to run remote Notebook with Pycharm and Vagrant?
I'm using python3.5 and jupyter 4.0.6. I launched the jupyter notebook, and get the following output:
[I 21:47:27.021 NotebookApp] Serving notebooks from local directory: /home/nitrous
[I 21:47:27.021 NotebookApp] 0 active kernels
[I 21:47:27.021 NotebookApp] The Jupyter Notebook is running at: http://localhost:8888/
[I 21:47:27.022 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[W 21:47:27.023 NotebookApp] No web browser found: could not locate runnable browser.
on my firefox browser, I typed the specified localhost url: http://localhost:8888/ but I get unable to connect error message. What am I missing? is the hint of the problem on this line:
[W 21:47:27.023 NotebookApp] No web browser found: could not locate runnable browser.
it is my first time using the notebook. I tried to put the below code in the jupyter_notebook_config.py file but to no avail:
c.NotebookApp.open_browser = True
c.NotebookApp.browser = 'firefox'
it also says 0 active kernels. is that a problem?
If you are running your jupyter notebook in a VM (mostly on a linux vm on a windows guest) and trying to access from guest, you will need to launch Jupyter Notebook with right options. Please try to run it like below and it should work.
jupyter notebook --ip=0.0.0.0 --no-browser
Please check the ip of the VM before you enter the URL in firefox. you can use
ifconfig
command for this.
While running Jupyter on my ec2 instance I had faced the same issue.
I resolved it by executing the following command :
jupyter notebook --no-browser
Initially it will show 0 active kernels, once we start creating notebooks on different environments like Python,R etc it will show us the number of active kernels.
I had a similar problem but with Chrome. I dont know in firefox but in chrome, Settings-> Enable guest browsing.
I have a similar problem at first. I solve it by changing the following line in jupyter_notebook_config.py:
c.NotebookApp.browser = r"C:/Program Files (x86)/Google/Chrome/Application/chrome.exe --app=%s"
I guess adding prefix "r" prefix to convert the string to raw string has actually solved this problem.
I've been trying to get ipython/jupyter notebook running as a notebook server on Google Compute Engine (Ubuntu 14.04 VM), but can't connect to the Python kernel.
I started with the latest Anaconda distribution. I can connect to the notebook server from my local machine via the web, browse the directory tree of the VM, and create new notebook, but never connects to the kernel (orange message in the upper right of Jupyter). The notebook port (8888) is open in the firewall settings. iPython notebook returns the following:
[I 19:21:10.152 NotebookApp] Using MathJax from CDN: https://cdn.mathjax.org/mathjax/latest/MathJax.js
[I 19:21:10.169 NotebookApp] Serving notebooks from local directory: /home/rattlerray
[I 19:21:10.170 NotebookApp] 0 active kernels
[I 19:21:10.170 NotebookApp] The IPython Notebook is running at: https://[all ip addresses on your system]:8888/
[I 19:21:10.170 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[I 19:22:13.835 NotebookApp] 302 GET / (98.169.96.157) 0.85ms
[I 19:22:31.902 NotebookApp] Creating new notebook in
[I 19:22:33.634 NotebookApp] Kernel started: 011d8a15-0e4a-448a-b02a-4121780e4bb6
If anyone has run into this before and can offer some things to check, I'd appreciate it. Googling around hasn't helped and I'd really like to get this working on the cloud. Thanks!
I am using EC2 but encounter same problem.
I uses SSL told in the tutorial, after login and open a notebook in Safari always showing "Connecting to kernel". Then I try Chrome, which gives warning about certificate but works fine.
Then I comment the certificate in config file, then open in Safari it works fine.
If you are using Firefox, may be this issue Unable to contact kernel in Firefox after updating to ipython 3.0
Hope this will help.
Update:
The issue is reported on Github: https://github.com/ipython/ipython/issues/8621. As said in the post, this is the limitation of Safari. If you still want to use SSL to visit Notebook, you have to add the certificate into your keychain and trust it. Check the help posted by Apple: https://support.apple.com/kb/PH18677?locale=en_US
Another way you can connect to your IPython (Jupyter) Notebook server is by opening an SSH tunnel with port forwarding. I wrote a how-to post describing my own workflow. I'm using Google Compute Engine from within Chrome but many of the steps are analogous to other setup variations:
https://stharrold.github.io/20151208-ipynb-on-gce-from-chrome.html
Excerpt:
"""
Brief setup routine:
* Start [your] virtual machine instance.
* Start a Jupyter Notebook server on the instance:
$ jupyter notebook --ip=0.0.0.0 --port=8888 --no-browser &
$ disown 1234 (where 1234 is the process ID)
* Create an SSH tunnel to forward a local port to the server's port on the instance:
$ ssh -f -N -L localhost:8888:0.0.0.0:8888 samuel_harrold#123.123.123.123
For [the] Chrome [app] Secure Shell, omit -f to keep the tunnel open (see screenshot [in post]).
* View the server at http://localhost:8888
"""
Additional details are in the post.