my goal is connect to goocle colab gpu from my local machine via jupyter notebook
done this things from documentation:
pip install jupyter_http_over_ws
jupyter serverextension enable --py jupyter_http_over_ws
opening jupyter with this command:
jupyter notebook --NotebookApp.allow_origin='https://colab.research.google.com' --port=8888 --NotebookApp.port_retries=0
after i'm connecting to colab with my url from terminal like this:
after this i've got this in my terminal
[I 18:12:04.374 NotebookApp] 302 GET /?token=HERE IS MY TOKEN (MY IP) 0.000000ms
actually idk what is 302 GET
finally, if i'm using os.getcwd() in Colab that shows me my LOCAL directory from my PC
and print(torch.cuda.is_available()) is False
so i've connected to Colab from my machine, but goal is vice versa, to get GPU on my local machine. so maybe i'm doing something wrong.
I think you get it wrong.
Colab is only GUI which runs in web browser and on normal Google web server (with some access to Google Drive), not on special Google server with GPU.
Colab can connect to hosted runtime which means Google server (hardware) with GPU - and then you can directly access files on this server and you can run code on hardware with GPU.
Or it can connect to local runtime which means your local computer (hardware) without GPU - and then you can directly access local files and you can run code only on local hardware.
You don't have access to both runtimes (hardwares) at the same.
This is how I see it
Connect to Google Server with GPU/TPU:
Connect to Local Computer without GPU:
Related
It's the first time that I am trying to use google credits, so I apologize if it's a basic question. I am trying to see how to connect google credits into google colab by this site https://medium.com/#senthilnathangautham/colab-gcp-compute-how-to-link-them-together-98747e8d940e (you can open it by creating a new incognito window).
I am stucked in step 3 because I can't see any SSH in my google cloud. Also the numbers after the -L are fixed? If not, how can I found them?
gcloud compute ssh colab-backend --zone=europe-west4-a -L 8081:locahost:8081
EDIT: I am trying to run the above line of code in Google Cloud SDK Shell, but I have this error.
Also, I can't type in the terminal jupiter notebook. If I run the above code in the python 3 jupyter notebook I have this strange error.
Actually, I had a very silly basic mistake. First of all your server should be linux and then write the following code in the windows command:
cloud compute ssh colab-backend --zone=europe-west4-a -L 8080:locahost:8081, change it into the zone of your project and port
type
jupyter notebook --NotebookApp.allow_origin='https://colab.research.google.com' --port=8888 --NotebookApp.port_retries=0 --no-browser
copy past the link into 'local runtime' in your google colab.
I'm on a Windows 10 machine. I have GPU running on the Google Cloud Platform to train deep learning models.
Historically, I have been running Jupyter notebooks on the cloud server without problem, but recently began preferring to run Python notebooks in VS Code instead of the server based Jupyter notebooks. I'd like to train my VS Code notebooks on my GPUs but I don't have access to my google instances from VS Code, I can only run locally on my CPU.
Normally, to run a typical model, I spin up my instance on the cloud.google.com Compute Engine interface. I use the Ubuntu on the Windows Subsystem for Linux installation and I get in like this:
gcloud compute ssh --zone=$ZONE jupyter#$INSTANCE_NAME -- -L 8080:localhost:8080
I have tried installing the Cloud Code extension so far on VS Code, but as I go through the tutorials, I always sort of get stuck. One error I keep experiencing is that gcloud won't work on anything EXCEPT my Ubuntu terminal. I'd like it to work in the terminal inside VS Code.
Alternatively, I'd like to run the code . command on my Ubuntu command line so I can open VS Code from there, and that won't work. I've googled a few solutions, but they lead me to these same problems with neither gcloud not working, nor code . working.
Edit: I just tried the Google Cloud SDK installer from https://cloud.google.com/sdk/docs/quickstart-windows
and then I tried running gcloud compute ssh from the powershell from within VSCODE. This is the new error I got:
(base) PS C:\Users\user\Documents\dev\project\python> gcloud compute ssh --zone=$ZONE jupyter#$INSTANCE_NAME -- -L 8080:localhost:8080
WARNING: The PuTTY PPK SSH key file for gcloud does not exist.
WARNING: The public SSH key file for gcloud does not exist.
WARNING: The private SSH key file for gcloud does not exist.
WARNING: You do not have an SSH key for gcloud.
WARNING: SSH keygen will be executed to generate a key.
ERROR: (gcloud.compute.ssh) could not parse resource []
It still runs from Ubuntu using WSL, I logged in fine. I guess I just don't know entirely enough about how they're separated, what's shared, and what is missing, and to how to get all my command lines using the same stuff.
It seems as if your ssh key paths are configured correctly for your Ubuntu terminal but not for the VS Code one. If your account is not configured to use OS Login, with which Compute Engine stores the generated key with your user account, local SSH keys are needed. SSH keys are specific to each instance you want to access and here is where you can find them. Once you have find them you can specify their path using the --ssh-key-file flag.
Another option is to use OS Login as I have mentioned before.
Here you have another thread with a similar problem than yours.
Remotely accessed Jupyter notebook from a linux machine has working too slowly and ambari server hosts shows the components are down ?
Working jupyter notebook can connected via putty from that machine using no-browser commnd.
Can you tell me what is the use of jupyter cluster. I created jupyter cluster,and established its connection.But still I'm confused,how to use this cluster effectively?
Thank you
With Jupyter Notebook cluster, you can run notebook on the local machine and connect to the notebook on the cluster by setting the appropriate port number. Example code:
Go to Server using ssh username#ip_address to server.
Set up the port number for running notebook. On remote terminal run jupyter notebook --no-browser --port=7800
On your local terminal run ssh -N -f -L localhost:8001:localhost:7800 username#ip_address of server.
Open web browser on local machine and go to http://localhost:8001/
I've been trying to get ipython/jupyter notebook running as a notebook server on Google Compute Engine (Ubuntu 14.04 VM), but can't connect to the Python kernel.
I started with the latest Anaconda distribution. I can connect to the notebook server from my local machine via the web, browse the directory tree of the VM, and create new notebook, but never connects to the kernel (orange message in the upper right of Jupyter). The notebook port (8888) is open in the firewall settings. iPython notebook returns the following:
[I 19:21:10.152 NotebookApp] Using MathJax from CDN: https://cdn.mathjax.org/mathjax/latest/MathJax.js
[I 19:21:10.169 NotebookApp] Serving notebooks from local directory: /home/rattlerray
[I 19:21:10.170 NotebookApp] 0 active kernels
[I 19:21:10.170 NotebookApp] The IPython Notebook is running at: https://[all ip addresses on your system]:8888/
[I 19:21:10.170 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[I 19:22:13.835 NotebookApp] 302 GET / (98.169.96.157) 0.85ms
[I 19:22:31.902 NotebookApp] Creating new notebook in
[I 19:22:33.634 NotebookApp] Kernel started: 011d8a15-0e4a-448a-b02a-4121780e4bb6
If anyone has run into this before and can offer some things to check, I'd appreciate it. Googling around hasn't helped and I'd really like to get this working on the cloud. Thanks!
I am using EC2 but encounter same problem.
I uses SSL told in the tutorial, after login and open a notebook in Safari always showing "Connecting to kernel". Then I try Chrome, which gives warning about certificate but works fine.
Then I comment the certificate in config file, then open in Safari it works fine.
If you are using Firefox, may be this issue Unable to contact kernel in Firefox after updating to ipython 3.0
Hope this will help.
Update:
The issue is reported on Github: https://github.com/ipython/ipython/issues/8621. As said in the post, this is the limitation of Safari. If you still want to use SSL to visit Notebook, you have to add the certificate into your keychain and trust it. Check the help posted by Apple: https://support.apple.com/kb/PH18677?locale=en_US
Another way you can connect to your IPython (Jupyter) Notebook server is by opening an SSH tunnel with port forwarding. I wrote a how-to post describing my own workflow. I'm using Google Compute Engine from within Chrome but many of the steps are analogous to other setup variations:
https://stharrold.github.io/20151208-ipynb-on-gce-from-chrome.html
Excerpt:
"""
Brief setup routine:
* Start [your] virtual machine instance.
* Start a Jupyter Notebook server on the instance:
$ jupyter notebook --ip=0.0.0.0 --port=8888 --no-browser &
$ disown 1234 (where 1234 is the process ID)
* Create an SSH tunnel to forward a local port to the server's port on the instance:
$ ssh -f -N -L localhost:8888:0.0.0.0:8888 samuel_harrold#123.123.123.123
For [the] Chrome [app] Secure Shell, omit -f to keep the tunnel open (see screenshot [in post]).
* View the server at http://localhost:8888
"""
Additional details are in the post.