It seems that DataSpell is trying to execute this command: C:\Windows\system32\wsl.exe --distribution Debian --exec /bin/sh -c "export LANGUAGE='' && export LC_ALL=en_US.UTF-8 && export LANG=en_US.UTF-8 && /usr/bin/python3 -m jupyter notebook --no-browser '--notebook-dir=/mnt/c/Users/Andy Zhou/Desktop/Year 2 stuff/GPT-2/code/SERI MATS IOI' --ip=172.22.246.59"
However, when I directly execute the part after --exec on WSL it works.
Adding some additional information regarding your problem, such as the error code that DataSpell likely returned to you, or whether or not htop shows a running Jupyter server, would make providing an accurate answer much easier.
As such, I believe your question could be read two ways, and I've provided an answer for each.
Server starts but will not connect
When DataSpell launches a local WSL-based Jupyter server it makes certain assumptions about how the connection should work, it also uses the LAN address for your WSL instance to attempt to connect. The default Jupyter config assumes a local connection, so DataSpell's connection via an external IP address is rejected immediately.
Steps to resolve this issue:
In WSL run jupyter notebook --generate-config, it will print out the path of your new config file
Vim into the new file and set the following values:
# Please note that the below values can be unsafe, consider changing these values to only allow your IP address to connect; alternatively you could require authentication to access the server.
## The IP address the notebook server will listen on.
c.NotebookApp.ip = '0.0.0.0'
## Set the Access-Control-Allow-Origin header
c.NotebookApp.allow_origin = '*'
## Allow requests where the Host header doesn't point to a local server
c.NotebookApp.allow_remote_access = True
Configure a WSL Python interpreter, detailed here: https://www.jetbrains.com/help/dataspell/using-wsl-as-a-remote-interpreter.html
Change the Jupyter connection for your project to use the interpreter you just set up, detailed here: https://www.jetbrains.com/help/dataspell/using-wsl-as-a-remote-interpreter.html
Run a cell in your notebook, a server should start up automatically and connect just fine.
A good resource on this is the following question: Why I can't access remote Jupyter Notebook server?
Sever does not start when using WSL
Unfortunately this is far more broad, and will almost certainly require more information to solve, but the following issues are highly likely causes:
Your WSL installation didn't include rsync: https://www.jetbrains.com/help/dataspell/using-wsl-as-a-remote-interpreter.html#prereq
WSl does not have Jupyter installed:
Install Jupyter via pip, or conda: {pip|conda} install jupyter
WSL isn't running when DataSpell attempts to run a notebook
Unfortunately without more information, or at least an error code, it isn't possible to give you a definitive answer; but hopefully this helped you in the right direction!
Related
I need some advice. So I am a big fan of VS Code and I always use its embedded notebooks. I built a remote Jupyter Server on Oracle Cloud hoping I could connect from vscode. To create the server I based on this article, but migrating as advised by Jupyter to JupyterServer. I've also used miniconda isntead of venv.
The server seems to work correctly, I can access it from my browser and in my Windows Terminal SSH, open Jupyter Lab, create and run noteboooks in it, etc. The problem is when I try to use it with VS Code, when I try to specify de Jupyter Server for connections, it allows me to do it, it even prompts me that it is an insecure connection (I use self signed ssl certificate), and it does mark Jupyter Server: Remote BUT, when I try to select my interpreter, change my kernel, it only shows my local conda envs. if I run !hostname it shows me my local hostname, not my remotes, it isn't really connecting or using the remote Jupyter server to run the cells.
I've looked around and can`t find a way to make it work, I really want it to work with VS Code, any help?
This has no impact on the actual use of jupyter. Your confusion is actually a misunderstanding caused by the definition of names.
As stated in the official document, when you connect to a remote server, everything runs in the server ather than the local computer.
At present, there is an issue for changing the naming on GitHub, which you can read in detail.
I want to use a remote Jupyter server via SSH with VSCode but I get an error whenever I try to specify the URI of any server. This also happens with local instances of Jupyter. Any server that is not started by VS Code seems to be unusable.
I am just starting a server like this
$ jupyter notebook --no-browser --port 8080
Then I enter the address in VSCode with the correct port and token
Nothing happens and I get this error, as you can see in the log below.
Command 'Python: Specify local or remote Jupyter server for connections' resulted in an error (Running the contributed command: 'python.datascience.selectjupyteruri' failed.)
Has anyone else been experiencing anything similar? I honestly have no idea how to troubleshoot this. It is worth noting that selecting the Default: VS Code will automatically start a server for you on localhost option raises the same error.
It looks like you're running a fairly old version of the Python extension, as the python.datascience.selectjupyteruri command no longer exists in the Python extension—it is now provided in a standalone Jupyter extension. Please consider upgrading to the latest version of the Python and Jupyter extensions. If that doesn't resolve the problem, please file an issue at https://github.com/microsoft/vscode-jupyter/issues and we'll be happy to help.
We had to restart our R Server (EC2 box), and following its restart, my Jupyter notebooks are throwing errors that regularly-used, imported modules cannot be found:
(in Jupyter Notebook cell)
import MySQLdb as mysql
ImportError: No module named 'MySQLdb'
But when I used the command line to verify that the module is imported (based off of the suggestion here), I saw that it was still present in the system. Is there something I'm missing, here?
(base)$ python -c "import MySQLdb"
(base)$ echo $?
0
For context, the Jupyter file used was using the MySQLdb module prior to the restart of the server. Also, not sure if it's important to know, but I had to reconnect my hard drive from the command line post-reboot as well:
sudo mount /dev/xvdf /mnt/ebs
After doing some back-tracking, we were able to resolve the issue!
Jupyter was initiated prior to mounting the server to the hard drive. Our Python modules are within the Anaconda environment in the server, and without the hard drive's connection to the server, the imported Jupyter pointed to the computer's python environment.
Posting steps on how we resolved the issue in case it comes up for others:
We rebooted the server again. This was to give the reconnection channels a re-start.
After a reboot:
➜ ~ ssh # onto the EC2 box
(base):~$ sudo mount /dev/xvdf /mnt/ebs # reattach external drive
(base):~$ screen -S somenameforscreensession # start a screen session (persistent session)
(base):~$ jupyter notebook --port=#### # start jupyter within that persistent session
I am leading a team of analysts and want to introduce them to Jupyter Notebook as a window into Python programming.
We have Anaconda downloaded and installed on our Linux server. I've asked our IT to help set it up to run on Google Chrome and they have been able to only provide the following steps:
source /R_Data/anaconda3/etc/profile.d/conda.sh
this kicks off Anaconda on the server, must run in PUTTY. We stored the installation in the same location as RStudio hence the R_Data in the filepath.
/R_Data/anaconda3/bin/jupyter-notebook --ip 0.0.0.0 --port 8889
This sets up the port 8889 with a token generated each time from scratch. We then need to grab the token id and paste into Chrome with the full URL per step 3
http://localhost:8889/?token=ea97e502a7f45d....
When I paste this in Chrome it loads Jupyter.
While this gets the job done it seems less than ideal for an entire team of analysts to have to do this each time. We also have RStudio installed on the same server but that simply opens from Chrome using a URL since I assume it is always running in the background. Jupyter and Anaconda seem to only run once they are kicked off first in PUTTY and I would like a way to bypass those steps.
I am familiar with the Jupyter config file however my limited understanding as a non-developer tells me it applies only to each user and cannot be applied to all users simultaneously (i.e. as a root user on the server or something to that effect).
I am hoping someone here might point me in the right direction. I should also point out that as a Redhat user I can't follow instructions based in Ubuntu since that syntax seems different.
Many thanks for the help.
Yoni
A convenient way is to run jupyter notebook --no-browser --port=12345 on your server while connecting through the ssh tunel as ssh -N -f -L 12345:localhost:12345 myserveralias. Now jupyter is on your 12345 localhost. Things like AutoSSH or Keep Alive will help with an erratic network, however, take security into account.
Dropdown menu in the top-right of the UI on a local machine (PC):
Kernel->
Change kernel->
Python 2 (on a local PC)
Python 3 (on a local PC)
My new kernel (on a remote PC)
The IPython notebook talks to the kernels over predefined ports. To talk to a remote kernel, you just need to forward the ports to the remote machine as part of the kernel initialisation, the notebook doesn't care where the kernel is as long as it can talk to it.
You could either set up a wrapper script that gets called in the kernel spec file (https://ipython.org/ipython-doc/dev/development/kernels.html#kernel-specs) or use a module that can help you set up and manage different kinds of remote kernels: (pip install remote_ikernel; https://bitbucket.org/tdaff/remote_ikernel).
If you are using remote_ikernel, and have ssh access to the machine, the following command will set up the entry in the drop down list:
remote_ikernel manage --add \
--kernel_cmd="ipython kernel -f {connection_file}" \
--name="Remote Python" --interface=ssh \
--host=my_remote_machine
IPython use kernel is a file in ~/.ipython/kernel/<name> that describe how to launch a kernel. If you create your own kernel (remote, or whatever) it's up to you to have the program run the remote kernel and bind locally to the port the notebook is expected.
Remote jupyter kernel/kernels administration utility (the rk):
https://github.com/korniichuk/rk
Install the rk from GitHub:
$ sudo pip install git+git://github.com/korniichuk/rk#egg=rk
Setup SSH for auto login without a password:
$ rk ssh
Install a template of a remote jupyter kernel:
$ rk install-template
Change the kernel.json file:
$ sudo gedit /usr/local/share/jupyter/kernels/template/kernel.json
For example from remote_username#remote_host to albert#192.168.0.1.
Click: Quickstart and YouTube video (less than 3 min).
remote_ikernel in an earlier answer hasn't been updated in a while. It may still work fine, but as a more recent/updated option with some additional features, I've just tested ssh_ipykernel, and it works well for this. I'm using it to run a kernel on a remote machine with a GPU, connecting via SSH.