I have a custom Jupyter kernel which runs IPython using a custom IPython profile which uses a matplotlib stylesheet.
I know to run this successfully normally I would put:
The matplotlib stylesheet in ~/.config/matplotlib/stylelib/
The IPython profile in ~/.ipython/
The kernel json in ~/.jupyter/kernels/my_kernel/
But I am doing this as part of larger program which runs in a virtualenv, and if I put the things as above then any notebook server running on the computer will be able to see the custom kernels, even if it is running outside the venv. I don't what this because I don't want my program to interfere with other notebooks on the computer.
I think what I need to do is put the things above somewhere equivalent inside the venv but I can't figure out where they should go. Doe anyone know where they would go? Or is this just a thing IPython/Jupiter can't/won't do?
It's probably worth mentioning that in the case of the stylesheet for example I don't want to just put it in the working directory of my program (which is one option matplotlib offers).
You can put kernelspecs in VIRTUAL_ENV/share/jupyter/kernels/ and they will be made available if the notebook server is running in that env. In general, <sys.prefix>/share/jupyter/kernels is included in the path to look for kernelspecs.
You can see the various locations Jupyter will look, you can see the output of jupyter --paths:
$ jupyter --paths
config:
/Users/you/.jupyter
/Users/you/env/etc/jupyter
/usr/local/etc/jupyter
/etc/jupyter
data:
/Users/you/Library/Jupyter
/Users/you/env/share/jupyter
/usr/local/share/jupyter
/usr/share/jupyter
runtime:
/Users/you/Library/Jupyter/runtime
Kernelspecs are considered data files, and will be found in any of those directories listed under data:, in a kernels subdirectory, e.g. /usr/local/share/jupyter/kernels.
Related
Summary:
I am trying to set up the Jupyter Notebook extension for VSCode but I can’t get my Python code snippets to run. There is no "Run" button to select. I can't select a Python/ Jupyter kernel to work with.
The problem in greater detail:
If you check out this YouTube clip at exactly 1 minute and 46 seconds (here - - you can pause it to see the still) this is what I am trying to accomplish.
But here is what my VSCode actually looks like:
As you can see in my screenshot, the “Play” icon to the left of my code snippet is missing. Along the top border, the other “Play All” button is missing.
In Microsoft’s official VSC doc for setting up Jupyter Notebooks, right near the top under this specific heading: “Create or open a Jupyter Notebook” their interface has way more options at the top of the cells when compared to mine. They have buttons named: “Variables”, “Restart”, “Interrupt”. That is expected for Jupyter Notebook funcionality. Mine is missing all of that. Clearly there is something wrong with my setup.
Other SO users report having ms-python extension installed (which includes Jupyter functionality) in addition to stand-alone Jupyter Notebook extensions installed. Apparently in situations when you have multiple Python extensions installed, they can interfere with each other. So after much wrangling, I resolved to uninstalling all my Jupyter extensions and keeping only the main ms-python extension.
The issue persists.
Here is a complete list of all my VSC installed extensions:
$ code --list-extensions --show-versions
arcticicestudio.nord-visual-studio-code#0.19.0
batisteo.vscode-django#1.8.0
bibhasdn.django-html#1.3.0
Compulim.indent4to2#0.1.2
Equinusocio.vsc-community-material-theme#1.4.4
Equinusocio.vsc-material-theme#33.5.0
equinusocio.vsc-material-theme-icons#2.3.1
esbenp.prettier-vscode#9.8.0
formulahendry.code-runner#0.11.8
GulajavaMinistudio.mayukaithemevsc#3.2.3
icrawl.discord-vscode#5.8.0
janisdd.vscode-edit-csv#0.6.10
lonefy.vscode-JS-CSS-HTML-formatter#0.2.3
m-thorsen.vscode-materia#0.4.4
ms-python.python#2022.12.1
NikolaosGeorgiou.html-fmt-vscode#0.6.0
PKief.material-icon-theme#4.20.0
samuelcolvin.jinjahtml#0.17.0
streetsidesoftware.code-spell-checker#2.7.0
yzhang.markdown-all-in-one#3.5.0
zamerick.black-ocean#1.0.2
zhuangtongfa.material-theme#3.15.2
There is a bit of noise in that list. I'm not sure what all of them refer to. But for certain there is only one Python app (ms-python) and nothing related to Jupyter.
Some SO members elsewhere have suggested selecting / setting the base Python version/environment. When I click the “Select Kernel” button at the top right (as illustrated in the screenshot below), it reveals an empty menu (see at the top left):
Here is “Python v3.8.5 (conda)” showing as selected at the bottom and slightly to the left:
Another exhaustive question and answer thread elsewhere on Stack Overflow involving an issue similar to mine includes many, many creative potential solutions: jupyter server : not started, no kernel in vs code
In that Stack Overflow question, one prominent member answers:
How I solved it:
Press Command+Shift+P to open a new command pallete
Type >Python: Select Intepreter to start jupyter notebook server
Open the notebook again
Someone else suggests:
Press Command+Shift+P on mac, Ctrl+Shift+p on windows
Type Jupyter: Select Interpreter to start Jupyterserver
It would show you a dropdown of python versions installed.
I chose python 3.7.5 and it worked for me. you can choose the python version installed on your machine.
Another member says:
I have seen all possible solutions but not work, finally I just
upgrade jupyter, notebook, and jupyterlab, like pip install -U jupyterlab, and I can choose the kernel in VScode!
None of the above resolves my issue.
Here is a quirky new twist: After installing jupyterlab with pip, after restarting VSC, I encountered this weird error:
It’s a condition box. I clicked: “Yes”.
Jupyter is still not integrated properly.
UPDATED: Based on JialeDu’s answer, I followed his instructions. I created a video shared below. It doesn’t quite capture every step, but I did follow all the steps. What you can see in the gif video is that my current working directory is a folder called Test, a new terminal has been opened, and there is an active virtual environment with $ source .venv/bin/activate. Next, as you can see in the .gif, I am able to select the latest Python interpreter v3.10.5 showing as “Recommended”. So that is progress. I couldn’t do that before. Going the virtual environment route was a great idea. However, I am still not able to get the Jupyter Notebook interface as the end product shared by JialeDu. When I invoke Ctrl + Shift + P again and type > create jup a new Jupyter Notebook file is created but the kernel I selected previously does not show and the expected Jupyter interface buttons and options are still not showing. Here is the video demo in .gif formatting:
Please note: Here you can see a list of 4 available Jupyter extensions:
I avoided installing any of these separate Jupyter extensions in the store because according to the instructor (and as confirmed on the official Microsoft Python extension ms-python page which I already have), it includes Jupyter built in. From the ms-python extension entry, it explains: "This Python extension will automatically install the Pylance and Jupyter extensions to give you the best experience when working with Python files and Jupyter notebooks."
Even though other Stack Overflow users indicated that mixing the above extensions such as ms-python with ms-toolsai was a bad idea, I experimented with having them both installed anyway. Didn’t fix the issue. I tried uninstalling ms-python and keeping ms-toolsai. Still no dice.
By the way, I’ve also got the latest version (as of this writing) of VSCode v1.70.1 installed:
VS Code has prompted you to install the suggested extension. Python and Jupyter.
Also it helps to make sure your vscode is up to date.
The play button of the interface is provided by Jupyter, but if you want to run the code (select the kernel), you must also install the Python extension.
It’s a condition box. I clicked: “Yes”.
You don't have to click Yes because there is a time delay. By the time you click Yes, the environment may not be ready.
If you are using a virtual environment, you can select the virtual environment's interpreter and then create a new terminal to activate the environment.
Complete creation steps:
Prerequisites:
The machine has python installed,
and vscode has Python and Jupyter extensions installed.
steps
Create a new working folder Test
Open the folder Test with vscode
new terminal
Type the command python -m venv .venv to create a virtual environment
After the creation is complete, use the command .venv/scripts/activate to activate the virtual environment
Or Ctrl+Shift+P --> Python:Select Interpreter, select the interpreter of the virtual environment and then create a new terminal to activate the environment
Create new jupyter file using command palette command Create:New Jupyter Notebook
Or right click --> New File --> name ends with .ipynb
The virtual environment you just selected will be enabled as the kernel by default.
I discovered a very simple answer: I installed the VSC package in the official Snap repo.
VSC and Python-Jupyter look to be all playing nice together now:
In my Pycharm projects I use Jupyter to experiment and when something is mature enough I move it to Python files within the project, and make the notebook use it.
From time to time, while running a notebook I change that Python code but am forced to manually restart the server in order to make the notebook use the changed code.
I'd like Pycharm to automatically do it for me.
Is there a way?
I'm still learning how this all works, so please bear with me.
I'm running conda 4.8.5 on my Windows 10 machine. I've already installed all necessary Jupyter extensions, I think (Jupyter Lab, Jupyter Notebook, Jupyter Book, Node.js, and their dependencies).
The problem might have to do with the fact that I've installed Miniconda on a separate (D:/) drive.
I've set up a virtual environment (MyEnv) with all the packages I might need for this project. These are the steps I follow:
Launch CMD window
$ conda activate MyEnv
$ jupyter-lab --notebook-dir "Documents/Jupyter Books"
At this point a browser tab opens running Jupyter Lab
From the launcher within Jupyter Lab, open a terminal
$ cd "Documents/Jupyter Books"
$ jb create MyCoolBook
New folder with template book contents gets created in this directory (Yay!)
Without editing anything: $ jb build MyCoolBook
A folder gets added to MyCoolBook called _build, but it doesn't contain much more than a few CSS files.
The terminal throws this error traceback which wasn't very helpful to me. The issue may be obvious to an experienced user.
I am not sure how to proceed. I've reset the entire environment a few times trying to get this to work. What do you suggest? I'm considering submitting a bug report but I want to rule out the very reasonable possibility that I'm being silly.
I asked around in the Github page/forum for Jupyter Book. Turns out it's a matter of text encoding in Windows (I could have avoided this by reading deep into the documentation).
If anyone runs across this issue just know that it can be fixed by reverting to some release, Python 3.7.*, and setting an environment variable (PYTHONUTF8=1) but this is not something I would recommend because some other packages might require the default system encoding. Instead, follow the instructions in this section of the documentation.
I am trying to use the guidance on http://www.slideshare.net/fullscreen/randyzwitch/ipython-ec2/12 to install public Ipython notebooks in an AWS instance. One problem that i encounter is, when i try to create a profile, i do not not observe the creation of an ipython_notebook_config_py file (as explained in the tutoral, as per the screenshot), but i only get a ipython_kernel_config.py file, which has very different contents, and cannot be edited in the way as explained in the tutoral. Can someone help me to understand why this happens, and what i should do subsequently? Many thanks.
The notebook server is no longer part of IPython; it's a separate Jupyter project, which has its own config directory ~/.jupyter. The config file for the notebook server is ~/.jupyter/jupyter_notebook_config.py. You can use this to configure the notebook server. If you want to keep multiple configurations of the notebook server, you can use the environment variable JUPYTER_CONFIG_DIR to specify that a different directory should be used:
JUPYTER_CONFIG_DIR=~/jupyter_nbserver jupyter notebook
Note: IPython has not lost its profiles or config files, so ~/.ipython/profile_default/ipython_config.py and startup files, etc. continue to work as before for configuring the IPython kernel, just not the notebook server itself.
References:
Running a public notebook server
Jupyter configuration
Migration from IPython to Jupyter
There was official(?) recommendation of running an IPython Notebook server, and creating a profile via
$ ipython profile create nbserver
as recommended in http://ipython.org/ipython-doc/1/interactive/public_server.html. This allowed for very different and very useful behavior when starting an IPython Notebook via ipython notebook and ipython notebook --profile=nbserver.
With Jupyter 4.0, there's a change and there are no longer profiles. I've found the conversation https://gitter.im/ipython/ipython/archives/2015/05/29 which has user minrk saying:
The .ipython directory has several things in it:
multiple config directories (called profiles)
one 'data' directory, containing things like kernelspecs, nbextensions
runtime info scattered throughout, but mostly in profiles
Jupyter follows more platform-appropriate conventions:
one config dir at JUPYTER_CONFIG_DIR, default: .jupyter
one data dir at JUPYTER_DATA_DIR, default: platform-specific
one runtime dir at JUPYTER_RUNTIME_DIR, default: platform-specific
And a rather cryptic remark:
If you want to use different config, specify a different config directory with JUPYTER_CONFIG_DIR=whatever
What's the best way to get different behavior (say, between when running as a server vs normal usage)?
Will it involve running something like:
$ export JUPYTER_CONFIG_DIR=~/.jupyter-nbserver
$ jupyter notebook
whenever a server 'profile' needs to be run? and
$ export JUPYTER_CONFIG_DIR=~/.jupyter
$ jupyter notebook
whenever a 'normal' profile needs to run? Because that seems terrible. What's the best way to do this in Jupyter 4.0?
Using some code from this blog post http://www.svds.com/jupyter-notebook-best-practices-for-data-science/ and updating it. The easiest solution appears to be to create an alias, like:
alias jupyter-nbserver='JUPYTER_CONFIG_DIR=~/.jupyter-nbserver jupyter notebook'
So now you can run the jupyter notebook with a different config via the simple command jupyter-nbserver.
A more robust solution might involve creating a bash function that changes the environment variable, checks whether there's a config file, if not creating one, then executing, but that's probably overkill. The answer that I give on this related question https://stackoverflow.com/a/32516200/246856 goes into creating the initial config files for a new 'profile'.