I am new to using multiple Python environments and for some reason the base environment in my system just will not work for Jupyter notebooks. When I try to open a notebook from the base environment, I am getting the error phrase '500: Internal Server Error'. However, I am able to get notebooks opened if I create a new environment but even after installing the necessary modules, Python does not seem to recognise them. Another issue is that, once I terminate a session in the newly created environment, it too stops working giving the error phrase '500: Internal Server Error'. Basically, I am having to create environments again and again since the base environment itself is dysfunctional yet all the environments break upon termination.
I have also noted that '!python --version' and 'sys.version' returns two different values despite correct activation of the required environment (which is not the base environment) and opening a notebook from the activated environment.
Any solution for this problem will be appreciated.
The long-short:
One day in July, noticed that Jupyter wasn’t importing the version of Seaborn I installed to my Conda env. It was downloading an older Seaborn from a global dir. Same for all other packages when I checked versions. After various attempts at fixing this, Jupyter doesn’t even import packages now. I’ve tried with pipenv too. In both Conda and pipenv, sys.path reveals more path variables than I know what to do with, sometimes including the desired env path, sometimes not. But either way Jupyter imports are ignoring the env path I want to use, instead looking for global packages that I have deleted since July in attempt to solve the issue. On top of this (but probably intertwined in a way I just don’t understand yet), I’m not getting kernels that connect Jupyter to the desired env directories where packages are installed. Global python gets used instead of the env’s python instance. Not sure exactly how kernels are created, but I can tell that they are either not being created for some new envs, not accessing env-specific python & packages, or failing to connect (loads a stale kernel.json file and fails to start.)
Desired outcome:
How do I get JupyterLab to import the intended package version from the intended env directories? Even deeper, How do I get environments back to their former functionality of automatically [1] creating their own python instance, [2] creating their own kernel recognized by Jupyter, [3] creating their own path to the env, and [4] initializing all of that in Jupyter Lab?
Things I have tried:
Deactivated the base env, which admittedly I wasn’t doing for the first few weeks of July until I remembered that is a must for Windows Conda… but still now that I’m regularly deactivating base, why would these issues still persist?
Uninstalled Anaconda and reinstalled Miniconda
Deleted (I believe) all stray / older pythons from my machine, reinstalled a fresh user-level Python 3.9
Verified that packages are installing with the conda list command. They just won’t import properly in JupyterLab
Some tweaks to Path variables in Windows settings, but was very cautious and have no idea how to / if I am revising those properly. Clearly not though seeing as the issue is still alive 3 months later 0.o No idea if I should be editing user variables or system variables, or how to trim the paths cautiously.
Reinstalled jupyter and jupyterlab
Reinstalled ipykernel
Noted that sys.executable, sys.path, and !where python give different outputs in shell python versus Jupyter Lab python
Switching kernels manually in JupyterLab (usually the manually selected kernel is DOA)
Tried manually rewriting sys.path in notebook.
Tried launching JupyterLab from a fresh pipenv instead of Miniconda
Got a pipenv working correctly for a Streamlit applet in August, using raw python in VS Code (This may isolate the issue to Jupyter? I had to continue the other Jupyter-based project in Google collab since JupyterLab started choking in July.)
Deeper details:
Sheesh this post is already getting long… but here I’ve picked out the 4 most prominent code / error snippets. There are more where that came from, but hopefully there might be something in here that you might recognize as the needle in the haystack.
[1]
When the kernel-session.json causes a stillborn kernel, it is created as a totally blank json file. Running jupyter lab in the conda terminal yields this error message, buried in the output:
Failed to load connection file:
'C:\\Users\\David.000\\AppData\\Roaming\\jupyter\\runtime\\kernel-a6082e80-0b65-48f1-b370-7c2918030185.json'
I’ve found past kernel-session.json files that were not stillborn, such as this one called kernel-b146b600-81e3-418e-a55f-5a3fbbc13471.json, which looks like this, auto-populated:
{
"shell_port": 50877,
"iopub_port": 50878,
"stdin_port": 50879,
"control_port": 50880,
"hb_port": 50881,
"ip": "127.0.0.1",
"key": "caa915a4-00a599e8b6c4db6417bcca77",
"transport": "tcp",
"signature_scheme": "hmac-sha256",
"kernel_name": ""
}
[2]
sys.executable, when run in shell python, yields the proper python location within the env:
>>> sys.executable
'C:\\Users\\David.000\\miniconda3\\envs\\aucu_ml\\python.exe'
But sys.executable, when run in Jupyter Lab, is latching onto the global Python:
sys.executable
'C:\\Python39\\python.exe'
[3]
sys.path, when run in shell python, seems to yield the env specific paths I want (though why are there so many??)
>>> sys.path
['', 'C:\\Users\\David.000\\miniconda3\\envs\\aucu_ml\\python39.zip',
'C:\\Users\\David.000\\miniconda3\\envs\\aucu_ml\\DLLs',
'C:\\Users\\David.000\\miniconda3\\envs\\aucu_ml\\lib',
'C:\\Users\\David.000\\miniconda3\\envs\\aucu_ml',
'C:\\Users\\David.000\\AppData\\Roaming\\Python\\Python39\\site-packages',
'C:\\Users\\David.000\\miniconda3\\envs\\aucu_ml\\lib\\site-packages',
'C:\\Users\\David.000\\miniconda3\\envs\\aucu_ml\\lib\\site-packages\\win32',
'C:\\Users\\David.000\\miniconda3\\envs\\aucu_ml\\lib\\site-packages\\win32\\lib',
'C:\\Users\\David.000\\miniconda3\\envs\\aucu_ml\\lib\\site-packages\\Pythonwin']
But sys.path, when run in Jupyter Lab, is also showing a butt-load of paths, many of which are unwanted globals:
sys.path
[> ['C:\\Users\\David.000\\Desktop\\Civic Innovation Corps\\Miami\\Predictive_Analytics_Business_Licensing',
'C:\\Python39\\python39.zip',
'C:\\Python39\\DLLs',
'C:\\Python39\\lib',
'C:\\Python39',
'',
'C:\\Users\\David.000\\AppData\\Roaming\\Python\\Python39\\site-packages',
'C:\\Python39\\lib\\site-packages',
'C:\\Python39\\lib\\site-packages\\win32',
'C:\\Python39\\lib\\site-packages\\win32\\lib',
'C:\\Python39\\lib\\site-packages\\Pythonwin',
'C:\\Python39\\lib\\site-packages\\IPython\\extensions',
'C:\\Users\\David.000\\.ipython']
I'm probably not qualified to answer this but what the heck.
I have several kernels associated with different environments that are configured to run different programs. I created them in the terminal and I make it a practice to install any module/program etc, within the terminal in the activated environment that I need it in instead of installing within the notebook.
If I'm using a desktop, then before opening Juypter I select the appropriate environment within anaconda. If I'm using a virtual machine then within the terminal I activate the environment and once I'm in the notebook, select the appropriate kernel.
While I can't be sure, I think your problem is occurring because of how/where you're installing or updating packages.
I'm still learning how this all works, so please bear with me.
I'm running conda 4.8.5 on my Windows 10 machine. I've already installed all necessary Jupyter extensions, I think (Jupyter Lab, Jupyter Notebook, Jupyter Book, Node.js, and their dependencies).
The problem might have to do with the fact that I've installed Miniconda on a separate (D:/) drive.
I've set up a virtual environment (MyEnv) with all the packages I might need for this project. These are the steps I follow:
Launch CMD window
$ conda activate MyEnv
$ jupyter-lab --notebook-dir "Documents/Jupyter Books"
At this point a browser tab opens running Jupyter Lab
From the launcher within Jupyter Lab, open a terminal
$ cd "Documents/Jupyter Books"
$ jb create MyCoolBook
New folder with template book contents gets created in this directory (Yay!)
Without editing anything: $ jb build MyCoolBook
A folder gets added to MyCoolBook called _build, but it doesn't contain much more than a few CSS files.
The terminal throws this error traceback which wasn't very helpful to me. The issue may be obvious to an experienced user.
I am not sure how to proceed. I've reset the entire environment a few times trying to get this to work. What do you suggest? I'm considering submitting a bug report but I want to rule out the very reasonable possibility that I'm being silly.
I asked around in the Github page/forum for Jupyter Book. Turns out it's a matter of text encoding in Windows (I could have avoided this by reading deep into the documentation).
If anyone runs across this issue just know that it can be fixed by reverting to some release, Python 3.7.*, and setting an environment variable (PYTHONUTF8=1) but this is not something I would recommend because some other packages might require the default system encoding. Instead, follow the instructions in this section of the documentation.
VScode for python runs perfectly, without any error, problems starts whenever I try to run Jupyter notebook inside VScode.
It starts with Ipython Kernel loading and then it shows Command Failed Error. Any fix for this ? I have no idea what's the problem and I don't know much about the environment and stuffs also.
Thank You.
From your picture, VSCode said Command failed, and it's a very long command. It mainly includes two parts: activate the conda base environment and change the vscode python environment through "Python" extension. And from the top right of the picture, it shows the jupyter server start failed.
So you can try to do two things:
First one: change the environment. Instead of using conda, download and install a global python from the official website. And recommend creating a virtual environment. Then select the environment instead of conda base environment.
Second one: reinstall or downgrade the 'Python' extension(if the first one not work).
This problem can be caused by the package in the environment, so I recommend you just change to another environment.
VS Code is unable to detect the torch package that I installed exclusively in one of the Conda environments. If I launch VS Code through Anaconda Launcher, however, then it detects torch and everything works fine.
I have done many things in order to make the Conda environment work on VS Code PowerShell terminal, and so far Conda is getting activated on VS Code terminal properly regardless of whether I launch it with or without the Anaconda Launcher. But when trying to run the the program via Code Runner, it always shows the error that the torch module is not found.
But doing the same thing in terminal or launching VS Code through Anaconda launcher works just fine.
I don't know what to do, it is so annoying. I really wanted to work with Python on VS Code as it looks neat and clean compared to other editors and also since it is also light and fast. Unfortunately, making Conda work on VS Code is a nightmare, and it seems like the only way to make it work is through launching it via Anaconda Launcher, but opening Anaconda Launcher takes too much time compared to VS Code -_-
Please help assist with launching the code through VS Code.
It's because conda activation sets various environment variables which are required for a conda environment to function appropriately. And I suspect the Code Runner extension simply doesn't understand the concept of a conda environment, and so it isn't doing any activating which makes the import search fail.
You can try executing code directly through the Python extension instead of Code Runner to see if that works better for you.