I'm new using Jupyter on Miniconda and I was having a problem while importing packages (ImportError: DLL load failed ), looking for answers the solution was to initialize a base environment in my bash.
I used to initialize jupyter typing jupyter notebook in bash, but using the solution given, I have to activate conda activate bash and then type jupyter notebook. What is the difference between starting Jupyter the way I used to and this new way?
conda activate command activates a virtual environment. It is an isolated environment so all packages you installed in the virtual environment cannot be used outside it. When you start bash, you are in the base environment and it seems that you installed your Jupiter in bash environment so you cannot use bash's Jupiter in base environment and vice versa. It may be a little annoying at the beginning, but it can let you use different environments for different purposes. For example, since pip only allows one version of a specific package to be installed, different environments can let you test a new version of a package without breaking the functionality of the original program.
Related
I am having problems installing modules and then importing them into specific Jupyter Notebook kernels. I want to install them directly into the kernel as opposed to throughout anaconda to separate dependencies in projects. Here is how the problem goes:
I firstly want a package, for example, nltk
I navigate to and activate the conda environment (called python3) and run 'conda install nltk'
I then load that environment into Jupyter using ipykernel with the command 'python -m ipykernel install --user --name python3'
When trying to import the package into the notebook it tells me that it cannot be found
I have been struggling with this for a while. Where am I going wrong? I greatly appreciate all the help.
NOTE: I have somehow managed to install and import many packages into notebooks using the aforementioned process. I'd really like a method to do this in a foolproof manner.
Not entirely clear where things go wrong, but perhaps clarifying some of the terminology could help:
"navigate to...the conda environment" - navigating has zero effect on anything. Most end-users should never enter or directly write to any environment directories.
"...and activate the conda environment" - activation is unnecessary - a more robust installation command is always to use a -n,--name argument:
conda install -n python3 nltk
This is more robust because it is not context-sensitive, i.e., it doesn't matter what (if any) environment is currently activated.
"load that environment into Jupyter using ipykernel" - that command registers the environment as a kernel at a user-level. That only ever needs to be run once per kernel - not after each new package installation. Loading the kernel happens when you are creating (or changing the settings of) a notebook. That is, you choose the kernel in the Jupyter GUI.
Even better, keep jupyter in a dedicated environment with an installation of nb_conda_kernels and Jupyter (launched from that dedicated environment) will auto-discover all Conda environments that have valid kernels installed (e.g., ipykernel, r-irkernel).
I have a conda environment containing all packages for jupyter notebook (say it's called jupyter_env. In a different conda environment I have R installed including r-irkernel (say the env is called R_env).
For python kernels I can easily make a python kernel in a specific environment (called e.g. pyth27) available to my jupyter installation in a different environment:
(pyth27) > python -m ipykernel install --prefix=/path/to/jupyter/env --name "python27"
Is there anything similar possible for the R kernel? So far I can only run the R kernel using a jupyter installation within the same environment(R_env).
One solution might be the nb-conda_kernels package. However there I'm not clear if it always adds all available kernels from all environments or whether I can specify which environments should be searched.
My question is similar to this one https://github.com/jupyter/jupyter/issues/397. Only that I don't want to use the base environment to start jupyter but a dedicated environment.
As described on https://github.com/IRkernel/IRkernel, the r-ikernel package provides a mechanism similar to python -m ipykernel install, to be run in R:
R> IRkernel::installspec()
To run this from Bash, you can do
(R_env)> Rscript -e "IRkernel::installspec()"
Now the tricky part, due to Jupyter and R being in different environments: According to https://github.com/IRkernel/IRkernel/issues/499, IRkernel::installspec() requires the jupyter-kernelspec command. I've tested two methods to provide it (to be done before issuing the above commands):
jupyter-kernelspec is part of Jupyter and hence in the file tree of jupyter_env, so add its path to PATH (I found it's better to add to the end so as to not disrupt other path lookups during the Rscript call)
(R_env)> export PATH="$PATH:</path/to/conda>/envs/jupyter_env/bin"
jupyter-kernelspec is included in the jupyter_client conda package, so you can do
(R_env)> conda install jupyter_client
Caveat: this installs a number of dependencies, including Python.
I opted for the first method to keep R_env free of Python packages.
On my latest clean install, I elected to run Jupyter in it's own virtual environment.
I just made a new venv, installed Jupyter and it's dependencies into it, and then ran jupyter lab from the activated venv. Everything worked fine, and it wasn't until later when I was reading up on some other stuff I noticed the recommended practice is usually to install a new IPython kernel into the environment and use that.
See here:https://medium.com/#eleroy/jupyter-notebook-in-a-virtual-environment-virtualenv-8f3c3448247
And here: https://anbasile.github.io/programming/2017/06/25/jupyter-venv/
My install seems to work fine, and this virtual environment is the only place where I have Jupyter/IPython installed. What is the difference between the way I'm running and the methods suggested in these articles?
Edited to address comments:
Method from links:
Create venv
Install dependencies into venv
Install new IPython kernel into venv
Fire up Jupyter and select newly created IPython kernel for notebooks.
My method:
Create venv
Install Jupyter into venv
Install dependencies into venv
Fire up Jupyter from inside activated venv. Only a single kernel is available.
This Real Python site had a decent explanation of the "why virtual" question. Here's a salient quote:
At its core, the main purpose of Python virtual environments is to
create an isolated environment for Python projects. This means that
each project can have its own dependencies, regardless of what
dependencies every other project has.
--EDIT--
(Since the OP added the steps, side-by-side, the answer is being updated to provided a side-by-side comparison.)
Because Basile created the virtual environment first, you would be able to create different dependency stacks for a new Python project or new Jupyter notebook.
(Speculation here, since I have not gone through the OP's steps:) Since the OP's method placed the Jupyter notebook inside the virtual environment, you'll be limited to that single set of dependencies.
I'm pretty new to programming so forgive my ignorance.
When installing certain python packages/modules in the cmd prompt I am able to import them fine when using Jupyter Notebook. But some modules Jupyter Notebook cannot import without installing via Conda first. Why is this?
The problem seems to be related to system and environment that you are using and not the programming :)
Since you are a beginner, let us understand the concepts first rather than solving the problem.
Python code is run on an interpreter that is installed on your machine.
Jupyter is a web application that takes code using a given language's interpreter. So Jupyter, on its own doesn't run your code. It uses the interpreter installed on a system(your local machine).
Conda is a package manager and also an environment manager. That means using conda, you can create a virtual environment on your machine and that virtual environment can have its own installation of an interpreter. This virtual environment can have its own copy of the packages/modules as well.
Now comes the best part: The jupyter notebook can be asked to use any of the interpreters including the ones installed on a virtual environment.
So most likely, you are running the jupyter notebook from an environment which doesn't have the required dependencies. So either run the jupyter notebook outside the environment or install the required packages in the environment where your jupyter notebook is running.
To know which environment is being used by your jupyter notebook, run below lines from the jupyter notebook cell:
import sys
sys.executable
If you don't get something like /usr/bin/python then the jupyter is running inside an environment. So you have to install all the packages/modules inside that environment only.
I've looked at all the responses in the search and specifically Use Conda environment in pycharm without resolving my problem.
I'm on Win10 using the latest Anaconda and PyCharm for Python 3.6. The situation is that I've created a new environment using Conda which uses the Python version available in the base environment. Therefore, there is no python.exe in the newly created environment. In PyCharm, when I try to select the Conda environment, the dialog apparently does not see a Python there and refuses to set that as the environment.
In PyCharm, I create a new Project (pure Python) and set the location for the project in my PyCharmProjects directory. I then try to select existing interpreter and navigate to the appropriate Conda environment which I'm unable to select.
I'm not sure if there is something I don't understand or there is actually a problem here, but I'm getting a bit frustrated trying
to find the proper approach. Any help appreciated.
--Don
When you create the conda environment from cmd prompt do you explicitly tell it what python you want to use?
conda create --environmentName python=3.6
That should place a python.exe in your environment.