I am working in notebooks provided in the Workbench section of Vertex AI. I need an updated version of Python, but I only have access to Python 3.7 in these notebooks. I have successfully followed these steps and if I run python3.8 --version in terminal, I get Python 3.8.2, which is good, but python --version still returns Python 3.7.12. If, following this answer and restarting notebook's kernel, I run
from platform import python_version
print(python_version())
in a notebook, and I get 3.7.12.
How do I get a notebook in Vertex AI supporting an up-to-date Python version?
#create a new conda env:
$ conda create -n python38 python=3.8
#Activate your new Python 3.8 environment:
$ conda activate python38
#install ipykernel when logged in the new env:
(python38)$ conda install ipykernel
Refresh the page and the new python38 env will be avaiable:
For me, #ewertonvsilva's answer wasn't working.
I had to install the ipython kernel at user level then deactivate the kernel before refreshing the page to make the environment appearing on Jupyter Lab :
All commands combined :
# create a new conda env:
$ conda create -n python38 python=3.8
# Activate your new Python 3.8 environment:
$ conda activate python38
#install ipykernel when logged in the new env:
(python38)$ conda install ipykernel
# install the ipython kernel at user level
(python38)$ ipython kernel install --user --name=python38
# Deactivate the new environment
(python38)$ conda deactivate
Then refresh the page.
Sources :
How to add conda environment to jupyter lab
Installing the IPython kernel - IPython Documentation
After a few trials and errors, I found that:
Changes under /opt/conda will not be preserved when you change/stop instances.
Custom conda env needs to be saved to $HOME.
conda activate $ENV will not work, since conda init was not executed.
pip install installs to /opt/conda/lib/python3.7/site-packages, which will not be preserved.
pip install --user installs to ~/.local/lib/python3.7, which will be applied to every python3.7 kernel, kinda messy.
In notebook, !pip uses pip in shell, not in current kernel.
In notebook, %pip uses pip in current kernel.
In custom env, don't use --user flag, or it installs to ~/.local/lib.
Run the commands in system terminal:
VENV=new_env
# create new env in `$HOME`
conda create -y -q -p $HOME/conda_env/$VENV python=3.8 ipykernel
# activate env
source /opt/conda/bin/activate ~/conda_env/$VENV
# register kernel to `$HOME/.local/share/jupyter/kernels`, so it will be preserved
python -m ipykernel install --user --name $VENV
# install your packages, WITHOUT `--user`
pip install numpy==1.22
# check package installation path
pip list -v
Now you can change kernel in Launcher (takes a few minutes to refresh), or the box on the top right of notebook. The kernel will be preserved.
In notebook with custom env:
# use `%pip` in notebook, instead of `!pip`
%pip install numpy==1.22 # `/home/jupyter/conda_env/$VENV/lib/python3.8/site-packages`
!pip install numpy==1.22 # `/opt/conda/lib/python3.7/site-packages`
!pip install --user numpy==1.22 # `~/.local/lib/python3.7/site-packages`
# you may see the difference in custom env
%pip list -v
!pip list -v
Related
I created a new python env using Anaconda Prompt by using the following steps:
conda create --name py3-TF2.0 python=3
conda activate py3-TF2.0
conda install tensorflow
pip install --upgrade tensorflow
pip install ipykernel
then i restarted anaconda and opened jupyter notebook.
The problem is when i open a new notebook and go to kernel to choose the new env i could not find the env i created (p3-TF2.0)
however, when I use conda info --envs, i find the new env created.
what is missing here?
thanks in advance
After activating the environment, try running:
python -m ipykernel install --user --name py3-TF2.0 --display-name "py3-TF2.0"
If this still isn't pointing to the correct environment (i.e. you can't import packages you know you installed in that environment), remove the py3-TF2.0 kernel that's broken with jupyter kernelspec remove py3-TF2.0.
Then run <your_path_to>/anaconda3/envs/py3-TF2.0/bin/python -m ipykernel install --user --name py3-TF2.0 --display-name "py3_TF2.0".
Additional help on Jupyter Kernels.
I have used !{sys.executable} -m pip install to install packages i want to use in my jupyter notebooks.
Now i also want to install some packaged by conda and use in the same notebooks.
This is the command i run and the error message
They ask us to create a separate conda environment. I am confused about this.
Do we need to create a new conda enviroment everytime we need new packages?
What is the correct way to install and access conda packages on jupyter python notebook?
You shouldn't in principle install from the jupyter notebook but from the terminal/cmd.
You can create an enviroment >conda create --name myenv
And activate it: >conda activate myenv
now everything you install will be restricted to myenv. for example conda install numpy
To use the packages in myenv in jupyter simply open jupyter with your enviroment active:
>conda activate myenv
>jupyter notebook
Note that you have to install jupyter in your enviroment too: conda install jupyter
I have 2 conda environments installed:
- env1: base environment where jupyter-notebook is installed and started from
- env2: project environment with ipykernel installed
I manually added kernelspecs for the 2 environments following this guide.
Everything works fine. sys.executable in 2 kernels show separate, correct paths. But for terminal commands (i.e. !which python), no matter which kernel I'm running in the environment defaults to env1.
Is there any way to have the notebook automatically change this to the kernel's environment?
P.S. I already tried installing nb_conda, nb_conda_kernels
install nb_conda and nb_conda_kernels into your base.
conda install nb_conda nb_conda_kernels -n env1
This should give you the ability to change kernel in jupyter, and use the env2 kernel.
I would install jupyter notebook in the base env (not env1, not env2)
Then install nb_conda_kernels in the base
in env1 and env2, install ipykernel
in env1 and env2, run this:
python -m ipykernel install --user --name env1 --display-name "env1 env"
Check this out for more info:
New Conda environment with latest Python Version for Jupyter Notebook
I am trying to test some code on Python 3.6 but my system default python version is python 3.5 I have installed python 3.6 and have tried to follow the jupyter documentation to install a new python kernel
python3.6 -m pip install ipykernel
python3.6 -m ipykernel install --user
But it didn't work since it continues to show a single kernel in the menu: Python3
Has anyone managed to have both 3.5 and 3.6 in the same jupyter installation?
one option is to create a kernel that you can use in jupyter notebook.
you can do this inside the virtual environment:
Open up your terminal and enter the following line by line
virtualenv -p python3.6 py_36_env
source py_36_env/bin/activate
pip install ipykernel
python -m ipykernel install --user --name=py_36_env
jupyter notebook
Then in jupyter notebook you can select the 3.6 environment (py_36_env) from the 'New' drop down menu shown above or from the 'Kernel' drop down menu within a given jupyter notebook.
Steps to install Python 3.6 in Windows
Open command prompt
conda install ipykernel
conda create -n Python3.6Test python=3.6
Activate Python3.6Test
pip install ipykernel
python -m ipykernel install --name Python3.6Test
Here you go with 3.6
C:\ProgramData\jupyter\kernels\Python3.6Test
Now open Jupitor Notebook and see, you will get the Python3.6Test option
Following worked for me:
Commands are executed in Jupyter Notebook (OS: Ubuntu 16.04 LTS)
Upgrade pip:
!pip install --upgrade pip
Install virtual environment:
!pip install virtualenv
Select version of Python you want to use in new environment:
I wanted to create an environment with Python version 3.6 Naming it as Python_3_6:
!virtualenv -p python3.6 Python_3_6
After execution, this will create a folder with the same name in the current working directory (i.e. the location where Jupyter notebook is present)
Create a new option with the name of the created environment
And finally, run the following command:
!python -m ipykernel install --user --name=Python_3_6
This will create a new option with the name Python_3_6 in the menu from where we create a new notebook.
NOTE: One can run above commands from the terminal as well just don't use '!' before the commands.
This is an old question, but like some commenters here I wanted to change the default jupyter completely without using a venv. It seems jupyter keeps this in a json file called kernel.json:
{
"display_name": "Python 3",
"language": "python",
"argv": [
"/path/to/some/python",
"-m",
"ipykernel_launcher",
"-f",
"{connection_file}"
]
}
There may be several paths containing this file which are searched in order. To find out what they are on your machine, run:
ipython kernel install --prefix tmp
ipython will throw a warning that the config file is being created in tmp and might not be found, but it will also output the paths it uses to find kernel.json, which you can then edit manually.
You should create a virtualenv for each Python version you are using. Create one for Python 3.5:
virtualenv -p /usr/bin/python3.5 py35
source py35/bin/activate
pip install jupyter
jupyter # Jupyter running Python 3.5
deactivate # Leave virtualenv
And create one for Python 3.6:
virtualenv -p /usr/bin/python3.6 py36
source py36/bin/activate
pip install jupyter
jupyter # Jupyter running Python 3.6
upper solution does not work in my case!!
I want to execute code in python3.8 but unfortunately I previous install pytohn3.6
I create, activate virtualenv of python3.8 but during lib installation all package installed in virtualenv of python3.6
Solution is that I deleted python3.6 env & Setup a new way of Jupyter kernel. After that mine code execute happily & required lib install in virtualenv of python3.8
For Windows users:
To add python 3.x ipykernel to jupyter notebook with name "Python 3.x", shown as 'Python 3.x', open your terminal (such as Anaconda Prompt), in base env and enter the following line by line.
For Python 3.10 ipykernel:
conda create -n py310 python=3.10 ipykernel
conda activate py310
python -m ipykernel install --user --name=py310 --display-name "Python 3.10"
For Python 3.7.7 ipykernel:
conda create -n py377 python=3.7.7 ipykernel
conda activate py377
python -m ipykernel install --user --name=py377 --display-name "Python 3.7"
Then close the specific env and go back to the base env, and type in jupyter notebook. Once it opens you'll see the following when you click the "New" drop down menu in the top right corner:
Then to check if the correct Python is installed, open a new notebook and run:
from platform import python_version
print(python_version())
For Python 3.10:
For Python 3.7.7
To remove a specific kernel, you can run the following in your terminal in the base env:
jupyter kernelspec uninstall <kernel name>
I use IPython notebooks and would like to be able to select to create a 2.x or 3.x python notebook in IPython.
I initially had Anaconda. With Anaconda a global environment variable had to be changed to select what version of python you want and then IPython could be started. This is not what I was looking for so I uninstalled Anaconda and now have set up my own installation using MacPorts and PiP. It seems that I still have to use
port select --set python <python version>
to toggle between python 2.x and 3.x. which is no better than the anaconda solution.
Is there a way to select what version of python you want to use after you start an IPython notebook, preferably with my current MacPorts build?
The idea here is to install multiple ipython kernels. Here are instructions for anaconda. If you are not using anaconda, I recently added instructions using pure virtualenvs.
Anaconda >= 4.1.0
Since version 4.1.0, anaconda includes a special package nb_conda_kernels that detects conda environments with notebook kernels and automatically registers them. This makes using a new python version as easy as creating new conda environments:
conda create -n py27 python=2.7 ipykernel
conda create -n py36 python=3.6 ipykernel
After a restart of jupyter notebook, the new kernels are available over the graphical interface. Please note that new packages have to be explicitly installed into the new environments. The Managing environments section in conda's docs provides further information.
Manually registering kernels
Users who do not want to use nb_conda_kernels or still use older versions of anaconda can use the following steps to manually register ipython kernels.
configure the python2.7 environment:
conda create -n py27 python=2.7
conda activate py27
conda install notebook ipykernel
ipython kernel install --user
configure the python3.6 environment:
conda create -n py36 python=3.6
conda activate py36
conda install notebook ipykernel
ipython kernel install --user
After that you should be able to choose between python2
and python3 when creating a new notebook in the interface.
Additionally you can pass the --name and --display-name options to ipython kernel install if you want to change the names of your kernels. See ipython kernel install --help for more informations.
If you’re running Jupyter on Python 3, you can set up a Python 2 kernel like this:
python2 -m pip install ipykernel
python2 -m ipykernel install --user
http://ipython.readthedocs.io/en/stable/install/kernel_install.html
These instructions explain how to install a python2 and python3 kernel in separate virtual environments for non-anaconda users. If you are using anaconda, please find my other answer for a solution directly tailored to anaconda.
I assume that you already have jupyter notebook installed.
First make sure that you have a python2 and a python3 interpreter with pip available.
On ubuntu you would install these by:
sudo apt-get install python-dev python3-dev python-pip python3-pip
Next prepare and register the kernel environments
python -m pip install virtualenv --user
# configure python2 kernel
python -m virtualenv -p python2 ~/py2_kernel
source ~/py2_kernel/bin/activate
python -m pip install ipykernel
ipython kernel install --name py2 --user
deactivate
# configure python3 kernel
python -m virtualenv -p python3 ~/py3_kernel
source ~/py3_kernel/bin/activate
python -m pip install ipykernel
ipython kernel install --name py3 --user
deactivate
To make things easier, you may want to add shell aliases for the activation command to your shell config file. Depending on the system and shell you use, this can be e.g. ~/.bashrc, ~/.bash_profile or ~/.zshrc
alias kernel2='source ~/py2_kernel/bin/activate'
alias kernel3='source ~/py3_kernel/bin/activate'
After restarting your shell, you can now install new packages after activating the environment you want to use.
kernel2
python -m pip install <pkg-name>
deactivate
or
kernel3
python -m pip install <pkg-name>
deactivate
With a current version of the Notebook/Jupyter, you can create a Python3 kernel. After starting a new notebook application from the command line with Python 2 you should see an entry „Python 3“ in the dropdown menu „New“. This gives you a notebook that uses Python 3. So you can have two notebooks side-by-side with different Python versions.
The Details
Create this directory: mkdir -p ~/.ipython/kernels/python3
Create this file ~/.ipython/kernels/python3/kernel.json with this content:
{
"display_name": "IPython (Python 3)",
"language": "python",
"argv": [
"python3",
"-c", "from IPython.kernel.zmq.kernelapp import main; main()",
"-f", "{connection_file}"
],
"codemirror_mode": {
"version": 2,
"name": "ipython"
}
}
Restart the notebook server.
Select „Python 3“ from the dropdown menu „New“
Work with a Python 3 Notebook
Select „Python 2“ from the dropdown menu „New“
Work with a Python 2 Notebook
A solution is available that allows me to keep my MacPorts installation by configuring the Ipython kernelspec.
Requirements:
MacPorts is installed in the usual /opt directory
python 2.7 is installed through macports
python 3.4 is installed through macports
Ipython is installed for python 2.7
Ipython is installed for python 3.4
For python 2.x:
$ cd /opt/local/Library/Frameworks/Python.framework/Versions/2.7/bin
$ sudo ./ipython kernelspec install-self
For python 3.x:
$ cd /opt/local/Library/Frameworks/Python.framework/Versions/3.4/bin
$ sudo ./ipython kernelspec install-self
Now you can open an Ipython notebook and then choose a python 2.x or a python 3.x notebook.
From my Linux installation I did:
sudo ipython2 kernelspec install-self
And now my python 2 is back on the list.
Reference:
http://ipython.readthedocs.org/en/latest/install/kernel_install.html
UPDATE:
The method above is now deprecated and will be dropped in the future. The new method should be:
sudo ipython2 kernel install
Following are the steps to add the python2 kernel to jupyter notebook::
open a terminal and create a new python 2 environment: conda create -n py27 python=2.7
activate the environment: Linux source activate py27 or windows activate py27
install the kernel in the env: conda install notebook ipykernel
install the kernel for outside the env: ipython kernel install --user
close the env: source deactivate
Although a late answer hope someone finds it useful :p
Use sudo pip3 install jupyter for installing jupyter for python3 and sudo pip install jupyter for installing jupyter notebook for python2. Then, you can call ipython kernel install command to enable both types of notebook to choose from in jupyter notebook.
I looked at this excellent info and then wondered, since
i have python2, python3 and IPython all installed,
i have PyCharm installed,
PyCharm uses IPython for its Python Console,
if PyCharm would use
IPython-py2 when Menu>File>Settings>Project>Project Interpreter == py2 AND
IPython-py3 when Menu>File>Settings>Project>Project Interpreter == py3
ANSWER: Yes!
P.S. i have Python Launcher for Windows installed as well.
Under Windows 7 I had anaconda and anaconda3 installed.
I went into \Users\me\anaconda\Scripts and executed
sudo .\ipython kernelspec install-self
then I went into \Users\me\anaconda3\Scripts and executed
sudo .\ipython kernel install
(I got jupyter kernelspec install-self is DEPRECATED as of 4.0. You probably want 'ipython kernel install' to install the IPython kernelspec.)
After starting jupyter notebook (in anaconda3) I got a neat dropdown menu in the upper right corner under "New" letting me choose between Python 2 odr Python 3 kernels.
If you are running anaconda in virtual environment.
And when you create a new notebook but i's not showing to select the virtual environment kernel.
Then you have to set it into the ipykernel using the following command
$ pip install --user ipykernel
$ python -m ipykernel install --user --name=test2