I'm trying to run Jupyter notebooks with a globally installed version of Jupyter from within virtual environments (using virtualenvwrapper, because I want to manage versions of installed packages). And I do not what to use Anaconda.
The problem is when I run jupyter notebook from within the virtualenv, it cannot find the packages installed in the env, it only finds the packages installed globally.
How do I set up Jupyter to check for packages installed within the virtual environment instead of globally?
Here is what I get when I run which python and which jupyter:
globally:
which python >>> /usr/local/bin/python
which jupyter >>> /usr/local/bin/jupyter
from within virtualenv:
which python >>> /Users/brianclifton/.virtualenvs/test/bin/python
which jupyter >>> /usr/local/bin/jupyter
running jupyter notebook from within the virtualenv:
which python >>> /usr/local/bin/python
which jupyter >>> /usr/local/bin/jupyter
Also, here is my .bash_profile:
export VISUAL=vim
export EDITOR="$VISUAL"
export PS1="\\[\[\e[38;5;94m\][\u] \[\e[38;5;240m\]\w:\[\e[m\] \$(__git_ps1 '(%s)')$ "
export CLICOLOR=1
export LSCOLORS=ExFxBxDxCxegedabagacad
export PATH=/usr/local/bin/python:/usr/local/bin:$PATH
alias ls='ls -GFh'
alias pserv="python -m SimpleHTTPServer"
alias ipynb="jupyter notebook"
export WORKON_HOME=/Users/brianclifton/.virtualenvs
export PROJECT_HOME=/Users/brianclifton/dev
source /usr/local/bin/virtualenvwrapper.sh
if [ -f $(brew --prefix)/etc/bash_completion ]; then
. $(brew --prefix)/etc/bash_completion
fi
alias branch='git rev-parse --abbrev-ref HEAD'
function frameworkpython {
if [[ ! -z "$VIRTUAL_ENV" ]]; then
PYTHONHOME=$VIRTUAL_ENV /usr/local/bin/python "$#"
else
/usr/local/bin/python "$#"
fi
}
Another solution from virtualenv doc
workon test
pip install ipykernel
python -m ipykernel install --prefix=/usr/local --name test-kernel
Then your kernel should appear when you run jupyter from your other virtualenv, and all the packages installed in test would be available from it. Change prefix value according to the doc if you prefer a per-user installation instead of system-wide
One possible solution is to prefix your virutalenv's bin directory to your path. This way jupyter will find the virtualenv's libraries. You can do this by running export PATH:`which python`:$PATH after you activate your environment. It would be easy enough to alias.
However, a better solution may be to add this line to the postactivate hook/script. To find the location of this script do ls $WORKON_HOME after activate virtualenvwrapper and edit $WORKON_HOME/<virtualenv_name>/bin/postactivate.
Related
is there a way to easily switch from conda environments to my system's native python environment? When I have conda activated, and I run bash or exec bash it doesn't seem to work, and when I run python it uses conda's python and it's not using the system python path. The only way I've been able to get this to work is by restarting the terminal entirely. Is there a faster way to do this?
(I've actually removed the conda init block from my ~/.bashrc and I run it manually every time I want to use conda's python.)
You have two options:
Just deactivate the base environment:
(base) $ conda deactivate
$
Configure your conda to not activate the base environment on startup.
(base) $ conda config --set auto_activate_base false
<restart shell>
$
# use the system python
$ python
# to use conda you'll need to activate it first
$ conda activate
(base) $ python
The conda command you're looking for is deactivate.
Suppose you have activated the myproject environment.
$ conda deactivate
Now your environment is base, as seen by which python,
or conda info --envs.
$ conda deactivate
Now you're not using conda at all, and which python shows the system
interpreter, likely /usr/bin/python.
This may be a bit of a hack, but what about adding an alias pointing to the system python before the conda block in .bashrc? Assuming you don't want to use "conda deactivate" and lose the environment completely.
alias syspython=`which python`
# >>> conda initalize >>>
...
You should then be able to use the system python in a conda env like so:
syspython file.py ...
I am having trouble with pyenv after updating my macOS to 11.5. This is what I tried:
$ pyenv install 3.8.11 --skip-existing
$ pyenv virtualenv 3.8.11 >virtual env name<
$ pyenv local >virtual env name<
All of this works fine. Running pyenv local results in printing my virtual env's name. But I can't install dependancies or do anything with pip:
$ pip freeze
python >virtual env name< not installed
I tried following this answer, but I still have the same problem.
Note: This is also happening with virtual envs that were previously working pre-update.
Turns out the issue was with updating pyenv and not macOS. All I had to do was add the following lines to my ~./zshrc (or ~/.zprofile:
export PYENV_ROOT="$HOME/.pyenv"
export PATH="$PYENV_ROOT/bin:$PATH"
eval "$(pyenv init --path)"
How can I load in these packages I've just installed in Anaconda Python? I've installed Anaconda, and installed a package with the following commands in my mac terminal:
conda activate base
conda install pip
pip install datascience
pip install okpy
I can verify that these have been installed by running conda list, and both packages show up in that list.
I've searched around, and there appears to be some difficulty when I've had other versions of Python on my machine. I've tried to remove many of them, but when I type which python I get /usr/bin/python. Here's what my .bash_profile file looks like:
# added by Anaconda3 2019.07 installer
# >>> conda init >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$(CONDA_REPORT_ERRORS=false '/Users/adityakamath/anaconda3/bin/conda' shell.bash hook 2> /dev/null)"
if [ $? -eq 0 ]; then
\eval "$__conda_setup"
else
if [ -f "/Users/adityakamath/anaconda3/etc/profile.d/conda.sh" ]; then
. "/Users/adityakamath/anaconda3/etc/profile.d/conda.sh"
CONDA_CHANGEPS1=false conda activate base
else
\export PATH="/Users/adityakamath/anaconda3/bin:$PATH"
fi
fi
unset __conda_setup
# <<< conda init <<<
How can I change my settings so that, when I start up Jupyter Notebook using the Anaconda navigator, I will be able to import these libraries? The import statement I need to run is from client.api.notebook import Notebook.
If you have multiple versions of python you should run in a virtual environment. You can create a virtual environment by typying
conda create -n yourenvname python=x.x anaconda
where python=x.x is the version of python to use in your environment.
Then run
activate yourenvname
.
After you activate your environment then type
pip3 install client
I've been keeping all my python2.7 installs in my ~/.local/ directory that way I don't have to sudo every time I want to do a pip install. I also have $HOME/.local/lib/python2.7/site-packages on my $PYTHONPATH. This has worked well for years but now I find myself needing to run python3 programs more frequently. After much research it seems like virtualenv is the most recommended way to deal with python 2 and 3 on the same system. But I am running into troubles. I can spin up a python3 virtual environment but when I try to install new libs with pip, my old global path (i.e. ~/.local/) is still being searched by pip, which makes sense. However, this is even the case if I remove my ~/.local/bin/ directory from my $PATH and unset my $PYTHONPATH.
Here is are the steps I took:
First check the preliminaries before activating virtualenv. (I'm on Ubuntu 16.04 btw)
maddoxw#firefly:~$ echo $PATH
/usr/local/cuda-8.0/bin:/home/maddoxw/.node_modules_global/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/maddoxw/bin:/home/maddoxw/scripts
maddoxw#firefly:~$ echo $PYTHONPATH
maddoxw#firefly:~$ python --version
Python 2.7.12
maddoxw#firefly:~$ python3 --version
Python 3.5.2
maddoxw#firefly:~$ which pip
Since I removed my ~/.local/bin directory from my path, then I can be certain pip will not be found. Also, $PYTHONPATH is still empty. Now I create my virtualenv:
maddoxw#firefly:~$ mkdir test && cd test/
mkdir: created directory 'test'
maddoxw#firefly:~/test$ python3 -m venv .env
maddoxw#firefly:~/test$ source .env/bin/activate
(.env) maddoxw#firefly:~/test$ echo $PATH
/home/maddoxw/test/.env/bin:/usr/local/cuda-8.0/bin:/home/maddoxw/.node_modules_global/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/home/maddoxw/bin:/home/maddoxw/scripts
(.env) maddoxw#firefly:~/test$ echo $PYTHONPATH
(.env) maddoxw#firefly:~/test$ which python
/home/maddoxw/test/.env/bin/python
(.env) maddoxw#firefly:~/test$ python --version
Python 3.5.2
(.env) maddoxw#firefly:~/test$ which pip
/home/maddoxw/test/.env/bin/pip
Good. My ~/.local/ is still NOT on my $PATH, $PYTHONPATH is still empty, python points to the correct path and version, and pip is pointing to the correct location. Now lets try to pip install a fresh lib.
(.env) maddoxw#firefly:~/test$ pip install Cython
Requirement already satisfied: Cython in /home/maddoxw/.local/lib/python2.7/site-packages
Why is pip still looking in a non-$PATH path?
First, install pip3 to use with python3. You can install it with the following command and then use pip3 to install your packages.
sudo apt-get install python3-pip
[Solved]
When I initially set up my python2.7 environment way back when, I created for myself a handy little function wrapper around pip so that I wouldn't have to type out the --user every time I wanted to pip install
pip() {
if [ "$1" = "install" -o "$1" = "bundle" ]; then
cmd="$1"
shift
$HOME/.local/bin/pip $cmd --user $#
else
$HOME/.local/bin/pip $#
fi
}
I put this function in ~/.bash.d/bash_functions
and in my ~/.bashrc i added the line,
[ -f ~/.bash.d/bash_functions ] && source ~/.bash.d/bash_functions
So, although I removed $HOME/.local/ from my path. This wrapper function was still being called everytime I fired up a new terminal. Weather or not I was or was not in a virtualenv was irrelevant.
Solution?
Commented out (or delete completely) the function wrapper fixed it.
I trying install virtualenvwrapper from your official guide http://virtualenvwrapper.readthedocs.org/en/latest/install.html
I install virtualenvwrapper with pip and when I define the environment variables and source
export WORKON_HOME=$HOME/.virtualenvs
export PROJECT_HOME=$HOME/Devel
source /usr/local/bin/virtualenvwrapper.sh
I reboot my shell and I get the following output.
bgarcial#el-pug:~$ bash
/usr/local/bin/python: No module named virtualenvwrapper
virtualenvwrapper.sh: There was a problem running the initialization hooks.
If Python could not import the module virtualenvwrapper.hook_loader,
check that virtualenv has been installed for
VIRTUALENVWRAPPER_PYTHON=/usr/local/bin/python and that PATH is
set properly.
bgarcial#el-pug:~$
Despite this, the commands mkvirtualenv and others works. But .. What does this mean? How I can fix it? Thanks
What is the content of your ~/.bashrc file?
You may be calling "virtualenvwrapper" instead of "virtualenvwrapper.sh"
OR:
You may be exporting variables for the current shell that don't persist or propagate to subsequent shells.
The following script will install virtualenvwrapper and configure bash to persist the environment variables, which should in turn make virtualenvwrapper work as expected.
pip install virtualenvwrapper
configure_bashrc(){
echo '
export WORKON_HOME=$HOME/.virtualenvs
export PROJECT_HOME=$HOME/Devel
source /usr/local/bin/virtualenvwrapper.sh
' >> ~/.bashrc
source ~/.bashrc
}
[[ -z $(grep virtualenv ~/.bashrc) ]] && configure_bashrc