I'm creating a Conda environment and want to make sure it allows anyone to run the scripts in my repository without having to install anything separately. But, I have pip installed some packages on my machine, so, if I understand correctly, some of my imports might work because I pip-installed them and not because they're installed in the Conda environment.
For example, suppose I created an environment with conda create --name my_env python=3.8.8. This shouldn't install Pandas, but if I run import pandas as pd, it works. I believe this is because I've pip-installed Pandas. My question is:
How can I test whether a script works with only packages installed in the conda environment?
You can create a yaml file just from the history of your conda installations:
conda env export --from-history --name my_env > myenv.yml
From that you create a new conda environment, which you can use to test your script with:
conda env create --name conda_only --file myenv.yml
Python has flags to ignore PYTHONPATH (-E) and default system- or user-level site-packages (-S), as well as a combined flag (-I) that does both (see python -h). Try using such flags to isolate from the system. Something like,
conda run -n my_env python -I script.py
Also, it should be clarified that pip install is only an issue like this if you are using the --user flag, which triggers a user-level installation of packages, or some alternate location (e.g., --root or --prefix flags) and have PYTHONPATH set to make those accessible.
Related
I have miniconda3 installed and since I would like to have an environment with python version 3.3.0, I create it via
conda create -n "myenv" python=3.3.0
However when I activate the environment via
conda activate myenv
python has version 2.7.15 and path
/usr/bin/python
and ipython has python version 3.6.8 and path
/home/myname/.local/bin/ipython
I can access the correct python with python3 which is at
/home/myname/miniconda3/envs/myenv/bin/python3
however, ipython3 has python version 3.6.8 again.
conda install python=3.3.0
left the situation unchanged.
A solution would be to open IPython via
python3 -m IPython
however, while this works fine for python here I get the error message
/home/myname/miniconda3/envs/myenv/bin/python3: No module named IPython
Is it possible to access with the commands python and ipython both python version 3.3.0 in that specific environment, i.e. not by setting an alias in the .bashrc?
EDIT:
Turns out that this problem does not occur if you select version 3.3 instead of 3.3.0 together with #ilmarinen's answer
conda create -n "myenv" python=3.3 ipython
everything works fine and python as well as ipython result to version python 3.3.5.
You need to install ipython as well into your given environment
conda create -n "myenv" python=3.3.0 ipython
The conda environments are prepended to your PATH variable, so when you are trying to run the executable "ipython", Linux will not find "ipython" in your activated environment (since it doesn't exist there), but it will continue searching for it, and eventually find it wherever you have it installed.
To create an environment named py33 with python 3.3.0, using the channel conda-forge and a list of packages:
conda create -y --name py33 python==3.3.0
conda install -f -y -q --name py33 -c conda-forge --file requirements.txt
conda activate py33
...
conda deactivate
Alternatively you can use
conda env create -f environment.yml
for using an environment.yml file instead of requirements.txt:
name: py33
channels:
- conda-forge
dependencies:
- python==3.3.0
- ipython
Use this command to remove the environment:
conda env remove -n py33
I had similar issue. And I could't find many useful discussions.
The problem for me was I have alias pointing python to miniconda python hardcoded in my shell config file when I execute conda init zsh. Somehow the init process copies the alias and always reload that, thus overwrites the "correct" version.
After conda create -n py27 python=2.7 (my system default is 3.6), the version was correctly installed at miniconda3/envs/py27/bin/python. But the activated evironment python was not pointing to it, as indicated by which python,even if I deleted updated my shell config.
In the end it was solved by 'reverse' conda init (remove the generated conda function in .zshrc), remove alias, and re-init.
I guess other shell is using the same mechanism.
My system is ubuntu 18.04.
I have a pre-installed version 3 and 2 of python.
which python3
/usr/bin/python3
python3 -V
Python 3.6.9
which python
/usr/bin/python
python -V
Python 2.7.17
I need to create several virtual environments, one for python 2.7.15 and another for 2.6. how can I do that?
There are different ways of creating virtual python environments. Three popular ones are
virtualenv
pipenv
conda
I personally like conda a lot.
virtualenv
Assuming you have pip installed, you get virtualenv with
pip install virtualenv
Once installed, you can change into a directory of your choice and create a virtual environment like this
virtualenv myenvironmentname
If you want to use a different python version in your virtual environment, you can specify this with the --python flag.
virtualenv --python=/usr/bin/python2.6 myenvironmentname
However, please note that this requires you to have the python version you specify installed in advance, virtualenv will not take care of that for you (have a look at Use different Python version with virtualenv for more details). So you'll need local installations of the versions you desire.
You then can activate the environment with
myenvironmentname/bin/activate
and go ahead to use pip to install packages, etc. Have a look at
pip freeze --help
to find out on how to make your environment reusable.
pipenv
pipenv combines pip and virtualenv.
You can install it using
pip install --user pipenv
Pipenv takes care of dependencies on a project basis
cd myprojectfolder
pipenv install
This will create a Pipfile which will track dependencies and a virtualenv (see https://docs.python-guide.org/dev/virtualenvs/ for more details).
To create an environment using a specific version, you can do
pipenv install --python '/usr/bin/python2.6'
or
pipenv install --python 2.6
Cmp. Set python version when creating virtualenv using pipenv. If you also have pyenv installed, the second form will prompt pipenv to attempt to install non-existing versions, afaik.
conda
Anaconda Python is a python distribution (with a focus on data science) that comes with its own package and virtual environment manager named conda. Anaconda Python is not available in the official package repository of Ubuntu 18.04 LTS but needs to be installed in another way (the official documentation can be found here: https://docs.anaconda.com/anaconda/install/linux/).
To create an environment with conda, do
conda create --name myenvironmentname python=2.7.15
In contrast to virtualenv, the environments are by default not created in the present working directory, but installed into the envs directory in your conda directory. conda will also take care to install the proper python version, that is at least as long as it is part of the default channel (see below).
You can then activate said environment with
conda activate myenvironmentname
As I wrote above, the python version you specify needs to be available from the configured conda channels. python2.6 however, was removed from the default channel. To remedy this, you can add the free channel back to your default list (see https://docs.conda.io/projects/conda/en/latest/user-guide/configuration/free-channel.html for more details):
conda config --set restore_free_channel true
After that you can
conda create --name myotherenvironmentname python=2.6
And switch between the environments as you like
conda activate myotherenvironmentname
For python3 python -m venv <your_virtual_enviroment_path> for python2 virtualenv <your_virutal_enviroment_path>
The to activate source <your_virtual_environment_path>/bin/activate. And to deactivate deactivate. Finally to check what is activated echo $VIRTUAL_ENV
I strongly recommend for one virtual environment for each project.
I created an environment in a prefixed path using
conda create -p ~/myenv python=3.6
I activated it using
conda activate ~/myenv
but when I install a package to it, using
conda install pandas
It tries to install the package in my root folder rather than in my environment.
For anyone facing similar issues:
I determined that there was an older conda version installed in .local . Removing this older version using rm -r solved the issue. Caution: Be careful while using the rm -r.
I am trying to create a new empty evironment with conda using the command:
conda create -n name python=3.6
Then I activate the environment and export it with
conda env export > environment.yml
I have noticed that the generated yml files contains a lot of pip packages that I guess are installed system wide. I am a ROS user and all the python packages installed by ROS appear there. Is there a way to ensure that those pip packages are not included in my new environment?
Thanks a lot for your help.
Make sure that you created the environment not using
conda env create -n name python=3.6
but using instead the command
conda create -n name python=3.6
Using the first command will link all packages from your base environment.
Also note that your new environment will contain several default packages (such as Python 3.6, pip, etc.) if you use the flag python=3.6 when creating the environment.
I have anaconda3 using python 3.5. I wanted to experiment with something in python 2.7 so I opened the anaconda command prompt and ran:
conda create -n py27 python=2.7 anaconda
In retrospect that may have been overkill; I didn't need everything in anaconda. I now want to remove it. What I tried
I looked for a uninstall executable for the py27, but couldn't find one
the docs say to: conda - conda uninstall (but that returned: could not locate 'conda--'
I also tried looking in control manager (windows) to see if it was available to uninstall, but it was not
Question: How do I undo conda create -n py27 python=2.7 anaconda? That is to say, I want to go back to my original anaconda3 python 3.5 and no py27.
When you use the create command, you are not replacing your current environment, just creating a new one.
You can see a list of your environments using the following command:
conda info --envs
Then, you can activate a specific environment using (replace py27 by the name of the environment):
On windows: activate py27
On linux: source activate py27
To delete the newly created environment use the following command:
conda remove --name py27 --all
The "all" parameter will also delete any configuration and packages installed with the environment.
Edit
New environments are installed inside the envs folder of your anaconda root so after removing it you can check the folder in case there is anything left, e.g.:
C:\Anaconda3\envs\py27
More info: https://conda.io/docs/user-guide/tasks/manage-environments.html
You only have to delete the environment you do not need any longer:
$ conda env remove -n <env_name>
In your case:
$ conda env remove -n py27
more info here