I received the following error when trying to install geopandas in my environment after it failed in the terminal via the conda-forge command. Is there a way to get this updated and fixed?
Output in format: Requested package -> Available versions
Package geopandas-base conflicts for:
geopandas -> geopandas-base==0.9.0=py_1
geopandas-base
Package ca-certificates conflicts for:
python=3.9 -> openssl[version='>=1.1.1q,<1.1.2a'] -> ca-certificates
geopandas -> python -> ca-certificates```
You’re installing geopandas into an existing environment with conflicting packages. This can happen when the other env has packages pinned to specific incompatible packages, or if packages have been installed from incompatible channels, e.g. if you’re installing into your base env using anaconda, or (though this happens less often these days) if it’s just been a really long time since you updated your packages.
Instead, create a new environment with the packages you need, e.g.:
conda create -n geopandas -c conda-forge python=3.10 geopandas gdal pygeos pandas scipy […]
If you’d like to delete the previous environment you can do so with
conda env remove -n other-env-name
If you continue to have issues and you’re using anaconda, I’d strongly suggest deleting anaconda, which bundles packages from the defaults channel which is incompatible with conda-forge, and install miniforge or it’s much faster compiled (but slightly less user friendly) cousin mambaforge.
$ conda install -c anaconda pip
$ pip install geopandas
I'm looking for a solution where environments do inherit from root, but searching for the answer there seems to be a lot of confusion. Many OP questions believe they are inheriting packages when they are not. So, the search results find these questions, but the answer has the counter solution (or just explain they are mistaken).
That said, one OP actually has a similar objective. Can packages be shared across Anaconda environments? This OP says they are running out of space on their HDD. The implication being "sharing" should use the same installed packages in the new environment. The answer (not accepted) is to use --clone.
I also found this post, Do newly created conda envs inherit all packages from the base env? which says --clone does not share packages. In this post the OP believed their new environment "shared" packages, and then concludes "shared" packages don't exist.
What is the use of non-separated anaconda environments?
I tested both the --clone flag, and the Conda Docs instructions to "build identical environments" options. Both env directories have the same identical size: 2G+.
(base) $ conda list --explicit > spec-file.txt
# Produced Size On Disk: 2.14 GB (2,305,961,984 bytes)
(base) conda create --name myclone --clone root
# Produced Size On Disk, clone: 2.14 GB (2,304,331,776 bytes)
The only difference was building identical environment downloaded the packages again, and clone coppied the local file taking much less time.
I use Miniconda to deploy CLI tools to coworker workstations. Basically, the tools all use the same packages, with the occasional exception, when I need to add a particular module which I don't want in the base install.
The goal is to use conda create for environments that extend the base packages similar to virtualenv --system-site-packages, and not to duplicate their installation.
UPDATE 2020-02-08
Responding to #merv and his link to this post (Why are packages installed rather than just linked to a specific environment?) which says Conda venvs inherit base packages by default. I had another opportunity this weekend with the problem. Here is the base case:
Downloaded the Miniconda installer. Installed with settings
Install for me
Install location: (C:\Users\xtian\Miniconda3_64)
NOTE: I added the _64
Advanced Options
Add Anaconda to the system PATH variable, False
Register Anaconda as the system Python 3.7, True
I updated pip and setuptools,
conda update pip setuptools
Below, I list packages in base:
(base) C:\Users\xtian>conda list
# packages in environment at C:\Users\xtian\Miniconda3_64:
#
# Name Version Build Channel
asn1crypto 1.3.0 py37_0
ca-certificates 2020.1.1 0
certifi 2019.11.28 py37_0
cffi 1.13.2 py37h7a1dbc1_0
chardet 3.0.4 py37_1003
conda 4.8.2 py37_0
conda-package-handling 1.6.0 py37h62dcd97_0
console_shortcut 0.1.1 3
cryptography 2.8 py37h7a1dbc1_0
idna 2.8 py37_0
menuinst 1.4.16 py37he774522_0
openssl 1.1.1d he774522_3
pip 20.0.2 py37_1
powershell_shortcut 0.0.1 2
pycosat 0.6.3 py37he774522_0
pycparser 2.19 py37_0
pyopenssl 19.1.0 py37_0
pysocks 1.7.1 py37_0
python 3.7.4 h5263a28_0
pywin32 227 py37he774522_1
requests 2.22.0 py37_1
ruamel_yaml 0.15.87 py37he774522_0
setuptools 45.1.0 py37_0
six 1.14.0 py37_0
sqlite 3.31.1 he774522_0
tqdm 4.42.0 py_0
urllib3 1.25.8 py37_0
vc 14.1 h0510ff6_4
vs2015_runtime 14.16.27012 hf0eaf9b_1
wheel 0.34.2 py37_0
win_inet_pton 1.1.0 py37_0
wincertstore 0.2 py37_0
yaml 0.1.7 hc54c509_2
Then I successfully create new venv:
(base) C:\Users\xtian>conda create -n wsgiserver
Collecting package metadata (current_repodata.json): done
Solving environment: done
## Package Plan ##
environment location: C:\Users\xtian\Miniconda3_64\envs\wsgiserver
Proceed ([y]/n)? y
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Here I activate the new wsgiserver virtual environment, list packages, and finally test with pip--but there is no pip! I tested today with 64 and 32 bit installers:
(base) C:\Users\xtian>conda activate wsgiserver
(wsgiserver) C:\Users\xtian>conda list
# packages in environment at C:\Users\xtian\Miniconda3_64\envs\wsgiserver:
#
# Name Version Build Channel
(wsgiserver) C:\Users\xtian>pip
'pip' is not recognized as an internal or external command,
operable program or batch file.
Should Conda environments inherit base packages?
No. The recommended workflow is to use conda create --clone to create a new standalone environment, and then mutate that environment to add additional packages. Alternatively, one can dump the template environment to a YAML (conda env export > env.yaml), edit it to include or remove packages, and then create a new environment from that (conda env create -f env.yaml -n foo).
Concern about this wasting storage is unfounded in most situations.1 There can be a mirage of new environments taking up more space than they really do, due to Conda's use of hardlinks to minimize redundancy. A more detailed analysis of this can be found in the question, Why are packages installed rather than just linked to a specific environment?.
Can Conda environments inherit base packages?
It's not supported, but it's possible. First, let's explicitly state that nested activation of Conda environments via the conda activate --stack command does not enable or help to allow inheritance of Python packages across environments. This is because it does not manipulate PYTHONPATH, but instead only keeps the previous active environment on PATH and skips the deactivate scripts. A more detailed discussion of this is available in this GitHub Issue.
Now that we've avoided that red herring, let's talk about PYTHONPATH. One can use this environment variable to include additional site-packages directories to search. So, naively, something like
conda activate foo
PYTHONPATH=$CONDA_ROOT/lib/python3.7/site-packages python
should launch Python with the packages of both base and foo available to it. A key constraint for this to work is that the Python in the new environment must match that of base up to and including the minor version (in this case 3.7.*).
Thinking through the details
While this will achieve package inheritance, we need to consider: Will this actually conserve space? I'd argue that in practice it likely won't, and here's why.
Presumably, we don't want to physically duplicate the Python installation, but the new environment must have a Python installed in order to help constrain solving for the new packages we want. To do this, we should not only match the Python version (conda create -n foo python=3.7), but rather the exact same build as base:
# first check base's python
conda list -n base '^python$'
# EXAMPLE RESULT
# Name Version Build Channel
python 3.7.6 h359304d_2
# use this when creating the environment
conda create -n foo python=3.7.6=h359304d_2
This will let Conda do its linking thing and use the same physical copy in both environments. However, there is no guarantee that Python's dependencies will also reuse the packages in base. In fact, if any compatible newer versions are available, it will download and install those.
Furthermore, let's say that we now install scikit-learn:
conda install -n foo scikit-learn
This again is going to check for the newest versions of it and its dependencies, independent of whether older but compatible versions of those dependencies are already available through base. So, more packages are unnecessarily being installed into the package cache.
The pattern here seems to be that we really want to find a way to have the foo env install new packages, but use as many of the existing packages to satisfy dependencies. And that is exactly what conda create --clone already does.2
Hence, I lose the motivation to bother with inheritance altogether.
Note
I'd speculate that for the special case of pure Python packages it may be plausible to use pip install --target from the base environment to install packages compatible with base to a location outside of base. The user could then add this directory to PYTHONPATH before launching python from base.
This would not be my first choice. I know the clone strategy is manageable; I wouldn't know what to expect with this going long-term.
[1] This will hold as long as the locations of the package cache (pkgs_dirs) and where the environment is created (which defaults to envs_dirs) are on the same volume. Configurations with multiple volumes should be using softlinks, which will ultimately have the same effect. Unless one has manually disabled linking of both types, Conda will do a decent job at silently minimizing redundancy.
[2] Technically, one might also have a stab at using the --offline flag to force Conda to use what it already has cached. However, the premise of OP is that the additional package is new, so it may not be wise to assume we already have a compatible version in the cache.
I have installed dash from official conda package management system. That version is severely outdated (from 2019 year). Now I need to update that installed dash and its all dependencies with version from conda-forge (which is latest dash version https://anaconda.org/conda-forge/dash). What is the proper way of doing that?
I've installed Anaconda 4.2.9 on my linux machine, and I work with a couple developpers on it, we work on offline mode so we need to download each package from Conda depository and then install it manually.
We face now some compatibility problems between some packages. so I would know from which channel a given package is installed. when I run conda list I can see only the local channel I've created:
sas7bdat 2.0.6 py35_0 file:///opt/Anaconda/Anaconda3-4.2.0/channel
scikit-image 0.12.3 np111py35_1
scikit-learn 0.17.1 np111py35_2
scipy 0.18.1 np111py35_0
to know which channel is used in your case, you have just to take the entire name of your package you have in conda list --explicit (as #pilu mentioned in his answer) and google it with "".
for example:
file:///opt/Anaconda/Anaconda3-4.2.0/channel/linux-64/sas7bdat-2.0.6-py35_0.tar.bz2
you have to google: "sas7bdat-2.0.6-py35_0.tar.bz2"
you will get the UPLOADER of package (Anaconda for this example).
Try the explicit flag:
conda list --explicit
If nothing is listed, then it is part of the default installation. The default installation packages are associated with the anaconda channel.
I was lately trying to update all my Anaconda packages:
conda update conda
conda update anaconda
Some of them still were not updated, so I also did conda update --all.
Right now my conda version : 4.3.16, which seems to be the latest release. However, for some reason, matplotlib still has old version:
conda list
matplotlib 1.5.1 np111py35_0
According to the Anaconda changelog, it should be updated to 2.0 version: matplotlib from 1.5.3 to 2.0.0.
My installed python version is 3.5.2, if this matters.
UPD: Seems that spyder-app is blocking update because of problem with dependencies:
conda install anaconda=4.3.1
UnsatisfiableError: The following specifications were found to be in conflict:
- anaconda 4.3.1* -> spyder 3.1.2 py34_0
- spyder-app -> spyder 2.3.3
Use "conda info <package>" to see the dependencies for each package.
conda install matplotlib=2
UnsatisfiableError: The following specifications were found to be in conflict:
- matplotlib 2.0*
- spyder-app -> spyder 2.3.0 -> pyqt 4.* -> qt >=4.8.6,<5.0
- spyder-app -> spyder 2.3.0 -> pyqt 4.* -> sip >=4.16.4,<4.18
Use "conda info <package>" to see the dependencies for each package.
UPD2: Problem was solved by removing spyder-app:
conda remove spyder-app
conda update anaconda
Matlotlib was updated to the 2.0 version, among other updated packages. I also checked if Spyder IDE still works, and it still is able to run.
It probably is a dependency conflict. The anaconda package is just a meta-package that bundles packages, if some package can't be updated because of a package you installed (maybe requiring matplotlib < 2, or even indirectly by requiring a package and version that matplotlib depends on) then you probably can't install the newest version of the anaconda package. You could try to install the newest anaconda package (currently 4.3.1) yourself:
conda install anaconda=4.3.1
But you can also try to install a specific version of matplotlib manually and see what happens (there should be an instructive message if it can't install it because of dependencies):
conda install matplotlib=2
In your case it's spyder-app that is responsible. It's a deprecated package and not included in newer versions of anaconda so one possibility is to remove it:
conda remove spyder-app
and then try the update of anaconda or matplotlib.
Another way would be to create a new environment and then you can choose which environment to use when:
conda install -n mynewenvironment python=3.5 anaconda
and then use the new environment when you need matplotlib 2 and the old environment if you need spyder-app (and don't care about the matplotlib version).