Develop Reference

Can't find NVidia cudatoolkit package when creating environment from .yml

Hopefully someone can help me, I've tried to search the internet for a solution, but can't seem to find a solution..
I'm trying to create an environment using conda env create -n deltaconv -f environment.yml and somehow conda I'm getting this response:
[Folder]>conda env create -n deltaconv -f environment.yml
Collecting package metadata (repodata.json): done
Solving environment: failed
ResolvePackageNotFound:
- nvidia::cudatoolkit=11.3
I've just freshly installed Miniconda for the task and the environment looks like this:
channels:
- nvidia
- pytorch
- pyg
dependencies:
- pip=21.2.4
- python=3.9.12
- setuptools=52.0.0
- wheel=0.36.2
- protobuf~=3.19.0
- nvidia::cudatoolkit=11.3
- pytorch::pytorch=1.11.0
- pytorch::torchvision
- pytorch::torchaudio
- pyg::pyg=2.0.4
- pip:
- numpy==1.21.5
- progressbar2==4.0.0
- tensorboard==2.8.0
- jupyter==1.0.0
- openmesh==1.2.1
- h5py==3.6.0
- pytest==7.1.2
- deltaconv==1.0.0
Does anyone know why conda is unable to find cudatoolkit 11.3 from the nvidia channel?

Installing OpenCV From Conda Environment YML File

I'm trying out OpenCV with Python bindings for which I'm using the following YML file:
name: opencv-python-sandbox
channels:
- menpo
- conda-forge
- defaults
dependencies:
- jupyter=1.0.0
- jupyterlab=0.34.9
- keras=2.9.0
- matplotlib=3.5.2
- numpy=1.23.1
- opencv-python==4.6.0.66
- pandas=1.4.3
- python=3.8.0
- scikit-learn=1.1.1
- scipy=1.8.1
- tensorboard=2.9.1
- tensorflow=2.9.1
When I rain it threw some errors and says that it is not able to resolve OpenCV and Tensorflow:
(ml-sandbox) joesan#joesan-InfinityBook-S-14-v5:~/Projects/Private/ml-projects/ml-sandbox/opencv-python-sandbox$ conda env create -f environment.yml
Collecting package metadata (repodata.json): done
Solving environment: failed
ResolvePackageNotFound:
- tensorflow=2.9.1
- opencv-python==4.6.0.66
How to get this fixed? Do I need to add pip to my environment.yml and then manually install opencv via pip after activating the conda environment?

Not sure why this was not answered by anyone else as this seems to be a very common problem. Nevertheless, I was able to solve this by adding pip as a dependency in my environment.yml and use pip to install OpenCV and any other libraries that won't resolve with Conda.
My environment.yml looks like this:
name: ml-sandbox
channels:
- menpo
- conda-forge
- defaults
dependencies:
- jupyter=1.0.0
- jupyterlab=0.34.9
- keras=2.9.0
- matplotlib=3.5.2
- pandas=1.4.3
- python=3.8.0
- pip=22.1.2
- scikit-learn=1.1.1
- scipy=1.8.1
- tensorboard=2.9.1
- pip:
- numpy==1.23.1
- opencv-contrib-python==4.6.0.66

You have fixed it yourself by moving the requirements to the pip section, which results in an installation from Pypi. I just wanted to add explanation why your original attempt did not work and suggestions in case you want to strictly stick to using conda. Note that for both tensorflow and opencv, the packages provided on conda-forge are not maintained by the respective developers, often resulting in them lacking behind in versions.
The python bindings for openCV are called py-opencv on conda forge and have different version strings, so you would need to put py-opencv==4.6.0 in your yml
tensorflow on conda-forge goes only up to 2.8.1. So when strictly sticking to conda, you would need to downgrade the version
You can always check available versions for packages by using conda search -c <channel> <package-name> from your terminal

conda environment and channels

According to the documentation, if I use
conda env export > file.yml
I am able to share the environment with others. For better cross-platform compatibility, a better way would be:
conda env export --from-history > file.yml
listing only the packages explicitly requested (and not their associated dependencies).
That is what I did, I created a requirement yml file with the second command. Here it is:
name: torch
channels:
- defaults
dependencies:
- python=3.8
- humanize
- nltk
- pandas
- lxml
- numpy
- bs4
- fire
- neptune-client
- tqdm
- pyyaml
- torchaudio
- pytorch
- cudatoolkit=11.3
- torchvision
Among those packages, some were installed from the channel conda-forge: channels seem to be lost in the yaml file.
Indeed, if I try and use that file for cloning the environment (same machine):
conda env create -n torch2 --file=file.yml
I get an error for the packages installed from conda-forge (I explicitly installed from conda-forge only neptune-client and fire):
Collecting package metadata (repodata.json): done
Solving environment: failed
ResolvePackageNotFound:
- torchaudio
- neptune-client
- fire
However, it seems that channels should be included in the yml. For example, on this github issue page I read:
Currently, conda env export does include channels information.
that closes the issue.
What am I missing?
NOTE: pytorch was installed with
conda install pytorch torchvision torchaudio cudatoolkit=11.3 -c pytorch
from the official web page.

What am I missing?
The docs are wrong - or misleading (at best) when it is communicated that conda env export --from-history exports the channels that packages are installed from, or even any channels at all. This is not the behavior you get, nor is it what I get myself:
$ conda env export | head -n 8
name: smithy
channels:
- bioconda
- conda-forge
- defaults
dependencies:
- _libgcc_mutex=0.1=conda_forge
- _openmp_mutex=4.5=1_gnu
$ conda env export --from-history | head -n 8
name: smithy
channels:
- defaults
dependencies:
- mamba
- constructor
- cookiecutter
- conda-build
Note that conda env export does include channel information, but in a highly-pinned way that's almost guaranteed to not work across platforms. So that's not going to work for your use case. I'm not sure if this is a bug or an oversight, but it's clearly not producing the desired result for the user.
Now to offer a (opinionated) recommendation on how to proceed: your best bet is to semi-manually curate an environment YAML file yourself and use that as a single source of truth. It looks like you can use your name: torch ... file a s a starting point, adding in the channels and maybe some other details as you go. Don't forget you can tie an individual package to a channel with the channel::package syntax a la
name: torch
channels:
- defaults
- conda-forge
- torch
dependencies:
- python=3.8
<SNIP>
- pytorch::torchaudio
- pytorch::pytorch
- pytorch::cudatoolkit=11.3
- pytorch::torchvision

Anaconda is unable to update/install packages

When trying to upgrade my anaconda distribution I keep getting the following error:
Collecting package metadata (repodata.json): done
Solving environment: failed
PackagesNotFoundError: The following packages are not available from current channels:
- pkgs/free/win-64::anaconda==4.4.0=np112py36_0 -> bzip2==1.0.6=vc14_3
- pkgs/free/win-64::anaconda==4.4.0=np112py36_0 -> cryptography==1.8.1=py36_0
[loads of similarly cryptic packages names]
- pkgs/free/win-64::anaconda==4.4.0=np112py36_0 -> vs2015_runtime==14.0.25123=0
Current channels:
- https://repo.anaconda.com/pkgs/main/win-64
- https://repo.anaconda.com/pkgs/main/noarch
- https://repo.anaconda.com/pkgs/r/win-64
- https://repo.anaconda.com/pkgs/r/noarch
- https://repo.anaconda.com/pkgs/msys2/win-64
- https://repo.anaconda.com/pkgs/msys2/noarch
- https://conda.anaconda.org/conda-forge/win-64
- https://conda.anaconda.org/conda-forge/noarch
To search for alternate channels that may provide the conda package you're
looking for, navigate to
https://anaconda.org
and use the search bar at the top of the page.
This is raised no matter what or how I try to install/update:
It is raised both with conda update --all and conda update numpy or any other package I tried, so this should not be due to some specific package that is not in the channel.
It is even raised when I try to update anaconda itself by conda update -n root conda.
I have tried updating numpy from Anaconda navigator.
Because of this question I added conda-forge to my channels - to no avail.
Grateful for any help.
EDIT: I finally solved this by completely removing and reinstalling Anaconda3, but I still lack an understanding of what exactly has originally caused the problem.

How to install fiona 1.6?

I am trying to install fiona=1.6 but I get the following error
conda install fiona=1.6
WARNING: The conda.compat module is deprecated and will be removed in a future release.
Collecting package metadata: done
Solving environment: -
The environment is inconsistent, please check the package plan carefully
The following packages are causing the inconsistency:
- conda-forge/noarch::flask-cors==3.0.7=py_0
- conda-forge/osx-64::blaze==0.11.3=py36_0
- conda-forge/noarch::flask==1.0.2=py_2
failed
PackagesNotFoundError: The following packages are not available from current channels:
- fiona=1.6 -> gdal==1.11.4
Current channels:
- https://conda.anaconda.org/conda-forge/osx-64
- https://conda.anaconda.org/conda-forge/noarch
To search for alternate channels that may provide the conda package you're
looking for, navigate to
https://anaconda.org
If I try to install gdal==1.11.4, I get the following
conda install -c conda-forge gdal=1.11.4
WARNING: The conda.compat module is deprecated and will be removed in a future release.
Collecting package metadata: done
Solving environment: |
The environment is inconsistent, please check the package plan carefully
The following packages are causing the inconsistency:
- conda-forge/noarch::flask-cors==3.0.7=py_0
- conda-forge/osx-64::blaze==0.11.3=py36_0
- conda-forge/noarch::flask==1.0.2=py_2
failed
PackagesNotFoundError: The following packages are not available from current channels:
- gdal=1.11.4
Current channels:
- https://conda.anaconda.org/conda-forge/osx-64
- https://conda.anaconda.org/conda-forge/noarch
- https://repo.anaconda.com/pkgs/main/osx-64
- https://repo.anaconda.com/pkgs/main/noarch
- https://repo.anaconda.com/pkgs/free/osx-64
- https://repo.anaconda.com/pkgs/free/noarch
- https://repo.anaconda.com/pkgs/r/osx-64
- https://repo.anaconda.com/pkgs/r/noarch
To search for alternate channels that may provide the conda package you're
looking for, navigate to
https://anaconda.org
and use the search bar at the top of the page.
This is the result of conda info
conda info
active environment : base
active env location : /anaconda3
shell level : 1
user config file : /Users/massaro/.condarc
populated config files : /Users/massaro/.condarc
conda version : 4.6.11
conda-build version : 3.17.8
python version : 3.6.8.final.0
base environment : /anaconda3 (writable)
channel URLs : https://conda.anaconda.org/conda-forge/osx-64
https://conda.anaconda.org/conda-forge/noarch
package cache : /anaconda3/pkgs
/Users/massaro/.conda/pkgs
envs directories : /anaconda3/envs
/Users/massaro/.conda/envs
platform : osx-64
user-agent : conda/4.6.11 requests/2.21.0 CPython/3.6.8 Darwin/17.5.0 OSX/10.13.4
UID:GID : 502:20
netrc file : None

Python Versions
The Conda Forge channel only has gdal v1.11.4 for Python 2.7, 3.4, and 3.5. You either need to use a newer version of Fiona (current is 1.8) or make a new env that includes one of those older Python versions.
For example,
conda create -n fiona_1_6 fiona=1.6 python=3.5
Channel defaults is Required
Another issue you face is that you have removed the defaults channel from your configuration (as per your conda info). It is impossible to install fiona=1.6 with only the conda-forge channel. My recommendation would be to have both conda-forge and defaults in your configuration, but just set conda-forge to have higher priority (if that's what you want). You can do this like so,
conda config --append channels defaults
If you really don't want to include defaults, but just want a temporary workaround, then you can simply run the first command with a --channels | -c flag
conda create -n fiona_1_6 -c conda-forge -c defaults fiona=1.6 python=3.5
This will still give conda-forge precedence, but allow missing dependencies to be sourced from defaults.
Environment File
If you have more than just Fiona that you require, it may be cleaner to put together a requirements file, like so
fiona_1_6.yaml
name: fiona_1_6
channels:
- conda-forge
- defaults
dependencies:
- python=3.5
- fiona=1.6
- osmnx
Then create the new environment with this:
conda env create -f fiona_1_6.yaml

Doing what the error message told me to,
To search for alternate channels that may provide the conda package you're
looking for, navigate to https://anaconda.org
and typing in gdal in the search box led me to https://anaconda.org/conda-forge/gdal which has this installation instruction:
conda install -c conda-forge gdal=1.11.4
Try that to install the gdal dependency, maybe?