Fist of all, I'm a total newbie, please bear my idiocy :)
I run this:
conda env create -f env.yml
Here's the yml file:
name: DAND
channels: !!python/tuple
- defaults
dependencies:
- _nb_ext_conf=0.3.0=py27_0
- anaconda-client=1.6.0=py27_0
- appnope=0.1.0=py27_0
- backports=1.0=py27_0
- backports_abc=0.5=py27_0
- beautifulsoup4=4.5.1=py27_0
- clyent=1.2.2=py27_0
- configparser=3.5.0=py27_0
- cycler=0.10.0=py27_0
- decorator=4.0.10=py27_1
- entrypoints=0.2.2=py27_0
- enum34=1.1.6=py27_0
- freetype=2.5.5=1
- functools32=3.2.3.2=py27_0
- get_terminal_size=1.0.0=py27_0
- icu=54.1=0
- ipykernel=4.5.2=py27_0
- ipython=5.1.0=py27_1
- ipython_genutils=0.1.0=py27_0
- ipywidgets=5.2.2=py27_0
- jinja2=2.8=py27_1
- jsonschema=2.5.1=py27_0
- jupyter=1.0.0=py27_3
- jupyter_client=4.4.0=py27_0
- jupyter_console=5.0.0=py27_0
- jupyter_core=4.2.1=py27_0
- libpng=1.6.22=0
- markupsafe=0.23=py27_2
- matplotlib=1.5.3=np111py27_1
- mistune=0.7.3=py27_1
- mkl=11.3.3=0
- nb_anacondacloud=1.2.0=py27_0
- nb_conda=2.0.0=py27_0
- nb_conda_kernels=2.0.0=py27_0
- nbconvert=4.2.0=py27_0
- nbformat=4.2.0=py27_0
- nbpresent=3.0.2=py27_0
- nltk=3.2.1=py27_0
- notebook=4.3.0=py27_0
- numpy=1.11.2=py27_0
- openssl=1.0.2j=0
- pandas=0.19.1=np111py27_0
- path.py=8.2.1=py27_0
- pathlib2=2.1.0=py27_0
- pexpect=4.0.1=py27_0
- pickleshare=0.7.4=py27_0
- pip=9.0.1=py27_1
- prompt_toolkit=1.0.9=py27_0
- ptyprocess=0.5.1=py27_0
- pygments=2.1.3=py27_0
- pymongo=3.3.0=py27_0
- pyparsing=2.1.4=py27_0
- pyqt=5.6.0=py27_1
- python=2.7.12=1
- python-dateutil=2.6.0=py27_0
- python.app=1.2=py27_4
- pytz=2016.10=py27_0
- pyyaml=3.12=py27_0
- pyzmq=16.0.2=py27_0
- qt=5.6.2=0
- qtconsole=4.2.1=py27_1
- readline=6.2=2
- requests=2.12.3=py27_0
- scikit-learn=0.17.1=np111py27_2
- scipy=0.18.1=np111py27_0
- seaborn=0.7.1=py27_0
- setuptools=27.2.0=py27_0
- simplegeneric=0.8.1=py27_1
- singledispatch=3.4.0.3=py27_0
- sip=4.18=py27_0
- six=1.10.0=py27_0
- sqlite=3.13.0=0
- ssl_match_hostname=3.4.0.2=py27_1
- terminado=0.6=py27_0
- tk=8.5.18=0
- tornado=4.4.2=py27_0
- traitlets=4.3.1=py27_0
- unicodecsv=0.14.1=py27_0
- wcwidth=0.1.7=py27_0
- wheel=0.29.0=py27_0
- widgetsnbextension=1.2.6=py27_0
- xlrd=1.0.0=py27_0
- yaml=0.1.6=0
- zlib=1.2.8=3
- pip:
- backports-abc==0.5
- backports.shutil-get-terminal-size==1.0.0
- backports.ssl-match-hostname==3.4.0.2
- ipython-genutils==0.1.0
- jupyter-client==4.4.0
- jupyter-console==5.0.0
- jupyter-core==4.2.1
- nb-anacondacloud==1.2.0
- nb-conda==2.0.0
- nb-conda-kernels==2.0.0
- prompt-toolkit==1.0.9
prefix: /Users/mat/anaconda/envs/DAND
The error I run into:
Collecting package metadata (repodata.json): done
Solving environment: failed
ResolvePackageNotFound:
- jupyter_console==5.0.0=py27_0
- freetype==2.5.5=1
- pyzmq==16.0.2=py27_0
- configparser==3.5.0=py27_0
- scipy==0.18.1=np111py27_0
- libpng==1.6.22=0
- ...then the list goes on and list all of the dependencies in the yml file, except the ones under pip
Things I've attempted:
I got this yaml file from a Udacity online class I'm taking, I downloaded from the website, so I don't think conda env export --no-builds > env.yml method applies to me.
I tried the solution in here, I simply moved everything under pip block, and run into a new error. Maybe I'm misunderstanding the solution.
The new error I run into:
Warning: you have pip-installed dependencies in your environment file, but you do not list pip itself as one of your conda dependencies. Conda may not use the correct pip to install your packages, and they may end up in the wrong place. Please add an explicit pip dependency. I'm adding one for you, but still nagging you.
Collecting package metadata (repodata.json): done
Solving environment: done
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Ran pip subprocess with arguments:
['/Users/yulia/anaconda3/envs/DAND/bin/python', '-m', 'pip', 'install', '-U', '-r', '/Users/yulia/data analysis -uda/condaenv.mo_ctuap.requirements.txt']
Pip subprocess output:
Pip subprocess error:
ERROR: Double requirement given: backports_abc==0.5=py27_0 (from -r /Users/yulia/data analysis -uda/condaenv.mo_ctuap.requirements.txt (line 12)) (already in backports-abc==0.5 (from -r /Users/yulia/data analysis -uda/condaenv.mo_ctuap.requirements.txt (line 1)), name='backports-abc')
CondaEnvException: Pip failed
I read some other posts suggesting to use pip to install the requirements.txt file, and some posts about "CondaEnvException: Pip failed" situation. But they didn't write explicit solutions, most of the time I'm really confused about those solutions.
Please let me know what I'm missing here, this is getting frustrating as I cannot set up the proper environment to continue the class. Thank you so much in advance!
UPDATE
It seems that things might work better in the end when you skip using the env file. Instead, create an env with required dependencies manually, this way libraries are up-to-date and notebooks appear to work properly.
$ conda create -n DAND python=2 numpy pandas matplotlib seaborn
Look for required libraries in your course's "Setting up your system" (or similar) section. The ones in my example are based on Udacity's "Intro to Data Analysis" course.
Older answer
I had a similar problem and what eventually worked for me was adding two more channels in the channels section of this YAML file.
Before:
channels: !!python/tuple
- defaults
After:
channels: !!python/tuple
- defaults
- conda-forge
- anaconda
Then all the packages even with the version restrictions were found.
In case you get some errors about conflicting version, make sure to set conda config channel_priority to false:
$ conda config --set channel_priority false
Related
I am trying to install nvidial toolkit with cuda 11.6.0
It keeps giving me this message, I have tried several methods to fix it but none of them worked.
I am using this command: conda install cuda --channel nvidia/
label/cuda-11.6.0
Collecting package metadata (current_repodata.json): done
Solving environment: \
The environment is inconsistent, please check the package plan carefully
The following packages are causing the inconsistency:
- defaults/win-64::anaconda==custom=py39_1
- defaults/win-64::anaconda-navigator==2.1.1=py39_0
- defaults/win-64::bcrypt==3.2.0=py39h196d8e1_0
- defaults/noarch::black==19.10b0=py_0
- defaults/noarch::bleach==4.0.0=pyhd3eb1b0_0
- defaults/win-64::bokeh==2.4.1=py39haa95532_0
- defaults/noarch::conda-token==0.3.0=pyhd3eb1b0_0
- defaults/noarch::dask==2021.10.0=pyhd3eb1b0_0
- defaults/noarch::dask-core==2021.10.0=pyhd3eb1b0_0
- defaults/win-64::distributed==2021.10.0=py39haa95532_0
- defaults/noarch::ipywidgets==7.6.5=pyhd3eb1b0_1
- defaults/win-64::jupyter==1.0.0=py39haa95532_7
- defaults/noarch::jupyterlab==3.2.1=pyhd3eb1b0_1
- defaults/noarch::jupyterlab_server==2.8.2=pyhd3eb1b0_0
- defaults/win-64::jupyter_server==1.4.1=py39haa95532_0
- defaults/noarch::nbclassic==0.2.6=pyhd3eb1b0_0
- defaults/win-64::notebook==6.4.12=py39haa95532_0
- defaults/noarch::numpydoc==1.1.0=pyhd3eb1b0_1
- defaults/noarch::paramiko==2.7.2=py_0
- defaults/win-64::pytest==6.2.4=py39haa95532_2
- defaults/noarch::python-lsp-black==1.0.0=pyhd3eb1b0_0
- defaults/win-64::scikit-image==0.18.3=py39hf11a4ad_0
- defaults/noarch::sphinx==4.2.0=pyhd3eb1b0_1
- defaults/win-64::spyder==5.1.5=py39haa95532_1
- pytorch/win-64::torchaudio==0.11.0=py39_cu113
- defaults/win-64::widgetsnbextension==3.5.1=py39haa95532_0
- defaults/win-64::_anaconda_depends==2021.11=py39_0
- defaults/win-64::_ipyw_jlab_nb_ext_conf==0.1.0=py39haa95532_0
I work with conda environments and need some pip packages as well, e.g. pre-compiled wheels from ~gohlke.
At the moment I have two files: environment.yml for conda with:
# run: conda env create --file environment.yml
name: test-env
dependencies:
- python>=3.5
- anaconda
and requirements.txt for pip which can be used after activating above conda environment:
# run: pip install -i requirements.txt
docx
gooey
http://www.lfd.uci.edu/~gohlke/pythonlibs/bofhrmxk/opencv_python-3.1.0-cp35-none-win_amd64.whl
Is there a possibility to combine them in one file (for conda)?
Pip dependencies can be included in the environment.yml file like this (docs):
# run: conda env create --file environment.yml
name: test-env
dependencies:
- python>=3.5
- anaconda
- pip
- numpy=1.13.3 # pin version for conda
- pip:
# works for regular pip packages
- docx
- gooey
- matplotlib==2.0.0 # pin version for pip
# and for wheels
- http://www.lfd.uci.edu/~gohlke/pythonlibs/bofhrmxk/opencv_python-3.1.0-cp35-none-win_amd64.whl
It also works for .whl files in the same directory (see Dengar's answer) as well as with common pip packages.
One can also use the requirements.txt directly in the YAML. For example,
name: test-env
dependencies:
- python>=3.5
- anaconda
- pip
- pip:
- -r requirements.txt
Basically, any option you can run with pip install you can run in a YAML. See the Advanced Pip Example for a showcase of other capabilities.
Important Note
A previous version of this answer (and Conda's Advanced Pip Example) used a substandard file URI syntax:
- -r file:requirements.txt
Pip v21.2.1 introduced stricter behavior for URI parsing and no longer supports this. See this answer for details.
Just want to add that adding a wheel in the directory also works. I was getting this error when using the entire URL:
HTTP error 404 while getting http://www.lfd.uci.edu/~gohlke/pythonlibs/f9r7rmd8/opencv_python-3.1.0-cp35-none-win_amd64.whl
Ended up downloading the wheel and saving it into the same directory as the yml file.
name: test-env
dependencies:
- python>=3.5
- anaconda
- pip
- pip:
- opencv_python-3.1.0-cp35-none-win_amd64.whl
If you want to do it automatically it seems that if you do:
conda env export > environment.yml
already has the pip things you need. No need to run pip freeze > requirements4pip.txt separately for me or include it as a
- pip:
- -r file:requirements.txt
as another answer has mentioned.
See my yml file:
$ cat environment.yml
name: myenv
channels:
- pytorch
- dglteam
- defaults
- conda-forge
dependencies:
- _libgcc_mutex=0.1=main
- absl-py=0.12.0=py38h06a4308_0
- aiohttp=3.7.4=py38h27cfd23_1
- async-timeout=3.0.1=py38h06a4308_0
- attrs=20.3.0=pyhd3eb1b0_0
- beautifulsoup4=4.9.3=pyha847dfd_0
- blas=1.0=mkl
- blinker=1.4=py38h06a4308_0
- brotlipy=0.7.0=py38h27cfd23_1003
- bzip2=1.0.8=h7b6447c_0
- c-ares=1.17.1=h27cfd23_0
- ca-certificates=2021.4.13=h06a4308_1
- cachetools=4.2.1=pyhd3eb1b0_0
- cairo=1.14.12=h8948797_3
- certifi=2020.12.5=py38h06a4308_0
- cffi=1.14.0=py38h2e261b9_0
- chardet=3.0.4=py38h06a4308_1003
- click=7.1.2=pyhd3eb1b0_0
- conda=4.10.1=py38h06a4308_1
- conda-build=3.21.4=py38h06a4308_0
- conda-package-handling=1.7.3=py38h27cfd23_1
- coverage=5.5=py38h27cfd23_2
- cryptography=3.4.7=py38hd23ed53_0
- cudatoolkit=11.0.221=h6bb024c_0
- cycler=0.10.0=py38_0
- cython=0.29.23=py38h2531618_0
- dbus=1.13.18=hb2f20db_0
- decorator=4.4.2=pyhd3eb1b0_0
- dgl-cuda11.0=0.6.1=py38_0
- dill=0.3.3=pyhd3eb1b0_0
- expat=2.3.0=h2531618_2
- filelock=3.0.12=pyhd3eb1b0_1
- fontconfig=2.13.1=h6c09931_0
- freetype=2.10.4=h7ca028e_0
- fribidi=1.0.10=h7b6447c_0
- gettext=0.21.0=hf68c758_0
- glib=2.66.3=h58526e2_0
- glob2=0.7=pyhd3eb1b0_0
- google-auth=1.29.0=pyhd3eb1b0_0
- google-auth-oauthlib=0.4.4=pyhd3eb1b0_0
- graphite2=1.3.14=h23475e2_0
- graphviz=2.40.1=h21bd128_2
- grpcio=1.36.1=py38h2157cd5_1
- gst-plugins-base=1.14.0=h8213a91_2
- gstreamer=1.14.0=h28cd5cc_2
- harfbuzz=1.8.8=hffaf4a1_0
- icu=58.2=he6710b0_3
- idna=2.10=pyhd3eb1b0_0
- importlib-metadata=3.10.0=py38h06a4308_0
- intel-openmp=2021.2.0=h06a4308_610
- jinja2=2.11.3=pyhd3eb1b0_0
- joblib=1.0.1=pyhd3eb1b0_0
- jpeg=9b=h024ee3a_2
- kiwisolver=1.3.1=py38h2531618_0
- lcms2=2.12=h3be6417_0
- ld_impl_linux-64=2.33.1=h53a641e_7
- libarchive=3.4.2=h62408e4_0
- libffi=3.2.1=hf484d3e_1007
- libgcc-ng=9.1.0=hdf63c60_0
- libgfortran-ng=7.3.0=hdf63c60_0
- libglib=2.66.3=hbe7bbb4_0
- libiconv=1.16=h516909a_0
- liblief=0.10.1=he6710b0_0
- libpng=1.6.37=h21135ba_2
- libprotobuf=3.14.0=h8c45485_0
- libstdcxx-ng=9.1.0=hdf63c60_0
- libtiff=4.1.0=h2733197_1
- libuuid=1.0.3=h1bed415_2
- libuv=1.40.0=h7b6447c_0
- libxcb=1.14=h7b6447c_0
- libxml2=2.9.10=hb55368b_3
- lz4-c=1.9.2=he1b5a44_3
- markdown=3.3.4=py38h06a4308_0
- markupsafe=1.1.1=py38h7b6447c_0
- matplotlib=3.3.4=py38h06a4308_0
- matplotlib-base=3.3.4=py38h62a2d02_0
- mkl=2020.2=256
- mkl-service=2.3.0=py38h1e0a361_2
- mkl_fft=1.3.0=py38h54f3939_0
- mkl_random=1.2.0=py38hc5bc63f_1
- multidict=5.1.0=py38h27cfd23_2
- ncurses=6.2=he6710b0_1
- networkx=2.5.1=pyhd3eb1b0_0
- ninja=1.10.2=hff7bd54_1
- numpy=1.19.2=py38h54aff64_0
- numpy-base=1.19.2=py38hfa32c7d_0
- oauthlib=3.1.0=py_0
- olefile=0.46=pyh9f0ad1d_1
- openssl=1.1.1k=h27cfd23_0
- pandas=1.2.4=py38h2531618_0
- pango=1.42.4=h049681c_0
- patchelf=0.12=h2531618_1
- pcre=8.44=he6710b0_0
- pillow=8.2.0=py38he98fc37_0
- pip=21.0.1=py38h06a4308_0
- pixman=0.40.0=h7b6447c_0
- pkginfo=1.7.0=py38h06a4308_0
- protobuf=3.14.0=py38h2531618_1
- psutil=5.8.0=py38h27cfd23_1
- py-lief=0.10.1=py38h403a769_0
- pyasn1=0.4.8=py_0
- pyasn1-modules=0.2.8=py_0
- pycosat=0.6.3=py38h7b6447c_1
- pycparser=2.20=py_2
- pyjwt=2.0.1=pyhd8ed1ab_1
- pyopenssl=20.0.1=pyhd3eb1b0_1
- pyparsing=2.4.7=pyhd3eb1b0_0
- pyqt=5.9.2=py38h05f1152_4
- pysocks=1.7.1=py38h06a4308_0
- python=3.8.2=hcf32534_0
- python-dateutil=2.8.1=pyhd3eb1b0_0
- python-libarchive-c=2.9=pyhd3eb1b0_1
- python_abi=3.8=1_cp38
- pytorch=1.7.1=py3.8_cuda11.0.221_cudnn8.0.5_0
- pytz=2021.1=pyhd3eb1b0_0
- pyyaml=5.4.1=py38h27cfd23_1
- qt=5.9.7=h5867ecd_1
- readline=8.1=h27cfd23_0
- requests=2.25.1=pyhd3eb1b0_0
- requests-oauthlib=1.3.0=py_0
- ripgrep=12.1.1=0
- rsa=4.7.2=pyhd3eb1b0_1
- ruamel_yaml=0.15.100=py38h27cfd23_0
- scikit-learn=0.24.1=py38ha9443f7_0
- scipy=1.6.2=py38h91f5cce_0
- setuptools=52.0.0=py38h06a4308_0
- sip=4.19.13=py38he6710b0_0
- six=1.15.0=pyh9f0ad1d_0
- soupsieve=2.2.1=pyhd3eb1b0_0
- sqlite=3.35.4=hdfb4753_0
- tensorboard=2.4.0=pyhc547734_0
- tensorboard-plugin-wit=1.6.0=py_0
- threadpoolctl=2.1.0=pyh5ca1d4c_0
- tk=8.6.10=hbc83047_0
- torchaudio=0.7.2=py38
- torchtext=0.8.1=py38
- torchvision=0.8.2=py38_cu110
- tornado=6.1=py38h27cfd23_0
- typing-extensions=3.7.4.3=0
- typing_extensions=3.7.4.3=py_0
- urllib3=1.26.4=pyhd3eb1b0_0
- werkzeug=1.0.1=pyhd3eb1b0_0
- wheel=0.36.2=pyhd3eb1b0_0
- xz=5.2.5=h7b6447c_0
- yaml=0.2.5=h7b6447c_0
- yarl=1.6.3=py38h27cfd23_0
- zipp=3.4.1=pyhd3eb1b0_0
- zlib=1.2.11=h7b6447c_3
- zstd=1.4.5=h9ceee32_0
- pip:
- aioconsole==0.3.1
- lark-parser==0.6.5
- lmdb==0.94
- pexpect==4.6.0
- progressbar2==3.39.3
- ptyprocess==0.7.0
- pycapnp==1.0.0
- python-utils==2.5.6
- sexpdata==0.0.3
- tqdm==4.56.0
prefix: /home/miranda9/miniconda3/envs/myenv
Note that at the time of this writing doing conda env create --file environment.yml to create the yml env results in an error:
$ conda env create --file environment.yml
CondaValueError: prefix already exists: /home/miranda9/miniconda3/envs/myenv
I would like to know how to install python libraries using yml file without making a new environment. I already have tensorflow environment in conda. I want to install list of libraries into this tensorflow environment. It is the only way I know manually add each of these libraries but it is very hard to do this list. Please give me solution for that
This is yml file:
name: virtual_platform
channels:
- menpo
- conda-forge
- peterjc123
- defaults
dependencies:
- ffmpeg=3.2.4=1
- freetype=2.7=vc14_1
- imageio=2.2.0=py35_0
- libtiff=4.0.6=vc14_7
- olefile=0.44=py35_0
- pillow=4.2.1=py35_0
- vc=14=0
- alabaster=0.7.10=py35_0
- astroid=1.5.3=py35_0
- babel=2.5.0=py35_0
- bleach=1.5.0=py35_0
- certifi=2016.2.28=py35_0
- cffi=1.10.0=py35_0
- chardet=3.0.4=py35_0
- colorama=0.3.9=py35_0
- decorator=4.1.2=py35_0
- docutils=0.14=py35_0
- entrypoints=0.2.3=py35_0
- html5lib=0.9999999=py35_0
- icu=57.1=vc14_0
- imagesize=0.7.1=py35_0
- ipykernel=4.6.1=py35_0
- ipython=6.1.0=py35_0
- ipython_genutils=0.2.0=py35_0
- isort=4.2.15=py35_0
- jedi=0.10.2=py35_2
- jinja2=2.9.6=py35_0
- jpeg=9b=vc14_0
- jsonschema=2.6.0=py35_0
- jupyter_client=5.1.0=py35_0
- jupyter_core=4.3.0=py35_0
- lazy-object-proxy=1.3.1=py35_0
- libpng=1.6.30=vc14_1
- markupsafe=1.0=py35_0
- mistune=0.7.4=py35_0
- mkl=2017.0.3=0
- nbconvert=5.2.1=py35_0
- nbformat=4.4.0=py35_0
- numpy=1.13.1=py35_0
- numpydoc=0.7.0=py35_0
- openssl=1.0.2l=vc14_0
- pandocfilters=1.4.2=py35_0
- path.py=10.3.1=py35_0
- pickleshare=0.7.4=py35_0
- pip=9.0.1=py35_1
- prompt_toolkit=1.0.15=py35_0
- psutil=5.2.2=py35_0
- pycodestyle=2.3.1=py35_0
- pycparser=2.18=py35_0
- pyflakes=1.6.0=py35_0
- pygments=2.2.0=py35_0
- pylint=1.7.2=py35_0
- pyqt=5.6.0=py35_2
- python=3.5.4=0
- python-dateutil=2.6.1=py35_0
- pytz=2017.2=py35_0
- pyzmq=16.0.2=py35_0
- qt=5.6.2=vc14_6
- qtawesome=0.4.4=py35_0
- qtconsole=4.3.1=py35_0
- qtpy=1.3.1=py35_0
- requests=2.14.2=py35_0
- rope=0.9.4=py35_1
- setuptools=36.4.0=py35_1
- simplegeneric=0.8.1=py35_1
- singledispatch=3.4.0.3=py35_0
- sip=4.18=py35_0
- six=1.10.0=py35_1
- snowballstemmer=1.2.1=py35_0
- sphinx=1.6.3=py35_0
- sphinxcontrib=1.0=py35_0
- sphinxcontrib-websupport=1.0.1=py35_0
- spyder=3.2.3=py35_0
- testpath=0.3.1=py35_0
- tornado=4.5.2=py35_0
- traitlets=4.3.2=py35_0
- vs2015_runtime=14.0.25420=0
- wcwidth=0.1.7=py35_0
- wheel=0.29.0=py35_0
- win_unicode_console=0.5=py35_0
- wincertstore=0.2=py35_0
- wrapt=1.10.11=py35_0
- zlib=1.2.11=vc14_0
- opencv3=3.1.0=py35_0
- pytorch=0.1.12=py35_0.1.12cu80
- torch==0.1.12
- torchvision==0.1.9
- pip:
- ipython-genutils==0.2.0
- jupyter-client==5.1.0
- jupyter-core==4.3.0
- prompt-toolkit==1.0.15
- pyyaml==3.12
- rope-py3k==0.9.4.post1
- torch==0.1.12
- torchvision==0.1.9
- win-unicode-console==0.5
You can use the conda env update command:
conda env update --name <your env name> -f <your file>.yml
or, if the environment you want to update is already activated, then
conda env update -f <your file>.yml
If you want to create the environment from your yml file:
conda env create -f environment.yml
The name of your environment is virtual_platform. If you want another name, just edit your yml name to desired name.
It is not recommended to install packages to your base environment but if that is what you want, and I believe you should not, you need to create a requirement.txt from dependencies listed on your yml.
Copy and paste all the dependencies
packages and there version to requirements.txt as:
python ==3.5
ffmpeg=3.2.4
freetype=2.7
imageio=2.2.0
...
Then do:
conda install --yes --file requirements.txt
The problem is that this will fail if any dependence fail to install. So I will recommend installing using yml which means having an environment separate from the rest.
I just need to import an Anaconda .yml environmental file in virtualenv virtual environment.
The reason I need to do this is because on nVidia Jetson TX2 developer board I cannot install and run Anaconda distribution (It is not compatible with ARM architecture). Virtualenv and Jupyter, instead, are installed and run flawlessly.
The .yml file is listed like this:
name: tfdeeplearning
channels:
- defaults
dependencies:
- bleach=1.5.0=py35_0
- certifi=2016.2.28=py35_0
- colorama=0.3.9=py35_0
- cycler=0.10.0=py35_0
- decorator=4.1.2=py35_0
- entrypoints=0.2.3=py35_0
- html5lib=0.9999999=py35_0
- icu=57.1=vc14_0
- ipykernel=4.6.1=py35_0
- ipython=6.1.0=py35_0
- ipython_genutils=0.2.0=py35_0
- ipywidgets=6.0.0=py35_0
- jedi=0.10.2=py35_2
- jinja2=2.9.6=py35_0
- jpeg=9b=vc14_0
- jsonschema=2.6.0=py35_0
- jupyter=1.0.0=py35_3
- jupyter_client=5.1.0=py35_0
- jupyter_console=5.2.0=py35_0
- jupyter_core=4.3.0=py35_0
- libpng=1.6.30=vc14_1
- markupsafe=1.0=py35_0
- matplotlib=2.0.2=np113py35_0
- mistune=0.7.4=py35_0
- mkl=2017.0.3=0
- nbconvert=5.2.1=py35_0
- nbformat=4.4.0=py35_0
- notebook=5.0.0=py35_0
- numpy=1.13.1=py35_0
- openssl=1.0.2l=vc14_0
- pandas=0.20.3=py35_0
- pandocfilters=1.4.2=py35_0
- path.py=10.3.1=py35_0
- pickleshare=0.7.4=py35_0
- pip=9.0.1=py35_1
- prompt_toolkit=1.0.15=py35_0
- pygments=2.2.0=py35_0
- pyparsing=2.2.0=py35_0
- pyqt=5.6.0=py35_2
- python=3.5.4=0
- python-dateutil=2.6.1=py35_0
- pytz=2017.2=py35_0
- pyzmq=16.0.2=py35_0
- qt=5.6.2=vc14_6
- qtconsole=4.3.1=py35_0
- requests=2.14.2=py35_0
- scikit-learn=0.19.0=np113py35_0
- scipy=0.19.1=np113py35_0
- setuptools=36.4.0=py35_1
- simplegeneric=0.8.1=py35_1
- sip=4.18=py35_0
- six=1.10.0=py35_1
- testpath=0.3.1=py35_0
- tk=8.5.18=vc14_0
- tornado=4.5.2=py35_0
- traitlets=4.3.2=py35_0
- vs2015_runtime=14.0.25420=0
- wcwidth=0.1.7=py35_0
- wheel=0.29.0=py35_0
- widgetsnbextension=3.0.2=py35_0
- win_unicode_console=0.5=py35_0
- wincertstore=0.2=py35_0
- zlib=1.2.11=vc14_0
- pip:
- ipython-genutils==0.2.0
- jupyter-client==5.1.0
- jupyter-console==5.2.0
- jupyter-core==4.3.0
- markdown==2.6.9
- prompt-toolkit==1.0.15
- protobuf==3.4.0
- tensorflow==1.3.0
- tensorflow-tensorboard==0.1.6
- werkzeug==0.12.2
- win-unicode-console==0.5
prefix: C:\Users\Marcial\Anaconda3\envs\tfdeeplearning
pip can install from a requirements.txt file, which would look like
the items in the sequence that is the value for the key
pip in your .yml file, but without the dashes:
ipython-genutils==0.2.0
jupyter-client==5.1.0
jupyter-console==5.2.0
jupyter-core==4.3.0
markdown==2.6.9
prompt-toolkit==1.0.15
protobuf==3.4.0
tensorflow==1.3.0
tensorflow-tensorboard==0.1.6
werkzeug==0.12.2
win-unicode-console==0.5
Assuming that the end of your file actually looks like:
.
.
.
- wincertstore=0.2=py35_0
- zlib=1.2.11=vc14_0
- pip:
- ipython-genutils==0.2.0
- jupyter-client==5.1.0
- jupyter-console==5.2.0
- jupyter-core==4.3.0
- markdown==2.6.9
- prompt-toolkit==1.0.15
- protobuf==3.4.0
- tensorflow==1.3.0
- tensorflow-tensorboard==0.1.6
- werkzeug==0.12.2
- win-unicode-console==0.5
prefix: C:\Users\Marcial\Anaconda3\envs\tfdeeplearning
(i.e. the entry for pip is indented to make this a valid YAML file),
and is named anaconda-project.yml, you can do:
import ruamel.yaml
yaml = ruamel.yaml.YAML()
data = yaml.load(open('anaconda-project.yml'))
requirements = []
for dep in data['dependencies']:
if isinstance(dep, str):
package, package_version, python_version = dep.split('=')
if python_version == '0':
continue
requirements.append(package + '==' + package_version)
elif isinstance(dep, dict):
for preq in dep.get('pip', []):
requirements.append(preq)
with open('requirements.txt', 'w') as fp:
for requirement in requirements:
print(requirement, file=fp)
resulting in a requirement.txt file, which can be used with:
pip install -r requirements.txt
Please note:
the non-pip packages might not be available from PyPI
the current pip version is 18.1, the one in that requirements list is old
that according to the official YAML FAQ, using .yml as an
extension for your YAML file should only be done if the recommended
.yaml extension. On modern filesystems that is never the case. I
don't know if Anaconda is, as so often, non-conform, or that you
have a choice in the matter.
since the introduction of binary wheels a few years ago, and many
packages supporting them, it is often (and for me always) possible
to just use virtualenvs and pip. And thereby circumventing the
problems caused by Anaconda not being 100% compliant and not being
up-to-date with all its packages (compared to PyPI).
I work with conda environments and need some pip packages as well, e.g. pre-compiled wheels from ~gohlke.
At the moment I have two files: environment.yml for conda with:
# run: conda env create --file environment.yml
name: test-env
dependencies:
- python>=3.5
- anaconda
and requirements.txt for pip which can be used after activating above conda environment:
# run: pip install -i requirements.txt
docx
gooey
http://www.lfd.uci.edu/~gohlke/pythonlibs/bofhrmxk/opencv_python-3.1.0-cp35-none-win_amd64.whl
Is there a possibility to combine them in one file (for conda)?
Pip dependencies can be included in the environment.yml file like this (docs):
# run: conda env create --file environment.yml
name: test-env
dependencies:
- python>=3.5
- anaconda
- pip
- numpy=1.13.3 # pin version for conda
- pip:
# works for regular pip packages
- docx
- gooey
- matplotlib==2.0.0 # pin version for pip
# and for wheels
- http://www.lfd.uci.edu/~gohlke/pythonlibs/bofhrmxk/opencv_python-3.1.0-cp35-none-win_amd64.whl
It also works for .whl files in the same directory (see Dengar's answer) as well as with common pip packages.
One can also use the requirements.txt directly in the YAML. For example,
name: test-env
dependencies:
- python>=3.5
- anaconda
- pip
- pip:
- -r requirements.txt
Basically, any option you can run with pip install you can run in a YAML. See the Advanced Pip Example for a showcase of other capabilities.
Important Note
A previous version of this answer (and Conda's Advanced Pip Example) used a substandard file URI syntax:
- -r file:requirements.txt
Pip v21.2.1 introduced stricter behavior for URI parsing and no longer supports this. See this answer for details.
Just want to add that adding a wheel in the directory also works. I was getting this error when using the entire URL:
HTTP error 404 while getting http://www.lfd.uci.edu/~gohlke/pythonlibs/f9r7rmd8/opencv_python-3.1.0-cp35-none-win_amd64.whl
Ended up downloading the wheel and saving it into the same directory as the yml file.
name: test-env
dependencies:
- python>=3.5
- anaconda
- pip
- pip:
- opencv_python-3.1.0-cp35-none-win_amd64.whl
If you want to do it automatically it seems that if you do:
conda env export > environment.yml
already has the pip things you need. No need to run pip freeze > requirements4pip.txt separately for me or include it as a
- pip:
- -r file:requirements.txt
as another answer has mentioned.
See my yml file:
$ cat environment.yml
name: myenv
channels:
- pytorch
- dglteam
- defaults
- conda-forge
dependencies:
- _libgcc_mutex=0.1=main
- absl-py=0.12.0=py38h06a4308_0
- aiohttp=3.7.4=py38h27cfd23_1
- async-timeout=3.0.1=py38h06a4308_0
- attrs=20.3.0=pyhd3eb1b0_0
- beautifulsoup4=4.9.3=pyha847dfd_0
- blas=1.0=mkl
- blinker=1.4=py38h06a4308_0
- brotlipy=0.7.0=py38h27cfd23_1003
- bzip2=1.0.8=h7b6447c_0
- c-ares=1.17.1=h27cfd23_0
- ca-certificates=2021.4.13=h06a4308_1
- cachetools=4.2.1=pyhd3eb1b0_0
- cairo=1.14.12=h8948797_3
- certifi=2020.12.5=py38h06a4308_0
- cffi=1.14.0=py38h2e261b9_0
- chardet=3.0.4=py38h06a4308_1003
- click=7.1.2=pyhd3eb1b0_0
- conda=4.10.1=py38h06a4308_1
- conda-build=3.21.4=py38h06a4308_0
- conda-package-handling=1.7.3=py38h27cfd23_1
- coverage=5.5=py38h27cfd23_2
- cryptography=3.4.7=py38hd23ed53_0
- cudatoolkit=11.0.221=h6bb024c_0
- cycler=0.10.0=py38_0
- cython=0.29.23=py38h2531618_0
- dbus=1.13.18=hb2f20db_0
- decorator=4.4.2=pyhd3eb1b0_0
- dgl-cuda11.0=0.6.1=py38_0
- dill=0.3.3=pyhd3eb1b0_0
- expat=2.3.0=h2531618_2
- filelock=3.0.12=pyhd3eb1b0_1
- fontconfig=2.13.1=h6c09931_0
- freetype=2.10.4=h7ca028e_0
- fribidi=1.0.10=h7b6447c_0
- gettext=0.21.0=hf68c758_0
- glib=2.66.3=h58526e2_0
- glob2=0.7=pyhd3eb1b0_0
- google-auth=1.29.0=pyhd3eb1b0_0
- google-auth-oauthlib=0.4.4=pyhd3eb1b0_0
- graphite2=1.3.14=h23475e2_0
- graphviz=2.40.1=h21bd128_2
- grpcio=1.36.1=py38h2157cd5_1
- gst-plugins-base=1.14.0=h8213a91_2
- gstreamer=1.14.0=h28cd5cc_2
- harfbuzz=1.8.8=hffaf4a1_0
- icu=58.2=he6710b0_3
- idna=2.10=pyhd3eb1b0_0
- importlib-metadata=3.10.0=py38h06a4308_0
- intel-openmp=2021.2.0=h06a4308_610
- jinja2=2.11.3=pyhd3eb1b0_0
- joblib=1.0.1=pyhd3eb1b0_0
- jpeg=9b=h024ee3a_2
- kiwisolver=1.3.1=py38h2531618_0
- lcms2=2.12=h3be6417_0
- ld_impl_linux-64=2.33.1=h53a641e_7
- libarchive=3.4.2=h62408e4_0
- libffi=3.2.1=hf484d3e_1007
- libgcc-ng=9.1.0=hdf63c60_0
- libgfortran-ng=7.3.0=hdf63c60_0
- libglib=2.66.3=hbe7bbb4_0
- libiconv=1.16=h516909a_0
- liblief=0.10.1=he6710b0_0
- libpng=1.6.37=h21135ba_2
- libprotobuf=3.14.0=h8c45485_0
- libstdcxx-ng=9.1.0=hdf63c60_0
- libtiff=4.1.0=h2733197_1
- libuuid=1.0.3=h1bed415_2
- libuv=1.40.0=h7b6447c_0
- libxcb=1.14=h7b6447c_0
- libxml2=2.9.10=hb55368b_3
- lz4-c=1.9.2=he1b5a44_3
- markdown=3.3.4=py38h06a4308_0
- markupsafe=1.1.1=py38h7b6447c_0
- matplotlib=3.3.4=py38h06a4308_0
- matplotlib-base=3.3.4=py38h62a2d02_0
- mkl=2020.2=256
- mkl-service=2.3.0=py38h1e0a361_2
- mkl_fft=1.3.0=py38h54f3939_0
- mkl_random=1.2.0=py38hc5bc63f_1
- multidict=5.1.0=py38h27cfd23_2
- ncurses=6.2=he6710b0_1
- networkx=2.5.1=pyhd3eb1b0_0
- ninja=1.10.2=hff7bd54_1
- numpy=1.19.2=py38h54aff64_0
- numpy-base=1.19.2=py38hfa32c7d_0
- oauthlib=3.1.0=py_0
- olefile=0.46=pyh9f0ad1d_1
- openssl=1.1.1k=h27cfd23_0
- pandas=1.2.4=py38h2531618_0
- pango=1.42.4=h049681c_0
- patchelf=0.12=h2531618_1
- pcre=8.44=he6710b0_0
- pillow=8.2.0=py38he98fc37_0
- pip=21.0.1=py38h06a4308_0
- pixman=0.40.0=h7b6447c_0
- pkginfo=1.7.0=py38h06a4308_0
- protobuf=3.14.0=py38h2531618_1
- psutil=5.8.0=py38h27cfd23_1
- py-lief=0.10.1=py38h403a769_0
- pyasn1=0.4.8=py_0
- pyasn1-modules=0.2.8=py_0
- pycosat=0.6.3=py38h7b6447c_1
- pycparser=2.20=py_2
- pyjwt=2.0.1=pyhd8ed1ab_1
- pyopenssl=20.0.1=pyhd3eb1b0_1
- pyparsing=2.4.7=pyhd3eb1b0_0
- pyqt=5.9.2=py38h05f1152_4
- pysocks=1.7.1=py38h06a4308_0
- python=3.8.2=hcf32534_0
- python-dateutil=2.8.1=pyhd3eb1b0_0
- python-libarchive-c=2.9=pyhd3eb1b0_1
- python_abi=3.8=1_cp38
- pytorch=1.7.1=py3.8_cuda11.0.221_cudnn8.0.5_0
- pytz=2021.1=pyhd3eb1b0_0
- pyyaml=5.4.1=py38h27cfd23_1
- qt=5.9.7=h5867ecd_1
- readline=8.1=h27cfd23_0
- requests=2.25.1=pyhd3eb1b0_0
- requests-oauthlib=1.3.0=py_0
- ripgrep=12.1.1=0
- rsa=4.7.2=pyhd3eb1b0_1
- ruamel_yaml=0.15.100=py38h27cfd23_0
- scikit-learn=0.24.1=py38ha9443f7_0
- scipy=1.6.2=py38h91f5cce_0
- setuptools=52.0.0=py38h06a4308_0
- sip=4.19.13=py38he6710b0_0
- six=1.15.0=pyh9f0ad1d_0
- soupsieve=2.2.1=pyhd3eb1b0_0
- sqlite=3.35.4=hdfb4753_0
- tensorboard=2.4.0=pyhc547734_0
- tensorboard-plugin-wit=1.6.0=py_0
- threadpoolctl=2.1.0=pyh5ca1d4c_0
- tk=8.6.10=hbc83047_0
- torchaudio=0.7.2=py38
- torchtext=0.8.1=py38
- torchvision=0.8.2=py38_cu110
- tornado=6.1=py38h27cfd23_0
- typing-extensions=3.7.4.3=0
- typing_extensions=3.7.4.3=py_0
- urllib3=1.26.4=pyhd3eb1b0_0
- werkzeug=1.0.1=pyhd3eb1b0_0
- wheel=0.36.2=pyhd3eb1b0_0
- xz=5.2.5=h7b6447c_0
- yaml=0.2.5=h7b6447c_0
- yarl=1.6.3=py38h27cfd23_0
- zipp=3.4.1=pyhd3eb1b0_0
- zlib=1.2.11=h7b6447c_3
- zstd=1.4.5=h9ceee32_0
- pip:
- aioconsole==0.3.1
- lark-parser==0.6.5
- lmdb==0.94
- pexpect==4.6.0
- progressbar2==3.39.3
- ptyprocess==0.7.0
- pycapnp==1.0.0
- python-utils==2.5.6
- sexpdata==0.0.3
- tqdm==4.56.0
prefix: /home/miranda9/miniconda3/envs/myenv
Note that at the time of this writing doing conda env create --file environment.yml to create the yml env results in an error:
$ conda env create --file environment.yml
CondaValueError: prefix already exists: /home/miranda9/miniconda3/envs/myenv