Conda-pack: CondaPackError: files managed by conda were - python

So, I have a python project where all my tests run but then I do:
conda install -y conda-pack
which succeeds and when I run:
conda-pack
I get a lengthy complaint:
Collecting packages...
CondaPackError:
Files managed by conda were found to have been deleted/overwritten in the
following packages:
- conda-pack 0.6.0:
lib/python3.1/site-packages/conda_pack-0.6.0.dist-info/INSTALLER
lib/python3.1/site-packages/conda_pack-0.6.0.dist-info/LICENSE.txt
lib/python3.1/site-packages/conda_pack-0.6.0.dist-info/METADATA
+ 19 others
- types-requests 2.26.0:
lib/python3.1/site-packages/requests-stubs/METADATA.toml
lib/python3.1/site-packages/requests-stubs/__init__.pyi
lib/python3.1/site-packages/requests-stubs/adapters.pyi
+ 41 others
- jsonschema 4.2.1:
lib/python3.1/site-packages/jsonschema-4.2.1.dist-info/COPYING
lib/python3.1/site-packages/jsonschema-4.2.1.dist-info/INSTALLER
lib/python3.1/site-packages/jsonschema-4.2.1.dist-info/METADATA
+ 39 others
- types-setuptools 57.4.2:
lib/python3.1/site-packages/pkg_resources-stubs/METADATA.toml
lib/python3.1/site-packages/pkg_resources-stubs/__init__.pyi
lib/python3.1/site-packages/pkg_resources-stubs/py31compat.pyi
+ 56 others
<snip>
I see no issues in conda list and I can still run my tests. I see no issues with my environment. Ideas?

This should be related to this issue: https://github.com/conda/conda-pack/issues/198
The root cause is that the image is trying to use python3.10 and conda-pack parse it as python3.1.
They claimed to have fixed it in conda-pack 0.7.0 (release note) but I was using 0.7.0 and still see this problem.
One workaround is to specify another python version in a new env:
conda create -n new_env python=3.9 ${other_packages}
conda install conda-pack
conda-pack -n new_env ${your_other_options}

I think the problem is still there in conda-pack 0.7.0 in the unix-world and will only appear for python 3.10 and upward. It might be dependent on the conda version, but many users do not have the rights to update conda (as is the case for me). Reason are the symbolic links generated by python, which direct python 3.1 packages/programs to python 3.10. Conda-pack does not understand that correctly und searches for python3.1 files.
Solution is a) to update conda if possible b) if that fails to exclude those python3.1 references (running conda-pack in the environment, say myenv, you want to pack - will generate myenv.tar.gz):
conda-pack -f --ignore-missing-files --exclude lib/python3.1

Related

Warning: 2 possible package resolutions only showing differing packages anaconda

Well a similar question has been asked already: resolving package resolutions in conda
Solving environment: /
Warning: 2 possible package resolutions (only showing differing packages):
- defaults/noarch::path.py-12.0.2-py_0, defaults/osx-64::path-13.2.0-py37_0
- defaults/noarch::path.py-12.4.0-0, defaults/osx-64::path-13.1.0-py37done
Already tried the following:
conda update --strict-channel-priority --all
conda update --all
conda update anaconda
conda update conda
Nothing seems to resolve this! Will really appreciate any help.
Conda info ==> http://dpaste.com/2951Y1J
conda version: 4.8.3
conda build version: 3.18.11
OS: Mac OS 10.14 Mojave
TIA
Hi recently I had the same problem, I'm just a newbie, but I found a way to solve it.
For example:
Warning: 2 possible package resolutions (only showing differing packages):
defaults/noarch::sphinx-3.5.3-pyhd3eb1b0_0, defaults/win-64::docutils-0.17-py38haa95532_1
defaults/noarch::sphinx-3.5.4-pyhd3eb1b0_0, defaults/win-64::docutils-0.16-py38done
The solution is: conda update sphinx, and then conda update docutils just in case, use conda update (name). depending on the name
Alright, found the solution
conda install anaconda-clean
anaconda-clean --yes
conda update --all
Should resolve the issue.
Just encountered this. What worked for me was:
conda update conda
conda update anaconda
conda update conda
conda update anaconda
Note that in my case this needed to be run in this specific order (with repeats).

Anaconda Navigator installing an older version (0.7.3) of Sympy

I'm new to Python and just downloaded Anaconda Python. Anaconda comes with Sympy 1.4 but I need to use Sympy 0.7.3.
I see that Anaconda Navigator has the option to replace 1.4 with other versions (specifically 1.1.1 or 1.2 or 1.3) however 0.7.3 is not in the list. I have found an archive with version 0.7.3 at https://anaconda.org/anaconda/sympy/files?version=0.7.3 and I tried adding that path to the channel list but it was flagged as an invalid channel. How do I import Sympy 0.7.3?
I'm running on Windows and am doing all this because I want to run Symoro.
Any help is very much appreciated!
Anaconda recently dropped support for what was called their free channel and this has resulted in many older packages becoming inaccessible by default. Unfortunately, that older version of SymPy is in that part of the repository. You have two options:
Download the tarball and install from that.
Enable access to the free channel.
SymPy 0.7.3 is only available for Python 2.6, 2.7, or 3.3. If you want to use Python 2.7 then downloading should be the easiest route; for Python 3.3 (or 2.6 for some reason), I'd recommend the latter option, because almost all the other packages for these versions are also in the free channel.
In either case, you will use Anaconda Prompt (or another shell) and need to create a new environment, because it is generally a bad idea to force major changes into the Anaconda base environment.
Option 1: Installing from the Tarball
First, you need create the new Python 2.7 env. Let's call it my_env:
conda create --name my_env python=2.7
If there is other software you know you'll want, then you can also include that after the python=2.7.
Download the file sympy-0.7.3-py27_0.tar.bz2 (this is the win-64 version - others are where you pointed out already).
Install it:
conda install -n my_env sympy-0.7.3-py27_0.tar.bz2
Test your new env:
conda activate my_env
python -c "import sympy; sympy.doctest('polynomial')"
Option 2: (Temporarily) Enabling the free Channel
Just note that I wouldn't generally recommend keeping this enabled since it really tends to slow down the Conda solver (see "Why we removed the free channel in Conda 4.7). I'll recommend disabling it again (step 3), but keep in mind that if you ever need to make adjustments to your SymPy 0.7.3 env, you'll probably need to re-enable it.
Enable the free channel:
conda config --set restore_free_channel True
Create your new env:
conda create --name my_env python=3.3 sympy=0.7.3
Disable free channel
conda config --set restore_free_channel False
Test your SymPy env
conda activate my_env
python -c "import sympy; sympy.doctest('polynomial')"
Have a look at installing packages on anaconda
To install a specific version of a package such as SciPy:
conda install scipy=0.15.0
So try
conda install sympy=0.7.3
Seems to do the trick for me. It's in the default conda channel so you shouldn't need to change or add a channel

conda update -n base -c defaults conda won't update to 4.6.x

Hopefully the title says most of it. When I use conda, it informs me that there's a newer version; however, when I go to install conda, it refuses to install anything beyond 4.5.11. This problem is occurring on my laptop (running OS X Mojave). The Ubuntu virtual machine I'm running does not have this problem. It's been 9 days and no one has replied to my conda Github issue, so I was hoping that someone here might be able to help out. That link contains relevant configuration details that may offer some clues.
$ conda update -n base -c defaults conda
Solving environment: done
==> WARNING: A newer version of conda exists. <==
current version: 4.5.11
latest version: 4.6.4
Please update conda by running
$ conda update -n base -c defaults conda
# All requested packages already installed.
I encountered a similar problem, except that conda was reporting that I needed to update from 4.6.x to 4.7.x and that I needed to update by running:
conda update -n base -c defaults conda
None of the other answers (as of the time I'm typing this) did the job immediately:
My system only has Python 3, so a 2/3 issue wasn't the problem.
Checking on torch and torchvision, I found that they were not part of my Anaconda installation, so that was not the problem either.
Just trying to update Python didn't resolve the issue, as attempts to update conda still did not do anything.
On something of a whim, I ran
conda update anaconda
That did a a lot of updates (far more than should be listed here), which I then followed with:
conda update python
Again, there were many updates done; looking through the list, it was in this step that the conda update actually got done. Nonetheless, I still ran:
conda update conda
A few more updates were done at this point and these turned out to be the last ones. However, for good measure I ran:
conda update -n base -c defaults conda
To update conda to the most recent version you have to update python:
(base) self#home:~$ conda update python
This updates many packages including:
conda: 4.5.11-py35_0 --> 4.6.11-py36_0
This issue was due to a conflict with a PyTorch installation in base. I removed torch and torchvision from the base env and this fixed the dependency snag.
Loop in conda update problem, and finally I typed which python and found python2.7.
Maybe it's python version problem, you just need to update the python to python3.

Why does conda create try to install weird packages?

I am trying to install a new conda environment that will be totally separate from my other environments, so I run:
conda create --name foot35 python=3.5
Anaconda then asks for my approval to install these NEW packages:
asn1crypto: 0.22.0-py35he3634b9_1
ca-certificates: 2017.08.26-h94faf87_0
cachecontrol: 0.12.3-py35h3f82863_0
certifi: 2017.7.27.1-py35hbab57cd_0
cffi: 1.10.0-py35h4132a7f_1
chardet: 3.0.4-py35h177e1b7_1
colorama: 0.3.9-py35h32a752f_0
cryptography: 2.0.3-py35h67a4558_1
distlib: 0.2.5-py35h12c42d7_0
html5lib: 0.999999999-py35h79d4e7f_0
idna: 2.6-py35h8dcb9ae_1
lockfile: 0.12.2-py35h667c6d9_0
msgpack-python: 0.4.8-py35hdef45cb_0
openssl: 1.0.2l-vc14hcac20b0_2 [vc14]
packaging: 16.8-py35h5fb721f_1
pip: 9.0.1-py35h69293b5_3
progress: 1.3-py35ha84af61_0
pycparser: 2.18-py35h15a15da_1
pyopenssl: 17.2.0-py35hea705d1_0
pyparsing: 2.2.0-py35hcabcaab_1
pysocks: 1.6.7-py35hb30ac0d_1
python: 3.5.4-hedc2606_15
requests: 2.18.4-py35h54a615f_1
setuptools: 36.5.0-py35h21a22e4_0
six: 1.10.0-py35h06cf344_1
urllib3: 1.22-py35h8cc84eb_0
vc: 14-h2379b0c_1
vs2015_runtime: 14.0.25123-hd4c4e62_1
webencodings: 0.5.1-py35h5d527fb_1
wheel: 0.29.0-py35hdbcb6e6_1
win_inet_pton: 1.0.1-py35hbef1270_1
wincertstore: 0.2-py35hfebbdb8_0
I don't know why it suggests these specific ones. I looked up lockfile and its website says:
Note: This package is deprecated.
Here is a screenshot of my command prompt as additional information.
I am trying to do a clean install that is unrelated/independent to the root environment.
Why is conda trying to install these things and how do I fix it?
conda create will "Create a new conda environment from a list of specified packages." ( https://conda.io/docs/commands/conda-create.html )
What list??!? The .condarc file is the conda configuration file.
https://conda.io/docs/user-guide/configuration/use-condarc.html#overview
The .condarc file can change many parameters, including:
Where conda looks for packages.
If and how conda uses a proxy server.
Where conda lists known environments.
Whether to update the bash prompt with the current activated environment name.
Whether user-built packages should be uploaded to Anaconda.org.
**Default packages or features to include in new environments.**
Additionally, if you ever typed conda config, even accidentally...
The .condarc file is not included by default, but it is automatically created in your home directory the first time you run the conda config command.
A .condarc file may also be located in the root environment, in which case it overrides any in the home directory.
If you would like a single clean env then Boshika's recommendation of --no-default-packages flag for an instance though, you can check and modify the default packages for all further envs. ( https://conda.io/docs/user-guide/configuration/use-condarc.html#always-add-packages-by-default-create-default-packages )
Always add packages by default (create_default_packages)
When creating new environments, add the specified packages by default. The default packages are installed in every environment you create. You can override this option at the command prompt with the --no-default-packages flag. The default is to not include any packages.
EXAMPLE:
create_default_packages:
- pip
- ipython
- scipy=0.15.0
Lockfile may be there due to legacy requirements across all operating systems. Hopefully, you have the tools to remove it if you choose.
To avoid conda from installing all default packages you can try this
conda create --name foot35 --no-deps python=3.5
please don't loose the hope it's very weird for me also.
What you have to do just follow the steps: -
1.Download the anaconda for you system from it's official site and Install it : https://repo.continuum.io
After the Installation process, you can select your own package from there and please don't need to download anything from anywhere, it's full of packages over the internet.
3.If you want to work on python download Syder IDE its very useful for the Machine learning library.
Don't create other environment instead of root by defaults otherwise you have to duplicate all the file again, if there is any error while installing in root so close the window and again run as administration and after that its works fine.
Cause all the file in your root environment so you don't worry about the path in future and you can install and uninstall the packages : like - numpy , pandas, tensorflow and its gpu , scikit-learn etc from there eaisly.
Thank you
These packages are generally useful if you wish to pip install ... anything. Without many of them doing a pip install requests could result in errors such as these (and more)
No Module named Setuptools
pip: command not found
pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available
The issue that the conda create ... exposes is that the packages it wants to pull down are variable (based on when you create the environment). If you wish to maintain the same environment for you and for those who may collaborate with you, then freezing or pinning conda create's default installed package may be necessary.
One way of doing this is creating your environment with conda env create using a conda environment YAML file such as this example:
dependencies:
- ca-certificates=2018.03.07
- certifi=2018.4.16
- libedit=3.1.20170329
- libffi=3.2.1
- ncurses=6.1
- openssl=1.0.2o
- pip=10.0.1
- python=3.6.6
- readline=7.0
- setuptools=40.0.0
- sqlite=3.24.0
- tk=8.6.7
- wheel=0.31.1
- xz=5.2.4
- zlib=1.2.11
conda env create -n <NAME_OF_ENVIRONMENT> -f <PATH_TO_CONDA_REQUIREMENTS_FILE>
(note it's conda env create not conda create)

anaconda update all possible packages?

I tried the conda search --outdated, there are lots of outdated packages, for example the scipy is 0.17.1 but the latest is 0.18.0. However, when I do the conda update --all. It will not update any packages.
update 1
conda update --all --alt-hint
Fetching package metadata .......
Solving package specifications: ..........
# All requested packages already installed.
# packages in environment at /home/user/opt/anaconda2:
#
update 2
I can update those packages separately. I can do conda update scipy. But why I cannot update all of them in one go?
TL;DR: dependency conflicts: Updating one requires (by it's requirements) to downgrade another
You are right:
conda update --all
is actually the way to go1. Conda always tries to upgrade the packages to the newest version in the series (say Python 2.x or 3.x).
Dependency conflicts
But it is possible that there are dependency conflicts (which prevent a further upgrade). Conda usually warns very explicitly if they occur.
e.g. X requires Y <5.0, so Y will never be >= 5.0
That's why you 'cannot' upgrade them all.
Resolving
Update 1: since a while, mamba has proven to be an extremely powerful drop-in replacement for conda in terms of dependency resolution and (IMH experience) finds solutions to problems where conda fails. A way to invoke it without installing mamba is via the --solver=libmamba flag (requires conda-libmamba-solver), as pointed out by matteo in the comments.
To add: maybe it could work but a newer version of X working with Y > 5.0 is not available in conda. It is possible to install with pip, since more packages are available in pip. But be aware that pip also installs packages if dependency conflicts exist and that it usually breaks your conda environment in the sense that you cannot reliably install with conda anymore. If you do that, do it as a last resort and after all packages have been installed with conda. It's rather a hack.
A safe way you can try is to add conda-forge as a channel when upgrading (add -c conda-forge as a flag) or any other channel you find that contains your package if you really need this new version. This way conda does also search in this places for available packages.
Considering your update: You can upgrade them each separately, but doing so will not only include an upgrade but also a downgrade of another package as well. Say, to add to the example above:
X > 2.0 requires Y < 5.0, X < 2.0 requires Y > 5.0
So upgrading Y > 5.0 implies downgrading X to < 2.0 and vice versa.
(this is a pedagogical example, of course, but it's the same in reality, usually just with more complicated dependencies and sub-dependencies)
So you still cannot upgrade them all by doing the upgrades separately; the dependencies are just not satisfiable so earlier or later, an upgrade will downgrade an already upgraded package again. Or break the compatibility of the packages (which you usually don't want!), which is only possible by explicitly invoking an ignore-dependencies and force-command. But that is only to hack your way around issues, definitely not the normal-user case!
1 If you actually want to update the packages of your installation, which you usually don't. The command run in the base environment will update the packages in this, but usually you should work with virtual environments (conda create -n myenv and then conda activate myenv). Executing conda update --all inside such an environment will update the packages inside this environment. However, since the base environment is also an environment, the answer applies to both cases in the same way.
To answer more precisely to the question:
conda (which is conda for miniconda as for Anaconda) updates all but ONLY within a specific version of a package -> major and minor. That's the paradigm.
In the documentation you will find "NOTE: Conda updates to the highest version in its series, so Python 2.7 updates to the highest available in the 2.x series and 3.6 updates to the highest available in the 3.x series."
doc
If Wang does not gives a reproducible example, one can only assist.
e.g. is it really the virtual environment he wants to update or could Wang get what he/she wants with
conda update -n ENVIRONMENT --all
*PLEASE read the docs before executing "update --all"!
This does not lead to an update of all packages by nature. Because conda tries to resolve the relationship of dependencies between all packages in your environment, this can lead to DOWNGRADED packages without warnings.
If you only want to update almost all, you can create a pin file
echo "conda ==4.0.0" >> ~/miniconda3/envs/py35/conda-meta/pinned
echo "numpy 1.7.*" >> ~/miniconda3/envs/py35/conda-meta/pinned
before running the update. conda issues not pinned
If later on you want to ignore the file in your env for an update, you can do:
conda update --all --no-pin
You should not do update --all. If you need it nevertheless you are saver to test this in a cloned environment.
First step should always be to backup your current specification:
conda list -n py35 --explicit
(but even so there is not always a link to the source available - like for jupyterlab extensions)
Next you can clone and update:
conda create -n py356 --clone py35
conda activate py356
conda config --set pip_interop_enabled True # for conda>=4.6
conda update --all
conda config
update:
Currently I would use mamba (or micromamba) as conda pkg-manager replacement
update:
Because the idea of conda is nice but it is not working out very well for complex environments I personally prefer the combination of nix-shell (or lorri) and poetry [as superior pip/conda .-)] (intro poetry2nix).
Alternatively you can use nix and mach-nix (where you only need you requirements file. It resolves and builds environments best.
On Linux / macOS you could use nix like
nix-env -iA nixpkgs.python37
to enter an environment that has e.g. in this case Python3.7 (for sure you can change the version)
or as a very good Python (advanced) environment you can use mach-nix (with nix) like
mach-nix env ./env -r requirements.txt
(which even supports conda [but currently in beta])
or via api like
nix-shell -p nixFlakes --run "nix run github:davhau/mach-nix#with.ipython.pandas.seaborn.bokeh.scikit-learn "
Finally if you really need to work with packages that are not compatible due to its dependencies, it is possible with technologies like NixOS/nix-pkgs.
Imagine the dependency graph of packages, when the number of packages grows large, the chance of encountering a conflict when upgrading/adding packages is much higher. To avoid this, simply create a new environment in Anaconda.
Be frugal, install only what you need. For me, I installed the following packages in my new environment:
pandas
scikit-learn
matplotlib
notebook
keras
And I have 84 packages in total.
I agree with Mayou36.
For example, I was doing the mistake to install new packages in the base environment using conda for some packages and pip for some other packages.
Why this is bad?
1.None of this is going to help with updating packages that have been > installed >from PyPI via pip, or any packages installed using python
setup.py install. conda list will give you some hints about the
pip-based Python packages you have in an environment, but it won't do
anything special to update them.
And I had all my projects in the same one environment! And I used update all -which is bad and did not update all-.
So, the best thing to do is to create a new environment for each project. Why?
2. A Conda environment is a directory that contains a specific collection of Conda packages that you have installed. For example, you
may be working on a research project that requires NumPy 1.18 and its
dependencies, while another environment associated with an finished
project has NumPy 1.12 (perhaps because version 1.12 was the most
current version of NumPy at the time the project finished). If you
change one environment, your other environments are not affected. You
can easily activate or deactivate environments, which is how you
switch between them.
So, to wrap it up:
Create a new environment for each project
Be aware for the differences in conda and pip
3.Only include the packages that you will actually need and update them properly only if necessary.
if working in MS windows, you can use Anaconda navigator. click on the environment, in the drop-down box, it's "installed" by default. You can select "updatable" and start from there
To update all possible packages I used conda update --update-all
It works!
I solved this problem with conda and pip.
Firstly, I run:
conda uninstall qt and conda uninstall matplotlib and conda uninstall PyQt5
After that, I opened the cmd and run this code that
pip uninstall qt , pip uninstall matplotlib , pip uninstall PyQt5
Lastly, You should install matplotlib in pip by this code that pip install matplotlib

Categories

Resources