I recently ran into a problem with a Conda environments and I believe I solved it using this. Is there a way to check if what I did actually solved the problem. Can I check if there are still any inconsistencies?
This was meant to be a general question and I did not think it would be dependent on the actual issue I had. It all started when I was having issues with Keras and TensorFlow for GPU in a Conda environment. I came to the conclusion that something was corrupt (I couldn't even run basic Keras commands) and decided to try to update everything and if that didn't fix my issue I would reinstall the packages I needed in that environment. As soon as I tried to update TensorFlow for GPU, the first thing that appeared was:
The environment is inconsistent, please check the package plan carefully The following packages are causing the inconsistency:
Followed by a bunch of packages and version numbers. As I said before I believe I solved the inconsistency issues by force updating Conda but am uncertain how to check if the environment is now consistent.
Related
Whenever I run conda install/remove/update <package>, it tells me it's "Solving environment" for some time before telling me the list of things it's going to download/install/update. Presumably it's looking for dependencies for <package>, but why does it sometimes remove packages after doing this operation? For example, as I was trying to install Mayavi, it decided it needed to remove Anaconda Navigator.
Furthermore it does not provide an option to perform only a subset of the suggested operations. Is there a way to specify that I don't want a package removed?
You can add --debug option to the conda command and see the output from console(or terminal). For example, type conda update --debug numpy.
From the output, we can see that the client requests repodata.json from channel list and do some computation locally in the Solving Environment Step.
As a side note on the "Solving Environment" step...
Lack of administrator privileges may affect whether or where you can install python packages.
I observed that my installs would hang on the "Solving Environment" step and never get through when attempting to install packages while logged in as a non-administrator.
Getting switched to admin was possible for me on the machine I was stuck on, so I just did that and it solved the problem.
Commenter explains workaround when this is not possible.
JUST WAIT! I wasted hours trying to fix this. It turns out, it just took around 45 minutes :/
The short answer is: use mamba as a drop-in replacement for conda, it's much much faster at solving environments, no more waiting for minutes. mamba has been officially endorsed by the conda team.
Mamba also allows you to configure more precisely which packages you require to be installed and allows you to pin versions, as conda does. For a more detailed comparison of conda and mamba see this Stackoverflow answer: https://stackoverflow.com/a/68043228/7483211
The long answer is: Solving conda environments with more than a few packages that each have dependencies on their own quickly ends up becoming a quite complicated SAT problem (see Boolean satisfiability problem and dependency hell)
With good algorithms, even fairly big SAT problems can be solved fast. In contrast to mamba's solver which is written in C++ and designed to be fast, it seems that conda's solver is not very high performance. It worked well enough when people used small environments in the past, but with bigger and bigger environments, conda has started to struggle.
I made the switch about a year ago and I have not once looked back. The open source project I'm working for (Nextstrain) has also started to recommend mamba in place of conda for new users. I have not seen anyone advocating against using mamba in place of conda.
conda install --prune <<package>> helped me to install the right channel.
Suspecting environment used are for zipline and channel used not compatible with existing one. prune takes a lot of time but helped me in solving the environment issues.
i'm actally facing a probleme since last friday and didn't find a solution for the moment.
First of all, you need to know that i'm a beginner on linux,i'm trying to do some deep learning in my internship, and i discovered that even if my company have a 1080 Ti, keras wasn't using it, so have the job to correct this.
I am trying to use Keras with GPU. I installer Tensorflow by following these steps : https://www.tensorflow.org/install/install_linux
I also installer CUDA,cuDNN.
I found on my machine an older installation of CUDA (version 7.5). I installed version 9.2 without uninstalling version 7.5. I added the PATH variables but it seems like it is not taking in account : [][https://i.stack.imgur.com/B3Pqm.png]
I tried to uninstall CUDA version 7.5 but i don't know how to do it, since in the usr/local folder, there is no cuda-7.5 folder.
When i enter nvidia-smi in the prompt, it works correctly. I installed tensorflow and tensorflow-gpu, but i does not work : [][https://i.stack.imgur.com/78gPd.png]
Did anyone know how to help me? i Guess the solution of my probleme is not really complicated for someone who knows Ubuntu, and i feel like i'm loosing a lot of times doing something i don't really understand.
If someone need some further informations in order to help me, feel free to ask.
Thank you
Uninstall tensorflow and install only tensorflow-gpu. You should not install both. If you are using keras, then install keras-gpu.
Let's say you are working with conda and you want to tidy up all this. Do
conda remove keras
conda remove tensorflow*
conda install keras-gpu
If you are not, then i highly recommend Anaconda for dealing with these issues which you seem to be having stress-free.
I'm trying to update/upgrade my Python with conda. I run the command:
conda update conda
but it crashes my Python:
Python has stopped working
A problem caused the program to stop working correctly. Windows will close the program and notify you if a solution is available.
My installation information is:
conda version : 4.5.2
conda-build version : 3.0.27
python version : 3.6.3.final.0
I googled the issue and got this github issue report. Unfortunately none of the solutions worked for me. I would appreciate if you could help me know what is the problem and how I can solve it.
I tried uninstalling and reinstalling conda and the problem disappeared temporarily. But then the issue happened again while I was trying to use matplotlib.
I experienced the exact same issue one a different machine when running conda update --all so either there is something wrong with conda or I'm making one mistake twice! (conda/python versions were exactly the same as above.)
I tried updating/upgrading all of the packages running the command in this post on admin-cmd. It took a long time to finish and the problem was still there.
There seemed to be issues with Qt. So I tried to uninstall and reinstall Qt and pyqt but it did not solve the problem either.
temporary workaround: I completely uninstalled Anaconda and installed stock Python. So far this has solved the problem both on my home and work computers.
Try conda update --all or if you're trying to update / upgrade specifically python then try conda update python. Faced such problem and it worked for me.
As was suggested by the mods, here I post my temporary solution. Considering that none of the other attempts (reinstalling Qt, Anaconda, updating with pip...) worked, I ended up uninstalling Anaconda and everything pythony and installed the stock / vanilla python from the official website. This has solved the issue so far. Please consider this is not a solution to the problem mentioned above but just to get the python working for the time being.
In tensorflow installation guide it is said, that I should use "environment" to install tensorflow: https://www.tensorflow.org/install/install_windows#installing_with_anaconda
Why? Can't I just install with pip?
If installed with environment, should I "activate" it each time I use tensorflow?
If I use tensorflow from within other thing like keras and/or PyCharm, then how can I activate environment?
The question is about Windows. I assume you installed python using anaconda. Then you have a default environment, called root. You can create as many environments as you want, think of each as a separate installation of python. Using conda or pip installs stuff at your current installation. Conda stuff is kind of pre-compiled to work with your machine/anaconda environment, while pip stuff is usually compiled on the spot. I assume compiling tensorflow might not be completely trivial...
'Activate' changes from one environment to the other, so unless you have multiple environments you shouldn't need it. You run all these on command prompt.
Bottom line is, unless you have multiple environments (I highly recommend it so you can try different things) I cannot see you using activate. Install tensorflow and keras on the same one and only root environment you have. You should be able to access both (it is also possible just installing keras would install tensorflow, if its a dependancy)
If you see no prompt, it is the default, root environment. You can see all your environments with: conda info --envs But unless you create some environment (using e.g. conda create --name py Python=2) you probably only have root. One of the nice things with environments is you can have one with Python=2 (latest python 2), one with Python=3, another with Python=2.7 etc
On your follow-up, If you have multiple environments, you can switch between them on Pycharm by changing the interpreter. On the image you see me selecting e.g. py2_olv
Professional answer:
Quote from https://machinelearningspace.com/installing-tensorflow-2-0-in-anaconda-environment/:
What is Anaconda and why I recommend it?
...
[dropped intro to Anaconda]
...
For a Python developer or a data science researcher, using Anaconda
has a lot of advantages, such as independently installing/updating
packages without ruining the system. So, we no need to worry about the
system library or anything like that. This can save time and energy
for other things.
Anaconda can be used across different platforms, Windows, macOS, and
Linux. If we want to use a different Python version or package
libraries, just create a different environment and play around without
any risk of crashing the system library.
####
Unprofessional research:
Now in addition my own research. I am not a professional, I have little knowledge of the seemingly chaotic world of different install methods. This refers to some first research at https://superuser.com/questions/1572640/do-i-need-to-install-cuda-separately-after-installing-the-nvidia-display-driver/1572762#1572762. Mind that I am guessing a lot here. Please comment if I am wrong.
We see that at the moment, Pytorch supports version 10.2, Tensorflow supports 10.1, and it is not just the version that differs: mind that "CUDA Toolkit" (standalone) and cudatoolkit (conda binary install) are different! One is a a standalone / executable install, the other is a binary install. And tensorflow needs tensorflow-gpu to reach the standalone cuda install.
Therefore you should consider a separate environment for both Tensorflow and Pytorch, since any update of the conda cudatoolkit to version 11.0 could harm the dependency condition of Pytorch (Though this is not completely right. Pytorch uses a cuda that is installed inside Pytorch. It is still the approach to understand the recommended different envs). For tensorflow, you have to install version CUDA Toolkit 10.1 although 11.0 is already available, so that your whole card must run on a lower version than possible only to support Tensorflow - even if some games would like to have version 11.0.
Unprofessional answer:
If all of the dependencies are so important and so easily wrong when updated separately, like you could do with pip, any install that you do by yourself using pip might crash your sensitive tensorflow install. Therefore it is recommended to keep to a full service approach which Anaconda offers, where all dependencies are kept right, even if you enter conda install --all. That is why you better search for an Anaconda guide, for example https://machinelearningspace.com/installing-tensorflow-2-0-in-anaconda-environment/.
If you would have read through the entire document, it would have stated that the Anaconda installation is community supported, not officially supported. They want you to install TensorFlow using native pip through Python 3.5.x. That being said, from personal experience, I will tell you that if you are looking to run basic level TensorFlow Python scripts, such as training and testing an MNIST model, a Windows installation will be fine, or using a model that has already been trained for some purpose will also be fine. However, if you want to train advanced models such as Inception, which are the state-of-the-art image classifiers with less than 5% error for normal images, Windows is not suitable. You should try using Linux installation for any training purposes. I would recommend using VirtualBox, having used it in the past.
As for activating the environment, as long as, in any script / in the bash, you include the line "import tensorflow as tf", you should be fine, at least for native pip installation.
Good luck!
I have a virtualenv running python 2.7.7. It has a pretty extensive set of libraries which support a pretty complicated set of proprietary modules. In other words, the virtualenv needs to maintain its integrity. That is of course the whole point of virtualenv.
Recently, I encountered a number of problems that are very easily solved by using Anaconda. I tried it out in a test environment and it worked quite well. Now I'm tasked with incorporating this new configuration into production. It isn't clear to me how to incorporate Anaconda into a virtualenv, or whether this is even a good idea. In fact, it almost seems to me like I should use the anaconda install as the new source and desconstruct the old virtualenv... merging the libraries it held into the conda.
Does anyone have a recommendation as to the best approach? If merging the environments is called for, can anyone point to an explanation of how to go about it?
It doesn't really make sense to merge Anaconda and a virtualenv, as Anaconda is a completely independent installation of Python. You can do it, typically by setting your PYTHONPATH, but things have a good chance of breaking when you do this sort of thing, and I would recommend against it.
If there are libraries in your virtualenv, you can use them with Anaconda by making conda packages for them. They may already have conda packages (search with conda search and search https://binstar.org/). Otherwise, you can build a package using a conda recipe. See http://conda.pydata.org/docs/build.html and https://github.com/conda/conda-recipes for some example recipes.