Develop Reference

how to install the arch package with anaconda?

I am trying to install the arch package https://pypi.org/project/arch/ using Anaconda.
The suggested install runs fine
(base) C:\Users\john>conda install arch-py -c conda-forge
Collecting package metadata (current_repodata.json): done
Solving environment: done
## Package Plan ##
environment location: C:\Users\john\anaconda3
added / updated specs:
- arch-py
The following NEW packages will be INSTALLED:
arch-py conda-forge/win-64::arch-py-4.18-py38h294d835_0
cython conda-forge/win-64::cython-0.29.22-py38h885f38d_0
icc_rt pkgs/main/win-64::icc_rt-2019.0.0-h0cc432a_1
patsy conda-forge/noarch::patsy-0.5.1-py_0
property-cached conda-forge/noarch::property-cached-1.6.4-py_0
scipy pkgs/main/win-64::scipy-1.6.1-py38h14eb087_0
statsmodels conda-forge/win-64::statsmodels-0.12.2-py38h347fdf6_0
The following packages will be UPDATED:
certifi pkgs/main::certifi-2020.12.5-py38haa9~ --> conda-forge::certifi-2020.12.5-py38haa244fe_1
The following packages will be SUPERSEDED by a higher-priority channel:
ca-certificates pkgs/main::ca-certificates-2021.1.19-~ --> conda-forge::ca-certificates-2020.12.5-h5b45459_0
conda pkgs/main::conda-4.9.2-py38haa95532_0 --> conda-forge::conda-4.9.2-py38haa244fe_0
openssl pkgs/main::openssl-1.1.1j-h2bbff1b_0 --> conda-forge::openssl-1.1.1j-h8ffe710_0
Proceed ([y]/n)? y
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
(base) C:\Users\john>spyder
Unfortunately, I cannot import the package correctly when I start Spyder.
from arch import arch_model
ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 88 from C header, got 80 from PyObject
What should I do?
Thanks!

The package requires the most recent version of numpy. I tried to remove numpy and reinstall version 1.20.0 (the version needed) without success. Anaconda would stick to 1.19
Ultimately, I did what I should have done a long time ago. Download miniconda (not anaconda) and install only the packages I need. That way, no annoying conflicts when updating packages with conda!

Get UnsatisfiableError when Installing OpenCV for Python through Anaconda on Windows

I've tried installing OpenCV on Windows through Anaconda Navigator and Anaconda Prompt, but I get the same error:
UnsatisfiableError: The following specifications were found to be incompatible with each other:
Package hdf5 conflicts for:
pytables -> hdf5[version='>=1.10.1,<1.10.2.0a0,>=1.10.2,<1.10.3.0a0,>=1.10.4,<1.10.5.0a0,>=1.8.18,<1.8.19.0a0']
hdf5
opencv -> hdf5[version='>=1.10.2,<1.10.3.0a0,>=1.8.18,<1.8.19.0a0,>=1.8.20,<1.9.0a0']
anaconda==2019.03=py37_0 -> hdf5==1.10.4=h7ebc959_0
h5py -> hdf5[version='>=1.10.1,<1.10.2.0a0,>=1.10.2,<1.10.3.0a0,>=1.8.18,<1.8.19.0a0,>=1.8.20,<1.9.0a0']
Package mkl-service conflicts for:
mkl-service
Package zipp conflicts for:
importlib_metadata -> zipp[version='>=0.3.2,>=0.5']
zipp
anaconda==2019.03=py37_0 -> importlib_metadata==0.8=py37_0 -> zipp[version='>=0.3.2']
Package importlib_metadata conflicts for:
anaconda==2019.03=py37_0 -> importlib_metadata==0.8=py37_0

I was even getting conflicts while trying to install opencv via conda.
Therefore I tried to use the below pip command here to install opencv.
pip install opencv-python
You will see opencv starts getting installed on your system, the size is approx 37MB
To check the version of opencv and other installed packages
Use the command:
conda list
It will list all installed packages with their version number
To use opencv in python script,
use the below import:
import cv2
print(cv2.__version__) #prints the version of opencv installed

Anaconda 2019.07 , same problems , i figure it out ,that may help you.
try using
conda update --all
update your packages in Conda, then there will be some notifications, which means some packages need to be installed to satisfy the requests. just input 'y' to comfirm.
y
then opencv should be started installing
after that use import cv2 cv2.__version__can print opencv's version '3.4.1'

Removing hdf5 is not a good idea since it will also remove the jupyter packages:
> conda uninstall hdf5
Collecting package metadata (repodata.json): done
Solving environment: done
[..]
removed specs:
- hdf5
[..]
The following packages will be REMOVED:
alabaster-0.7.12-py37_0
anaconda-2019.07-py37_0
anaconda-project-0.8.3-py_0
[..]
jupyter-1.0.0-py37_7
jupyter_console-6.0.0-py37_0
[..and a lot more]
It will remove 155 packages in total, among them the jupyter packages and you will then not be able to run Jupyter Notebooks, getting errors like
ModuleNotFoundError Traceback (most recent call last)
<ipython-input-1-c65cee9c4793> in <module>
1 get_ipython().run_line_magic('clear', 'all')
----> 2 get_ipython().run_line_magic('matplotlib', 'inline')
3
4 import numpy as np
5 import cv2
I am currently also looking into the problem with opencv and Anaconda 2019.07; one alternative is to install Anaconda 2019.03 where opencv will only clash with a few other (unimportant) packages.

I was also encountered with the same problem like you. And I finally addressed it. You can firstly remove this package. Then install the OpenCV. During the course of installing the openCV, anaconda will help you install other realted packages again. Therefore, you need not to worry about lossing packages. And how to remove this package conflicted with opencv?
You are recommanded to input "conda remove hdf5". Hope to help u!

No module named 'absl' error when I import tensorflow

I have anaconda installed on my Win 7 machine with a GTX1070.
I have been trying to install tensorflow gpu version for the past hour with no luck. I followed the tutorials on the tensorflow website to no success. They ask for you to install CUDA 8.0 and Cudll 6.0 which I do. I then add the location of cudnn64_6.dll, cudart64_80.dll and the libnvvp folder to my path variables. I then set up a virtual environment in conda and run the following command:
(tensorflow)C:> pip install --ignore-installed --upgrade tensorflow-gpu
I believe when I do this, it installs tensorflow 1.5 which seems to require CUDA tools 9.0 because I get an error saying that cudart64_80.dll cannot be found. So I update my CUDA tools to 9.0 and download the corresponding Cuda library. I then change my path variables. Now I dont get any missing dll errors however, I am now faced with the following error
ModuleNotFoundError: No module named 'absl'
Does anyone have any suggestions as to what I can do from here on?

This was caused by a Python version issue for me. I had the absl package installed on my Python 2.x, but my Python 3.x didn't have it. So I just made sure that both Pythons on my machine had the package installed:
pip install absl-py
pip3 install absl-py

Here is how I solved it in my end.
I was getting the error even though absl-py was already installed.
When I used pip install absl-py I got the following error:
Requirement already satisfied: absl-py in c:\users\stack\appdata\local\programs\python\python38\lib\site-packages (0.9.0)
Requirement already satisfied: six in c:\users\stack\appdata\local\programs\python\python38\lib\site-packages (from absl-py) (1.15.0)
I fixed it by reinstalling, i.e. by using:
pip uninstall absl-py
pip install absl-py
Then I faced a similar error with wrapt:
ModuleNotFoundError: No module named 'wrapt'
Fixed wrapt as well by uninstalling and then installing it again.
pip uninstall wrapt
pip install wrapt

Try:
conda install tensorflow
conda install -c anaconda absl-py
This worked for me.
For more information see: https://anaconda.org/anaconda/absl-py

I was facing the same issues and pip (or pip3) install absl-py was not working.
python --version showed I was using the wrong python version (2.7). So I changed the python version to 3.5 (https://askubuntu.com/questions/320996/how-to-make-python-program-command-execute-python-3)
This worked for me.

You can use the Anaconda GUI to install absl-py pack to solve the problem:
Select the uninstall columnand search absl-py to apply it.

Try another version of absl-py. My default absl version is 0.8.1 and python is 3.6:
$pip install absl-py==0.8.0
works for me.
Or change my environment python version to 3.7 also works.

This is quite late but still worth posting. What they don't tell you on the NVidia website is that there is one more path you need to add to your environment variables. The path is;
c:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\extras\CUPTI\libx64
It may not be exactly the same on your installation as it depends on where you installed your CUDA tools.
You could also just use pip to install TF GPU version but pip installs TF 1.1 which is fairly old.

For me, "pip3 install absl-py" don't work:
I received: "ModuleNotFoundError: No module named 'absl' ", when I was trying import tensorflow:
import tensorflow
Traceback (most recent call last):
File "", line 1, in
File "/opt/anaconda3/lib/python3.6/site-packages/tensorflow/init.py", line 22, in
from tensorflow.python import pywrap_tensorflow # pylint: disable=unused-import
File "/opt/anaconda3/lib/python3.6/site-packages/tensorflow/python/init.py", line 63, in
from tensorflow.python.framework.framework_lib import * # pylint: disable=redefined-builtin
File "/opt/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/framework_lib.py", line 25, in
from tensorflow.python.framework.ops import Graph
File "/opt/anaconda3/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 55, in
from tensorflow.python.platform import app
File "/opt/anaconda3/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 24, in
from tensorflow.python.platform import flags
File "/opt/anaconda3/lib/python3.6/site-packages/tensorflow/python/platform/flags.py", line 25, in
from absl.flags import * # pylint: disable=wildcard-import
ModuleNotFoundError: No module named 'absl'
But if I compile absl-py from scratch, all goes right:
jk#D17:~/py/004/abseil-py-master$ python setup.py install
running install
running bdist_egg
running egg_info
.......
Using /opt/anaconda3/lib/python3.6/site-packages
Finished processing dependencies for absl-py==0.4.0
and then:
import tensorflow
is OK

Find the absl-py related folder in your site-packages folder and delete it. Try reinstalling pip3 install absl-py. I solved this way, I hope to be useful to you.

If you are using Anaconda make sure you install jupyter notebook from Home. if not you can import it from python CLI but it will give "No module named 'absl' error when importing TensorFlow"

In hopes that this might help someone in the future. I had similar issues installing/using tensorflow and by extension the absl python package. There were some issues relating to a pure pip installation so I had to find an alternate route.
This is the process I followed.
Download and install miniconda:
wget https://repo.continuum.io/miniconda/Miniconda3-3.7.0-Linux-x86_64.sh -O ~/miniconda.sh
bash ~/miniconda.sh -b -p $HOME/miniconda
add conda to PATH (Optional):
export PATH="$HOME/miniconda/bin:$PATH"
activate the conda environment:
source $HOME/miniconda/bin/activate
At this point you should have (base) as your sourced condo environment. From this environment perform the following:
conda create -n tensorflow python=3.7
activate tensorflow
Just to note, at this point you should be working in the (tensorflow) environment. It would have replaced the base environment. It is also the env where you would run your tensorflow related job (that failed and brought you here).
pip install --ignore-installed --upgrade tensorflow
pip install absl-py
At this point you might need to install additional dependencies such as numpy.
My goal was to be able to run tensorflow tools and this worked for my ends. Hope it may be useful to someone else in the future.

I got the No module named absl error when I tried to install tf-nightly 2.9.0.dev20220203 on my Ubuntu 20.04.3 machine.
Reinstalling absl-py did not directly worked.
While reinstalling absl, I got this error: ERROR: tf-nightly 2.9.0.dev20220203 has requirement gast<=0.4.0,>=0.2.1, but you'll have gast 0.5.3 which is incompatible.
So I uninstalled absl-py:
pip uninstall absl-py
Then downgraded gast:
pip uninstall gast
pip install gast==0.4.0
Finally,
pip install absl-py

Anaconda installation not finding packages

This one is best shown through what I'm trying to do and what is going on.
In my python file, I want to import seaborn.
It's available to install via conda.
(me_dev)[me#ip-***]$ conda install seaborn
Fetching package metadata: ....
Solving package specifications: ............................
# All requested packages already installed.
# packages in environment at /home/me/miniconda/envs/me_dev:
#
seaborn 0.7.0 py27_0
(me_dev)[me#ip-***** ****]$ which python
~/miniconda/bin/python
(me_dev)[me#ip-****]$ which ipython
~/miniconda/bin/ipython
Now if I start iPython and import seaborn:
In [1]: import seaborn
---------------------------------------------------------------------------
ImportError Traceback (most recent call last)
<ipython-input-1-085c0287ecb5> in <module>()
----> 1 import seaborn
ImportError: No module named seaborn
Also, pip is not working either. It's not reading packages from pip.
(me_dev)[me#ip-*****]$ pip install jellyfish
Requirement already satisfied (use --upgrade to upgrade): jellyfish in /home/me/miniconda/lib/python2.7/site-packages
Yet:
from jellyfish import jaro_winkler ImportError: No module named jellyfish
Edit:
sys.path looks like so in iPython
['',
'/home/me/miniconda/bin',
'/home/me/miniconda/lib/python27.zip',
'/home/me/miniconda/lib/python2.7',
'/home/me/miniconda/lib/python2.7/plat-linux2',
'/home/me/miniconda/lib/python2.7/lib-tk',
'/home/me/miniconda/lib/python2.7/lib-old',
'/home/me/miniconda/lib/python2.7/lib-dynload',
'/home/me/miniconda/lib/python2.7/site-packages/setuptools-19.6.2-py2.7.egg',
'/home/me/miniconda/lib/python2.7/site-packages',
'/home/me/miniconda/lib/python2.7/site-packages/cryptography-1.0.2-py2.7-linux-x86_64.egg',
'/home/me/miniconda/lib/python2.7/site-packages/IPython/extensions',
'/home/me/.ipython']
Here's my .bashsrc
(me_dev)[me#ip-**** ~]$ cat .bashrc
# .bashrc
# Source global definitions
if [ -f /etc/bashrc ]; then
. /etc/bashrc
fi
# added by Miniconda 3.16.0 installer
export PATH="/home/me/miniconda/bin:$PATH"
# User specific aliases and functions
source activate me_dev
# added by Miniconda2 3.19.0 installer
export PATH="/home/me/miniconda/bin:$PATH"

I think which python (and which ipython) should be pointing at ~/miniconda/envs/me_dev/bin/, not at ~/miniconda/bin/ directory. When you start an python session with those binaries, they can't see your seaborn library, which is probably only installed in the environment site packages.
I'm not sure exactly how you ended up in this state, but I would recommend trying to make a new environment and see if you end up pointed at the correct binaries.
The organization your .bashrc is the problem. You're activating your virtual environment, but then giving the main miniconda bin directory precedence in your $PATH, which has the effect of partially inactivating the virtual environment.

Build wheel for a package (like scipy) lacking dependency declaration

I think it doesn't make a difference here but I'm using Python 2.7.
So the general part of my question is the following: I use a separate virtualenv for each of my projects. I don't have administrator access and I don't want to mess with system-installed packages anyway. Naturally, I want to use wheels to speed up package upgrades and installations across the virtualenvs. How can I build a wheel whose dependencies are only met within a specific virtualenv?
Specifically, issuing
pip wheel -w $WHEELHOUSE scipy
fails with
Building wheels for collected packages: scipy
Running setup.py bdist_wheel for scipy
Destination directory: /home/moritz/.pip/wheelhouse
Complete output from command /home/moritz/.virtualenvs/base/bin/python -c "import setuptools;__file__='/home/moritz/.virtualenvs/base/build/scipy/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" bdist_wheel -d /home/moritz/.pip/wheelhouse:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/home/moritz/.virtualenvs/base/build/scipy/setup.py", line 237, in <module>
setup_package()
File "/home/moritz/.virtualenvs/base/build/scipy/setup.py", line 225, in setup_package
from numpy.distutils.core import setup
ImportError: No module named numpy.distutils.core
----------------------------------------
Failed building wheel for scipy
Failed to build scipy
Cleaning up...
because numpy is not globally present and while building the wheel works when a virtualenv with numpy installed is active, it seems like a terrible idea to have the wheel depend on a specific virtualenv's version of numpy.
pandas which also depends on numpy appears to install its own components of numpy but I'm not sure that's the best solution.
I could install numpy with --user and use that to build the scipy wheel. Are there better options?

Problem description
Have a python package (like scipy), which is dependent on other packages (like numpy) but setup.py is not declaring that requirement/dependency.
Building a wheel for such a package will succeed in case, current environment provides the package(s) which are needed.
In case, required packages are not available, building a wheel will fail.
Note: Ideal solution is to correct the broken setup.py by adding there required package declaration. But this is mostly not feasible and we have to go another way around.
Solution: Install required packages first
The procedure (for installing scipy which requires numpy) has two steps
build the wheels
use the wheels to install the package you need
Populate wheelhouse with wheels you need
This has to be done only once and can be then reused many times.
have properly configured pip configuration so that installation from wheels is allowed, wheelhouse directory is set up and overlaps with download-cache and find-links as in following example of pip.conf:
[global]
download-cache = /home/javl/.pip/cache
find-links = /home/javl/.pip/packages
[install]
use-wheel = yes
[wheel]
wheel-dir = /home/javl/.pip/packages
install all required system libraries for all the packages, which have to be compiled
build a wheel for required package (numpy)
$ pip wheel numpy
set up virtualenv (needed only once), activate it and install there numpy:
$ pip install numpy
As a wheel is ready, it shall be quick.
build a wheel for scipy (still being in the virtualenv)
$ pip wheel scipy
By now, you will have your wheelhouse populated with wheels you need.
You can remove the temporary virtualenv, it is not needed any more.
Installing into fresh virtualenv
I am assuming, you have created fresh virtualenv, activated it and wish to have scipy installed there.
Installing scipy from new scipy wheel directly would still fail on missing numpy. This we overcome by installing numpy first.
$ pip install numpy
And then finish with scipy
$ pip install scipy
I guess, this could be done in one call (but I did not test it)
$ pip install numpy scipy
Repeatedly installing scipy of proven version
It is likely, that at one moment in future, new release of scipy or numpy will be released and pip will attempt to install the latest version for which there is no wheel in your wheelhouse.
If you can live with the versions you have used so far, you shall create requirements.txt stating the versions of numpy and scipy you like and install from it.
This shall ensure needed package to be present before it is really used.