Develop Reference

importing jax fails on mac with m1 chip

For python 3.8.8 and using the new mac air (with the m1 chip), in jupyter notebooks and in python terminal, import jax raises this error
Python 3.8.8 (default, Apr 13 2021, 12:59:45)
[Clang 10.0.0 ] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import jax
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/steve/Documents/code/jax/jax/__init__.py", line 37, in <module>
from . import config as _config_module
File "/Users/steve/Documents/code/jax/jax/config.py", line 18, in <module>
from jax._src.config import config
File "/Users/steve/Documents/code/jax/jax/_src/config.py", line 26, in <module>
from jax import lib
File "/Users/steve/Documents/code/jax/jax/lib/__init__.py", line 63, in <module>
cpu_feature_guard.check_cpu_features()
RuntimeError: This version of jaxlib was built using AVX instructions, which your CPU and/or operating system do not support. You may be able work around this issue by building jaxlib from source.
I suspect it occurs because of the m1 chip.
I tried using jax with pip install jax, then I built it from source as suggested by the comment, by cloning their repository and following the instructions given here, but the same error message shows.

Thanks #jakevdp I looked at the issue you linked and found a workaround :
Thanks to Noah who mentioned in issue #5501 that you could just use a previous version of jax and jaxlib, for my purposes jaxlib==0.1.60 and jax==0.2.10 work just fine!

JAX does not yet provide pre-built jaxlib wheels that are compatible with M1 chips. The best source of information I know on building jaxlib on M1 is probably this github issue: https://github.com/google/jax/issues/5501, which also tracks improving this support.
Hopefully M1 support will be improved in the near future, but it's taking a while for the scientific computing infrastructure up and down the stack to catch up with the requirements of the new chips.

I had a similar problem. Since I already had Anaconda installed and didn't want to clutter up my space with Anaconda + miniconda + homebrew versions of python and package management and whatever, I hunted around for a simple solution. What ended up working for me was first uninstalling jax and jaxlib and then installing jax and jaxlib via conda-forge directly:
pip uninstall jax jaxlib
conda install -c conda-forge jaxlib
conda install -c conda-forge jax

As of now (January 2022), jax is available for M1 Macs. Make sure to uninstall jax and jaxlib and then install the new packages via pip:
pip install --upgrade jax jaxlib
Afterwards, you can use jax without problems.
--Edit--
I am running on a machine with the following specs:
ProductName: macOS
ProductVersion: 12.1
BuildVersion: 21C52
and with Python 3.9.6 within a conda environment.

Installing tensorflow with virtualenv pip in Anaconda

I'm trying to install tensorflow with pip in virtualenv in Anaconda for Windows 10. After activating my virtualenv, when I run pip install tensorflow, I get an error:
ERROR: Could not find a version that satisfies the requirement tensorflow (from versions: none)
ERROR: No matching distribution found for tensorflow
This might be because tensorflow doesn't support Python3.8 which I am using. The update of the post says that it should now work, but it doesn't for me. I tried what the answers suggest with no better luck.
So, I tried the following to downgrade to Python3.7:
conda install python=3.7.1
setting a virtualenv with Python3.7 version
creating a conda environment with Python3.7
installing an older version of Anaconda that contains Python3.7
But any of these attempts incredibly see Python switch back to version 3.8 when I activate the virtualenv. Even if I destroy the environment and create it again in the older Anaconda.
I also tried py -3.7 -m pip install tensorflow in the Anaconda with Python3.7 (it doesn't find the correct version of Python when using the more recent Anaconda). It lead to the error:
Cannot uninstall 'wrapt'. It is a distutils installed project and thus
we cannot accurately determine which files belong to it which would
lead to only a partial uninstall.
I fixed it running conda update wrapt and launched the tensorflow installation command again. After many package installations and removals, it displayed the following error:
ERROR: After October 2020 you may experience errors when installing or
updating packages. This is because pip will change the way that it
resolves dependency conflicts.
We recommend you use --use-feature=2020-resolver to test your packages
with the new resolver before it becomes the default.
pytest-doctestplus 0.2.0 requires pytest>=2.8.0, but you'll have
pytest 0.0.0 which is incompatible. pytest-astropy 0.5.0 requires
pytest>=3.1.0, but you'll have pytest 0.0.0 which is incompatible.
And in effect, I can see that the installation didn't work:
pip freeze > requirements.txt creates an empty requirements.txt
in the Python console, when I try to import keras, it resolves to:
Traceback (most recent call last):
File "C:\Users\my_private_path\venv\lib\site-packages\keras\__init__.py", line 3, in <module>
from tensorflow.keras.layers.experimental.preprocessing import RandomRotation
ModuleNotFoundError: No module named 'tensorflow'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\my_private_path\venv\lib\site-packages\keras\__init__.py", line 5, in <module>
raise ImportError(
ImportError: Keras requires TensorFlow 2.2 or higher. Install TensorFlow via `pip install tensorflow`
It seems that no matter how many paths I try, tensorflow doesn't like virtualenv in Anaconda. Would someone have a path that works?

I use virtual environments with tensorflow all the time and it works fine. First use Anaconda to create a new environment. From the Anaconda home page click on Environments.
When the Environment page appears click on create. A new window will pop up. Give the environment a name (for example call it tf), check the python checkbox. From the dropdown select python 3.7. A new environment called tf is created with python 3.7 installed.
Now go to the Anaconda prompt which is conda. type in conda activate tf. Now use conda to install tensorflow (do not use pip). Conda automatically installs cudnn 7.6.5 and CUDA Toolkit 10.1.243 and tensorflow 2.1.0. If you want tensorflow 2.2 first install 2.1.0 as described. Then use pip to install 2.2 with pip install tensorflow ==2.2.0. 2.2 is compatible with the toolkit and cudnn versions installed with 2.1. Conda can only install tensorflow up to version 2.1.0. Install whatever other packages you need using either pip or conda.

Python setup.py: Could not find suitable distribution for Requirement.parse('tensorflow')

I have tensorflow listed as a requirement in the install_requires section of the setup.py of my project.
When I attempt to install my project into a new Anaconda environment I get the following error:
$ python setup.py install
...
Searching for tensorflow
Reading https://pypi.org/simple/tensorflow/
No local packages or working download links found for tensorflow
error: Could not find suitable distribution for Requirement.parse('tensorflow')
I can get past this by installing tensorflow "manually" via conda:
$ conda install tensorflow
Once I do this the install via setup.py works without a hitch.
Am I mistaken in assuming that something is amiss with my environment? If not then what is going on and how can I avoid this issue? (My concern is that users of my package will not be able to install from source using setup.py)
Another oddity that I assume is related or may provide a clue is that the version of TensorFlow listed in my Anaconda environment is 2.0 but if I import it when running Python it appears to instead be using version 1.15. For example:
$ conda list tensorflow
# packages in environment at /home/james/miniconda3/envs/cvdata_test:
#
# Name Version Build Channel
tensorflow 2.0.0 mkl_py37h66b46cc_0
tensorflow-base 2.0.0 mkl_py37h9204916_0
tensorflow-estimator 2.0.0 pyh2649769_0
$ python
Python 3.7.6 | packaged by conda-forge | (default, Jan 7 2020, 22:33:48)
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf
>>> tf.__version__
'1.15.0-rc2'
This is on a Dell laptop running Ubuntu 18.04 without a GPU, so perhaps the version shown in the interpreter is akin to tensorflow-cpu? If I run pip freeze I see tensorflow==2.0.0 and tensorflow-cpu==1.15.0rc2, which is a bit confusing...

This appears to have been caused by using Python version 3.8, which is currently an unsupported version of Python. Once I created a new Anaconda environment with Python version 3.7 this issue went away.
The only remaining issue is this error that I see when I run pip install -e . for my project which includes tensorflow:
ERROR: tensorflow-cpu 1.15.0rc2 has requirement tensorboard<1.16.0,>=1.15.0, but you'll have tensorboard 2.1.0 which is incompatible.
ERROR: tensorflow-cpu 1.15.0rc2 has requirement tensorflow-estimator==1.15.1, but you'll have tensorflow-estimator 2.1.0 which is incompatible.
So the issue of tensorflow-cpu version 1.15.0rc2 actually being installed when version 2.1.0 shows as being the installed version is still a mystery. To wit:
$ conda list tensorflow
# packages in environment at /home/james/miniconda3/envs/cvd:
#
# Name Version Build Channel
tensorflow 2.1.0 pypi_0 pypi
tensorflow-estimator 2.1.0 pypi_0 pypi
$ python -c "import tensorflow as tf; print(tf.__version__)"
1.15.0-rc2

Can't import module installed with pip (anaconda python)

I'm trying to use the ReadIM package with an anaconda environment (anaconda 4.7.12, python 3.8, pip 19.3.1), thus requiring me to use pip install readIM which succeeds, with the package found in conda list and pip list.
I have read a lot of posts talking about using the correct python and pip paths, my python where shows only my anaconda path, where I ran conda install pip beforehand, and then pip -V shows my pip is in my conda path.
I have tried conda update --all and pip uninstall readIM. I also read that it could be an issue with Jupyter notebook, hence in the conda command prompt I tried python then import readIM, which still gave me the same import error.
As advised on the source page, I tried installing by running python setup.py build install/python setup.py install in the source directory, however this gave me an error
cygwin TypeError: '>=' not supported between instances of 'NoneType' and 'str'
I am at a loss, any ideas?
Thanks,
Mustafa.
UPDATE:
Managed to compile it from source successfully, but still get the same error:
(base) C:>python -m pip install ReadIM-0.8.2.tar.gz
Processing c:\readim-0.8.2.tar.gz
Building wheels for collected packages: ReadIM
Building wheel for ReadIM (setup.py) ... done
Created wheel for ReadIM: filename=ReadIM-0.8.2-cp37-cp37m-win_amd64.whl size=219546 sha256=90f8960a6f1f80ae62dc18eab4bcd31fb2a6dfd7da364a5c15fc37e6e2ce0360
Stored in directory: C:\Users\mi4517\AppData\Local\pip\Cache\wheels\d6\a9\11\936e986255027bb654601b322a3431f9bcc3fde72ebb406835
Successfully built ReadIM
Installing collected packages: ReadIM
Successfully installed ReadIM-0.8.2
(base) C:>python
Python 3.7.5 (default, Oct 31 2019, 15:18:51) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
import readIM
Traceback (most recent call last):
File "", line 1, in
ModuleNotFoundError: No module named 'readIM'>
Following their instructions (python setup.py build install then python setup.py test) yields a successful build but a failed test:
C:\ReadIM-0.8.2>python setup.py test
Traceback (most recent call last):
File "setup.py", line 41, in
assert os.path.isdir(testFileDir)
AssertionError
SOLUTION
It was installing correctly, just that I was importing it incorrectly in Python. I was doing import readIM or import readim when it should be import ReadIM I did not know it was case sensitive!
Thank you for your help, apologies for the trivial mistake on my part.

I had faced a similar issue. With your conda environment activated, type
which pip
Check if it shows the path to pip in your environment. In my case, it was not. It was showing path to some other pip. So pip install was installing in that environment. To fix use the full path of pip, like below
~/anaconda3/envs/my_env/bin/pip install ...

To avoid this issue follow these steps
open Anaconda prompt
type this command conda install -c conda-forge imread
after installation open jupiter notebook.
type import imread then run.

It was installing correctly, just that I was importing it incorrectly in Python. I was doing import readIM or import readim when it should be import ReadIM I did not know it was case sensitive!
Thank you for your help, apologies for the trivial mistake on my part.

"import torch" giving error "from torch._C import *, DLL load failed: The specified module could not be found"

I am currently using Python 3.5.5 on Anaconda and I am unable to import torch. It is giving me the following error in Spyder:
Python 3.5.5 |Anaconda, Inc.| (default, Mar 12 2018, 17:44:09) [MSC v.1900
64 bit (AMD64)]
Type "copyright", "credits" or "license" for more information.
IPython 6.2.1 -- An enhanced Interactive Python.
import torch
Traceback (most recent call last):
File "<ipython-input-1-eb42ca6e4af3>", line 1, in <module>
import torch
File "C:\Users\trish\Anaconda3\envs\virtual_platform\lib\site-
packages\torch\__init__.py", line 76, in <module>
from torch._C import *
ImportError: DLL load failed: The specified module could not be found.
Many suggestions on the internet say that the working directory should not be the same directory that the torch package is in, however I've manually set my working directory to C:/Users/trish/Downloads, and I am getting the same error.
Also I've already tried the following: reinstalling Anaconda and all packages from scratch, and I've ensured there is no duplicate "torch" folder in my directory.
Pls help! Thank you!

I had this similar problem in windows 10...
Solution:
Download win-64/intel-openmp-2018.0.0-8.tar.bz2 from https://anaconda.org/anaconda/intel-openmp/files
Extract it and put the dll files in Library\bin into C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v9.0\bin
Make sure your cuda directory is added to your %PATH% environment variable

I had the same problem. In my case I didn't want the GPU version of pytorch.
I uninstalled it. The version was pytorch: 0.3.1-py36_cuda80_cudnn6he774522_2 peterjc123.
The problem is that cuda and cudnn . then installed with the following command and now it works!
conda install -c peterjc123 pytorch-cpu

I also encountered the same problem when I used a conda environment with python 3.6.8 and pytorch installed by conda from channel -c pytorch.
Here is what worked for me:
1:) conda create -n envName python=3.6 anaconda
2:) conda update -n envName conda
3:) conda activate envName
4:) conda install pytorch torchvision cudatoolkit=9.0 -c pytorch
and then tested torch with the given code:
5:) python -c "import torch; print(torch.cuda.get_device_name(0))"
Note: 5th step will return your gpu name if you have a cuda compatible gpu
Summary: I just created a conda environment containing whole anaconda and then to tackle the issue of unmatched conda version I updated conda of new environment from the base environment and then installed pytorch in that environment and tested pytorch.
For CPU version, here is the link for my another answer: https://gist.github.com/peterjc123/6b804651288e76db7b5fabe5348e1f03#gistcomment-2842825
https://gist.github.com/peterjc123/6b804651288e76db7b5fabe5348e1f03#gistcomment-2842837

Had the same problem and fixed it by re-installing numpy with mkl (Intel's math kernel library)
https://www.lfd.uci.edu/~gohlke/pythonlibs/#numpy
Download the right .whl for your machine. For me it was numpy‑1.14.5+mkl‑cp36‑cp36m‑win_amd64.whl (python 3.6, windows, 64-bit)
and then install using pip.
pip install numpy‑1.14.5+mkl‑cp36‑cp36m‑win_amd64.whl

I am using a Windows 10 computer with an NVIDIA GeForce graphics card. NVIDIA showed I had CUDA 10.1, but I was getting this error when running import torch in Jupyter Lab and suspected it had something to do with CUDA support.
I fixed this problem by downloading and installing the CUDA Toolkit directly from NVIDIA. It installed all required Visual Studio components. When I returned to Jupyter Lab, import torch ran without error.

Make sure you installed the right version of pytorch for your enviroment. I had the same problem I was using pytorch on windows but I had the default package installed which was meant for cuda 8. So I reinstalled the pytorch package for cpu which was what I needed.

I had the same issue with running torch installed with pure pip and solved it by switching to conda.
Following steps:
uninstall python 3.6 from python.org (if exists)
install miniconda
install torch in conda ("conda install pytorch -c pytorch")
Issue with pip installation:
import torch
File "C:\Program Files\Python35\lib\site-packages\torch\__init__.py", line 78, in <module>
from torch._C import *
ImportError: DLL load failed: The specified module could not be found.
After switching to conda it works fine. I believe the issue was resolved by conda through installing the vs_redist 2017
vs2017_runtime 15.4.27004.2010 peterjc123
But I have tried it w/o conda and it did not help. Could not find how to check (and tweak) Python's vs_redist.

Windows10 Solution(This worked for my system):
I was having the same issue in my system. Previously I was using Python 3.5 and I created a virtual environment named pytorch_test using the virtualenv module because I didn't want to mess up my tensorflow installation(which took me a lot of time). I followed every instruction but it didn't seem to work. I installed python 3.6.7 added it to the path. Then I created the virtual environment using:
virtualenv --python=3.6 pytorch_test
Then go to the destination folder
cd D:\pytorch_test
and activate the virtual environment entering the command in cmd:
.\Scripts\activate
After you do this the command prompt will show:
(pytorch_test) D:\pytorch_test>
Update pip if you have not done it before using:
(pytorch_test) D:\pytorch_test>python -m pip install --upgrade pip
Then go for installing numpy+mkl from the site:
https://www.lfd.uci.edu/~gohlke/pythonlibs/#numpy
Choose the correct version from the list if you have python 3.6.7 go with the wheel file:
numpy‑1.15.4+mkl‑cp36‑cp36m‑win_amd64.whl (For 64 bit)
(Note if the whole thing doesnot work just go with simple numpy installation and mkl installation separately)
Then go for installing openmp using:
(pytorch_test) D:\pytorch_test>pip install intel-openmp
Now you are done with the prerequisites. To install pytorch go to the previous versions site:
https://pytorch.org/get-started/previous-versions/
Here select the suitable version from the list of Windows Binaries. For example I am having CUDA 9.0 installed in my system with python 3.6.7 so I went with the gpu version:
cu90/torch-1.0.0-cp36-cp36m-win_amd64.whl
(There are two available versions 0.4.0 and 1.0.0 for pytorch, I went with 1.0.0)
After downloading the file install it using pip(assuming the whl file is in D:).You have to do this from the virtual environment pytorch_test itself:
(pytorch_test) D:\pytorch_test>pip install D:\torch-1.0.0-cp36-cp36m-win_amd64.whl
Prerequisites like six, pillow will be installed automatically.
Then once everything is done, install the models using torchvision.
Simply type :
(pytorch_test) D:\pytorch_test>pip install torchvision
To check everything is working fine try the following script:
import torch
test = torch.rand(4, 7)
print(test)
If everything was good then it wont be an issue. Whenever there is an issue like this it is related to version mismatch of one or more dependencies. This also occurred during tensorflow installation.
Deactivate the following virtual environment using the command deactivate in the cmd:
(pytorch_test) D:\pytorch_test>deactivate
This is the output of pip list in my system:
Package Version
------------ -----------
intel-openmp 2019.0
mkl 2019.0
numpy 1.16.2
Pillow 6.0.0
pip 19.0.3
setuptools 41.0.0
six 1.12.0
torch 1.0.0
torchvision 0.2.2.post3
wheel 0.33.1
Hope this helps. This is my first answer in this community, hope you all find it helpful. I setup pytorch today in the afternoon after trying all sorts of combinations. The same import problem occurred to me while installing CNTK and tensorflow. Anyway I kept them separate in different virtual environments so that I can use them anytime.