M1 Mac Tensorflow VS Code Rosetta2 - python

I'm struggling to install tensorflow with a M1 mac. I've got python 3.9.7 and Monterrey 12.3 and apple silicon visual studio code. There is an apple solution involving miniconda apple dependancies and tensorflow-macos and tensorflow-metal. However this solution is not good for me as I have to use Rosetta2 emulator for multiple packages including PyQt5 etc. I was wondering if anyone has been able to use their M1 macs and pip installed tensorflow in a venv rosetta terminal. Thank you.
Kevin

Running TensorFlow on miniforge + conda-forge (arm64)
TensorFlow can run natively on M1 (arm64) macs. A highly recommended, easy way to install TensorFlow on arm64 macs is to via conda-forge. You should install python via miniforge or miniconda, because there is an arm64 (Apple Sillicon) distribution. With this, as of today, you can install the latest version 2.10.0 of TensorFlow:
$ lipo -archs $(which python3) # python3 is running natively as arm64
arm64
$ conda install -c conda-forge tensorflow
Note: tensorflow-macos 2.4.0 is obsolete so you shouldn't be using that.
But still want Rosetta 2? Try conda-forge.
If you really need to have python running on Rosetta 2 (x86_64) in cases where some packages does not support arm64,
you can still install TensorFlow with a macOS x86_64 release via conda. Installing via pip and PyPI repository won't work here, because you will run into Illegal hardware instruction segfault because Google's official TF macos-x86_64 wheel releases on PyPI assumes a target platform that has AVX instructions.
$ lipo -archs $(which python3) # x86_64 means Rosetta 2
x86_64
$ conda install -c conda-forge tensorflow # install via conda
$ python -c 'import tensorflow; print(tensorflow.__version__)'

Related

'Expected in: /usr/lib/libc++.1.dylib': Installing Tensorflow on M1 MacBook Pro

I am trying to install Tensorflow on my MacBook Pro with the M1 chip. The operating system of my MacBook is MacOS Big Sur Version 11.0.
In order to install Tensorflow to use it with Python, I have followed this tutorial, which says that I have to do the following:
Install Homebrew.
Download MiniForge3 for macOS arm64 chips (link provided in the webpage).
Install MiniForge3 using:
chmod +x ~/Downloads/Miniforge3-MacOSX-arm64.sh
sh ~/Downloads/Miniforge3-MacOSX-arm64.sh
source ~/miniforge3/bin/activate
Create a folder to set up an environment for Tensorflow.
mkdir tensorflow-test
cd tensorflow-test
Make and activate Conda environment.
conda create --prefix ./env python=3.9.7
conda activate ./env
Install Tensorflow dependencies.
conda install -c apple tensorflow-deps
python -m pip install tensorflow-macos
python -m pip install tensorflow-metal
After this, I open a Jupyter Notebook and I try to import tensorflow, but this error shows up:
OSError: dlopen(/Users/blancoarnau/tensorflow-test/env/lib/python3.9/site-packages/tensorflow/python/platform/../../core/platform/_cpu_feature_guard.so, 6): Symbol not found: __ZNKSt3__115basic_stringbufIcNS_11char_traitsIcEENS_9allocatorIcEEE3strEv
Referenced from: /Users/blancoarnau/tensorflow-test/env/lib/python3.9/site-packages/tensorflow/python/platform/../../core/platform/_cpu_feature_guard.so (which was built for Mac OS X 12.3)
Expected in: /usr/lib/libc++.1.dylib
As you can see in this screenshot:
Do you have an idea why this is happening?
check the message details:
(which was built for Mac OS X 12.3)
you need to upgrade macOS to 12.3

Tensorflow < 2.4 chip M1

I am working on a project which requirements need python 3.7 and TensorFlow 2.3.1. The problem, I have a MacBook Pro with M1 chip. I was able to install and run TF 2.4.
However, I am running into more complicated compatibility issues.
Does anyone know how can I solve this?
M1 has a compatibility issue with TensorFlow. There is a workaround provided by Apple and other blogs. I have recently tried the same and have provided the summary below:
Tensorflow:
OS: BigSur(11.2.3)
Install command-line tools:
xcode-select --install
Install miniforge:
https://github.com/conda-forge/miniforge#miniforge3
After download run
chmod +x ~/<dir>/Miniforge3-MacOSX-arm64.sh
sh ~/<dir>/Miniforge3-MacOSX-arm64.sh
dir: directory to which miniforge is downloaded.
Or
brew install miniforge
After download change source to miniforge3
source ~/miniforge3/bin/activate
Download the environment.yml file from
https://github.com/mwidjaja1/DSOnMacARM
Setup a new conda environment using the yml file:
conda env create --file=environment.yml --name env_name
conda activate env_name
Install tensorflow dependencies:
conda install -c apple tensorflow-deps
Install base tensorflow:
python -m pip install tensorflow-macos
Install tensorflow-metal plugin:
python -m pip install tensorflow-metal
Possible Issues:
Miniforge uses conda-forge to install packages
Method to route channel to conda-forge
conda install -c conda-forge matplotlib
conda install or pip install might not work
Packages available for libraries in conda-forge:
https://anaconda.org/conda-forge
Only these will work.
Sources:
https://developer.apple.com/metal/tensorflow-plugin/
https://towardsdatascience.com/installing-tensorflow-on-the-m1-mac-410bb36b776
https://github.com/mwidjaja1/DSOnMacARM

How to run sklearn library with native tensorflow on m1 mac

I have installed TensorFlow using a virtual environment running python 3.8 as described by Apple. This should theoretically run natively and utilise the GPU. I tried installing TensorFlow using miniforge last time and it was not able to use the GPU as miniforge uses python 3.9 and Tensorflow for m1 macs currently require python 3.8.
On sklearns website, the only way to install sklearn libraries currently is by using conda install sklearn which is through miniforge.
Is there a way to install sklearn on a tensorflow environment created using
python3 -m venv TFGPU
I have already tried pip. I was able to install most other libraries other than sklearn which I use for pre-processing.
Hi and welcome to SO :)
I'm a pip/virtualvenv user, so I had to fix my venv to work with my M1 mac using the conda/miniforge, without switching to conda's venv. So, I believe this should work for you as well:
# if not yet installed
xcode-select --install
git clone git://github.com/scikit-learn/scikit-learn.git
cd scikit-learn
# mac / mac m1 specific
brew install libomp
brew install miniforge
wget https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-MacOSX-arm64.sh
bash Miniforge3-MacOSX-arm64.sh
conda init bash
conda create -n conda-sklearn-dev -c conda-forge python numpy scipy cython joblib threadpoolctl pytest compilers llvm-openmp
conda activate conda-sklearn-dev
pip install cython
pip install --verbose --no-build-isolation --editable .
Now:
Test it on the conda venv. You should get version 0.24.2 (at the
time of writing)
Deactivate all conda venv-s
Activate your regular venv and test again. you should get version .dev0
In case sklearn is missing - do a simple pip install for it - it should grab the compiled one

Can't install Tensorflow 2.3 on Raspberry Pi 4

I can't currently install TensorFlow 2.3 on a Raspberry Pi 4.
Unfortunately, the pip doesn't return any TensorFlow version:
pip install tensorflow==
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
ERROR: Could not find a version that satisfies the requirement tensorflow==
ERROR: No matching distribution found for tensorflow==
I read somewhere that TensorFlow only supports the 64-bit version of Python and only the versions between 3.5 to 3.8 so I made sure that I have these versions installed as well.
The current Linux distro that I have is the following aarch64.
pip and python have the following versions:
pip -V python
pip 20.3.1 from /home/pi/envs/awe/lib/python3.7/site-packages/pip (python 3.7)
python -V
Python 3.7.3
And I also have a 64-bit version of Python:
platform.architecture()
('64bit', 'ELF')
Why can't pip find a compatible TensorFlow of version 2.3?
For the Raspberry Pi 4 ARM64 platform TensorFlow is not available as pre-built version for pip and must be compiled manually.
Follow these instructions and cross compile it on a different more powerful environment, not on the Pi 4. The instructions are not up to date anymore, I have completed them here. These are the steps to cross compile TensorFlow 2.3.1 on an Ubuntu Linux:
git clone git#github.com:tensorflow/tensorflow.git
cd tensorflow
The old version 2.3.1 does not know Python 3.8. When using Python 3.7 then this might not be necessary.
wget https://github.com/tensorflow/tensorflow/blob/master/tensorflow/tools/ci_build/Dockerfile.pi-python38 -o tensorflow/tools/ci_build/Dockerfile.pi-python38
CMake is now also required, I'm using here the master branch, which is corresponding to version 2.5.0 (in case this does not work anymore in the future with the master version):
wget -O tensorflow/tools/ci_build/install/install_cmake.sh https://raw.githubusercontent.com/tensorflow/tensorflow/master/tensorflow/tools/ci_build/install/install_cmake.sh
chmod +x tensorflow/tools/ci_build/install/install_cmake.sh
To support Python 3.8 add support for it, again taken from the master branch (version 2.5.0):
wget -O tensorflow/tools/ci_build/install/install_pi_python3x_toolchain.sh https://raw.githubusercontent.com/tensorflow/tensorflow/master/tensorflow/tools/ci_build/install/install_pi_python3x_toolchain.sh
chmod +x tensorflow/tools/ci_build/install/install_pi_python3x_toolchain.sh
You also need these files for the build:
wget -O tensorflow/tools/ci_build/pi/build_raspberry_pi.sh https://raw.githubusercontent.com/tensorflow/tensorflow/master/tensorflow/tools/ci_build/pi/build_raspberry_pi.sh
chmod +x tensorflow/tools/ci_build/pi/build_raspberry_pi.sh
There is an issue with newer numpy version. Use this cherry-pick from git:
git cherry-pick 75ea0b31477d6ba9e990e296bbbd8ca4e7eebadf
Now finally all can be compiled. This takes several hours most likely.
tensorflow/tools/ci_build/ci_build.sh PI-PYTHON38 tensorflow/tools/ci_build/pi/build_raspberry_pi.sh AARCH64
If something fails and some files have to be patched or added keep in mind to clean the Docker container to prevent using the Docker cache when restarting the build process. docker images, docker rmi <imageid> and docker rm <container> are your friends.

Installing tensorflow with anaconda in windows

I have installed Anaconda on Windows 64 bit. I have downloaded PyCharm for creating a project and in the terminal of PyCharm I have installed numpy, scipy, matplotlib using the following commands:
conda install numpy
conda install scipy
conda install matplotlib
I am not able to install Tensorflow in the same way I installed these other packages. How should I install it?
Google has recently launched a newer version of TensorFlow r0.12 which include support of Windows both CPU and GPU version can now be installed using Python >=3.5.2 (only 64-bit) version.
For CPU only version open command prompt and enter follow command
pip install --upgrade https://storage.googleapis.com/tensorflow/windows/cpu/tensorflow-0.12.0rc0-cp35-cp35m-win_amd64.whl
Follow this TensorFlow on Windows for step-by-step instructions.
UPDATE
To install current latest version please run following command:
pip install tensorflow #CPU only
pip install tensorflow-gpu #For GPU support
UPDATE 2020
TensorFlow 2.0 now has a single package for both CPU and GPU version, simply run
pip install tensorflow
If you're using Anaconda you can install TensorFlow GPU version and all of its dependencies (CUDA, cuDNN) by running:
conda install -c tensorflow-gpu
To install TF on windows, follow the below-mentioned steps:
conda create --name tensorflow python=3.5
activate tensorflow
conda install jupyter
conda install scipy
pip install tensorflow-gpu
Use pip install tensorflow in place of pip install tensorflow-gpu, in case if you want to install CPU only version of TF.
Note: This installation has been tested with Anaconda Python 3.5 (64 bit). I have also tried the same installation steps with (a) Anaconda Python 3.6 (32 bit), (b) Anaconda Python 3.6 (64 bit), and (c) Anaconda Python 3.5 (32 bit), but all of them (i.e. (a), (b) and (c) ) failed.
Currently tensorflow has binaries only for Unix based OS i.e. Ubuntu Mac OS X - that's why no mention of Windows in setup docs.
There are long discussions on Github:
Open - Windows Support and Documentation
Closed - How to install TensorFlow on Windows
Closed - How to install/run/use TensorFlow on windows machines?
A SO answer - tensorflow — is it or will it (sometime soon) be compatible with a windows workflow?
Suggestion:
For now, on Windows, the easiest way to get started with TensorFlow
would be to use Docker:
http://tensorflow.org/get_started/os_setup.md#docker-based_installation
It should become easier to add Windows support when Bazel (the build
system we are using) adds support for building on Windows, which is on
the roadmap for Bazel 0.3. You can see the full Bazel roadmap here.
Or simply use a Linux VM (using VMPlayer), and the stated steps will setup it up for you.
For PyCharm - Once conda environment will be created, you'll need to set the new interpretor (in conda environment) as the interpretor to use in PyCharm:
Now to use the conda interpreter from PyCharm go to file > settings > project > interpreter, select Add local in the project interpreter field (the little gear wheel) and browse the interpreter or past the path.
The default location - the environment lives under conda_root/envs/tensorflow. The new python interpreter 'll be at conda_root/envs/tensorflow/bin/pythonX.X , such that the site-packages will be in conda_root/envs/tensorflow/lib/pythonX.X/site-packages.
Google has announced support for tensorflow on Windows. Please follow instructions at https://developers.googleblog.com/2016/11/tensorflow-0-12-adds-support-for-windows.html. Please note CUDA8.0 is needed for GPU installation.
If you have installed the 64-bit version of Python 3.5 (either from Python.org or Anaconda), you can install TensorFlow with a single command:
C:> pip install tensorflow
For GPU support, if you have CUDA 8.0 installed, you can install the following package instead:
C:> pip install tensorflow-gpu
I was able to install tensorflow on windows following the instructions on tensorflow.org, using the conda method of installation, as given here: https://www.tensorflow.org/get_started/os_setup#anaconda_installation.
There are small differences on how to activate an 'environment' on windows, you call 'activate' directly without the 'source'. So, for me after installing anaconda the steps where:
C:\Users\Dunschm>conda create -n tensorflow python=3.5
C:\Users\Dunschm>activate tensorflow
(tensorflow) C:\Users\Dunschm>conda install -c conda-forge tensorflow
activate tensorflow
conda install -c conda-forge tensorflow worked for me.
None of the other steps mentioned online helped, I found it here when trying to install an older version.
Eventhough the steps mentioned in the link seems to be for MAC OS X/Linux it worked in windows 7
You can install spyder along with this
conda install spyder
This worked for me:
conda create -n tensorflow python=3.5
activate tensorflow
conda install -c conda-forge tensorflow
Open Anaconda Navigator.
Change the dropdown of "Applications on" from "root" to "tensorflow"
see screenshot
Launch Spyder
Run a little code to validate you're good to go:
import tensorflow as tf
node1 = tf.constant(3, tf.float32)
node2 = tf.constant(4) # also tf.float32 implicitly
print(node1, node2)
or
hello = tf.constant('Hello, TensorFlow!')
sess = tf.Session()
print(sess.run(hello))
I have python 3.5 with anaconda. First I tried everything given above but it did not work for me on windows 10 64bit.
So I simply tried:-
Open the command prompt
Check for python version for which you want to install tensorflow, if you have multiple versions of python.
If you just have one version, then type in cmd:
C:/>conda install tensorflow
for multiple versions of python, type in cmd:
C:/>conda install tensorflow python=version(e.g.python=3.5)
It works, just give it a try.
After installation open ipython console and import tensorflow:
import tensorflow
If tensorflow installed properly then you are ready to go.
Enjoy machine learning:-)
I found a more recent blog post in Anaconda which instructs how to install the TF easily.
I used:
conda create -n tensorflow_env tensorflow
Or for the GPU version (Make sure that you have NVIDIA GPU)
conda create -n tensorflow_gpuenv tensorflow-gpu
This way you will have different environments for different TFs.
The following command from inside your command window (and preferably, conda environment) will work provided you have an Nvidia graphics card.
conda install tensorflow-gpu
1) Update conda
Run the anaconda prompt as administrator
conda update -n base -c defaults conda
2) Create an environment for python new version say, 3.6
conda create --name py36 python=3.6
3) Activate the new environment
conda activate py36
4) Upgrade pip
pip install --upgrade pip
5) Install tensorflow
pip install https://testpypi.python.org/packages/db/d2/876b5eedda1f81d5b5734277a155fa0894d394a7f55efa9946a818ad1190/tensorflow-0.12.1-cp36-cp36m-win_amd64.whl
If it doesn't work
If you have problem with wheel at the environment location, or pywrap_tensorflow problem,
pip install tensorflow --upgrade --force-reinstall
This is what I did for Installing Anaconda Python 3.6 version and Tensorflow on Window 10 64bit.And It was success!
Go to https://www.continuum.io/downloads to download Anaconda Python 3.6 version for Window 64bit.
Create a conda environment named tensorflow by invoking the following command:
C:> conda create -n tensorflow
Activate the conda environment by issuing the following command:
C:> activate tensorflow (tensorflow)C:> # Your prompt should change
Go to http://www.lfd.uci.edu/~gohlke/pythonlibs/enter code here download “tensorflow-1.0.1-cp36-cp36m-win_amd64.whl”. (For my case, the file will be located in “C:\Users\Joshua\Downloads” once after downloaded)
Install the Tensorflow by using the following command:
(tensorflow)C:>pip install C:\Users\Joshua\Downloads\ tensorflow-1.0.1-cp36-cp36m-win_amd64.whl
This is what I got after the installing:
Validate installation by entering following command in your Python environment:
import tensorflow as tf
hello = tf.constant('Hello, TensorFlow!')
sess = tf.Session()
print(sess.run(hello))
If the output you got is 'Hello, TensorFlow!',that means you have successfully install your Tensorflow.
Install Anaconda for Python 3.5 - Can install from here for 64 bit windows
Then install TensorFlow from here
(I tried previously with Anaconda for Python 3.6 but failed even after creating Conda env for Python3.5)
Additionally if you want to run a Jupyter Notebook and use TensorFlow in it. Use following steps.
Change to TensorFlow env:
C: > activate tensorflow
(tensorflow) C: > pip install jupyter notebook
Once installed, you can launch Jupyter Notebook and test
(tensorflow) C: > jupyter notebook
Open anaconda prompt
make sure your pip version is updated
and you have python 3.4 3.5 or 3.6
Just run the command
pip install --upgrade tensorflow
you can take help from the documentation and video
Goodluck
I use windows 10, Anaconda and python 2. A combination of mentioned solutions worked for me:
Once you installed tensorflow using:
C:\Users\Laleh>conda create -n tensorflow python=3.5 # use your python version
C:\Users\Laleh>activate tensorflow
(tensorflow) C:\Users\Laleh>conda install -c conda-forge tensorflow
Then I realized tensorflow can not be imported in jupyter notebook, although it can work in commad windows. To solve this issue first I checked:
jupyter kernelspec list
I removeed the Jupyter kernelspec, useing:
jupyter kernelspec remove python2
Now, the jupyter kernelspec list is pointing to the correct kernel. Again, I activate tensorflow and installed notebook in its environment:
C:\Users\Laleh>activate tensorflow
(tensorflow)C:> conda install notebook
Also if you want to use other libraries such as matplotlib, they should be installed separately in tensorflow environment
(tensorflow)C:> conda install -c conda-forge matplotlib
Now everything works fine for me.
This documentation link is helpful and worked for me. Installs all dependencies and produces a working Anaconda. Or this answer is also helpful if you want to use it with spyder
If you have anaconda version 2.7 installed on your windows, then go to anaconda prompt, type these two commands:
Create a conda environment for tensorflow using conda create -n tensorflow_env tensorflow
activate the tensorflow using conda activate tensorflow_env
If it is activated, then the base will be replaced by tensorflow_env i.e. now it will show (tensorflow_env) C:\Users>
You can now use import tensorflow as tf for using tensorflow in your code.
I tried many things but always faced some issue or other. Below steps with specific version only worked for me.
1> Create virtual env
#conda create -n tensorflow pip python=3.5
2> activate env
#activate tensorflow
#conda info --envs
3> Install tensorflow
#conda install -c conda-forge tensorflow
this will install tensorflow 1.10.0
#python -m pip install --upgrade pip
#pip install setuptools==39.1.0
3> Install keras
#pip install keras==2.2.2
Testing
(tensorflow) C:\WINDOWS\system32>python
Python 3.5.6 |Anaconda, Inc.| (default, Aug 26 2018, 16:05:27) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf
>>> import keras
Using TensorFlow backend.
>>>
The above steps
conda install -c conda-forge tensorflow
will work for Windows 10 as well but the Python version should be 3.5 or above. I have used it with Anaconda Python version 3.6 as the protocol buffer format it refers to available on 3.5 or above.
Thanks,
Sandip
"Conda" installs some of the special packages that might have compiled in C or other languages.
You can use "pip install tensorflow" and it will work.

Categories

Resources