How to run sklearn library with native tensorflow on m1 mac - python

I have installed TensorFlow using a virtual environment running python 3.8 as described by Apple. This should theoretically run natively and utilise the GPU. I tried installing TensorFlow using miniforge last time and it was not able to use the GPU as miniforge uses python 3.9 and Tensorflow for m1 macs currently require python 3.8.
On sklearns website, the only way to install sklearn libraries currently is by using conda install sklearn which is through miniforge.
Is there a way to install sklearn on a tensorflow environment created using
python3 -m venv TFGPU
I have already tried pip. I was able to install most other libraries other than sklearn which I use for pre-processing.

Hi and welcome to SO :)
I'm a pip/virtualvenv user, so I had to fix my venv to work with my M1 mac using the conda/miniforge, without switching to conda's venv. So, I believe this should work for you as well:
# if not yet installed
xcode-select --install
git clone git://github.com/scikit-learn/scikit-learn.git
cd scikit-learn
# mac / mac m1 specific
brew install libomp
brew install miniforge
wget https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-MacOSX-arm64.sh
bash Miniforge3-MacOSX-arm64.sh
conda init bash
conda create -n conda-sklearn-dev -c conda-forge python numpy scipy cython joblib threadpoolctl pytest compilers llvm-openmp
conda activate conda-sklearn-dev
pip install cython
pip install --verbose --no-build-isolation --editable .
Now:
Test it on the conda venv. You should get version 0.24.2 (at the
time of writing)
Deactivate all conda venv-s
Activate your regular venv and test again. you should get version .dev0
In case sklearn is missing - do a simple pip install for it - it should grab the compiled one

Related

Tensorflow < 2.4 chip M1

I am working on a project which requirements need python 3.7 and TensorFlow 2.3.1. The problem, I have a MacBook Pro with M1 chip. I was able to install and run TF 2.4.
However, I am running into more complicated compatibility issues.
Does anyone know how can I solve this?
M1 has a compatibility issue with TensorFlow. There is a workaround provided by Apple and other blogs. I have recently tried the same and have provided the summary below:
Tensorflow:
OS: BigSur(11.2.3)
Install command-line tools:
xcode-select --install
Install miniforge:
https://github.com/conda-forge/miniforge#miniforge3
After download run
chmod +x ~/<dir>/Miniforge3-MacOSX-arm64.sh
sh ~/<dir>/Miniforge3-MacOSX-arm64.sh
dir: directory to which miniforge is downloaded.
Or
brew install miniforge
After download change source to miniforge3
source ~/miniforge3/bin/activate
Download the environment.yml file from
https://github.com/mwidjaja1/DSOnMacARM
Setup a new conda environment using the yml file:
conda env create --file=environment.yml --name env_name
conda activate env_name
Install tensorflow dependencies:
conda install -c apple tensorflow-deps
Install base tensorflow:
python -m pip install tensorflow-macos
Install tensorflow-metal plugin:
python -m pip install tensorflow-metal
Possible Issues:
Miniforge uses conda-forge to install packages
Method to route channel to conda-forge
conda install -c conda-forge matplotlib
conda install or pip install might not work
Packages available for libraries in conda-forge:
https://anaconda.org/conda-forge
Only these will work.
Sources:
https://developer.apple.com/metal/tensorflow-plugin/
https://towardsdatascience.com/installing-tensorflow-on-the-m1-mac-410bb36b776
https://github.com/mwidjaja1/DSOnMacARM

How to install a package only within a conda environment, without affecting other environments?

I thought that this would simply be done by running pip install within the environment (after source activate). But in my case, pip install affects ALL my environments.
For example, I am trying to install an older version of torch only in env1, but if I run pip install torch==1.1.0 after source activate env1 all my environments have torch version 1.1.0.
I also tried manually running pip as described in this answer
but it didn't work.
How can I fix that?
Versions
Anaconda 3, 2019.07
Python 3.7

Install Tensorflow 2.0 in conda enviroment

I would like to know if anyone knows how can I install tensorflow==2.0.0-alpha0 in a conda enviroment using python 3.7. Is it possible to use python 3.7 or do I have to downgrade to 3.6. Either way what is the command I need to use because the following don't find any package
conda install tensorflow==2.0.0-alpha0
conda install tensorflow
conda install tensorflow=2.0.0-alpha0
I am using fedora 29 and conda 4.6.8
Thanks!
TENSORFLOW 2.0 release version is out!
Since 01/10/2019 I'm not talking beta but the release version.
Using Anaconda
Since 01/11/2019 Anaconda is supporting the Tensorflow 2.0.0.
Option 1: For what the easiest way is just:
conda install tensorflow or conda install tensorflow-gpu
For the gpu mode, anaconda will take care of all the CUDA everything you need to install for the tensorflow gpu mode to work so I strongly recommend using this method.
The only issue with this method is that anaconda might not have the last last version of TensorFlow. For example, at Feb 21 2021, conda has the version 2.3 whereas the PIP version is 2.4. You can check the current version of gpu or cpu.
Option 2 (virtual env): It is strongly recommended to use an environment on where to install tensorflow, for which you need the following command that will create an environment first and then install tensorflow within:
CPU: conda create -n <your_env_name> tensorflow
GPU: conda create -n <your_env_name> tensorflow-gpu
Change <your_env_name> by a meaningful name like tf-2
To use tensorflow run first conda activate <your_env_name>
Using pip
Using pip the tensorflow official instructions are quite complete.
Just install tensorflow using pip like:
# Current stable release for CPU-only
pip install tensorflow
I yet recommend before doing everything to install tensorflow in a new environment so the 3 steps would be (with anaconda):
conda create --n <our_env_name> pip
conda activate <your_env_name>
pip install tensorflow
Now for the GPU version it's harder with pip, I recommend you this link that explains the extra things you need to install (CUDA and others).
It could be the case that the package version you want is not available in conda-forge. What you could do is install packages with pip in your conda environment.
pip install tensorflow==2.0.0-alpha0
Also the requirements don't state python 3.7, you can try your luck or downgrade to python 3.6.
You can now install TF2 for Python 3.7 using conda. You can run the usual
$ conda install tensorflow=2.0 python=3.7
or
$ conda install tensorflow-gpu=2.0 python=3.7
for the GPU version.
My preferred approach however would be to manage the dependencies using an environment.yml file. You can find examples of how to do this for TF2 and dependencies in these template repos that I created on GitHub.
https://github.com/kaust-vislab/tensorflow-cpu-data-science-project
https://github.com/kaust-vislab/tensorflow-gpu-data-science-project
The problem is in conda install tensorflow.
conda does not have tensorflow. You will require to install tensorflow using pip. You do not need to downgrade your Python. It will work with Python 3.7.
Use this
$ pip install --upgrade tensorflow==2.0.0-beta0
Since the beta0 version is released, I mentioned that. You can choose other tf version.
I recommend going through this post on TowardsDataScience: Step-by-Step Guide to Install Tensorflow 2.0.
This post covers installation steps with conda.
You might want to take a look at this link: https://pypi.org/project/tf-nightly-2.0-preview/#files to see which python version and OS supports your package
I tried to install tensorflow v2 with conda install tensorflow or conda install tensorflow-gpu only to get lots of incompatible dependencies.
Just run
pip install -upgrade tensorflow-gpu
or
pip install tensorflow-gpu=2.0.0 for a specific version
Use ' pip install tensorflow-gpu '. This command does the job - downloads Tensorflow-gpu = 2.4.1

How to install Tensorflow on Windows 10 with anaconda?

I know that there exists a link for installing Tensorflow for python 3.5 on
Windows Installation link. There also a similar question on StackOverflow link also, but it case when I use this command:
pip install --ignore-installed --upgrade https://storage.googleapis.com/tensorflow/windows/cpu/tensorflow-1.0.0-cp35-cp35m-win_x86_64.whl
But it said Wheel package needs to be updated. So I updated it using pip and ran the application once again. The output was
tensorflow-1.0.0-cp35-cp35m-win_x86_64.whl is not a supported wheel on this platform.
So how do I solve this problem ? Please help.
I've tried to install the cpu version of TensorFlow.
conda upgrade conda
conda upgrade --all
conda create -n tensorflow python=3.5.
activate tensorflow
conda install -c conda-forge tensorflow
This will create an "environment" that will contain all of your packages you need (the example above is just tensorflow) and you will be able to import that library while you are in that environment anaconda is really just used to manage packages and segregate projects that require different packages
When your finished with your environment, to close out use:
deactivate
these commands are slightly different on OSX/Linux so be sure to look them up if you are on a different operating system
If you are using TF for some machine learning then you will probably want these packages in your environment as well:
conda install pandas matplotlib jupyter notebook scipy scikit-learn
Place that line between (activate tensorflow) and (conda install)
The executable, Anaconda for python 3.5 is not available on the official website.
An alternative to downloading that version is to download the latest version of Anaconda(3.6 as of 9 May, 2017), open your cmd shell in windows and execute the following commands. Note that the activate command is not fully supported in Windows Powershell. Click here to see why.
conda create --name tensorflow python=3.5
activate tensorflow
conda install -c conda-forge tensorflow=1.0.0
The answer has been borrowed from Anaconda Public Google Group.

Installing tensorflow with anaconda in windows

I have installed Anaconda on Windows 64 bit. I have downloaded PyCharm for creating a project and in the terminal of PyCharm I have installed numpy, scipy, matplotlib using the following commands:
conda install numpy
conda install scipy
conda install matplotlib
I am not able to install Tensorflow in the same way I installed these other packages. How should I install it?
Google has recently launched a newer version of TensorFlow r0.12 which include support of Windows both CPU and GPU version can now be installed using Python >=3.5.2 (only 64-bit) version.
For CPU only version open command prompt and enter follow command
pip install --upgrade https://storage.googleapis.com/tensorflow/windows/cpu/tensorflow-0.12.0rc0-cp35-cp35m-win_amd64.whl
Follow this TensorFlow on Windows for step-by-step instructions.
UPDATE
To install current latest version please run following command:
pip install tensorflow #CPU only
pip install tensorflow-gpu #For GPU support
UPDATE 2020
TensorFlow 2.0 now has a single package for both CPU and GPU version, simply run
pip install tensorflow
If you're using Anaconda you can install TensorFlow GPU version and all of its dependencies (CUDA, cuDNN) by running:
conda install -c tensorflow-gpu
To install TF on windows, follow the below-mentioned steps:
conda create --name tensorflow python=3.5
activate tensorflow
conda install jupyter
conda install scipy
pip install tensorflow-gpu
Use pip install tensorflow in place of pip install tensorflow-gpu, in case if you want to install CPU only version of TF.
Note: This installation has been tested with Anaconda Python 3.5 (64 bit). I have also tried the same installation steps with (a) Anaconda Python 3.6 (32 bit), (b) Anaconda Python 3.6 (64 bit), and (c) Anaconda Python 3.5 (32 bit), but all of them (i.e. (a), (b) and (c) ) failed.
Currently tensorflow has binaries only for Unix based OS i.e. Ubuntu Mac OS X - that's why no mention of Windows in setup docs.
There are long discussions on Github:
Open - Windows Support and Documentation
Closed - How to install TensorFlow on Windows
Closed - How to install/run/use TensorFlow on windows machines?
A SO answer - tensorflow — is it or will it (sometime soon) be compatible with a windows workflow?
Suggestion:
For now, on Windows, the easiest way to get started with TensorFlow
would be to use Docker:
http://tensorflow.org/get_started/os_setup.md#docker-based_installation
It should become easier to add Windows support when Bazel (the build
system we are using) adds support for building on Windows, which is on
the roadmap for Bazel 0.3. You can see the full Bazel roadmap here.
Or simply use a Linux VM (using VMPlayer), and the stated steps will setup it up for you.
For PyCharm - Once conda environment will be created, you'll need to set the new interpretor (in conda environment) as the interpretor to use in PyCharm:
Now to use the conda interpreter from PyCharm go to file > settings > project > interpreter, select Add local in the project interpreter field (the little gear wheel) and browse the interpreter or past the path.
The default location - the environment lives under conda_root/envs/tensorflow. The new python interpreter 'll be at conda_root/envs/tensorflow/bin/pythonX.X , such that the site-packages will be in conda_root/envs/tensorflow/lib/pythonX.X/site-packages.
Google has announced support for tensorflow on Windows. Please follow instructions at https://developers.googleblog.com/2016/11/tensorflow-0-12-adds-support-for-windows.html. Please note CUDA8.0 is needed for GPU installation.
If you have installed the 64-bit version of Python 3.5 (either from Python.org or Anaconda), you can install TensorFlow with a single command:
C:> pip install tensorflow
For GPU support, if you have CUDA 8.0 installed, you can install the following package instead:
C:> pip install tensorflow-gpu
I was able to install tensorflow on windows following the instructions on tensorflow.org, using the conda method of installation, as given here: https://www.tensorflow.org/get_started/os_setup#anaconda_installation.
There are small differences on how to activate an 'environment' on windows, you call 'activate' directly without the 'source'. So, for me after installing anaconda the steps where:
C:\Users\Dunschm>conda create -n tensorflow python=3.5
C:\Users\Dunschm>activate tensorflow
(tensorflow) C:\Users\Dunschm>conda install -c conda-forge tensorflow
activate tensorflow
conda install -c conda-forge tensorflow worked for me.
None of the other steps mentioned online helped, I found it here when trying to install an older version.
Eventhough the steps mentioned in the link seems to be for MAC OS X/Linux it worked in windows 7
You can install spyder along with this
conda install spyder
This worked for me:
conda create -n tensorflow python=3.5
activate tensorflow
conda install -c conda-forge tensorflow
Open Anaconda Navigator.
Change the dropdown of "Applications on" from "root" to "tensorflow"
see screenshot
Launch Spyder
Run a little code to validate you're good to go:
import tensorflow as tf
node1 = tf.constant(3, tf.float32)
node2 = tf.constant(4) # also tf.float32 implicitly
print(node1, node2)
or
hello = tf.constant('Hello, TensorFlow!')
sess = tf.Session()
print(sess.run(hello))
I have python 3.5 with anaconda. First I tried everything given above but it did not work for me on windows 10 64bit.
So I simply tried:-
Open the command prompt
Check for python version for which you want to install tensorflow, if you have multiple versions of python.
If you just have one version, then type in cmd:
C:/>conda install tensorflow
for multiple versions of python, type in cmd:
C:/>conda install tensorflow python=version(e.g.python=3.5)
It works, just give it a try.
After installation open ipython console and import tensorflow:
import tensorflow
If tensorflow installed properly then you are ready to go.
Enjoy machine learning:-)
I found a more recent blog post in Anaconda which instructs how to install the TF easily.
I used:
conda create -n tensorflow_env tensorflow
Or for the GPU version (Make sure that you have NVIDIA GPU)
conda create -n tensorflow_gpuenv tensorflow-gpu
This way you will have different environments for different TFs.
The following command from inside your command window (and preferably, conda environment) will work provided you have an Nvidia graphics card.
conda install tensorflow-gpu
1) Update conda
Run the anaconda prompt as administrator
conda update -n base -c defaults conda
2) Create an environment for python new version say, 3.6
conda create --name py36 python=3.6
3) Activate the new environment
conda activate py36
4) Upgrade pip
pip install --upgrade pip
5) Install tensorflow
pip install https://testpypi.python.org/packages/db/d2/876b5eedda1f81d5b5734277a155fa0894d394a7f55efa9946a818ad1190/tensorflow-0.12.1-cp36-cp36m-win_amd64.whl
If it doesn't work
If you have problem with wheel at the environment location, or pywrap_tensorflow problem,
pip install tensorflow --upgrade --force-reinstall
This is what I did for Installing Anaconda Python 3.6 version and Tensorflow on Window 10 64bit.And It was success!
Go to https://www.continuum.io/downloads to download Anaconda Python 3.6 version for Window 64bit.
Create a conda environment named tensorflow by invoking the following command:
C:> conda create -n tensorflow
Activate the conda environment by issuing the following command:
C:> activate tensorflow (tensorflow)C:> # Your prompt should change
Go to http://www.lfd.uci.edu/~gohlke/pythonlibs/enter code here download “tensorflow-1.0.1-cp36-cp36m-win_amd64.whl”. (For my case, the file will be located in “C:\Users\Joshua\Downloads” once after downloaded)
Install the Tensorflow by using the following command:
(tensorflow)C:>pip install C:\Users\Joshua\Downloads\ tensorflow-1.0.1-cp36-cp36m-win_amd64.whl
This is what I got after the installing:
Validate installation by entering following command in your Python environment:
import tensorflow as tf
hello = tf.constant('Hello, TensorFlow!')
sess = tf.Session()
print(sess.run(hello))
If the output you got is 'Hello, TensorFlow!',that means you have successfully install your Tensorflow.
Install Anaconda for Python 3.5 - Can install from here for 64 bit windows
Then install TensorFlow from here
(I tried previously with Anaconda for Python 3.6 but failed even after creating Conda env for Python3.5)
Additionally if you want to run a Jupyter Notebook and use TensorFlow in it. Use following steps.
Change to TensorFlow env:
C: > activate tensorflow
(tensorflow) C: > pip install jupyter notebook
Once installed, you can launch Jupyter Notebook and test
(tensorflow) C: > jupyter notebook
Open anaconda prompt
make sure your pip version is updated
and you have python 3.4 3.5 or 3.6
Just run the command
pip install --upgrade tensorflow
you can take help from the documentation and video
Goodluck
I use windows 10, Anaconda and python 2. A combination of mentioned solutions worked for me:
Once you installed tensorflow using:
C:\Users\Laleh>conda create -n tensorflow python=3.5 # use your python version
C:\Users\Laleh>activate tensorflow
(tensorflow) C:\Users\Laleh>conda install -c conda-forge tensorflow
Then I realized tensorflow can not be imported in jupyter notebook, although it can work in commad windows. To solve this issue first I checked:
jupyter kernelspec list
I removeed the Jupyter kernelspec, useing:
jupyter kernelspec remove python2
Now, the jupyter kernelspec list is pointing to the correct kernel. Again, I activate tensorflow and installed notebook in its environment:
C:\Users\Laleh>activate tensorflow
(tensorflow)C:> conda install notebook
Also if you want to use other libraries such as matplotlib, they should be installed separately in tensorflow environment
(tensorflow)C:> conda install -c conda-forge matplotlib
Now everything works fine for me.
This documentation link is helpful and worked for me. Installs all dependencies and produces a working Anaconda. Or this answer is also helpful if you want to use it with spyder
If you have anaconda version 2.7 installed on your windows, then go to anaconda prompt, type these two commands:
Create a conda environment for tensorflow using conda create -n tensorflow_env tensorflow
activate the tensorflow using conda activate tensorflow_env
If it is activated, then the base will be replaced by tensorflow_env i.e. now it will show (tensorflow_env) C:\Users>
You can now use import tensorflow as tf for using tensorflow in your code.
I tried many things but always faced some issue or other. Below steps with specific version only worked for me.
1> Create virtual env
#conda create -n tensorflow pip python=3.5
2> activate env
#activate tensorflow
#conda info --envs
3> Install tensorflow
#conda install -c conda-forge tensorflow
this will install tensorflow 1.10.0
#python -m pip install --upgrade pip
#pip install setuptools==39.1.0
3> Install keras
#pip install keras==2.2.2
Testing
(tensorflow) C:\WINDOWS\system32>python
Python 3.5.6 |Anaconda, Inc.| (default, Aug 26 2018, 16:05:27) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow as tf
>>> import keras
Using TensorFlow backend.
>>>
The above steps
conda install -c conda-forge tensorflow
will work for Windows 10 as well but the Python version should be 3.5 or above. I have used it with Anaconda Python version 3.6 as the protocol buffer format it refers to available on 3.5 or above.
Thanks,
Sandip
"Conda" installs some of the special packages that might have compiled in C or other languages.
You can use "pip install tensorflow" and it will work.

Categories

Resources