Develop Reference

Confused with setting up ML and DL on GPU

My goal is to set up my PC for machine and deep learning through my GPU. I've read about all the different components however I can not connect the dots for what I need to do.
OS: Ubuntu 20.04
GPU: Nvidia RTX 2070 Super
Anaconda: 4.8.3
I've installed the nvidia-cuda-toolkit (10.1.243), but now what?
How does this integrate with jupyter notebook?
The 3 python modules I want to work with are:
turicreate - I've gotten this to run off CPU but not GPU
scikit-learn
tensorflow
matlab
I know cuDNN and pyCUDA fit in there somewhere.
Any help is appreciated. Thanks

First of all - I have the experience limited to ubuntu 18.04 and 16.xx and python DL frameworks. But I hope some sugestions will be helpfull.
If I were familiar with docker I would rather consider to use docker instead of setting-up everything from scratch. This approach is described in section about tensorflow container
If you decided to setup all components yourself please see this guideline
I used some contents from it for 18.04, succesfully.
be carefull with automatic updates. After the configuration is finished and tested protect it from being overwritten with newest version of CUDAor TensorRT.
Answering one of your sub-questions - How does this integrate with jupyter notebook? - it does not, becuase it is unneccesary. CUDA library cooperates with a framework such as Tensorflow, not with the Jupyter. Jupyter is just an editor and execution controller on the server side.

EXE made from Python file which uses Tensorflow-GPU does not use GPU when deployed

I have a python file which uses tensorflow GPU in it. It uses GPU when i run the file from console using python MyFile.py.
However, when i convert it into exe using pyinstaller, it converts and runs successfully, But it does not use GPU anymore when i run the exe. This happens on a system which was not used for developing MyFile.py. Checking on the same system which was used in development, it uses just 40-50% GPU, which was 90% if i run the python script.
My application even has a small UI made using tkinter.
Though application runs fine on CPU, It is incredibly slow. (I am not using --one-file flag in pyinstaller.) Although having GPU, The application is not using it.
My questions are:
How do I overcome this issue? Do I need to install any CUDA or CuDnn toolkits in my Destination computer?
(Once the main question is solved) Can i use 1050ti in development and 2080ti in destination computer, if the CuDnn and CUDA versions are the same?
Tensorflow Version : 1.14.0 (I know 2.x is out there, but this works perfectly fine for me.)
GPU : GeForce GTX 1050 ti ( In development as well as deployment.)
CUDA Toolkit : 10.0
CuDnn : v7.6.2 for cuda 10.0
pyinstaller version : 3.5
Python version : 3.6.5

As I asnwered also here, according to the GitHub issues in the official repository (here and here for example) CUDA libraries are usually dynamically loaded at run-time and not at link-time, so they are typically not included in the final exe file (or folder) with the result that the generated exe file won't work on a machine without CUDA installed. The solution (please refer to the linked issues too) is to put the DLLs necessary to run the exe in its dist folder (if generated without the --onefile option) or install the CUDA runtime on the target machine.

Minimum required hardware component to install tensorflow-gpu in python

I'm tried many PC with different hardware capability to install tensorflow on gpu, they are either un-compatible or compatible but stuck in some point. I would like to know the minimum hardware required to install tensorflow-gpu. And also I would like to ask about some hardware, Is they are allowed or not:
Can I use core i5 instead of core i7 ??
Is 4 GB gpu enough for training the dataset??
Is 8 GB ram enough for training and evaluating the dataset ?? with most thanks.

TensorFlow (TF) GPU 1.6 and above requires cuda compute capability (ccc) of 3.5 or higher and requires AVX instruction support.
https://www.tensorflow.org/install/gpu#hardware_requirements.
https://www.tensorflow.org/install/pip#hardware-requirements.
Therefore you would want to buy a graphics card that has ccc above 3.5.
Here's a link that shows ccc for various nvidia graphic cards https://developer.nvidia.com/cuda-gpus.
However if your cuda compute capability is below 3.5 you have to compile TF from sources yourself. This procedure may or may not work depending on the build flags you choose while compiling and is not straightforward.
In my humble opinion, The simplest way is to use TF-GPU pre-built binaries to install TF GPU.
To answer your questions. Yes you can use TF comfortably on i5 with 4gb of graphics card and 8gb ram. The training time may take longer though, depending on task at hand.
In summary, the main hardware requirement to install TF GPU is getting a Nvidia graphics card with cuda compute capability more than 3.5, more the merrier.
Note that TF officially supports only NVIDIA graphics card.

You should find your answers here:
https://www.nvidia.com/en-gb/data-center/gpu-accelerated-applications/tensorflow/
From the link:
The GPU-enabled version of TensorFlow has the following requirements:
64-bit Linux
Python 2.7
CUDA 7.5 (CUDA 8.0 required for Pascal GPUs)
cuDNN v5.1 (cuDNN v6 if on TF v1.3)

Python pyopencl DLL load failed even with latest drivers

I've installed the latest CUDA and driver for my GPU. I'm using Python 2.7.10 on Win7 64bit.
I tried installing pyopencl from:
a. the unofficial windows binaries at http://www.lfd.uci.edu/~gohlke/pythonlibs/#pyopencl
b. by compiling my own after getting the sources from https://pypi.python.org/pypi/pyopencl
The installation was successful on both cases but I get the same error message once I try to import it:
>>> import pyopencl
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python27\lib\site-packages\pyopencl-2015.1-py2.7-win-amd64.egg\pyope
cl\__init__.py", line 30, in <module>
import pyopencl._cl as _cl
ImportError: DLL load failed: The specified procedure could not be found.
>>>
I have Visual C++ Redistributable for Visual Studio 2015 installed from https://www.microsoft.com/en-us/download/details.aspx?id=48145 .
I also tried with 2 different versions of the GPU driver (including latest). Same thing.
A lot of people seem to get the same error and on some forums I read that by updating the GPU drivers to latest, it works fine. But not for me.
Anyone knows how to fix this?

I'm affraid there isn't one right answer to this problem. Each case is different. It depends on what is installed in the OS.
To track down such problems I normally use Dependency Walker.
In this specific case I would open _cl.pyd (usually in C:\Python27\Lib\site-packages\pyopencl) in Dependency Walker to check if there aren't any missing dependencies or if for example OpenCL.dll is actually the one which should be used. OpenCL.dll may be installed by other programs and their path added to PATH. Also OpenCL.dll in System32 may be too old. Basically trial and error renaming all but one OpenCL.dll into OpenCL.dll.bak and/or removing paths from PATH may get you there.

I had this same problem and discovered it was caused by AMD OpenCL.dll not having a function introduced in OpenCL 2.1. The Gohlke site only has OpenCL 2.1 and 1.2, while AMD drivers support 2.0.
Because I wanted 2.0, the easy fix was to manually replace the AMD System32/OpenCL.dll with the one from Intel SDK with experimental 2.1 support.

I had the same problem here, the way I resolved it was:
Make sure you have downloaded and installed the right OpenCL SDK. For example
Intel
NVIDIA
Open the Windows Command Prompt cmd and set the LIB and INCLUDE environment variables. For example
Intel:
set INCLUDE=C:\Program Files (x86)\IntelSWTools\system_studio_2020\OpenCL\sdk\include
set LIB=C:\Program Files (x86)\IntelSWTools\system_studio_2020\OpenCL\sdk\lib\x64
NVIDIA:
set LIB=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v3.2\lib\x64
set INCLUDE=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v3.2\include
now run pip install pyopencl --no-cache-dir
open Python and test import pyopencl
there might be a way to install PyOpenCL via pipwin or by using the --global-option to set the include and library folders. But I haven't succeeded so far.
P.S. The above mentioned NVIDIA OpenCL SDK (i.e., CUDA toolkit) turns out to be very outdated. please don't use it. If you have that installed, please uninstall and install the newer versions.

Try both the versions 1.2 and 2.1 I was trying with later and got this issue. Switched the whl and it works but used the Intel GPU. NVidia OpenCL.dll is 2.0 and that is not working still.
Just checked the cl.get_platforms array and found them
0. Intel
1. NVidia
pyopencl.Platform Intel(R) OpenCL & pyopencl.Device Intel(R) Core(TM) ... Intel(R) OpenCL
pyopencl.Platform NVIDIA CUDA & pyopencl.Device Quadro ... NVIDIA CUDA

I had the same problem in my Lenovo yoga 720. It has NVidia Geforce GTX1050 and intel i7 630 CPU/GPU.
I installed a long time ago update drivers and SDK for Nvidia CUDA. But now I what to run python OpenGL and I install intel SDK also. Pip install pyopencl without problems but import pyopengl give me dll load failure.
Solution was to change Windows\system32\opencl.dll to a new one. The old one was NVidia signed (you can see it in properties of file opencl.dll). The new one is Microsoft signed version 2.1.1.0 Khronos OpenCL ICD
I hope this is useful for you. Solution arrived after a long time trying a lot of things... but nothing worked except the new opencl.dll file

Why doesn't run OpenCL on my GPU (Ubuntu)

I've set up pyopencl on my laptop by getting python-pyopencl from multiverse and installing the amd app sdk. To get the Nvidia ICDs I reinstalled the latest Nvidia driver from the driver manager.
My system is a Thinkpad t540p, i7 4700hq, Nvidia gt 730m, 64bit Ubuntu 14.04
To test the opencl installation I ran this pyopencl example: http://wiki.tiker.net/PyOpenCL/Examples/MatrixMultiply
Unfortunately the performance is very bad: Only 2 GFlop/s. Surely the laptop can do better. So I printed the vendor information. It's "GenuineIntel", apparently the kernel is not run on the GPU, but on the CPU. How can I change that ?
It seems like pyopencl doesn't find the GPU.
for dev in ctx.devices:
print dev.vendor
this returns only "GenuineIntel"
The context is created with:
import pyopencl as cl
ctx=cl.create_some_context()
UPDATE:
This seems to be a duplicate of: ERROR: pyopencl: creating context for specific device

There are two issues here.
First, you should specify GPU as the device to execute the kernel on. Replace:
ctx = cl.create_some_context()
with:
platform = cl.get_platforms()
gpus = platform[0].get_devices(device_type=cl.device_type.GPU)
ctx = cl.Context(devices=gpus)
Second, you appear to have Optimus switchable graphics, so the NVIDIA card is actually left in standby and all the graphic tasks are handled by the CPU for powersaving. You will need to activate the discrete GPU for your program by launching it using Bumblebee:
optirun python yourscript.py