I want to run Tensorflow GPU in Pycharm on Linux Mint.
I tried some guides like these
https://medium.com/#p.venkata.kishore/install-anaconda-tenserflow-gpu-keras-and-pycharm-on-windows-10-6bfb39630e4e
https://medium.com/#kekayan/step-by-step-guide-to-install-tensorflow-gpu-on-ubuntu-18-04-lts-6feceb0df5c0
I run this code
import tensorflow as tf
sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))
I have got this error
Traceback (most recent call last):
File "/home/alex/anaconda3/envs/TfTestGPU/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow.py", line 58, in <module>
from tensorflow.python.pywrap_tensorflow_internal import *
File "/home/alex/anaconda3/envs/TfTestGPU/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 28, in <module>
_pywrap_tensorflow_internal = swig_import_helper()
File "/home/alex/anaconda3/envs/TfTestGPU/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 24, in swig_import_helper
_mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
File "/home/alex/anaconda3/envs/TfTestGPU/lib/python3.6/imp.py", line 243, in load_module
return load_dynamic(name, filename, file)
File "/home/alex/anaconda3/envs/TfTestGPU/lib/python3.6/imp.py", line 343, in load_dynamic
return _load(spec)
ImportError: libcublas.so.10.0: cannot open shared object file: No such file or directory
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/alex/PycharmProjects/TfTestGPU/test.py", line 1, in <module>
import tensorflow as tf
File "/home/alex/anaconda3/envs/TfTestGPU/lib/python3.6/site-packages/tensorflow/__init__.py", line 24, in <module>
from tensorflow.python import pywrap_tensorflow # pylint: disable=unused-import
File "/home/alex/anaconda3/envs/TfTestGPU/lib/python3.6/site-packages/tensorflow/python/__init__.py", line 49, in <module>
from tensorflow.python import pywrap_tensorflow
File "/home/alex/anaconda3/envs/TfTestGPU/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow.py", line 74, in <module>
raise ImportError(msg)
ImportError: Traceback (most recent call last):
File "/home/alex/anaconda3/envs/TfTestGPU/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow.py", line 58, in <module>
from tensorflow.python.pywrap_tensorflow_internal import *
File "/home/alex/anaconda3/envs/TfTestGPU/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 28, in <module>
_pywrap_tensorflow_internal = swig_import_helper()
File "/home/alex/anaconda3/envs/TfTestGPU/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 24, in swig_import_helper
_mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
File "/home/alex/anaconda3/envs/TfTestGPU/lib/python3.6/imp.py", line 243, in load_module
return load_dynamic(name, filename, file)
File "/home/alex/anaconda3/envs/TfTestGPU/lib/python3.6/imp.py", line 343, in load_dynamic
return _load(spec)
ImportError: libcublas.so.10.0: cannot open shared object file: No such file or directory
Failed to load the native TensorFlow runtime.
See https://www.tensorflow.org/install/errors
for some common reasons and solutions. Include the entire stack trace
above this error message when asking for help.
Process finished with exit code 1
First Make sure CUDA and CuDNN has been installed successfully and Configuration should be verified.
CUDA driver version should be sufficient for CUDA runtime version.
Once done,
Open PyCharm
Goto File->Settings-> Project Interpreter
Select the appropriate Environment which has tensorflow-gpu installed
Select Run->Edit Configuration->Environment Variables
Since the code is searching for libcublas.so.10.0 ,
Assume like path you find "libcublas.so.10.0" is something like "/home/Alex/anaconda3/pkgs/cudatoolkit-10.0.130-0/lib/"
Add the lib path as LD_LIBRARY_PATH in environment variables as
Name : LD_LIBRARY_PATH
Value : /home/Alex/anaconda3/pkgs/cudatoolkit-10.0.130-0/lib/
Save it and then try to import tensorflow
It's kind of confusing in your installment, Do you install the tensorflow in your pycharm or anaconda, because you use pycharm, but there are some error about the anaconda.
Check the environment variable configuration, both for Linux
and pycharm. Be careful the cuda-x in the path. x is the version cuda such as 10.0
Check the versions of the tensorflow, cuda, cudnn, according
to this site.
Make sure you can find the libcublas.so.10.0 in this folder
/usr/local/cuda-10.0/lib64. If not, you should reinstall the cudnn.
Sometimes I encounter this problem, but after restarting the
computer, it works.
I followed this guide with no need for Anaconda.
https://www.youtube.com/watch?v=IubEtS2JAiY
I installed MS Visual Studio, CUDA and Cudnn and pycharm started recognicing the GPUs. Run speed tests on CPU and GPU with faster results on my available GPU.
Be very carefull with the Tensorflow/CUDA/Cudnn versions.
Related
I installed anaconda 3-5.2 from the anaconda install archive and I can't seem to get tensorflow working. When I try to import it this is the error message I get. I recently updated brew and deleted anaconda and reinstalled it also.
ImportError: Traceback (most recent call last):
File "/Users/admin/.local/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow.py", line 41, in <module>
from tensorflow.python.pywrap_tensorflow_internal import *
File "/Users/admin/.local/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 28, in <module>
_pywrap_tensorflow_internal = swig_import_helper()
File "/Users/admin/.local/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 24, in swig_import_helper
_mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
File "/Users/admin/usr/local/opt/anaconda3/lib/python3.6/imp.py", line 243, in load_module
return load_dynamic(name, filename, file)
File "/Users/admin/usr/local/opt/anaconda3/lib/python3.6/imp.py", line 343, in load_dynamic
return _load(spec)
ImportError: dlopen(/Users/admin/.local/lib/python3.6/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so, 10): Library not loaded: #rpath/libcublas.8.0.dylib
Referenced from: /Users/admin/.local/lib/python3.6/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so
Reason: image not found
Failed to load the native TensorFlow runtime.
See https://www.tensorflow.org/install/install_sources#common_installation_problems
for some common reasons and solutions. Include the entire stack trace
above this error message when asking for help.
You need to install using conda install tensorflow so that the cuda and cudnn libraries are install.
I am trying to run Gpt2 on my machine, i am following this guide : https://lambdalabs.com/blog/run-openais-new-gpt-2-text-generator-code-with-your-gpu/
I created a new environment on Conda to make sure it would be isolated and all, and downloaded the correct versions of the libs.
My environment is :
Windows 10
Python 3.6
tensorflow-gpu 1.12 (installed with pip)
CUDA v9, v10, v11
cuDNN for v9 and for v11 (i first installe dht elatest, but then i was told tensorflow was only happy with version 9 so i installed it)
Whenever i try to import tensorflow, either with from tensorflow import * or import tensorflow as tf
I get :
Traceback (most recent call last):
File "C:\Users\theo1\Anaconda3\envs\gpt2\lib\site-packages\tensorflow\python\pywrap_tensorflow.py", line 58, in <module>
from tensorflow.python.pywrap_tensorflow_internal import *
File "C:\Users\theo1\Anaconda3\envs\gpt2\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py", line 28, in <module>
_pywrap_tensorflow_internal = swig_import_helper()
File "C:\Users\theo1\Anaconda3\envs\gpt2\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py", line 24, in swig_import_helper
_mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
File "C:\Users\theo1\Anaconda3\envs\gpt2\lib\imp.py", line 243, in load_module
return load_dynamic(name, filename, file)
File "C:\Users\theo1\Anaconda3\envs\gpt2\lib\imp.py", line 343, in load_dynamic
return _load(spec)
ImportError: DLL load failed: The specified module could not be found.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\theo1\Anaconda3\envs\gpt2\lib\site-packages\tensorflow\__init__.py", line 24, in <module>
from tensorflow.python import pywrap_tensorflow # pylint: disable=unused-import
File "C:\Users\theo1\Anaconda3\envs\gpt2\lib\site-packages\tensorflow\python\__init__.py", line 49, in <module>
from tensorflow.python import pywrap_tensorflow
File "C:\Users\theo1\Anaconda3\envs\gpt2\lib\site-packages\tensorflow\python\pywrap_tensorflow.py", line 74, in <module>
raise ImportError(msg)
ImportError: Traceback (most recent call last):
File "C:\Users\theo1\Anaconda3\envs\gpt2\lib\site-packages\tensorflow\python\pywrap_tensorflow.py", line 58, in <module>
from tensorflow.python.pywrap_tensorflow_internal import *
File "C:\Users\theo1\Anaconda3\envs\gpt2\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py", line 28, in <module>
_pywrap_tensorflow_internal = swig_import_helper()
File "C:\Users\theo1\Anaconda3\envs\gpt2\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py", line 24, in swig_import_helper
_mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
File "C:\Users\theo1\Anaconda3\envs\gpt2\lib\imp.py", line 243, in load_module
return load_dynamic(name, filename, file)
File "C:\Users\theo1\Anaconda3\envs\gpt2\lib\imp.py", line 343, in load_dynamic
return _load(spec)
ImportError: DLL load failed: The specified module could not be found.
Failed to load the native TensorFlow runtime.
See https://www.tensorflow.org/install/errors
for some common reasons and solutions. Include the entire stack trace
above this error message when asking for help.
This error isn't helping at all, and i'm stuck there. Any ideas ?
Thank you.
EDIT :
For those running in the same problem, follow the github link, install everything with conda. I also ran into this problem : Tensorflow has no Attribute "sort" in GPT 2 Git Release? that was solved as well by udating tensorflow. Thanks everyone
Please try to install tensorflow with conda with below command.
conda install -c conda-forge tensorflow //tensorflow package
or for specific version
conda install -c conda-forge tensorflow=1.13.1
"tensorflow-cpu" works fine on my device with windows 10.
I was trying to install tensorflow-gpu and after installing it fails to import showing the following error.
tensorflow.python.pywrap_tensorflow_internal no module
I have installed the following
CUDA 10.1
Nvidia graphics driver Version 419
CUDnn for CUDA 10.1
VC++ 2015 REDIST
Could anyone say what am i missing here is complete error message.
Traceback (most recent call last):
File "D:\Program Files\Python\lib\site-packages\tensorflow\python\pywrap_tensorflow.py", line 58, in <module>
from tensorflow.python.pywrap_tensorflow_internal import *
File "D:\Program Files\Python\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py", line 28, in <module>
_pywrap_tensorflow_internal = swig_import_helper()
File "D:\Program Files\Python\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py", line 24, in swig_import_helper
_mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
File "D:\Program Files\Python\lib\imp.py", line 243, in load_module
return load_dynamic(name, filename, file)
File "D:\Program Files\Python\lib\imp.py", line 343, in load_dynamic
return _load(spec)
ImportError: DLL load failed: The specified module could not be found.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "D:\Program Files\Python\lib\site-packages\tensorflow\__init__.py", line 24, in <module>
from tensorflow.python import pywrap_tensorflow # pylint: disable=unused-import
File "D:\Program Files\Python\lib\site-packages\tensorflow\python\__init__.py", line 49, in <module>
from tensorflow.python import pywrap_tensorflow
File "D:\Program Files\Python\lib\site-packages\tensorflow\python\pywrap_tensorflow.py", line 74, in <module>
raise ImportError(msg)
ImportError: Traceback (most recent call last):
File "D:\Program Files\Python\lib\site-packages\tensorflow\python\pywrap_tensorflow.py", line 58, in <module>
from tensorflow.python.pywrap_tensorflow_internal import *
File "D:\Program Files\Python\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py", line 28, in <module>
_pywrap_tensorflow_internal = swig_import_helper()
File "D:\Program Files\Python\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py", line 24, in swig_import_helper
_mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
File "D:\Program Files\Python\lib\imp.py", line 243, in load_module
return load_dynamic(name, filename, file)
File "D:\Program Files\Python\lib\imp.py", line 343, in load_dynamic
return _load(spec)
ImportError: DLL load failed: The specified module could not be found.
If your are sure you have Visual C++ 2015 Redist installed on your Windows machine then probably it is the version conflict between CUDA version and the Tensorflow version. Because newer versions have some conflicts and more over tensorflow requires the exact version of CUDA and the CUDnn Library for it to work properly.
The best way to get it configured correctly is using Anaconda for installing and configuring tensorflow-gpu as conda will automatically download all the necessary versions of CUDA and CUDnn for the corresponding tensorflow
The problem is that the new Anaconda uses Python 3.7.x by default which is not supported by tensorflow or tensorflow-gpu
So install Anaconda You can download it here
Now create a virtual environment using conda with python 3.6.x configured by typing the following in Anaconda Prompt
conda create -n yourenvname python=3.6 anaconda
Now change switch to your environment by typing in
conda activate yourenvname
Now Check the python version by using which should show you something like this
> python --version
Python 3.6.7 :: Anaconda custom (64-bit)
Now finally install tensorflow-gpu
conda install tensorflow-gpu
Now open python and try importing tensorflow
>>> import tensorflow as tf
>>> sess = tf.Session(config=tf.ConfigProto(log_device_placement=True))
Congrats ! You are all set.
My purpose is setting up virtual environment to run tensorflow benchmark but I am getting an error at last statement after running following code. May I ask what I should do to resolve this issue? FYI: I am using Python 3.6.7 with only pip and virtualenv packages and CUDA 9.2/cuDNN7 installed (all dlls in system path).
virtualenv venv
venv\scripts\activate
#now you should be in virtual environment
#install using pip
pip install tf-nightly-gpu
python
import tensorflow
Error message:
>>> import tensorflow
Traceback (most recent call last):
File "C:\MyWork\testarea\testvirtualenv\venv\lib\site-packages\tensorflow\python\pywrap_tensorflow.py", line 58, in <module>
from tensorflow.python.pywrap_tensorflow_internal import *
File "C:\MyWork\testarea\testvirtualenv\venv\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py", line 28, in <module>
_pywrap_tensorflow_internal = swig_import_helper()
File "C:\MyWork\testarea\testvirtualenv\venv\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py", line 24, in swig_import_helper
_mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
File "C:\MyWork\testarea\testvirtualenv\venv\lib\imp.py", line 243, in load_module
return load_dynamic(name, filename, file)
File "C:\MyWork\testarea\testvirtualenv\venv\lib\imp.py", line 343, in load_dynamic
return _load(spec)
ImportError: DLL load failed: The specified module could not be found.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\MyWork\testarea\testvirtualenv\venv\lib\site-packages\tensorflow\__init__.py", line 24, in <module>
from tensorflow.python import pywrap_tensorflow # pylint: disable=unused-import
File "C:\MyWork\testarea\testvirtualenv\venv\lib\site-packages\tensorflow\python\__init__.py", line 49, in <module>
from tensorflow.python import pywrap_tensorflow
File "C:\MyWork\testarea\testvirtualenv\venv\lib\site-packages\tensorflow\python\pywrap_tensorflow.py", line 74, in <module>
raise ImportError(msg)
ImportError: Traceback (most recent call last):
File "C:\MyWork\testarea\testvirtualenv\venv\lib\site-packages\tensorflow\python\pywrap_tensorflow.py", line 58, in <module>
from tensorflow.python.pywrap_tensorflow_internal import *
File "C:\MyWork\testarea\testvirtualenv\venv\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py", line 28, in <module>
_pywrap_tensorflow_internal = swig_import_helper()
File "C:\MyWork\testarea\testvirtualenv\venv\lib\site-packages\tensorflow\python\pywrap_tensorflow_internal.py", line 24, in swig_import_helper
_mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
File "C:\MyWork\testarea\testvirtualenv\venv\lib\imp.py", line 243, in load_module
return load_dynamic(name, filename, file)
File "C:\MyWork\testarea\testvirtualenv\venv\lib\imp.py", line 343, in load_dynamic
return _load(spec)
ImportError: DLL load failed: The specified module could not be found.
Failed to load the native TensorFlow runtime.
See https://www.tensorflow.org/install/errors
for some common reasons and solutions. Include the entire stack trace
above this error message when asking for help.
>
Found an answer just after writing the question.
Downgrading CUDA 9.2 to 9.0 before copying cudnn64_7.dll (cuDNN 7.3.1 for CUDA 9.0) to CUDA bin directory made it work.
In the end, following script produced two different results from two different machines.
virtualenv venv
venv\scripts\activate
pip install tf-nightly-gpu
python tf_cnn_benchmarks.py --num_gpus=1 --batch_size=32 --model=resnet50 --variable_update=parameter_server
7th gen quad-core i7 with GTX 970
total images/sec: 78.16
AMD 1920X with GTX 1080
total images/sec: 147.77
I am trying to install tensorflow 1.10 on my old mac, but I run into the same problem every time. As soon as I start the python shell and I do get the error below.
I did try to install it in a virtualenv first, after that, I tried to install it just using pip and got the same error. Also when I tried to install it with conda, the same issue again. I googled and looked that up here, but couldn't solve it yet, maybe due to my noobish acting. I did uninstall anaconda and pip already and tried again, but I always run into the same error.
I do have CUDA installed, although I tried to install tensorflow with and without GPU.
The error results in: Failed to load the native TensorFlow runtime.
I hope someone is able to help, as I would really like to learn this.
(tensorflow) abc:~ me$ python
import tensorflow as tf
Traceback (most recent call last):
File "/Users/me/tensorflow/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow.py", line 58, in <module>
from tensorflow.python.pywrap_tensorflow_internal import *
File "/Users/me/tensorflow/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 28, in <module>
_pywrap_tensorflow_internal = swig_import_helper()
File "/Users/me/tensorflow/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 24, in swig_import_helper
_mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
File "/Users/me/tensorflow/lib/python3.6/imp.py", line 243, in load_module
return load_dynamic(name, filename, file)
File "/Users/me/tensorflow/lib/python3.6/imp.py", line 343, in load_dynamic
return _load(spec)
ImportError: dlopen(/Users/me/tensorflow/lib/python3.6/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so, 6): Symbol not found: _SecKeyCopyExternalRepresentation
Referenced from: /Users/me/tensorflow/lib/python3.6/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so (which was built for Mac OS X 10.12)
Expected in: /System/Library/Frameworks/Security.framework/Versions/A/Security
in /Users/me/tensorflow/lib/python3.6/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so
During handling of the above exception, another exception occurred:
>Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/me/tensorflow/lib/python3.6/site-packages/tensorflow/__init__.py", line 22, in <module>
from tensorflow.python import pywrap_tensorflow # pylint: disable=unused-import
File "/Users/me/tensorflow/lib/python3.6/site-packages/tensorflow/python/__init__.py", line 49, in <module>
from tensorflow.python import pywrap_tensorflow
File "/Users/me/tensorflow/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow.py", line 74, in <module>
raise ImportError(msg)
ImportError: Traceback (most recent call last):
File "/Users/me/tensorflow/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow.py", line 58, in <module>
from tensorflow.python.pywrap_tensorflow_internal import *
File "/Users/me/tensorflow/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 28, in <module>
_pywrap_tensorflow_internal = swig_import_helper()
File "/Users/me/tensorflow/lib/python3.6/site-packages/tensorflow/python/pywrap_tensorflow_internal.py", line 24, in swig_import_helper
_mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
File "/Users/me/tensorflow/lib/python3.6/imp.py", line 243, in load_module
return load_dynamic(name, filename, file)
File "/Users/me/tensorflow/lib/python3.6/imp.py", line 343, in load_dynamic
return _load(spec)
ImportError: dlopen(/Users/me/tensorflow/lib/python3.6/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so, 6): Symbol not found: _SecKeyCopyExternalRepresentation
Referenced from: /Users/me/tensorflow/lib/python3.6/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so (which was built for Mac OS X 10.12)
Expected in: /System/Library/Frameworks/Security.framework/Versions/A/Security
in /Users/me/tensorflow/lib/python3.6/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so
Failed to load the native TensorFlow runtime.
See https://www.tensorflow.org/install/install_sources#common_installation_problems
For some common reasons and solutions. Include the entire stack trace
above this error message when asking for help.
I think there is something wrong with TensorFlow 1.10 and OS X El Capitan (10.11.6)!
Just install 1.9.0 instead of 1.10 with the pip command:
pip3 install tensorflow==1.9.0
or its GPU version:
pip3 install tensorflow-gpu==1.9.0
It seems to be a version conflict. I can succesfully run Tensorflow 1.9.