Jupyter w/ Tensorflow pydotprint not found error - python

I'm using the tensorflow docker image run this way:
docker run -it --rm -v $(realpath ~/Documents/notebooks):/tf/notebooks -p 8888:8888 tensorflow/tensorflow:latest-py3-jupyter
I'm trying to generate dot files / plots of the model I've made, but all of the functions say:
Failed to import pydot. You must install pydot and graphviz for
pydotprint to work.
At the top of my notebook, I have:
!pip install pydot
!pip install graphviz
And I'm using Tensorflow 2.0.0
How do I get this to work? Everything I've seen says to install those modules with pip. I've done that, but it doesn't seem to help.

Use
conda install graphviz
conda install pydot
can solve the problem.
It seems like that graphviz is a system level package which should be installed by conda or apt/yum/brew

Related

How install kmapper?

I have installed kmapper since command prompt, but when I import kmapper in jupyter notebook
is visualized a error: No module named kmapper.
What can I do?
I should install kmapper, please help me.
I don't see a Conda version for this in a major Anaconda Cloud channel. Install with pip.
conda create -n my_kmapper_env python pip numpy matplotlib scikit-learn bokeh pillow
conda activate my_kmapper_env
pip install kmapper

Python Name Error ('XLMProphetNetTokenizer' is not defined)

I'm working on a cheminformatics project that requires quite a few packages to complete. I'm working in Google Colab.
TLDR
Google Colab throws error NameError: name 'XLMProphetNetTokenizer' is not defined even though I explicitly load the relevant package with !pip install transformers.
Long Explanation
To complete my project, I require packages such as RDKit, Moleculekit from Acellera, and the NLP tool allennlp. Google Colab seemed to have issues loading the cheminformatics package RDKit so I had to install using conda. I used !pip install for the rest of the packages. Code as follows:
# Installing RDKit
!wget -c https://repo.continuum.io/miniconda/Miniconda3-py37_4.8.3-Linux-x86_64.sh
!chmod +x Miniconda3-py37_4.8.3-Linux-x86_64.sh
!time bash ./Miniconda3-py37_4.8.3-Linux-x86_64.sh -b -f -p /usr/local
!time conda install -q -y -c conda-forge rdkit
import sys
sys.path.append('/usr/local/lib/python3.7/site-packages/')
!apt-get install python-rdkit librdkit1 rdkit-data -qq
# Installing ProphetNet Packages (Needed for ElmoEmbedder)
!pip install transformers
# Installing Moleculekit
!pip install moleculekit
# Installing AllenNLP
!pip install allennlp
However, when I call from allennlp.commands.elmo import ElmoEmbedder, Google Colab throws the following error: NameError: name 'XLMProphetNetTokenizer' is not defined
Looking on the following link, https://huggingface.co/transformers/master/model_doc/xlmprophetnet.html, it seems that I should be able to load the relavent XLMProphetNetTokenizer by using !pip install transformers. I'm not sure why it still throws the error. Regardless, would appreciate any help.

can't find graphviz module in azure notebook but says it's installed

I installed graphviz in my jupyter notebook (Azure studio).
I know it is installed because when i try to install it again like this:
!{sys.executable} -m pip install graphviz
this is printed out:
Requirement already satisfied: graphviz in /anaconda/envs/azureml_py36/lib/python3.6/site-packages (0.14.1)
However, when I then try to import stuff from graphviz / use it / look for it:
!type graphviz
I get this:
graphviz: not found
I thought it might be a pathing problem but it doesn't seem to be:
print(sys.path) gives:
['/anaconda/envs/azureml_py36/lib/python36.zip',
'/anaconda/envs/azureml_py36/lib/python3.6',
'/anaconda/envs/azureml_py36/lib/python3.6/lib-dynload',
'',
'/anaconda/envs/azureml_py36/lib/python3.6/site-packages',
'/anaconda/envs/azureml_py36/lib/python3.6/site-packages/xgboost-0.90-py3.6.egg',
'/anaconda/envs/azureml_py36/lib/python3.6/site-packages/IPython/extensions',
'/home/azureuser/.ipython',
'/anaconda/envs/azureml_py36/lib/python3.6/site-packages/azureml/_project/vendor']
Could somebody point me in the right direction? I am a little bit lost and have been trying to locate the issues for a while now...
With the command !{sys.executable} -m pip install graphviz you installed the python package graphiz.
You can use it in your python script by following the quickstart guide.
As you're trying the command !type graphviz it seems you want to have the graphviz binary though, not the python package.
The Graphviz download page has install instructions for different operating systems, e.g.:
sudo apt install graphviz
Alternatively you can install Graphviz via conda:
conda install -c anaconda graphviz

How to install mayavi on google Colab?

I tried installing mayavi on Colab using pip:
!pip install mayavi
This threw the following error:
Running setup.py bdist_wheel for mayavi ... error
The rest of the error output is available at the Colab document.
Solution: Work in Progress
Following the response from #Bob-Smith, I found that his solution needed a slight change for installing the dependencies:
!apt-get install vtk6
!apt-get install libvtk6-dev python-vtk6
Problems Faced and Workaround Found (PFWF)
PFWF-001 !apt-get install python-vtk throws the following error:
Package 'python-vtk' has no installation candidate
I found a command-reference for this:
!apt-get install libvtk5-dev python-vtk
However, this command also did not work. The package name had changed from libvtk5-dev to libvtk6-dev and the python binding for VTK has changed from python-vtk to python-vtk6. Clearly this kind of change will continue to happen in future and you may need to check the package name and the python binding for VTK before running the following statement:
!apt-get install libvtk6-dev python-vtk6
Note: If you are here looking to solve VTK installation problems for python and this does not solve that you may want to look here: installing-vtk-for-python
Installing mayavi still throws error:
Although the two steps above install the dependencies, the last line: !pip install mayavi spits out the following error:
Could not connect to any X display.
The latest progress on Mayavi installation can be found here.
https://colab.research.google.com/drive/1K_VIP9izNLKalD_IgBSiTowyNkU7aWcW
You'll first need to install deps. Run
!apt-get install vtk6
!apt-get install python-vtk
!pip install mayavi
If you've attempted to import myavi before installing the deps, you may need to restart you runtime before executing the !pip install myavi command using the Runtime -> Restart runtime menu.
I was trying to do the same thing and I was getting error like this. So I tried install vtk package with conda. You need conda to install vtk of course so:
!wget -c https://repo.anaconda.com/miniconda/Miniconda3-4.5.4-Linux-x86_64.sh
!chmod +x Miniconda3-4.5.4-Linux-x86_64.sh
!bash ./Miniconda3-4.5.4-Linux-x86_64.sh -b -f -p /usr/local
!conda install -q -y --prefix /usr/local python=3.6 ujson
import sys
sys.path.append('/usr/local/lib/python3.6/site-packages')
import ujson
print(ujson.dumps({1:2}))
!conda --version
Then set up vtk package with conda as:
!apt-get install vtk6
!conda install -c anaconda vtk
I was trying to install mayavi for mne package so:
!conda activate mne
!conda install gxx_linux-64=7.3
!pip install https://api.github.com/repos/enthought/mayavi/zipball/226189a6ad3dc3c01d031ef21d0d0cde554ac851
Be careful because you need to mne package to activate so before installing mayavi (I was trying to install mne as I said before):
!pip install mne

Keras: "RuntimeError: Failed to import pydot." after installing graphviz and pydot

I'm using Anaconda Python 2.7 on windows 10
I was planning on doing Keras visualization so (whilst spyder was open) I opened the Anaconda command prompt and pip installed graphviz and pydot. Now when I try run the following:
from keras.models import Sequential
or any sort of "from keras." , I get the error:
ImportError: cannot import name gof
I have uninstalled and reinstalled Keras, Graphviz and pydot. i am using the development version of theano. I cannot find a fix.
P.S
If I uninstall graphviz and pydot, keras works again
EDIT
After uninstalling anaconda and reinstalling it including theano, keras, graphviz and pydot I now get the following error:
from keras.utils.visualize_util import plot
Using Theano backend.
Using gpu device 0: GeForce GTX 970M (CNMeM is disabled, cuDNN not available)
Traceback (most recent call last):
File "<ipython-input-1-65016ddab3cd>", line 1, in <module>
from keras.utils.visualize_util import plot
File "C:\Anaconda2\lib\site-packages\keras\utils\visualize_util.py", line 8, in <module>
raise RuntimeError('Failed to import pydot. You must install pydot'
RuntimeError: Failed to import pydot. You must install pydot and graphviz for `pydotprint` to work.
I used pip install graphviz and pip install git+https://github.com/nlhepler/pydot.git
The error message is a bit misleading, as you can see here. The problem is that graphviz is not installed.
But you mention that graphviz was installed using pip. This is also misleading, since that graphviz package is just a python wrapper, and the graphviz binaries have to be installed separately for the python wrapper to work.
If you are using an Anaconda environment, you'd better install pydotplus and graphviz via conda install.
conda install graphviz
conda install pydotplus
Note:
You'd better update your Keras to the newest version (2.0.9+), it can automatically check and choose which one of pydotplus,pydot-ng,pydot to be used. pydot-ng has been unmaintained for a long time, and it only supports py3.4- and py2.7.
What I did is followed.
import keras
import pydotplus
from keras.utils.vis_utils import model_to_dot
keras.utils.vis_utils.pydot = pydot
plot_model(your_model_name, to_file='model.png')
That's worked for me.
On mac Anaconda python=3.6.8
Keras 2.0.6 looks for pydot-ng (better maintained) and then if it's not found, falls back on pydot. I resolved this issue by installing pydot-ng from source.
I had the same problem. I am using Anaconda python on Ubuntu. but it seems that Keras uses system's python not the Anaconda python. Initially, I installed pydot and graphviz using conda. When I installed pydot and graphviz in system's python(using /usr/bin/pip install pydot) it worked fine.
Install graphviz by brew in osx brew install graphviz, for ubuntu use apt-get install graphviz, do not need to install graphviz by pip.
I had similar problem with my Keras (without anaconda). I have solved my problem using this way
sudo pip install pydot
sudo pip install graphviz
sudo add-apt-repository ppa:gviz-adm/graphviz-dev
sudo apt-get update
sudo apt-get install graphviz-dev
1)Conda install graphviz
2)pip install graphviz
3)pip install pydot
then:
import os
os.environ["PATH"] += os.pathsep + AppData\\Local\\Continuum\\anaconda3\\envs\\tensorflow\\Library\\bin\\graphviz'
For Anaconda on Mac:
To install this package with conda run:
conda install -c anaconda graphviz
Install graphviz to the system. Download the package from here, or on Mac:
brew install graphviz
Install python pydot-ng and graphviz wrapper.
pip install pydot-ng graphviz
conda install -c anaconda pydot-ng #Anaconda user
Use pydot-ng in your code
import pydot_ng as pydot
If Keras visualization utils still uses pydot, try to change import pydot to import pydot_ng as pydot in visualize_util.py
The below works inside a jupyter notebook runing in a jupyter/tensorflow-notebook docker container.
!conda install -y graphviz pydotplus
import pydotplus
import keras.utils
keras.utils.vis_utils.pydot = pydotplus
keras.utils.plot_model(your_model_name, to_file='model.png', show_shapes=True)
You need to tell keras to use pydotplus
it is nothing related to pydot or graphviz if you have installed via pip.
You should go download graphviz brinary and install.
Don't forget adding the bin folder into your PATH: C:/Program Files (x86)/Graphviz2.38/bin/
And after that to close and reopen your console.
After installing pydot and graphviz and adding graphviz to the path, you can restart the IDE or terminal. see here.

Categories

Resources