ImportError: No module named 'graphframes' databricks - python

I am trying to import graphframes in to my databricks notebook
from graphframes import *
but failed with following error message
ImportError: No module named 'graphframes'
How can I add/import in to databricks notebook, any help appreciated.

Note: By default, "graphframes" is not installed on the databricks.
You need to install the package explicitly.
You can install the packages in different methods.
Method1: Installing external packages using pip cmdlet.
Syntax: %sh /databricks/python3/bin/pip install <packagename>
%sh
/databricks/python3/bin/pip install graphframes
Method2: Using Databricks library utilities
Syntax:
dbutils.library.installPyPI("pypipackage", version="version", repo="repo", extras="extras")
dbutils.library.restartPython() # Removes Python state, but some libraries might not work without calling this function
To install graphframes using databricks library utilities use the below command.
dbutils.library.installPyPI("graphframes")
Tried the examples available in this article GraphFrames Documentation.
Notebook output:
Hope this helps.

graphframes is not default dependency with python. You should install this dependency.

You need to install the graphframes module by opening your terminal and typing pip install graphframes

Related

Weird error of ModuleNotFoundError: No module named 'tqdm' when run pip install -e but works when using python setup install

when I git clone this github repo, I am able to install it via python setup.py install. However, the installation fails if tries to use pip install -e . with an error:
ModuleNotFoundError: No module named 'tqdm'
I looked up this issue and ensured tqdm has been installed in the environment.
I also noticed that travis service for this package cannot also pass its build and having the same error regarding tqdm: https://travis-ci.com/github/aristoteleo/dynamo-release/jobs/367588183
see [screenshot of the error message][1]
I managed to fix this issue. The culprit is simply because you cannot specify specific version of tqdm in your setup.py file. Meanwhile, the specification of package requirement between the pyproject.toml and the setup.py need to match up.

try to run airflow on databricks but got error

I am trying to use airflow on databricks.
I have installed apache-airflow 1.10.6 from https://pypi.org/project/apache-airflow/.
I am using python3.6 on databricks.
But, I got error:
import airflow
ModuleNotFoundError: No module named 'werkzeug.wrappers.json'; 'werkzeug.wrappers' is not a package
I have tried the followings:
Apache Airflow : airflow initdb results in "ImportError: No module named json"
Apache Airflow : airflow initdb throws ModuleNotFoundError: No module named 'werkzeug.wrappers.json'; 'werkzeug.wrappers' is not a package error
But, I still got the same problem.
Thanks
Note: By default, "Airflow" and its dependency is not installed on the databricks.
You need to install the package explicitly.
Dependency installation: Using Databricks library utilities.
dbutils.library.installPyPI("Werkzeug")
You can install the packages in different methods.
Method1: Installing external packages using pip cmdlet.
Syntax: %sh /databricks/python3/bin/pip install <packagename>
%sh
/databricks/python3/bin/pip install apache-airflow
Method2: Using Databricks library utilities
Syntax:
dbutils.library.installPyPI("pypipackage", version="version", repo="repo", extras="extras")
dbutils.library.restartPython() # Removes Python state, but some libraries might not work without calling this function
To install apache-airflow using databricks library utilities use the below command.
dbutils.library.installPyPI("apache-airflow")
Method3: GUI Method
Go to Clusters => Select Cluster => Libraries => Install New => Library Source "PyPI" => Package "apache-airflow" => Install
Hope this helps. Do let us know if you any further queries.
Do click on "Mark as Answer" and Upvote on the post that helps you, this can be beneficial to other community members.

No Module named "sklearn"

I am trying to import 'sklearn' using Python 3.4.3 on a Raspberry Pi 3 running Raspian.
I downloaded microconda3, which includes all the necessary modules to use scikit.
However, when I attempt to import 'sklearn' in IDLE, I receive an error stating that there is no module named 'sklearn'.
Try starting fresh and reinstall all of the necessary modules through miniconda3. Maybe the scikit-learn included on that install did not work.
You can then try
sudo apt-get install python3-scipy python3-sklearn
You can install the module with the following command using pip:
pip install -U scikit-learn.
See: Scikit-learn Docs

Why ImportError: No module named lightgbm

My OS is Ubuntu, and I've followed the official installation guide to install lightgbm. However, when I import it, this error is raised:
ImportError: No module named lightgbm
How can I solve this?
Do I also need to go to /python-package folder to run setup.py after running those linux commandlines?
I had the same problem, and solved running the installation directly on the notebook
!pip install lightgbm
Besides running those linux command lines. I also need to go to /python-package then run 'python setup.py install'.
conda install -c conda-forge lightgbm solved the problem for me
you need just run in a notebook cell before importing
For Windows users, VC runtime <https://go.microsoft.com/fwlink/?LinkId=746572> is needed if Visual Studio (2015 or 2017) is not installed.
Install wheel <http://pythonwheels.com> via pip install wheel first. After that download the wheel file and install from it:
pip install lightgbm
The following should do the trick:
export PATH=/usr/local/bin/anaconda3/bin${PATH:+:${PATH}}
PYTHONPATH=$PYTHONPATH:/usr/local/bin/anaconda3/lib/python3.6/site-packages
Note that you might need to change the paths if you are using a different Python version.
Thank you for above question and answers, had a similar issue.
Problem:
After successfull install of lightgbm, I was getting the error ImportError: No module named 'lightgbm' (in Jupyter Notebook on Google Cloud's Notebook Instance in AI Platform project).
Issue:
Realized that the install of lightgbm was in Python 2.7 even when the notebook was running in Python 3 (path: './.local/lib/python2.7/site-packages').
Solution:
The error was gone after the Jupyter Notebook was set to run on Python 2 instead of Python 3.
within Jupyter Notebook cell is: try running
import sys
!{sys.executable} -m pip install lightgbm
With python try, from pypi.org, this line
pip install lightgbm
or
pip3 install lightgbm
Also, you can try this one if you use anaconda
conda install lightgbm

jupyter notebook - ImportError: No module named 'bson'

Trying to play with some jupyter notebook, I have an issue with the bson module which seams not available as reported in the error: "ImportError: No module named 'bson'"
I'm running jupyter notebook 4.3.0 over a python 2.7 kernel which is available in a dedicated virtual environment. I tried to install manually the bson module through a simple "conda install bson" but it's not in the official/standard repo so I had to install OpenMDAO/bson. Finally, it didn't change the issue. I understand that the module might come from the pymongo package so I tried to install with the same result.
Any hint here? What's the link between bson and pymongo?
You can advise this answer: import error: no module named bson
Which states that:
If you don't get the no module named bson error but the EPOCH_AWARE import error, this is due to a name clash between bson and pymongo.
In this case, you should do these in order:
sudo pip uninstall bson
sudo pip uninstall pymongo
sudo pip install pymongo
In your case, try to start with a fresh virtualenv and do the above mentioned in the given order
That also answers the link between bson and pymongo.

Categories

Resources