Develop Reference

No module named 'utils_nlp' when using MS nlp_recipes in google colab

I want to use the utils_nlp provided in the nlp_recipes github repo from MS in my google colab project. However, I'm getting a "No module named 'utils_nlp'" error. This is what I have tried:
In the setup from nlp_recipes is stated that:
It is also possible to install directly from Github, which is the best way to utilize the utils_nlp package in external projects (while still reflecting updates to the source as it's installed as an editable '-e' package).
pip install -e git+git#github.com:microsoft/nlp-recipes.git#master#egg=utils_nlp
In colab I run
!pip install -e git+https://github.com/microsoft/nlp-recipes.git#master#egg=utils_nlp
Which works perfectly
Obtaining utils_nlp from git+https://github.com/microsoft/nlp
recipes.git#master#egg=utils_nlp Cloning
https://github.com/microsoft/nlp-recipes.git (to revision master) to ./src/utils-nlp
Running command git clone -q https://github.com/microsoft/nlp-recipes.git /content/src/utils-nlp
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... done
Installing collected packages: utils-nlp
Running setup.py develop for utils-nlp
Successfully installed utils-nlp
When I do !pip list I get
utils-nlp 2.0.0 /content/src/utils-nlp
When I want to import from utils-nlp, for example
from utils_nlp.dataset.preprocess import to_lowercase, to_spacy_tokens
I get a
No module named 'utils_nlp'
I have tried using sys.path.append("/content/src/") and many other paths to append but none of those seem to work.
Any idea?

Restart your runtime after install and prior to import.
Restart command is:
A full worked example is:

ModuleNotFoundError after successful pip install in Google Colaboratory

I'm trying to import a package I wrote into a Google Colaboratory notebook. I uploaded the package contents to my Drive, then ran:
[ ] from google.colab import drive
[ ] drive.mount('content/gdrive/')
[ ] ! pip install --user /content/gdrive/My\ Drive/my-package
Processing ./gdrive/My Drive/my-package
Building wheels for collected packages: my-package
Building wheel for my-package (setup.py) ... done
Created wheel for my-package:
Stored in directory:
Successfully built my-package
Installing collected packages: my-package
Successfully installed my-package-1.0.0.dev1
pip list shows the package has been successfully installed. However, imports of the package fail with a ModuleNotFoundError.
I've successfully pip installed and imported my-package on my local machine. I've also successfully installed and imported another Python package through the same Colab notebook using pip install --user. As mentioned here, I've also tried restarting the kernel.
This may be related to this related but unanswered question.

The github page you linked to about restarting the runtime was a little ambiguous IMO, so I just wanted to clarify:
You need to run the !pip install cell. Then "Restart Runtime." Then run your import statement cell.
I might suggest you "Reset all Runtimes" before performing these steps, just to make sure you have a clean slate.
-- If the above steps are what you already did:
Are you using a Python 2 or 3 notebook? (not 100% sure why that would matter, but more info would be good)
Did you pip install to your local machine from the google drive link? (If not, try to see if that works and report back)

The editable install (or setuptools development-mode) appends the module path to an easy-install.pth file. The site module processes these files when python is started and appends the paths to sys.path. That's why it works only after restarting the runtime.
One can avoid restarting the colab notebook by importing the site module and (re)running site.main().
%pip install -e pkg
import site
site.main()
import pkg
However in the example below this has the effect of removing the current directory from sys.path and replacing it with its absolute path.
%pip install -e "git+https://github.com/jd/tenacity#egg=tenacity"
print("\nTrying to import tenacity")
try:
import tenacity
except ImportError as exc:
print("ImportError")
print(exc)
print()
import sys, site
print("\n##### sys.path original #####")
for p in sys.path:
print(f"'{p}'")
print()
site.main()
print("\n##### sys.path after site.main() #####")
for p in sys.path:
print(f"'{p}'")
print()
import tenacity
print(f"\nImported tenacity from {tenacity.__file__}")
Prints
Obtaining tenacity from git+https://github.com/jd/tenacity#egg=tenacity
Cloning https://github.com/jd/tenacity to ./src/tenacity
Running command git clone -q https://github.com/jd/tenacity /content/src/tenacity
Requirement already satisfied: six>=1.9.0 in /usr/local/lib/python3.6/dist-packages (from tenacity) (1.15.0)
Installing collected packages: tenacity
Running setup.py develop for tenacity
Successfully installed tenacity
Trying to import tenacity
ImportError
No module named 'tenacity'
##### sys.path original #####
''
'/env/python'
'/usr/lib/python36.zip'
'/usr/lib/python3.6'
'/usr/lib/python3.6/lib-dynload'
'/usr/local/lib/python3.6/dist-packages'
'/usr/lib/python3/dist-packages'
'/usr/local/lib/python3.6/dist-packages/IPython/extensions'
'/root/.ipython'
##### sys.path after site.main() #####
'/content'
'/env/python'
'/usr/lib/python36.zip'
'/usr/lib/python3.6'
'/usr/lib/python3.6/lib-dynload'
'/usr/local/lib/python3.6/dist-packages'
'/usr/lib/python3/dist-packages'
'/usr/local/lib/python3.6/dist-packages/IPython/extensions'
'/root/.ipython'
'/content/src/tenacity'
Imported tenacity from /content/src/tenacity/tenacity/__init__.py
Example colab notebook: https://colab.research.google.com/drive/1S5EU-MirhaTWz1JJVdos3GcojmOSLC1C?usp=sharing
Stumbled onto this via the blog post: https://yidatao.github.io/2016-05-10/Python-easyinstall-generates-pth-file/

Thanks to #jojo's suggestion, I uninstalled and reinstalled the package on my local machine and was able to diagnose the problem. On both my local machine and in Colab I was able to successfully install and import the package only when including the -e (editable) flag in the pip command (pip install --user -e /content/gdrive/My\ Drive/my-package), and restarting the runtime after installation). I have no idea why including the -e flag would make a difference; please comment if you have any insight!

Checking if a package is installed specifically via pip

Checking if a particular package is available from within Python can be done via
try:
import requests
except ImportError:
available = False
else:
available = True
Additionally, I would like to know if the respective package has been installed with pip (and can hence been updated with pip install -U package_name).
Any hints?

I believe one way to figure out if a project has been installed by pip is by looking at the content of the INSTALLER text file in the distribution's dist-info directory for this project. With pkg_resources from setuptools this can be done programmatically like the following (error checking omitted):
import pkg_resources
pkg_resources.get_distribution('requests').get_metadata('INSTALLER')
This would return pip\n, in case requests was indeed installed by pip.

You said subprocess.call() is allowed, so
available = not(subprocess.call(["pip", "show", "requests"]))

No module named 'bankdate'(How to import just .py??)

Sorry that I have no idea how to describe this situation. The bigger package I like to install is "finance" (http://pydoc.net/finance/0.2502/finance.bankdate/). I downloaded it and unzipped to install using python setup.py install.
However, I cannot resolve importing another sub-module
bankdate(.py)
When I use finance module, there comes the error message, "ImportError: No module named 'bankdate'.(It is required in "__init__.py" under finance.) bankdate.py seems to be file under finance folder. How could I install "bankdate"?? Does anybody help me with this??
Thank you~!
cf) pip install bankdate, easy_install bankdate don't work in this case.

I do not know if you are working with Linux or Windows. But it would be a good start checking if the package was installed properly. You could use the following code to check installed packages and its versions:
import pip
installed_packages = pip.get_installed_distributions()
installed_packages_list = sorted(["%s==%s" % (i.key, i.version)
for i in installed_packages])
print(installed_packages_list)
By doing so packages installed using both setuptools and pip will appear in a list below. If your package does not appear in the list there you have, it was not installed.
Neverhteless try to import the module usign this:
from finance import bankdate
And see if the error continue. Hope it helps.

you can use pip install finance or you can download .whl file use pip intsall .whl
you can try

How to install zeroRPC (python) on windows

I would like to try zeroRPC but couldn't install the package properly. I am using the latest python_xy distribution (python 2.7.3) under windows 7 and I must say I don't have much experience with installing new modules since the distribution is allready pretty complete.
I pulled the master zeroRPC-python from gitHub and tried to do "python setup.py install"
I had a first problem with something like "impossible to locate vcvarsall.bat". I solved it by installing mingw as explained here error: Unable to find vcvarsall.bat
Then I could run the install untill the end, but now, when I import zerorpc, I get the following ImportError (only the end of the stack):
C:\Python27\lib\site-packages\gevent-0.13.8-py2.7-win32.egg\gevent\greenlet.py in <module>()
4 import traceback
5 from gevent import core
----> 6 from gevent.hub import greenlet, getcurrent, get_hub, GreenletExit, Waiter
7 from gevent.timeout import Timeout
8
C:\Python27\lib\site-packages\gevent-0.13.8-py2.7-win32.egg\gevent\hub.py in <module>()
28
29 try:
---> 30 greenlet = __import__('greenlet').greenlet
31 except ImportError:
32 greenlet = __import_py_magic_greenlet()
ImportError: No module named greenlet
I wonder more generally if I am following the right procedure to install new packages (under windows) or if there is a simpler way (safer with dependancies) that I would be overlooking (easy_install)? I must say I am very new to this and any hints or link to the relevant documentation would be appreciated.
Thanks in advance,
Samuel

I was struggling with this question myself for a while now. The solution involves several components, and many answers out there seem to relate to different versions of those components that don't always play well together.
Here is the complete solution that worked for me, starting from an empty virtualenv:
mkvirtualenv myenv
python -m pip install --upgrade pip==6.0.8 wheel==0.24.0
pip install gevent-1.0.1-cp27-none-win32.whl pyzmq-13.1.0-cp27-none-win32.whl zerorpc==0.4.4
The first step installs wheel and upgrades pip itself to support wheel package installations. The next step installs binary wheels for gevent-1.0.1 (downloadable from this unofficial but extremely useful python windows binaries page) and pyzmq-13.1.0 (available here), and the zerorpc-0.4.4 package from source in the usual way.
Note that I hard-coded source package versions here (pip 6.0.8, wheel 0.24.0, zerorpc 0.4.4) because as I said other versions don't always follow the same build patterns. This may not be necessary and future versions may prove to work just as well together.
The final result for me:
(myenv) C:\work>pip freeze
gevent==1.0.1
greenlet==0.4.5
msgpack-python==0.4.5
pyzmq==13.1.0
wheel==0.24.0
zerorpc==0.4.4

I used a slightly different way, I am using Anaconda + Jupyter to run my python notebooks.
I used this link to zerorpc package, and installed using
conda install -c groakat zerorpc
which installed following -