The import works when I call it from the command line, but when I call the import in the ipynb I get the following error:
ImportError: libmkl_intel_lp64.so: cannot open shared object file: No such file or directory
I have already added the path to libmkl_intel_lp64.so to my bashrc file, so I am not sure what else I can do to fix this import error in the notebook.
Your command line python interpreter,
and your jupyter notebook kernel,
have different environments.
Use one of these techniques to help you debug the path problem:
$ python -m site (from bash prompt)
import sys; from pprint import pp; pp(sys.path) (within a cell)
Do yourself a favor, and use conda to manage your environment.
https://docs.conda.io/en/latest/miniconda.html
Be sure to conda activate myfavoriteenvironment
before you start the jupyter kernel.
Folks, I am having this puzzle with a pybind11 library mytest.cp37-win_amd64.pyd put in C:\Temp.
Then I have this:
import sys
sys.path.insert(0, r"C:\Temp")
from mytest import *
Now here comes the problem, if I launch the python in command line:
>python
>>>import sys
>>>sys.path.insert(0, r"C:\Temp")
>>>from mytest import *
It works fine. Or I just put above code in test.py, then run:
>python test.py
It also works. If I put this piece of code in Spyder, it works as well. But if I put this piece of code in Jupyter, it will not work by saying:
ModuleNotFoundError: No module named 'mytest'
I am sure all my tests are conducted in the same python environment by printing it out:
import os
print(os.environ['CONDA_DEFAULT_ENV'])
Am I missing anything here?
With Wayne's help and a previous post, I finally found out the root cause, that is the version of the python running in the Jupyter kernel is different from that of the environment from which the Jupyter is launch and also the pybind11 library is build with.
After I reinstall the Jupyter in the environment again, the library is picked up successfully.
I am trying to set up IBPY and Python on my Mac to run with Interactive Brokers. I have installed Git. I created a subdirectory ibapi under Home. I downloaded IBPy using git clone https://github.com/blampe/IbPy from the ibapi directory.
I am now trying to run the Demo.py program in Spyder. When I choose Run, I receive the error message:
ImportError: No module named Ib.Message
The first few lines of the demo program are:
import os
import sys
import time
import Ib.Message
import Ib.Socket
import Ib.Type
I am also trying to run a sample program: ib_api_demo from http://www.quantstart.com/articles/Using-Python-IBPy-and-the-Interactive-Brokers-API-to-Automate-Trades. When I try to run this, I get the error message:
ImportError: No module named ib.ext.Contract
The first few lines are:
from ib.ext.Contract import Contract
from ib.ext.Order import Order
from ib.opt import Connection, message
I figured this out. The problem was I was launching Spyder from my Mac Finder. When I did this, I received the error messages. When I launched Spyder (actually Anaconda Python) by typing "Spyder" in the Terminal window, this launched Anaconda Python. From here, I could run all my programs successfully with no errors.
I want to check if a module exists, if it doesn't I want to install it.
How should I do this?
So far I have this code which correctly prints f if the module doesn't exist.
try:
import keyring
except ImportError:
print 'f'
import pip
def import_or_install(package):
try:
__import__(package)
except ImportError:
pip.main(['install', package])
This code simply attempt to import a package, where package is of type str, and if it is unable to, calls pip and attempt to install it from there.
Here is how it should be done, and if I am wrong, please correct me. However, Noufal seems to confirm it in another answer to this question, so I guess it's right.
When writing the setup.py script for some scripts I wrote, I was dependent on the package manager of my distribution to install the required library for me.
So, in my setup.py file, I did this:
package = 'package_name'
try:
return __import__(package)
except ImportError:
return None
So if package_name was installed, fine, continue. Else, install it via the package manager which I called using subprocess.
This approach of dynamic import work really well in cases you just want to print a message if module is not installed. Automatically installing a module SHOULDN'T be done like issuing pip via subprocess. That's why we have setuptools (or distribute).
We have some great tutorials on packaging, and the task of dependencies detection/installation is as simple as providing install_requires=[ 'FancyDependency', 'otherFancy>=1.0' ]. That's just it!
But, if you really NEED to do by hand, you can use setuptools to help you.
from pkg_resources import WorkingSet , DistributionNotFound
working_set = WorkingSet()
# Printing all installed modules
print tuple(working_set)
# Detecting if module is installed
try:
dep = working_set.require('paramiko>=1.0')
except DistributionNotFound:
pass
# Installing it (anyone knows a better way?)
from setuptools.command.easy_install import main as install
install(['django>=1.2'])
NOTE: Ipython / Jupyter specific solution.
While using notebooks / online kernels, I usually do it using systems call.
try:
import keyring
except:
!pip install keyring
import keyring
P.S. One may wish to call conda install or mamba install instead.
You can use os.system as follows:
import os
package = "package_name"
try:
__import__package
except:
os.system("pip install "+ package)
You can launch pip install %s"%keyring in the except part to do this but I don't recommend it. The correct way is to package your application using distutils so that when it's installed, dependencies will be pulled in.
Not all modules can be installed so easily. Not all of them have easy-install support, some can only be installed by building them.. others require some non-python prerequisites, like gcc, which makes things even more complicated (and forget about it working well on Windows).
So I would say you could probably make it work for some predetermined modules, but there's no chance it'll be something generic that works for any module.
I made an import_neccessary_modules() function to fix this common issue.
# ======================================================================================
# == Fix any missing Module, that need to be installed with PIP.exe. [Windows System] ==
# ======================================================================================
import importlib, os
def import_neccessary_modules(modname:str)->None:
'''
Import a Module,
and if that fails, try to use the Command Window PIP.exe to install it,
if that fails, because PIP in not in the Path,
try find the location of PIP.exe and again attempt to install from the Command Window.
'''
try:
# If Module it is already installed, try to Import it
importlib.import_module(modname)
print(f"Importing {modname}")
except ImportError:
# Error if Module is not installed Yet, the '\033[93m' is just code to print in certain colors
print(f"\033[93mSince you don't have the Python Module [{modname}] installed!")
print("I will need to install it using Python's PIP.exe command.\033[0m")
if os.system('PIP --version') == 0:
# No error from running PIP in the Command Window, therefor PIP.exe is in the %PATH%
os.system(f'PIP install {modname}')
else:
# Error, PIP.exe is NOT in the Path!! So I'll try to find it.
pip_location_attempt_1 = sys.executable.replace("python.exe", "") + "pip.exe"
pip_location_attempt_2 = sys.executable.replace("python.exe", "") + "scripts\pip.exe"
if os.path.exists(pip_location_attempt_1):
# The Attempt #1 File exists!!!
os.system(pip_location_attempt_1 + " install " + modname)
elif os.path.exists(pip_location_attempt_2):
# The Attempt #2 File exists!!!
os.system(pip_location_attempt_2 + " install " + modname)
else:
# Neither Attempts found the PIP.exe file, So i Fail...
print(f"\033[91mAbort!!! I can't find PIP.exe program!")
print(f"You'll need to manually install the Module: {modname} in order for this program to work.")
print(f"Find the PIP.exe file on your computer and in the CMD Command window...")
print(f" in that directory, type PIP.exe install {modname}\033[0m")
exit()
import_neccessary_modules('art')
import_neccessary_modules('pyperclip')
import_neccessary_modules('winsound')
Here is my approach. The idea is loop until python has already installed all modules by built in module as "pip" .
import pip
while True:
try:
#import your modules here. !
import seaborn
import bokeh
break
except ImportError as err_mdl:
print((err_mdl.name))
pip.main(['install', err_mdl.name])
I tried installing transformers using the below method and it worked fine. Similarly, you can just replace your library name instead of "transformers".
import pip
try:
from transformers import pipeline
except ModuleNotFoundError:
pip.main(['install', "transformers"])
from transformers import pipeline
I tried this in a new virtual envoirnment with no packages installed and it installed the necessary package i.e. opencv-python
Example is given below
import os
try:
import cv2
except ImportError:
os.system('pip install opencv-python')
I need to import the multiprocessing module in Python 2.5.
I've followed the instructions here exactly: http://code.google.com/p/python-multiprocessing/wiki/Install
make and make test run without errors. I've also edited $PYTHONPATH to include the directory where the package is installed.
But 'import multiprocessing' still says: "ImportError: no module named multiprocessing".
What am I doing wrong? Is there some step missing from these instructions? I haven't installed a Python module before.
Navigate to the directory containing the package then type:
python setup.py install
This info was contained in the INSTALL.txt file.
http://code.google.com/p/python-multiprocessing/source/browse/trunk/INSTALL.txt
perhaps you can try:
import sys
sys.path.append('/path/to/processingdotpylibs/')
import processing