Develop Reference

C++ python module (based on Pybind11) import error: ModuleNotFoundError

A C++ python module based on pybind11 library cannot be imported anymore in python. It was working until a few weeks ago but not any more (may be since the installation of miniconda). Cannot track the exact point when this stopped working as I was not using it since many weeks. I start python in the same directory as the module, and tried to import it in the terminal. And I get the error:
ModuleNotFoundError: No module named 'ld_pybind_d'
In the mean while I also tried:
deleted the directory where miniconda was installed, rebuilt the module and linked it against python3.6m library.
Created an empty __ init__.py file in the module directory
Exported the current working directory in the PYTHONPATH environment variable
Other info:
Built 64 bit version module and I also have 64 bit version python
Python version 3.6.8
Nothing works .. Your help is very appreciated ...

As is, many times the case, if nothing really works, a restart would definitely do. I did that and seems to solve the case. Never the less, im just relieved it is loading now.

How do I get IDLE to find NLTK?

Programming noob here. I'm on Mac OS 10.5.8. I have Python 2.7.6 and have installed NLTK. If I run Python from Terminal, I can "import nltk" with no problem. But if I open IDLE (either from Terminal or by double-clicking on the application) and try the same thing there, I get an error message, "ImportError: No module named nltk". I assume this is a path problem, but what exactly should I do?
The directory where I installed NLTK is "My Documents/Python/nltk-2.0.4". But within this there are various other directories called build, dist, etc. Which of these is the exact directory that IDLE needs to be able to find? And how do I add that directory to IDLE's path?

Supplementing the answer above, when you install python packages they will install under the default version of python you are using. Since the module imports in python 2.7.6 make sure that you aren't using the Python 3 version of IDLE.

Please go through the link given below:
setting-up-nltk-with-python-idle-on-os-x-10-6
HTH! Thanks!

PyInstaller - ImportError: No module named win32api

I'm trying to compile a Python project (using Python 2.7 on Windows XP) into an EXE using PyInstaller with the default options. When I try to run the EXE, I get the message:
PyInstaller - ImportError: No module named win32api
I added the win32api path to the windows PATH environment variable (I do have Python Win32 Extensions installed) but it's not working. I'm pretty new to this and a little overwhelmed by all the options etc, and I really have no idea where to start (or what information would be useful to solving this problem.) I assume this is some little thing that I'm missing, but I haven't found anyone having precisely this problem online and any help would be greatly appreciated.

Ok, it looks like "import os" in one of my modules was causing this issue. I had no luck getting it to successfully use the win32api module, but since this was only being used to set the program name, I just commented this out and this particular issue is resolved. Thanks Luke for your help!

Python: import _io

I'm trying to determine which files in the Python library are strictly necessary for my script to run. Right now I'm trying to determine where _io.py is located. In io.py (no underscore), the _io.py module (with underscore) is imported on line 60.

Some modules are compiled directly into the interpreter -- there are no files corresponding to them. You can retrieve a list of these modules from sys.builtin_module_names. In my Pyton 3.1 installation, _io is included in this list.
You might want to have a look at snakefood to determine the dependencies of your script.

Not all Python modules are written in Python. Try looking for _io.so or _io.pyd.

Try the DLLs folder under your base python install directory if you are on windows. It contains .pyd modules Ignacio mentions. I had a similar problem with a portable install. Including the DLLs folder contents to my install fixed it. I am using python 2.5.

From python-list email archive: is "_io.py" missing from 2.7.4 ?, the situation for Python 2 and 3 is different:
In Python 2.7:
To find where the _io module lives, at the interactive interpreter run
this:
import _io
_io.__file__
Under Linux, you should get something like this:
'/usr/local/lib/python2.7/lib-dynload/_io.so'
and the equivalent under Windows.
In Python 3:
Note that in Python 3.3, the _io module is now built-in into the
compiler, so _io.__file__ no longer exists.

python module dlls

Is there a way to make a python module load a dll in my application directory rather than the version that came with the python installation, without making changes to the python installation (which would then require I made an installer, and be careful I didn't break other apps for people by overwrting python modules and changing dll versions globaly...)?
Specifically I would like python to use my version of the sqlite3.dll, rather than the version that came with python (which is older and doesn't appear to have the fts3 module).

If you're talking about Python module DLLs, then simply modifying sys.path should be fine. However, if you're talking about DLLs linked against those DLLs; i.e. a libfoo.dll which a foo.pyd depends on, then you need to modify your PATH environment variable. I wrote about doing this for PyGTK a while ago, but in your case I think it should be as simple as:
import os
os.environ['PATH'] = 'my-app-dir' + os.pathsep + os.environ['PATH']
That will insert my-app-dir at the head of your Windows path, which I believe also controls the load-order for DLLs.
Keep in mind that you will need to do this before loading the DLL in question, i.e., before importing anything interesting.
sqlite3 may be a bit of a special case, though, since it is distributed with Python; it's obviously kind of tricky to test this quickly, so I haven't checked sqlite3.dll specifically.

The answer with modifying os.environ['PATH'] is right but it didn't work for me because I use python 3.9.
Still I was getting an error:
ImportError: DLL load failed while importing module X: The specified
module could not be found.
Turned out since version python 3.8 they added a mechanism to do this more securely.
Read documentation on os.add_dll_directory https://docs.python.org/3/library/os.html#os.add_dll_directory
Specifically see python 3.8 what's new:
DLL dependencies for extension modules and DLLs loaded with ctypes on Windows are now resolved more securely. Only the system paths, the directory containing the DLL or PYD file, and directories added with add_dll_directory() are searched for load-time dependencies. Specifically, PATH and the current working directory are no longer used, and modifications to these will no longer have any effect on normal DLL resolution. If your application relies on these mechanisms, you should check for add_dll_directory() and if it exists, use it to add your DLLs directory while loading your library.
So now this is the new way to make it work in python 3.8 and later:
import os
os.add_dll_directory('my-app-dir')
Again, the old way is still correct and you will have to use it in python 3.7 and older:
import os
os.environ['PATH'] = 'my-app-dir' + os.pathsep + os.environ['PATH']
After that my module with a dll dependency has been successfully loaded.

Ok it turns out python always loads the dll in the same directory as the pyd file, regardless of what the python and os paths are set to.
So I needed to copy the _sqlite3.pyd from python/v2.5/DLLS to my apps directory where the new sqlite3.dll is, making it load my new dll, rather than the one that comes with python (since the pyd files seem to follow the PYTHONPATH, even though the actual dlls themselves don't).

If your version of sqlite is in sys.path before the systems version it will use that. So you can either put it in the current directory or change the PYTHONPATH environment variable to do that.

I had the same issue as administrative rights to the default python library is blocked in a corporate environment and its extremely troublesome to perform installations.
What works for me:
Duplicate the sqlite3 library in a new location
Put in the latest sqlite3.dll (version you want from sqlite3 web) and the old _sqlite3.pyd into the new location or the new sqlite3 library. The old _sqlite3.pyd can be found in the default python library lib/DLLs folder.
Go to the new sqlite3 library and amend the dbapi2.py as follows:
Change "from _sqlite3 import *" to "from sqlite3._sqlite3 import *"
Make sure python loads this new sqlite3 library first. Add the path to this library if you must.

I encountered the same problem for my .pyd that depends on cuda/cudnn/tensorrt.
I came with a little function I call before importing my module.
def add_cuda_to_path():
if os.name != "nt":
return
path = os.getenv("PATH")
if not path:
return
path_split = path.split(";")
for folder in path_split:
if "cuda" in folder.lower() or "tensorrt" in folder.lower():
os.add_dll_directory(folder)
It is the easiest workaround I found, so I don't need to hardcode any path.
I guess little improvement would to actually check for .dll file, but this snippet serves well my needs.
Then you can simply:
add_cuda_to_path()
import my_module_that_depends_on_cuda