Develop Reference

How to specify what directory to import a package from in Python?

I want to install updated packages in another directory and have Python grab the updated package instead of the older package.
I'm trying to find a way that I can specify which directory to import for when there are multiple identical packages in sys.path.
I started by running this code to make sure that the path for the second module is present:
import sys
print('\n'.join(sys.path))
Both paths are shown, so I know that Python could find the package from either location.
I run this to see what path Python is using:
import statsmodels
print(statsmodels.__file__)
It is using the path of the out of date version.
I've been looking into using importlib but I haven't figured out how to make that work.
I'm just looking for a way to import a package from a specified path, even when the package exists in another directory in sys.path.

as discussed in the commend you'd neeed to implement this solution. With that to further explain what it does, it points to another folder to consider files to import. Considering the mentioned code:
# some_file.py (this is this script you're running)
import sys
sys.path.insert(0, '/path/to/application/app/folder')
import file_name_inside_the_folder_above
You'd let the first argument 0 untouched and just edit the second argument, pointing to which folder the script you have access is. Then you just import as the it's file name.

Importing self-made package

I am testing my own package, but I am struggling to import it. My package file structure is as follows:
(You can also alternatively view my Github repository here)
In PyAdventures is my init.py with the content of
name="pyadventures"
So my question is, when I import it in another python file, how come it doesn't work?
I run:
import pyadventures
But I get the following error:
No module named pyadventures
It'd be great if you could answer my question!
It's important to note that the package is in my Python package directory, not the test one I showed
New discovery! In my PyAdventures folder (the one Python actually uses) the folder only has a few files, not the ones in the screenshot above.
You can install this with pip install pyadventures

Ah, like others remark in comments and answer: The camelcase might be the problem. (Why not name it just all in lower case: pyadventures? - Else users will be as confused as its developer now :D .)
Before, I thought, it might be a problem that you want to use your module without having installed it (locally). And my following answer is for this case (using a not installed package from a local folder):
In the docs you can read:
The variable sys.path is a list of strings that determines the
interpreter’s search path for modules. It is initialized to a default
path taken from the environment variable PYTHONPATH, or from a
built-in default if PYTHONPATH is not set. You can modify it using
standard list operations:
import sys
sys.path.append('/ufs/guido/lib/python')
thus, before import do:
import sys
sys.path.append('/absolute/or/relative/path/to/your/module/pyadventures')
# Now, your module is "visible" for the module loader
import pyadventures
Alternatively, you could just place your pyadventures module folder locally in your working directory where you start python and then just do
import pyadventures
However, it is much easier to manage to keep only one version in one place and refer to this from other places/scripts. (Else you have multiple copies, and have difficulties to track changes on the multiple copies).
As far as I know, your __init__.py doesn't need to contain anything. It works if it is empty. It is there just to mark your directory as a module.

Simple. Even though the package listed on pip is namd pyadventures, in your code the directory is called PyAdventures. So that's what python knows it as. I ran import PyAdventures and it worked fine.

Importing a module in resources/ into an XBMC plugin

I'm working on an XBMC plugin which requires a few Python modules not available via a requires tag in addon.xml (they are not in the XBMC repository as far as I'm aware). The documentation for plugin development seems to indicate that you can do this by adding the module(s) to the resources/lib/ subdirectory of your plugin directory.
I've done this and when testing it in XBMC I get an import error trying to import this module because it cannot be found.
I've read the other question I found on SO regarding this topic entitled Importing a python module into XBMC for use within an addon but the proposed solution there, of adding the module directory to the path before importing, doesn't work for me either. I get the same import error.
In fact I don't think that answer is correct because os.getcwd() in XBMC does not return your plugin directory path when called from within your plugin; so concatenating the path it gives with /resources/lib as the answer suggests won't yield a valid path. I modified the example to use getAddonInfo to find the plugin path from an Addon object via the xbmcaddon module and added that to the path concatenated with /resources/lib but it still did not work.
Putting the modules into the root of the plugin directory also doesn't work. I haven't been able to find specific documentation about how to do this correctly beyond the initial tutorial saying that I should add it to the resources/lib subdirectory.
So does anyone know how to do this or have an example of this being done successfully in another XBMC plugin?

Figured out my mistake. I wasn't paying attention to the path I was adding. I was adding the addon profile directory to sys.path using getAddonInfo('profile') when I should have been using getAddonInfo('path')
For future reference, if you want to add a subdirectory of your addon to the path, this is what I did:
import xbmcaddon
import os
...
my_addon = xbmcaddon.Addon('plugin.video.my_plugin')
addon_dir = xbmc.translatePath( my_addon.getAddonInfo('path') )
sys.path.append(os.path.join( addon_dir, 'resources', 'lib' ) )
import <whatever modules you want>
I guess that's yet another lesson in paying close attention to the content of an error message.

py2exe cannot import Module from other directory

I am bundling python source code with py2exe. The directory structure is as follows:
some_Mod.py
some_dir/another_dir/some_Mod.py
Inside the latter some_dir/another_dir/some_Mod.py I am trying to import the other Python Module with
from ..some_Mod import *
Using the import causes no problems with the python interpreter, but if I run the same constellation in the bundled package, I get an Exception:
ImportError: No module named some_Mod
Can somebody explain why?
Remark: Renaming the Modules is actually no problem, but I was just wondering, why py2exe cannot deal with this constellation.

If you have __init__.py files in each of those sub-directories then all import statements should work correctly.
Assuming that's not the problem, here's an excellent guide to importing best practices:
http://blog.habnab.it/blog/2013/07/21/python-packages-and-you/
In summary, never use relative imports - always absolute (see the link above for why).
Second (and I'm not entirely sure why), always keep your py2exe setup.py script in the exact folder where your main script is. I've tried modifying py2exe's 'script' option to allow my script to be somewhere else... but your exact problem happened to me. So, try making sure it is right where the main script is.
Finally, you can always give py2exe a little help. I usually have to add the root directory to the system path so the import statements are valid. Note, I'm not modifying sys.path in any of my application's code - only the py2exe script I use to build the exe.
At the top of my py2exe setup script:
import sys
sys.path.append(PATH_WHERE_PACKAGES_ARE)
# add any packages that need explicit importing here located in root directory:
import package1 # apparently it wasn't found...
import package2 # apparently same thing
Generally I don't import packages though, adding the project root where they exist usually is enough.

I'm not sure that py2exe now how to handle the from ..some_Mod import * syntax, check this to ensure that the some_Mod.py module is correctly packaged : python -m py2exe.mf -d some_dir/another_dir/some_Mod.py as explained in the py2exe FAQ

python module dlls

Is there a way to make a python module load a dll in my application directory rather than the version that came with the python installation, without making changes to the python installation (which would then require I made an installer, and be careful I didn't break other apps for people by overwrting python modules and changing dll versions globaly...)?
Specifically I would like python to use my version of the sqlite3.dll, rather than the version that came with python (which is older and doesn't appear to have the fts3 module).

If you're talking about Python module DLLs, then simply modifying sys.path should be fine. However, if you're talking about DLLs linked against those DLLs; i.e. a libfoo.dll which a foo.pyd depends on, then you need to modify your PATH environment variable. I wrote about doing this for PyGTK a while ago, but in your case I think it should be as simple as:
import os
os.environ['PATH'] = 'my-app-dir' + os.pathsep + os.environ['PATH']
That will insert my-app-dir at the head of your Windows path, which I believe also controls the load-order for DLLs.
Keep in mind that you will need to do this before loading the DLL in question, i.e., before importing anything interesting.
sqlite3 may be a bit of a special case, though, since it is distributed with Python; it's obviously kind of tricky to test this quickly, so I haven't checked sqlite3.dll specifically.

The answer with modifying os.environ['PATH'] is right but it didn't work for me because I use python 3.9.
Still I was getting an error:
ImportError: DLL load failed while importing module X: The specified
module could not be found.
Turned out since version python 3.8 they added a mechanism to do this more securely.
Read documentation on os.add_dll_directory https://docs.python.org/3/library/os.html#os.add_dll_directory
Specifically see python 3.8 what's new:
DLL dependencies for extension modules and DLLs loaded with ctypes on Windows are now resolved more securely. Only the system paths, the directory containing the DLL or PYD file, and directories added with add_dll_directory() are searched for load-time dependencies. Specifically, PATH and the current working directory are no longer used, and modifications to these will no longer have any effect on normal DLL resolution. If your application relies on these mechanisms, you should check for add_dll_directory() and if it exists, use it to add your DLLs directory while loading your library.
So now this is the new way to make it work in python 3.8 and later:
import os
os.add_dll_directory('my-app-dir')
Again, the old way is still correct and you will have to use it in python 3.7 and older:
import os
os.environ['PATH'] = 'my-app-dir' + os.pathsep + os.environ['PATH']
After that my module with a dll dependency has been successfully loaded.

Ok it turns out python always loads the dll in the same directory as the pyd file, regardless of what the python and os paths are set to.
So I needed to copy the _sqlite3.pyd from python/v2.5/DLLS to my apps directory where the new sqlite3.dll is, making it load my new dll, rather than the one that comes with python (since the pyd files seem to follow the PYTHONPATH, even though the actual dlls themselves don't).

If your version of sqlite is in sys.path before the systems version it will use that. So you can either put it in the current directory or change the PYTHONPATH environment variable to do that.

I had the same issue as administrative rights to the default python library is blocked in a corporate environment and its extremely troublesome to perform installations.
What works for me:
Duplicate the sqlite3 library in a new location
Put in the latest sqlite3.dll (version you want from sqlite3 web) and the old _sqlite3.pyd into the new location or the new sqlite3 library. The old _sqlite3.pyd can be found in the default python library lib/DLLs folder.
Go to the new sqlite3 library and amend the dbapi2.py as follows:
Change "from _sqlite3 import *" to "from sqlite3._sqlite3 import *"
Make sure python loads this new sqlite3 library first. Add the path to this library if you must.

I encountered the same problem for my .pyd that depends on cuda/cudnn/tensorrt.
I came with a little function I call before importing my module.
def add_cuda_to_path():
if os.name != "nt":
return
path = os.getenv("PATH")
if not path:
return
path_split = path.split(";")
for folder in path_split:
if "cuda" in folder.lower() or "tensorrt" in folder.lower():
os.add_dll_directory(folder)
It is the easiest workaround I found, so I don't need to hardcode any path.
I guess little improvement would to actually check for .dll file, but this snippet serves well my needs.
Then you can simply:
add_cuda_to_path()
import my_module_that_depends_on_cuda