python search paths order on Debian - python

I have two google.protobuf modules on my Debian(stretch).
/usr/local/lib/python2.7/dist-packages/google
/home/myuser/.local/lib/python2.7/site-packages/google (installed with pip --user)
I'd like to import 2, but python always gives me 1, while ipython imports 2. I've tried set PYTHONPATH such that /home/myuser/.local/lib/python2.7/site-packages/ is the first in it, but not working.
Is there any way I can force python to search my $HOME/.local/lib/python2.7/site-packages/ first?

Yes. See here for an official description of how python determines which module to import first: https://docs.python.org/2/tutorial/modules.html#the-module-search-pathkk
See here for a way to change that default behavior: http://www.hasenkopf2000.net/wiki/python/how-override-pythons-module-import-order/
Of the two solutions provided on the hasenkopf site, the second will be less problematic if you change your mind on which module you want to use. You just edit the file rather than having to remember which symbolic links you created. Briefly, the code is:
import sys
# Assume path to module is
# /path/to/recent/version/of/module.py
sys.path.insert(0,"/path/to/recent/version/of")
import module
You place it at the top of your script.

Related

Python: After separating out modules to their own package, local imports failing

I have a project which worked perfectly fine, but I wanted to separate some of the code into a separate project (with its own packages). I have successfully done that, but now in that newly separated package I need to explicitly import local modules using from . import <module> instead of import <module> - the latter of which had worked before separating the modules into their own package.
The new imports (where I include the dot representing the local dir) works but it seems like it should also just work without having to specify the module locations since the files are in the same directory as the module that is trying to import them. Am I missing something here?
Artificial example:
|-root
|--ProjectA
|---app.py
|-root
|--ProjectB
|---ProjectB
|----__init__.py
|----libA.py
|----libB.py
In app.py I am able to import a module from ProjectB (I have adjusted PYTHONPATH to include /root/ProjectB). So the following works just fine as I would expect:
from ProjectB import libA.py
But the entire thing fails because in libA.py the following does not work:
import libB
Instead I have to use:
from . import libB
I have tried to wrap my head around how python imports work across packages and I thought I had sorted it in my head, but apparently I am missing something. Unless this is the way it is supposed to work? (packages need to explicitly indicate that a module is local). But I thought python would always implicitly add the path of the current module that is doing the importing.
Any advice (or even just links to tutorials that explain this - I have watched a lot but they all tend to focus on different aspects of importing than this) is greatly appreciated.

How to specify what directory to import a package from in Python?

I want to install updated packages in another directory and have Python grab the updated package instead of the older package.
I'm trying to find a way that I can specify which directory to import for when there are multiple identical packages in sys.path.
I started by running this code to make sure that the path for the second module is present:
import sys
print('\n'.join(sys.path))
Both paths are shown, so I know that Python could find the package from either location.
I run this to see what path Python is using:
import statsmodels
print(statsmodels.__file__)
It is using the path of the out of date version.
I've been looking into using importlib but I haven't figured out how to make that work.
I'm just looking for a way to import a package from a specified path, even when the package exists in another directory in sys.path.
as discussed in the commend you'd neeed to implement this solution. With that to further explain what it does, it points to another folder to consider files to import. Considering the mentioned code:
# some_file.py (this is this script you're running)
import sys
sys.path.insert(0, '/path/to/application/app/folder')
import file_name_inside_the_folder_above
You'd let the first argument 0 untouched and just edit the second argument, pointing to which folder the script you have access is. Then you just import as the it's file name.

Importing self-made package

I am testing my own package, but I am struggling to import it. My package file structure is as follows:
(You can also alternatively view my Github repository here)
In PyAdventures is my init.py with the content of
name="pyadventures"
So my question is, when I import it in another python file, how come it doesn't work?
I run:
import pyadventures
But I get the following error:
No module named pyadventures
It'd be great if you could answer my question!
It's important to note that the package is in my Python package directory, not the test one I showed
New discovery! In my PyAdventures folder (the one Python actually uses) the folder only has a few files, not the ones in the screenshot above.
You can install this with pip install pyadventures
Ah, like others remark in comments and answer: The camelcase might be the problem. (Why not name it just all in lower case: pyadventures? - Else users will be as confused as its developer now :D .)
Before, I thought, it might be a problem that you want to use your module without having installed it (locally). And my following answer is for this case (using a not installed package from a local folder):
In the docs you can read:
The variable sys.path is a list of strings that determines the
interpreter’s search path for modules. It is initialized to a default
path taken from the environment variable PYTHONPATH, or from a
built-in default if PYTHONPATH is not set. You can modify it using
standard list operations:
import sys
sys.path.append('/ufs/guido/lib/python')
thus, before import do:
import sys
sys.path.append('/absolute/or/relative/path/to/your/module/pyadventures')
# Now, your module is "visible" for the module loader
import pyadventures
Alternatively, you could just place your pyadventures module folder locally in your working directory where you start python and then just do
import pyadventures
However, it is much easier to manage to keep only one version in one place and refer to this from other places/scripts. (Else you have multiple copies, and have difficulties to track changes on the multiple copies).
As far as I know, your __init__.py doesn't need to contain anything. It works if it is empty. It is there just to mark your directory as a module.
Simple. Even though the package listed on pip is namd pyadventures, in your code the directory is called PyAdventures. So that's what python knows it as. I ran import PyAdventures and it worked fine.

how python import packages with multiple copies

I was believing that when importing a package, it would search from sys.path and use the first hit for import. However, it seems not true:
import mpl_toolkits
print(mpl_toolkits.__path__)
And it outputs:
['/Library/Python/2.7/site-packages/matplotlib-1.5.0-py2.7-macosx-10.11-intel.egg/mpl_toolkits', '/usr/local/lib/python2.7/site-packages/mpl_toolkits']
Can someone please explain to me how exactly python looks for packages if it is installed multiple times in the machine (in different location searchable by sys.path)? Or a pointer to relevant reference would be good.
when you import a module, python uses PYTHON PATH (system variable that contains a list of folders) and loops to search for importable module.
Python will test if it is a package (folder containing init.py) or a module (*.py). it will stop on first module found if no module is found python raises an import error

Specify the developement version of a python module

I want to add a new class to PICOS, a python module. I installed it the normal way a long time ago. But now I have downloaded the source and I a am trying to make some changes.
The problem is that I cannot manage to ask python to load the module from the development folder and not the normal folder.
reload(picos.constraint)
Out[22]: <module 'picos.constraint' from '/home/optimi/bzffourn/python/lib/python2.7/site-packages/picos/constraint.pyc'>
while the source code are here:
/home/optimi/bzffourn/ZIB/python_scripts/pyMathProg/picos
So the changes I make are not considered.
This should help you do it: Override Python import order.
Just change your import to this:
import sys
sys.path.insert(0,"/home/optimi/bzffourn/ZIB/python_scripts/pyMathProg/picos")
import picos.constraint

Categories

Resources