How to do relative imports in Python? - python

Imagine this directory structure:
app/
__init__.py
sub1/
__init__.py
mod1.py
sub2/
__init__.py
mod2.py
I'm coding mod1, and I need to import something from mod2. How should I do it?
I tried from ..sub2 import mod2 but I'm getting an "Attempted relative import in non-package".
I googled around but found only "sys.path manipulation" hacks. Isn't there a clean way?
Edit: all my __init__.py's are currently empty
Edit2: I'm trying to do this because sub2 contains classes that are shared across sub packages (sub1, subX, etc.).
Edit3: The behaviour I'm looking for is the same as described in PEP 366 (thanks John B)

Everyone seems to want to tell you what you should be doing rather than just answering the question.
The problem is that you're running the module as '__main__' by passing the mod1.py as an argument to the interpreter.
From PEP 328:
Relative imports use a module's __name__ attribute to determine that module's position in the package hierarchy. If the module's name does not contain any package information (e.g. it is set to '__main__') then relative imports are resolved as if the module were a top level module, regardless of where the module is actually located on the file system.
In Python 2.6, they're adding the ability to reference modules relative to the main module. PEP 366 describes the change.
Update: According to Nick Coghlan, the recommended alternative is to run the module inside the package using the -m switch.

Here is the solution which works for me:
I do the relative imports as from ..sub2 import mod2
and then, if I want to run mod1.py then I go to the parent directory of app and run the module using the python -m switch as python -m app.sub1.mod1.
The real reason why this problem occurs with relative imports, is that relative imports works by taking the __name__ property of the module. If the module is being directly run, then __name__ is set to __main__ and it doesn't contain any information about package structure. And, thats why python complains about the relative import in non-package error.
So, by using the -m switch you provide the package structure information to python, through which it can resolve the relative imports successfully.
I have encountered this problem many times while doing relative imports. And, after reading all the previous answers, I was still not able to figure out how to solve it, in a clean way, without needing to put boilerplate code in all files. (Though some of the comments were really helpful, thanks to #ncoghlan and #XiongChiamiov)
Hope this helps someone who is fighting with relative imports problem, because going through PEP is really not fun.

main.py
setup.py
app/ ->
__init__.py
package_a/ ->
__init__.py
module_a.py
package_b/ ->
__init__.py
module_b.py
You run python main.py.
main.py does: import app.package_a.module_a
module_a.py does import app.package_b.module_b
Alternatively 2 or 3 could use: from app.package_a import module_a
That will work as long as you have app in your PYTHONPATH. main.py could be anywhere then.
So you write a setup.py to copy (install) the whole app package and subpackages to the target system's python folders, and main.py to target system's script folders.

"Guido views running scripts within a package as an anti-pattern" (rejected
PEP-3122)
I have spent so much time trying to find a solution, reading related posts here on Stack Overflow and saying to myself "there must be a better way!". Looks like there is not.

This is solved 100%:
app/
main.py
settings/
local_setings.py
Import settings/local_setting.py in app/main.py:
main.py:
import sys
sys.path.insert(0, "../settings")
try:
from local_settings import *
except ImportError:
print('No Import')

explanation of nosklo's answer with examples
note: all __init__.py files are empty.
main.py
app/ ->
__init__.py
package_a/ ->
__init__.py
fun_a.py
package_b/ ->
__init__.py
fun_b.py
app/package_a/fun_a.py
def print_a():
print 'This is a function in dir package_a'
app/package_b/fun_b.py
from app.package_a.fun_a import print_a
def print_b():
print 'This is a function in dir package_b'
print 'going to call a function in dir package_a'
print '-'*30
print_a()
main.py
from app.package_b import fun_b
fun_b.print_b()
if you run $ python main.py it returns:
This is a function in dir package_b
going to call a function in dir package_a
------------------------------
This is a function in dir package_a
main.py does: from app.package_b import fun_b
fun_b.py does from app.package_a.fun_a import print_a
so file in folder package_b used file in folder package_a, which is what you want. Right??

def import_path(fullpath):
"""
Import a file with full path specification. Allows one to
import from anywhere, something __import__ does not do.
"""
path, filename = os.path.split(fullpath)
filename, ext = os.path.splitext(filename)
sys.path.append(path)
module = __import__(filename)
reload(module) # Might be out of date
del sys.path[-1]
return module
I'm using this snippet to import modules from paths, hope that helps

This is unfortunately a sys.path hack, but it works quite well.
I encountered this problem with another layer: I already had a module of the specified name, but it was the wrong module.
what I wanted to do was the following (the module I was working from was module3):
mymodule\
__init__.py
mymodule1\
__init__.py
mymodule1_1
mymodule2\
__init__.py
mymodule2_1
import mymodule.mymodule1.mymodule1_1
Note that I have already installed mymodule, but in my installation I do not have "mymodule1"
and I would get an ImportError because it was trying to import from my installed modules.
I tried to do a sys.path.append, and that didn't work. What did work was a sys.path.insert
if __name__ == '__main__':
sys.path.insert(0, '../..')
So kind of a hack, but got it all to work!
So keep in mind, if you want your decision to override other paths then you need to use sys.path.insert(0, pathname) to get it to work! This was a very frustrating sticking point for me, allot of people say to use the "append" function to sys.path, but that doesn't work if you already have a module defined (I find it very strange behavior)

Let me just put this here for my own reference. I know that it is not good Python code, but I needed a script for a project I was working on and I wanted to put the script in a scripts directory.
import os.path
import sys
sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), "..")))

As #EvgeniSergeev says in the comments to the OP, you can import code from a .py file at an arbitrary location with:
import imp
foo = imp.load_source('module.name', '/path/to/file.py')
foo.MyClass()
This is taken from this SO answer.

Take a look at http://docs.python.org/whatsnew/2.5.html#pep-328-absolute-and-relative-imports. You could do
from .mod1 import stuff

From Python doc,
In Python 2.5, you can switch import‘s behaviour to absolute imports using a from __future__ import absolute_import directive. This absolute- import behaviour will become the default in a future version (probably Python 2.7). Once absolute imports are the default, import string will always find the standard library’s version. It’s suggested that users should begin using absolute imports as much as possible, so it’s preferable to begin writing from pkg import string in your code

I found it's more easy to set "PYTHONPATH" enviroment variable to the top folder:
bash$ export PYTHONPATH=/PATH/TO/APP
then:
import sub1.func1
#...more import
of course, PYTHONPATH is "global", but it didn't raise trouble for me yet.

On top of what John B said, it seems like setting the __package__ variable should help, instead of changing __main__ which could screw up other things. But as far as I could test, it doesn't completely work as it should.
I have the same problem and neither PEP 328 or 366 solve the problem completely, as both, by the end of the day, need the head of the package to be included in sys.path, as far as I could understand.
I should also mention that I did not find how to format the string that should go into those variables. Is it "package_head.subfolder.module_name" or what?

You have to append the module’s path to PYTHONPATH:
export PYTHONPATH="${PYTHONPATH}:/path/to/your/module/"

A hacky way to do it is to append the current directory to the PATH at runtime as follows:
import pathlib
import sys
sys.path.append(pathlib.Path(__file__).parent.resolve())
import file_to_import # the actual intended import
In contrast to another solution for this question this uses pathlib instead of os.path.

This method queries and auto populates the path:
import os
import inspect
currentdir = os.path.dirname(os.path.abspath(inspect.getfile(inspect.currentframe())))
parentdir = os.path.dirname(currentdir)
os.sys.path.insert(1, parentdir)
# print("currentdir = ", currentdir)
# print("parentdir=", parentdir)

What a debate!
Relative newcomer to python (but years of programming experience, and dislike of perl). Relative lay-person when it comes to the dark art of Apache setup, but I know what I (think I) need to get my little experimental projects working at home.
Here is my summary of what the situ seems to be.
If I use the -m 'module' approach, I need to:-
dot it all together;
run it from a parent folder;
lose the '.py';
create an empty (!) __init__.py file in every sub-folder.
How does that work in a cgi environment, where I have aliased my scripts directory, and want to run a script directly as /dirAlias/cgi_script.py??
Why is amending sys.path a hack? The python docs page states: "A program is free to modify this list for its own purposes." If it works, it works, right? The bean counters in Accounts don't care how it works.
I just want to go up one level and down into a 'modules' dir:-
.../py
/cgi
/build
/modules
so my 'modules' can be imported from either the cgi world or the server world.
I've tried the -m/modules approach but I think I prefer the following (and am not confused how to run it in cgi-space):-
Create XX_pathsetup.py in the /path/to/python/Lib dir (or any other dir in the default sys.path list). 'XX' is some identifier that declares an intent to setup my path according to the rules in the file.
In any script that wants to be able to import from the 'modules' dir in above directory config, simply import XX_pathsetup.py.
And here's my really simple XX_pathsetup.py:
import sys, os
pypath = sys.path[0].rsplit(os.sep,1)[0]
sys.path.insert( 0, pypath+os.sep+'modules' )
Not a 'hack', IMHO. 1 small file to put in the python 'Lib' dir, one import statement which declares intent to modify the path search order.

Related

Attempted relative import in non-package for optional package [duplicate]

Imagine this directory structure:
app/
__init__.py
sub1/
__init__.py
mod1.py
sub2/
__init__.py
mod2.py
I'm coding mod1, and I need to import something from mod2. How should I do it?
I tried from ..sub2 import mod2 but I'm getting an "Attempted relative import in non-package".
I googled around but found only "sys.path manipulation" hacks. Isn't there a clean way?
Edit: all my __init__.py's are currently empty
Edit2: I'm trying to do this because sub2 contains classes that are shared across sub packages (sub1, subX, etc.).
Edit3: The behaviour I'm looking for is the same as described in PEP 366 (thanks John B)
Everyone seems to want to tell you what you should be doing rather than just answering the question.
The problem is that you're running the module as '__main__' by passing the mod1.py as an argument to the interpreter.
From PEP 328:
Relative imports use a module's __name__ attribute to determine that module's position in the package hierarchy. If the module's name does not contain any package information (e.g. it is set to '__main__') then relative imports are resolved as if the module were a top level module, regardless of where the module is actually located on the file system.
In Python 2.6, they're adding the ability to reference modules relative to the main module. PEP 366 describes the change.
Update: According to Nick Coghlan, the recommended alternative is to run the module inside the package using the -m switch.
Here is the solution which works for me:
I do the relative imports as from ..sub2 import mod2
and then, if I want to run mod1.py then I go to the parent directory of app and run the module using the python -m switch as python -m app.sub1.mod1.
The real reason why this problem occurs with relative imports, is that relative imports works by taking the __name__ property of the module. If the module is being directly run, then __name__ is set to __main__ and it doesn't contain any information about package structure. And, thats why python complains about the relative import in non-package error.
So, by using the -m switch you provide the package structure information to python, through which it can resolve the relative imports successfully.
I have encountered this problem many times while doing relative imports. And, after reading all the previous answers, I was still not able to figure out how to solve it, in a clean way, without needing to put boilerplate code in all files. (Though some of the comments were really helpful, thanks to #ncoghlan and #XiongChiamiov)
Hope this helps someone who is fighting with relative imports problem, because going through PEP is really not fun.
main.py
setup.py
app/ ->
__init__.py
package_a/ ->
__init__.py
module_a.py
package_b/ ->
__init__.py
module_b.py
You run python main.py.
main.py does: import app.package_a.module_a
module_a.py does import app.package_b.module_b
Alternatively 2 or 3 could use: from app.package_a import module_a
That will work as long as you have app in your PYTHONPATH. main.py could be anywhere then.
So you write a setup.py to copy (install) the whole app package and subpackages to the target system's python folders, and main.py to target system's script folders.
"Guido views running scripts within a package as an anti-pattern" (rejected
PEP-3122)
I have spent so much time trying to find a solution, reading related posts here on Stack Overflow and saying to myself "there must be a better way!". Looks like there is not.
This is solved 100%:
app/
main.py
settings/
local_setings.py
Import settings/local_setting.py in app/main.py:
main.py:
import sys
sys.path.insert(0, "../settings")
try:
from local_settings import *
except ImportError:
print('No Import')
explanation of nosklo's answer with examples
note: all __init__.py files are empty.
main.py
app/ ->
__init__.py
package_a/ ->
__init__.py
fun_a.py
package_b/ ->
__init__.py
fun_b.py
app/package_a/fun_a.py
def print_a():
print 'This is a function in dir package_a'
app/package_b/fun_b.py
from app.package_a.fun_a import print_a
def print_b():
print 'This is a function in dir package_b'
print 'going to call a function in dir package_a'
print '-'*30
print_a()
main.py
from app.package_b import fun_b
fun_b.print_b()
if you run $ python main.py it returns:
This is a function in dir package_b
going to call a function in dir package_a
------------------------------
This is a function in dir package_a
main.py does: from app.package_b import fun_b
fun_b.py does from app.package_a.fun_a import print_a
so file in folder package_b used file in folder package_a, which is what you want. Right??
def import_path(fullpath):
"""
Import a file with full path specification. Allows one to
import from anywhere, something __import__ does not do.
"""
path, filename = os.path.split(fullpath)
filename, ext = os.path.splitext(filename)
sys.path.append(path)
module = __import__(filename)
reload(module) # Might be out of date
del sys.path[-1]
return module
I'm using this snippet to import modules from paths, hope that helps
This is unfortunately a sys.path hack, but it works quite well.
I encountered this problem with another layer: I already had a module of the specified name, but it was the wrong module.
what I wanted to do was the following (the module I was working from was module3):
mymodule\
__init__.py
mymodule1\
__init__.py
mymodule1_1
mymodule2\
__init__.py
mymodule2_1
import mymodule.mymodule1.mymodule1_1
Note that I have already installed mymodule, but in my installation I do not have "mymodule1"
and I would get an ImportError because it was trying to import from my installed modules.
I tried to do a sys.path.append, and that didn't work. What did work was a sys.path.insert
if __name__ == '__main__':
sys.path.insert(0, '../..')
So kind of a hack, but got it all to work!
So keep in mind, if you want your decision to override other paths then you need to use sys.path.insert(0, pathname) to get it to work! This was a very frustrating sticking point for me, allot of people say to use the "append" function to sys.path, but that doesn't work if you already have a module defined (I find it very strange behavior)
Let me just put this here for my own reference. I know that it is not good Python code, but I needed a script for a project I was working on and I wanted to put the script in a scripts directory.
import os.path
import sys
sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), "..")))
As #EvgeniSergeev says in the comments to the OP, you can import code from a .py file at an arbitrary location with:
import imp
foo = imp.load_source('module.name', '/path/to/file.py')
foo.MyClass()
This is taken from this SO answer.
Take a look at http://docs.python.org/whatsnew/2.5.html#pep-328-absolute-and-relative-imports. You could do
from .mod1 import stuff
From Python doc,
In Python 2.5, you can switch import‘s behaviour to absolute imports using a from __future__ import absolute_import directive. This absolute- import behaviour will become the default in a future version (probably Python 2.7). Once absolute imports are the default, import string will always find the standard library’s version. It’s suggested that users should begin using absolute imports as much as possible, so it’s preferable to begin writing from pkg import string in your code
I found it's more easy to set "PYTHONPATH" enviroment variable to the top folder:
bash$ export PYTHONPATH=/PATH/TO/APP
then:
import sub1.func1
#...more import
of course, PYTHONPATH is "global", but it didn't raise trouble for me yet.
On top of what John B said, it seems like setting the __package__ variable should help, instead of changing __main__ which could screw up other things. But as far as I could test, it doesn't completely work as it should.
I have the same problem and neither PEP 328 or 366 solve the problem completely, as both, by the end of the day, need the head of the package to be included in sys.path, as far as I could understand.
I should also mention that I did not find how to format the string that should go into those variables. Is it "package_head.subfolder.module_name" or what?
You have to append the module’s path to PYTHONPATH:
export PYTHONPATH="${PYTHONPATH}:/path/to/your/module/"
A hacky way to do it is to append the current directory to the PATH at runtime as follows:
import pathlib
import sys
sys.path.append(pathlib.Path(__file__).parent.resolve())
import file_to_import # the actual intended import
In contrast to another solution for this question this uses pathlib instead of os.path.
This method queries and auto populates the path:
import os
import inspect
currentdir = os.path.dirname(os.path.abspath(inspect.getfile(inspect.currentframe())))
parentdir = os.path.dirname(currentdir)
os.sys.path.insert(1, parentdir)
# print("currentdir = ", currentdir)
# print("parentdir=", parentdir)
What a debate!
Relative newcomer to python (but years of programming experience, and dislike of perl). Relative lay-person when it comes to the dark art of Apache setup, but I know what I (think I) need to get my little experimental projects working at home.
Here is my summary of what the situ seems to be.
If I use the -m 'module' approach, I need to:-
dot it all together;
run it from a parent folder;
lose the '.py';
create an empty (!) __init__.py file in every sub-folder.
How does that work in a cgi environment, where I have aliased my scripts directory, and want to run a script directly as /dirAlias/cgi_script.py??
Why is amending sys.path a hack? The python docs page states: "A program is free to modify this list for its own purposes." If it works, it works, right? The bean counters in Accounts don't care how it works.
I just want to go up one level and down into a 'modules' dir:-
.../py
/cgi
/build
/modules
so my 'modules' can be imported from either the cgi world or the server world.
I've tried the -m/modules approach but I think I prefer the following (and am not confused how to run it in cgi-space):-
Create XX_pathsetup.py in the /path/to/python/Lib dir (or any other dir in the default sys.path list). 'XX' is some identifier that declares an intent to setup my path according to the rules in the file.
In any script that wants to be able to import from the 'modules' dir in above directory config, simply import XX_pathsetup.py.
And here's my really simple XX_pathsetup.py:
import sys, os
pypath = sys.path[0].rsplit(os.sep,1)[0]
sys.path.insert( 0, pypath+os.sep+'modules' )
Not a 'hack', IMHO. 1 small file to put in the python 'Lib' dir, one import statement which declares intent to modify the path search order.

How to fix "Attempted relative import in non-package" even with __init__.py

I'm trying to follow PEP 328, with the following directory structure:
pkg/
__init__.py
components/
core.py
__init__.py
tests/
core_test.py
__init__.py
In core_test.py I have the following import statement
from ..components.core import GameLoopEvents
However, when I run, I get the following error:
tests$ python core_test.py
Traceback (most recent call last):
File "core_test.py", line 3, in <module>
from ..components.core import GameLoopEvents
ValueError: Attempted relative import in non-package
Searching around I found "relative path not working even with __init__.py" and "Import a module from a relative path" but they didn't help.
Is there anything I'm missing here?
To elaborate on Ignacio Vazquez-Abrams's answer:
The Python import mechanism works relative to the __name__ of the current file. When you execute a file directly, it doesn't have its usual name, but has "__main__" as its name instead. So relative imports don't work.
You can, as Igancio suggested, execute it using the -m option. If you have a part of your package that is meant to be run as a script, you can also use the __package__ attribute to tell that file what name it's supposed to have in the package hierarchy.
See http://www.python.org/dev/peps/pep-0366/ for details.
Yes. You're not using it as a package.
python -m pkg.tests.core_test
It depends on how you want to launch your script.
If you want to launch your UnitTest from the command line in a classic way, that is:
python tests/core_test.py
Then, since in this case 'components' and 'tests' are siblings folders, you can import the relative module either using the insert or the append method of the sys.path module.
Something like:
import sys
from os import path
sys.path.append( path.dirname( path.dirname( path.abspath(__file__) ) ) )
from components.core import GameLoopEvents
Otherwise, you can launch your script with the '-m' argument (note that in this case, we are talking about a package, and thus you must not give the '.py' extension), that is:
python -m pkg.tests.core_test
In such a case, you can simply use the relative import as you were doing:
from ..components.core import GameLoopEvents
You can finally mix the two approaches, so that your script will work no matter how it is called.
For example:
if __name__ == '__main__':
if __package__ is None:
import sys
from os import path
sys.path.append( path.dirname( path.dirname( path.abspath(__file__) ) ) )
from components.core import GameLoopEvents
else:
from ..components.core import GameLoopEvents
You can use import components.core directly if you append the current directory to sys.path:
if __name__ == '__main__' and __package__ is None:
from os import sys, path
sys.path.append(path.dirname(path.dirname(path.abspath(__file__))))
In core_test.py, do the following:
import sys
sys.path.append('../components')
from core import GameLoopEvents
Issue is with your testing method,
you tried python core_test.py
then you will get this error
ValueError: Attempted relative import in non-package
Reason: you are testing your packaging from non-package source.
so test your module from package source.
if this is your project structure,
pkg/
__init__.py
components/
core.py
__init__.py
tests/
core_test.py
__init__.py
cd pkg
python -m tests.core_test # dont use .py
or from outside pkg/
python -m pkg.tests.core_test
single . if you want to import from folder in same directory .
for each step back add one more.
hi/
hello.py
how.py
in how.py
from .hi import hello
incase if you want to import how from hello.py
from .. import how
If your use case is for running tests, and it seams that it is, then you can do the following. Instead of running your test script as python core_test.py use a testing framework such as pytest. Then on the command line you can enter
$$ py.test
That will run the tests in your directory. This gets around the issue of __name__ being __main__ that was pointed out by #BrenBarn. Next, put an empty __init__.py file into your test directory, this will make the test directory part of your package. Then you will be able to do
from ..components.core import GameLoopEvents
However, if you run your test script as a main program then things will fail once again. So just use the test runner. Maybe this also works with other test runners such as nosetests but i haven't checked it. Hope this helps.
My quick-fix is to add the directory to the path:
import sys
sys.path.insert(0, '../components/')
As Paolo said, we have 2 invocation methods:
1) python -m tests.core_test
2) python tests/core_test.py
One difference between them is sys.path[0] string. Since the interpret will search sys.path when doing import, we can do with tests/core_test.py:
if __name__ == '__main__':
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
from components import core
<other stuff>
And more after this, we can run core_test.py with other methods:
cd tests
python core_test.py
python -m core_test
...
Note, py36 tested only.
As you have already marked everything as a module, there's no need to use the relative reference if you launch as python module.
Instead of
from ..components.core import GameLoopEvents
simply
from pkg.components.core import GameLoopEvents
When you run from the parent of pkg, use the following
python -m pkg.tests.core_test
Old thread. I found out that adding an __all__= ['submodule', ...] to the
__init__.py file and then using the from <CURRENT_MODULE> import * in the target works fine.
You can use from pkg.components.core import GameLoopEvents, for example I use pycharm, the below is my project structure image, I just import from the root package, then it works:
This approach worked for me and is less cluttered than some solutions:
try:
from ..components.core import GameLoopEvents
except ValueError:
from components.core import GameLoopEvents
The parent directory is in my PYTHONPATH, and there are __init__.py files in the parent directory and this directory.
The above always worked in python 2, but python 3 sometimes hit an ImportError or ModuleNotFoundError (the latter is new in python 3.6 and a subclass of ImportError), so the following tweak works for me in both python 2 and 3:
try:
from ..components.core import GameLoopEvents
except ( ValueError, ImportError):
from components.core import GameLoopEvents
Try this
import components
from components import *
If someone is looking for a workaround, I stumbled upon one. Here's a bit of context. I wanted to test out one of the methods I've in a file. When I run it from within
if __name__ == "__main__":
it always complained of the relative imports. I tried to apply the above solutions, but failed to work, since there were many nested files, each with multiple imports.
Here's what I did. I just created a launcher, an external program that would import necessary methods and call them. Though, not a great solution, it works.
Here's one way which will piss off everyone but work pretty well. In tests run:
ln -s ../components components
Then just import components like you normally would.
For me only this worked: I had to explicitly set the value of package to the parent directory, and add the parent directory to sys.path
from os import path
import sys
if __package__ is None:
sys.path.append( path.dirname( path.dirname( path.abspath(__file__) ) ) )
__package__= "myparent"
from .subdir import something # the . can now be resolved
I can now directly run my script with python myscript.py.
python <main module>.py does not work with relative import
The problem is relative import does not work when you run a __main__ module from the command line
python <main_module>.py
It is clearly stated in PEP 338.
The release of 2.5b1 showed a surprising (although obvious in retrospect) interaction between this PEP and PEP 328 - explicit relative imports don't work from a main module. This is due to the fact that relative imports rely on __name__ to determine the current module's position in the package hierarchy. In a main module, the value of __name__ is always '__main__', so explicit relative imports will always fail (as they only work for a module inside a package).
Cause
Python Bug Tracker Issue1510172: Absolute/relative import not working?
The issue isn't actually unique to the -m switch. The problem is that relative imports are based on __name__, and in the main module, __name__ always has the value __main__. Hence, relative imports currently can't work properly from the main module of an application, because the main module doesn't know where it really fits in the Python module namespace (this is at least fixable in theory for the main modules executed through the -m switch, but directly executed files and the interactive interpreter are completely out of luck).
To understand further, see Relative imports in Python 3 for the detailed explanation and how to get it over.
I've had similar issues and as a software engineer, I think some of the suggested solutions here are not ideal. If you want relative imports, you should not have try/except and then sometimes do an absolute import. Also, to run a program, you should not have to change sys.path.
Furthermore, the program should always work, independent of your current working directory and independent of how you start it.
Thus, I've created a new, experimental import library: ultraimport
It allows file system based imports, no matter how you run your code.
From the original question, you would change your core_test.py to something like
import ultraimport
GameLoopEvents = ultraimport('__dir__/../components/core.py', 'GameLoopEvents')
print(GameLoopEvents)
and it would always find it, no matter how you run your tests.
$ python -m tests.core_test
<class 'core.GameLoopEvents'>
python ./tests/core_test.py
<class 'core.GameLoopEvents'>
I've also put this example into the examples folder in the git repo.
As the library is experimental, I am interested in feedback. It works for me but it not widely tested, yet.
If your project structure would look like this:
project
|
| --- module1
| |
| file1.py
|
|-----module2
| |
| file2.py
and you are going import file1.py from within file2.py,
you can do this in file2.py:
import sys
sys.path.append('.')
import file2
I still don't know why and how, but it worked for me.
This is very confusing and if you are using IDE like Pycharm, it's little more confusing.
What worked for me:
Make Pycharm project settings (if you are running python from a VE or from Python directory)
There is nothing wrong with the way you defined. Sometime it works with:
from folder1.file1 import class
if it does not work, use:
import folder1.file1
Your environment variable should be correctly mentioned in system or provide it in your command line argument.
Because your code contains if __name__ == "__main__", which doesn't be imported as a package, you'd better use sys.path.append() to solve the problem.

relative path not working even with __init__.py

I know that there are plenty of similar questions on stack overflow. But the common answer doesn't seem to be working for me.
I have a file structure like this
proj/
lib/
__init__.py
aa.py
bb.py
test/
__init__.py
aa_test.py
I figured that if I include the code in my test.py
import lib.aa
or
from lib import aa
I would be able to reference the modules in the lib/ directory. But that did not work.
So I tried to add to path, and it adds it correctly:
os.environ["PATH"] += ":%s" % os.path.abspath(os.path.join("..",""))
print os.environ["PATH"]
but even now when I try the import statements above... I keep getting the error
ImportError: No module named aa
or
ImportError: Importing from non-package <Something...>
Is there something obvious I am missing?
Is there a way to check if I have configured my __init__.py files correctly, or to see my package hierarchy?
You need to update your sys.path, which is where python looks for modules, as opposed to your system's path in the current environment, which is what os.environ["PATH"] is referring to.
Example:
import os, sys
sys.path.insert(0, os.path.abspath(".."))
import aa
After doing this, you can use your functions in aa like this: aa.myfunc()
There's some more information in the accepted answer for python: import a module from a directory
The lib directory needs to be in your python module search path, which isn't the same things as the search path used by your shell.
This will probably work for you:
import sys, os
sys.path.append(os.path.abspath(".."))
However, it is probably better to run your code from a context where the lib package is already on the path. Such as from the 'proj' directory.
Where is the code that you're trying to import lib.aa from? I'm guessing /proj/ is not your working directory and it would need to be as it's setup right now. Instead of PATH, you would want to add your directory to PYTHONPATH so it appears in the search path for an import. See http://docs.python.org/tutorial/modules.html#the-module-search-path
Also, please take a look at http://as.ynchrono.us/2007/12/filesystem-structure-of-python-project_21.html It strongly recommends you put an extra level of directory in place so instead of lib.aa, you would refer to it as my_proj.lib.aa.
I had similar problems and here is my advice.
Instead of changing sys.path, better run your test.py from being in proj (i.e. project root) directory. This way project dir will automatically be in sys.path and you will be able to import lib package.
And use absolute imports.
System PATH variable is not used by python import statement. It uses PYTHONPATH, but best way to add new directory to import search path is to modify sys.path.
If this does not help, add to the question your value of sys.path and value returned by os.getcwd().
In Sublime Text 3, abspath it didn't work for me.
I use this instead in the top "__ init __.py" file
Hope it works for you.
from os.path import dirname
from sys import path
path.insert( 0 , dirname( __file__ ) ) ;
from test import aa_test

Import paths - the right way?

I know there are A LOT of similar or the same questions, but i still cannot understand / find the right way for me to work with modules. Python is my favorite language, and i like everything in it except working with imports: recursive imports (when you try to reference a name that is not yet there), import paths, etc.
So, I have this kind of a project structure:
my_project/
package1/
__init__.py
module1
module2
package2/
__init__.py
module1
module2
Package1 may be used a standalone unit, but is also expected to be imported by package2.
What am i doing now, is that, for example, in package1.module1 i write from package1 import module2, i.e. using full path to imported module. I do this because if i use import module2 -- this will not work when the module will be imported from another package (package2). I also cannot use from . import module2 -- this will not work when running module1 directly.
OK, so for from package1 import module2 in package1.module1 to work in both cases (when running directly package1.module1 and when importing it from package2) i add these lines at the beginning of package1.module1:
import os, sys
currDir = os.path.dirname(os.path.realpath(__file__))
rootDir = os.path.abspath(os.path.join(currDir, '..'))
if rootDir not in sys.path: # add parent dir to paths
sys.path.append(rootDir)
For me this works, but i feel this is not pythonic. Am i doing something wrong?
Should i, instead, always run package1.module1 from project root? If so, this makes inconvenient to run it from an IDE -- i need somehow to set paths in it.
UPDATE: I tried to add a file root.pth to package1 dir with contents of ... But it didn't work -- i guess it's intended for something else.
CONCLUSIONS:
Always use absolute imports: import package1.module1
Add a bootstrapper to the root folder to start some of the modules as a standalone script. This solves running the script form an IDE and is a pythonic approach.
On 4/22/07, Brett Cannon wrote:
This PEP is to change the if __name__ == "__main__": ... idiom to
if __name__ == sys.main: ... so that you at least have a chance
to execute module in a package that use relative imports.
Ran this PEP past python-ideas. Stopped the discussion there when too
many new ideas were being proposed. =) I have listed all of them in
the Rejected Ideas section, although if overwhelming support for one
comes forward the PEP can shift to one of them.
I'm -1 on this and on any other proposed twiddlings of the __main__
machinery. The only use case seems to be running scripts that happen
to be living inside a module's directory, which I've always seen as an
antipattern. To make me change my mind you'd have to convince me that
it isn't.
--Guido van Rossum
What is the entry point for your program? Usually the entry point for a program will be at the root of the project. Since it is at the root, all the modules within the root will be importable, provided there is an __init__.py file in them.
So, using your example:
my_project/
main.py
package1/
__init__.py
module1
module2
package2/
__init__.py
module1
module2
main.py would be the entry point for your program. Because the file that is executed as main is automatically put on the PYTHONPATH, both package1 and package2 are available from the top level import.
# in main.py
from package1.module1 import *
from package1.module2 import *
# in package1.module1
import module2
from package2.module1 import *
# in package2.module1 import *
import module2
from package1.module1 import *
Note that in the above, package1 and package2 depend on each other. That should never be the case. But this is just an example of being able to import from anywhere.
main.py doesn't have to be anything fancy either. It can be very simple:
# main.py
if __name__ == '__main__':
from package1.module1 import SomeClass
SomeClass().start()
The point I'm trying to make, is that if a module needs to be accessible by other modules, that module should be available as a top level import. A module should not attempt to put itself as a top level import (directly on the PYTHONPATH).
It should be the responsibility of the project for ensuring that all imports can be satisfied if the module is included directly in the project. There are two ways to do this. The first is by creating a bootstrapper file such as main.py in the project folder. The other, is by creating a file that adds all relevant paths to PYTHONPATH, that is loaded by any entry points that may exist.
For example:
# setup.py
import sys
def load():
paths = ['/path1/','/path2/','/path3/']
for p in path:
sys.path.insert(0, p)
# entrypoint.py
from setup import load
load()
# continue with program
The main thing to take away, is that a module is not supposed to put itself on the path. The path should be determined automatically by the entry point into the program, or defined explicitly by a setup script that knows where all the relevant modules are.
I generally create each package as an installable package (ie, create a setup.py file), and then install them into a virtualenv just for this project, using pip.
You can even install the using pip -e if they are still under development.
I'm 5 years late here.. but just do export PYTHONPATH=/path1:/path2: (notice trailing ":") - this way your working directory (from which you run python) will be in the path.

Python project and package directories layout

I created a project in python, and I'm curious about how packages work in python.
Here is my directory layout:
top-level dir
\ tests
__init__.py
\ examples
__init__.py
example.py
module.py
How would I go about including module.py in my example.py module. I know I could set PYTHONPATH to the top-level directory, but that doesn't seem like a good solution. This is how pydev gets around this problem, but I'd like a solution that didn't require updating environment variables.
I could put something at the top of the example.py to update sys.path like this:
from os import path
import sys
sys.path.append( path.dirname(path.abspath(path.dirname(__file__))) )
I don't think this is an appropriate solution either.
I feel like I'm missing some basic part of python packages. I'm testing this on python 2.6. If any further clarification is needed, please let me know.
Python packages are very simple: a package is any directory under any entry in sys.path that has an __init__.py file. However, a module is only considered to be IN a package if it is imported via a relative import such as import package.module or from package import module. Note that this means that in general, someone must set up sys.path to contain the directories above any package you want to be importable, whether via PYTHONPATH or otherwise.
The primary wrinkle is that main modules (those run directly from the command line) are always __main__ no matter their location. They therefore have to do absolute imports, and either rely on PYTHONPATH being set up, or munge sys.path themselves.
In your case, I'd recommend either having a small Python script that runs your examples after setting up the correct path. Say you put it in the top level directory:
#!/usr/bin/env python
import sys
import os.path
sys.path.append(os.path.dirname(__file__))
example = __import__("examples", globals(), locals(), sys.argv[1])
Then your example can do "import module".
Alternately, if module.py is meant to also be in a package, add the PARENT of your "top level directory" to sys.path and then use the from .. import module syntax in your example modules. Also, change the first parameter to the __import__ in the wrapper to "tldname.examples" where tldname is the name of your top-level directory.
If you don't want to rely on an "example runner" module, each example needs sys.path boilerplate, or you need a PYTHONPATH setting. Sorry.
http://docs.python.org/tutorial/modules.html#intra-package-references
from .. import module

Categories

Resources