Currently have the following file hierarchy:
\package
__init__.py
run_everything.py
\subpackage
__init__.py
work.py
work1.py
work2.py
\test
__init__.py
test_work.py
test_work1.py
My first question is regarding relative imports. Suppose in \subpackage\work.py I have a function called custom_function(), and I would like to test that function in test_work.py. For some reason I can not figure out how to make this import from one module to another. Trying from .. subpackage.work1 import custom_function() does not seem to work, and yields the error Attempted relative import in non-package Is there any way to resolve this?
2)
I would like to run all test files from run_everything.py with one function, would adding a suite() function in each test_work*.py file, which adds each unit_testing class to suite.addTest(unittest.makeSuite(TestClass)), and finally importing them into the top-level run_everything.py be the most conventional way in Python2.7?
Here is a hack*
Insert the path's to "subpackage" and "test" to your python path in run_everything using:
import sys
sys.path.insert(0, '/path/to/package/subpackage')
sys.path.insert(0, '/path/to/package/test')
And then, you can import all your files using vanilla imports in run_everything:
import work, work1, work2
import test_work, test_work1
*This won't permanently affect your PYTHONPATH.
Related
The project has the same structure as in the picture: I'm trying to import from "mod.py " in "index.py "
from .. import mod
However, it gives the error: "ImportError: attempted relative import with no known parent package" If you use this option:
from pack1 import mod
Then error: "ModuleNotFoundError error: there is no module named 'pack1'"
enter image description here
PROJECT/
pack1/
__init__.py
mod.py
pack2/
__init__.py
index.py
What is the problem?
This is a recurring question on StackOverflow. And much of the confusion (in my opinion) comes from how Python interprets the files and folders it sees is based on where Python is run from. First, some terminology:
module: a file containing Python code.
package: a folder containing files with Python code and other folders.
When you start Python in a directory (folder), it doesn't "know" what the namespace of that directory should be. I.e., if you are working in Z:\path\to_my\project\ when you start Python:
it does NOT consider project to be a package.
any .py files you want to import from will be in their own namespace as modules.
any folders you want to import from will also be in their own namespace as packages.
What about __init__.py? Since version 3.3, Python has implicit namespace packages, which allows importing without needing to create an empty __init__.py file.
Consider #2: if you have two files: first.py and second.py:
path/
to_my/
project/
>>Python is running here<<
first.py
second.py
with these contents:
# first.py
first_var = 'hello'
# second.py
from .first import first_var
second_var = first_var + ' world'
if you try to import like this:
>>> import second
Python basically does the following:
"ok, I see second.py"
"Reading that in as a module, chief!"
"Ok, it wants to import .first
"The . means get the package (folder) that contains first.py"
"Wait, I don't have a parent package for first.py!"
"Better raise an error."
The same rules apply for #3 as well. If we add a few packages to the project like this:
path/
to_my/
project/
>>Python is running here<<
first.py
second.py
pack1/
mod.py
other_mod.py
pack2/
index.py
with the following contents:
# pack1/mod.py
mod_var = 1234
# pack1/other_mod.py
from .mod import mod_var
other_var = mod_var * 10
# pack2/index.py
from ..pack1 import mod
and when you try to import like this:
>>> from pack2 import index.py
The import in pack2/index.py is going to fail for the same reason second.py, Python will work its way up the import chain of dots like this:
"Reading in in index.py as a module."
"Looks like it wants to import mod from ..pack1.
"Ok, . is the pack2 parent package namespace of index.py, found that."
"So, .. is the parent package of pack2."
"But, I don't have a parent package for pack2!"
"Better raise an error."
How do we make it work? Two thing.
First, move where Python is running up one level so that all of the .py files and subfolders are considered to be part of the same package namespace, which allows the file to reference each other using relative references.
path/
to_my/
>>Python is running here now<<
project/
first.py
second.py
pack1/
mod.py
other_mod.py
pack2/
index.py
So now Python sees project as a package namespace, and all of the files within can use relative references up to that level.
This changes how you import when you are in the Python interpreter:
>>> from project.pack2 import index.py
Second, you make explicit references instead of relative references. That can make the import statements really long, but if you have several top-level modules that need to pull from one another, this is how you can do it. This is useful when you are defining your functions in one file and writing your script in another.
# first.py
first_var = 'hello'
# second.py
from first import first_var # we dropped the dot
second_var = first_var + ' world'
I hope this helps clear up some of the confusion about relative imports.
I am working with the following directory in Python3.8:
package/
__init__.py
/folder1
__init__.py
file1.py
/folder2
__init__.py
file2.py
/folder3
__init__.py
file3.py
I would like to import a function from file3 into file2. What is the easiest way to do it? I would also like to avoid manually appending to PYTHONPATH, to make this as smooth as possible for the next person pulling the project.
So far tried relative imports, which didn't work, possibly because I did not specify something in the init, which is currently empty. The error I am getting using:
from ..package.folder3.file3 import function_name
is
ImportError: attempted relative import with no known parent package
Thanks for all help!
The answer to your question is pretty simple, you need to add your package path to the system path.
Here is a complete example:
In file3.py, let's create a simple function
def func():
print("Hello from file3")
In file2.py, we can import func function like so:
import os
import sys
sys.path.append(os.path.abspath('../../..'))
# import func now from file3
from package.folder3.file3 import func
func() #should return "Hello from file3"
Hopefully, this answers your question!
I spent some time researching this and I just cannot work this out in my head.
I run a program in its own directory home/program/core/main.py
In main.py I try and import a module called my_module.py thats located in a different directory, say home/program/modules/my_module.py
In main.py this is how I append to sys.path so the program can be run on anyone's machine (hopefully).
import os.path
import sys
# This should give the path to home/program
sys.path.append(os.path.join(os.path.abspath(os.path.dirname(__file__), '..'))
# Which it does when checking with
print os.path.join(os.path.abspath(os.path.dirname(__file__), '..')
# So now sys.path knows the location of where modules directory is, it should work right?
import modules.my_module # <----RAISES ImportError WHY?
However if I simply do:
sys.path.append('home/program/modules')
import my_module
It all works fine. But this is not ideal as it now depends on the fact that the program must exist under home/program.
that's because modules isn't a valid python package, probably because it doesn't contain any __init__.py file (You cannot traverse directories with import without them being marked with __init__.py)
So either add an empty __init__.py file or just add the path up to modules so your first snippet is equivalent to the second one:
sys.path.append(os.path.join(os.path.abspath(os.path.dirname(__file__), '..','modules'))
import my_module
note that you can also import the module by giving the full path to it, using advanced import features: How to import a module given the full path?
Although the answer can be found here, for convenience and completeness here is a quick solution:
import importlib
dirname, basename = os.path.split(pyfilepath) # pyfilepath: /my/path/mymodule.py
sys.path.append(dirname) # only directories should be added to PYTHONPATH
module_name = os.path.splitext(basename)[0] # /my/path/mymodule.py --> mymodule
module = importlib.import_module(module_name) # name space of defined module (otherwise we would literally look for "module_name")
Now you can directly use the namespace of the imported module, like this:
a = module.myvar
b = module.myfunc(a)
I am having trouble using my classes that I've defined in a module. I've looked at this stackoverlfow post, and the answer seems to be "you don't need imports." This is definitely not the behavior I'm experiencing. I'm using Python 3.3. Here is my directory structure:
root/
__init__.py
mlp/
__init__.py
mlp.py
layers/
__init__.py
hidden_layer.py
dropout_layer.py
My problem is this: the class defined in dropout_layer.py extends the class in hidden_layer.py, but when I try to import hidden_layer, I sometimes get an error depending on the directory I execute my code from. For instance, from layers.hidden_layer import HiddenLayer then I run my code if I execute it from root/mlp. This import does not work, however, if I execute my code from root. This is strange behavior to me. How can I get this working correctly?
My only non-empty __init__.py file is in root/mlp/layers/:
# root/mlp/layers/__init__.py
__all__ = ['hidden_layer', 'dropout_layer']
In Python 3 you can prepend a . for an import relative to the location of the current module:
from .hidden_layer import HiddenLayer
I am attempting to import a python file(called test.py that resides in the parent directory) from within the currently executing python file(I'll call it a.py). All my directories involved have a file in it called init.py(with 2 underscores each side of init)
My Problem: When I attempt to import the desired file I get the following error
Attempted relative import in non-package
My code inside a.py:
try:
from .linkIO can_follow # error occurs here
except Exception,e:
print e
print success
Note: I know that if I were to create a file called b.py and import a.py(which in itself imports the desired python file) it all works, so whats going wrong?
For eg:
b.py:
import a
print "success 2"
As stated in PEP 328 all import must be absolute to prevent modules masking each other. Absolute means the module/package must be in the module-path sys.path. Relative imports (thats the dot for) are only allowed intra-packages wise, meaning if modules from the same package want to import each other.
So this leave you with following possibilities:
You make a package (which you seem to have made already) and add the package-path to sys. path
you just adjust sys.path for each module
you put all your custom modules into the same directory as the start-script/main-application
for 1. and 2. you may add a package/module to sys.path like this:
import sys
from os.path import dirname, join
sys.path.append(dirname(__file__)) #package-root-directory
or
module_dir = 'mymodules'
sys.path.append(join(dirname(__file__), module_dir)) # in the main-file
BTW:
from .linkIO can_follow
can't work! The import statement is missing!
As a reminder: if using relative imports you MUST use the from-version: from .relmodule import xyz. An import .XYZ without the from isn't allowed!