I'm a little confused about the import in a python project.
I used this as a model to create my project:
https://docs.python-guide.org/writing/structure/
at the moment, working in spyder, I set my working directory to MyProject/
MyProject
|
|
--- mymodule
| |
| |--- myclass1.py (contains def MyClass1 )
| |
| |--- myclass2.py (contains def MyClass2 )
|
|
|--- tests
| |
| |-- test_MyClass1.py (contains def TestMyClass1(unittest.TestCase)
| |
| |
| |-- test_MyClass2.py (contains def TestMyClass2(unittest.TestCase)
then I run test_MyClass1.py
the test_MyClass1.py references the MyClass1 this way:
from mymodule.myclass1 import MyClass1
and in the myclass1.py, I reference the MyClass2 this way:
from mymodule.myclass2 import MyClass2
I read about the __init__.py and the namespace packages, the more I read the more confused I get...
Basically I do not want to do :
mymodule.myfile.myclass
but rather:
import mymodule as mm
mm.MyClass1
or again:
from mymodule import *
a = MyClass1()
Still, I want one file by class.
You can add the import of MyClass1 and MyClass2 in mymodule/__init__.py.
Basically you will have the following files:
mymodule/
__init__.py
myclass1.py
myclass2.py
tests/
test_myclass1.py
test_myclass2.py
where:
mymodule/__init__.py contains the following lines:
from mymodule.myclass1 import MyClass1
from mymodule.myclass2 import MyClass2
mymodule/myclass1.py contains MyClass1 definition
mymodule/myclass2.py contains MyClass2 definition
Then in tests/test_myclass1.py you can import MyClass1 thanks to:
from mymodule import MyClass1
a = MyClass1()
or
import mymodule as mm
a = mm.MyClass1()
You can do the same for MyClass2
Related
Let's say I have the following directory
model_folder
|
|
------- model_modules
| |
| ---- __init__.py
| |
| ---- foo.py
| |
| ---- bar.py
|
|
------- research
| |
| ----- training.ipynb
| |
| ----- eda.ipynb
|
|
------- main.py
and I want to import model_modules into a script in research
I can do that with the following
import sys
sys.path.append('/absolute/path/model_folder')
from model_modules.foo import Foo
from model_modules.bar import Bar
However, let's say I don't explicitly know the absolute path of the root, or perhaps just don't want to hardcode it as it may change locations. How could I get the absolute path of module_folder from anywhere in the directory so I could do something like this?
import sys
sys.path.append(root)
from model_modules.foo import Foo
from model_modules.bar import Bar
I referred to this question in which one of the answers recommends adding the following to the root directory, like so:
utils.py
from pathlib import Path
def get_project_root() -> Path:
return Path(__file__).parent.parent
model_folder
|
|
------- model_modules
| |
| ---- __init__.py
| |
| ---- foo.py
| |
| ---- bar.py
|
|
|
------- src
| |
| ---- utils.py
|
|
|
|
|
------- research
| |
| ----- training.ipynb
| |
| ----- eda.ipynb
|
|
------- main.py
But then when I try to import this into a script in a subdirectory, like training.ipynb, I get an error
from src.utils import get_project_root
root = get_project_root
ModuleNotFoundError: No module named 'src'
So my question is, how can I get the absolute path to the root directory from anywhere within the directory in python?
sys.path[0] contain your root directory (the directory where the program is located). You can use that to add your sub-directories.
import sys
sys.path.append( sys.path[0] + "/model_modules")
import foo
and for cases where foo.py may exist elsewhere:
import sys
sys.path.insert( 1, sys.path[0] + "/model_modules") # put near front of list
import foo
I am building Python bindings for a standalone C library that I wrote. The file layout of the library is as following:
<project root>
|
`- cpython
| |
| `- module1_mod.c
| `- module2_mod.c
| `- module3_mod.c
|
`- include
| |
| `- module1.h
| `- module2.h
| `- module3.h
|
`- src
| |
| `- module1.c
| `- module2.c
| `- module3.c
|
`- setup.py
I want to obtain a Python package so I can import modules in a namespace such as my_package.module1, my_package.module2, etc.
This is my setup.py so far:
from os import path
from setuptools import Extension, setup
ROOT_DIR = path.dirname(path.realpath(__file__))
MOD_DIR = path.join(ROOT_DIR, 'cpython')
SRC_DIR = path.join(ROOT_DIR, 'src')
INCL_DIR = path.join(ROOT_DIR, 'include')
EXT_DIR = path.join(ROOT_DIR, 'ext')
ext_libs = [
path.join(EXT_DIR, 'ext_lib1', 'lib.c'),
# [...]
]
setup(
name="my_package",
version="1.0a1",
ext_modules=[
Extension(
"my_package.module1",
[
path.join(SRC_DIR, 'module1.c',
path.join(MOD_DIR, 'module1_mod.c',
] + ext_libs,
include_dirs=[INCL_DIR],
libraries=['uuid', 'pthread'],
),
],
)
Importing mypackage.module1 works but the problem is that the external libraries are also needed by module2 and module3 (not all of them for all the modules), and I assume that if I include the same external libs in the other modules, I would get a lot of bloat.
I looked around sample setups in Github but haven't found an example resolving this problem.
What is a good way to organize my builds?
EDIT: This is actually a more severe problem in that I have symbols in module1 that are needed in module2, etc. E.g. an object in module2 requires an object type defined in module1. If I create separate binaries without including all sources for each dependency, the symbols won't be available at linking time, thus increasing redundancy and complexity of keeping track of what is needed for which module.
After a couple of days of digging into Python bug reports and scarcely documented features, I found an answer to this, which resolved both the multiple external dependencies and the internal cross-linking.
The solution was to create a monolithic "module" with all the modules defined inside it, then exposing them with a few lines of Python code in a package initialization file.
To do this I changed the module source files to header files, maintaining most of their methods static and only exposing the PyTypeObject structs and my object type structs so they can be used in other modules.
Then I moved the PyMODINIT_FUNC functions defining all the modules in a "package" module (py_mypackage.c), which also defines an empty module. the "package" module is defined as _my_package.
Finally I added some internal machinery to an __init__.py script that extracts the module symbols from the .so file and exposes them as modules of the package. This is documented in the Python docs :
import importlib.util
import sys
import _my_package
pkg_path = _my_package.__file__
def _load_module(mod_name, path):
spec = importlib.util.spec_from_file_location(mod_name, path)
module = importlib.util.module_from_spec(spec)
sys.modules[mod_name] = module
spec.loader.exec_module(module)
return module
for mod_name in ('module1', 'module2', 'module3'):
locals()[mod_name] = _load_module(mod_name, pkg_path)
The new layout is thus:
<project root>
|
`- cpython
| |
| `- my_package
| |
| `- __init__.py
|
| `- py_module1.h
| `- py_module2.h
| `- py_module3.h
| `- py_mypackage.c
|
`- include
| |
| `- module1.h
| `- module2.h
| `- module3.h
|
`- src
| |
| `- module1.c
| `- module2.c
| `- module3.c
|
`- setup.py
And setup.py:
setup(
name="my_package",
version="1.0a1",
package_dir={'my_package': path.join(CPYTHON_DIR, 'my_package')},
packages=['my_package'],
ext_modules=[
Extension(
"_my_package",
"<all .c files in cpython folder + ext library sources>",
libraries=[...],
),
],
)
For the curious, the complete code is at https://notabug.org/scossu/lsup_rdf/src/e08da1a83647454e98fdb72f7174ee99f9b8297c/cpython (pinned at the current commit).
Im new to python and have looked at various stack overflow posts. i feel like this should work but it doesnt. How do you import a class from another file in python?
This folder structure
src/example/ClassExample
src/test/ClassExampleTest
I have this class
class ClassExample:
def __init__(self):
pass
def helloWorld(self):
return "Hello World!"
I have this test class
import unittest
from example import ClassExample
class ClassExampleTest(unittest.TestCase):
def test_HelloWorld(self):
hello = ClassExample()
self.assertEqual("Hello World!", hello.helloWorld())
if __name__ == '__main__':
unittest.main()
When the unit test runs the object is None:
AttributeError: 'NoneType' object has no attribute 'helloWorld'
What's wrong with this? How do you import a class in python?
If you're using Python 3, then imports are absolute by default. This means that import example will look for an absolute package named example, somewhere in the module search path.
So instead, you probably want a relative import. This is useful when you want to import a module that is relative the module doing the importing. In this case:
from ..example.ClassExample import ClassExample
I'm assuming that your folders are Python packages, meaning that they contain __init__.py files. So your directory structure would look like this.
src
|-- __init__.py
|-- example
| |-- __init__.py
| |-- ClassExample.py
|-- test
| |-- __init__.py
| |-- ClassExampleTest.py
I am creating a Python package/library. My directory structure looks like this:
my_package/
|-- my_package/
| |-- tests/
| | |-- __init__.py
| | |-- my_tests.py
| |
| |-- __init__.py
| |-- main.py
|
|-- setup.py
I have all my functions in the main.py file:
def sum_nums(a,b):
res = a + b
return(res)
def mult_nums(a,b):
res = a * b
return(res)
def sub_nums(a,b):
res = a - b
return(res)
my_tests.py looks like this:
from unittest import TestCase
import my_package
def test_sum():
assert sum_nums(3,4) == 7
def test_mult():
assert mult_nums(3,4) == 12
def test_sub():
assert sub_nums(3,4) == -1
When I run my tests from the package root directory as follows:
python setup.py test
... I get the following error:
NameError: name 'sum_nums' is not defined
Is my package directory structure correct?
Am I missing an _ init _.py file?
Does every directory require an _ init _.py file?
Is it okay to place all my functions inside a single main.py file
without using if name == "main"?
You need to indicate that the functions under test came for the my_package package like:
from unittest import TestCase
import my_package
def test_sum():
assert my_package.sum_nums(3,4) == 7
def test_mult():
assert my_package.mult_nums(3,4) == 12
def test_sub():
assert my_package.sub_nums(3,4) == -1
tests should not be in the package so move it up one dir. See sanic or any other module in github for example. Your functions need to be available in init.py. You can import them like is done in sanic.
https://github.com/huge-success/sanic
You also need
from my_package import sum_nums, mult_nums, sub_nums
Or prefix with my_package.
I'm having a problem with assigning static files containing resources.
My working directory structure is:
|- README.md
|- nlp
| |-- morpheme
| |-- |-- morpheme_builder.py
| |-- fsa_setup.py
| - tests
| |-- test_fsa.py
| - res
| |-- suffixes.xml
The code for fsa_setup.py is:
class FSASetup():
fsa = None
def get_suffixes():
list_suffix = list()
file = os.path.realpath("../res/suffixes.xml")
.....
if __name__ == "__main__":
FSASetup.get_suffixes()
The code for morpheme_builder.py is:
class MorphemeBuilder:
def get_all_words_from_fsa(self):
......
if __name__ == "__main__":
FSASetup.get_suffixes()
When it is called in fsa_setup.py, the file path's value is '\res\suffixes.xml' and that is correct, but when the other case realized, the file path value is '\nlp\res\suffixes.xml'.
I know how it works like this. So how can I give the path of the resource to the file.
The problem is that morpheme_builder.py is in the directory morphem. So when you say ../res/suffixes.xml it will go on directory back ... so it will go to nlp/res/suffixes.xml. What about if you use os.path.abspath("../res/suffixes.xml")?