this is my project structure (just an example to illustrate the problem):
.
├── hello_world
│ ├── __init__.py
│ └── some
│ └── very_nested
│ └── stuff.py
└── tests
└── test_stuff.py
The test_stuff.py file (for py.test):
from hello_world.some.very_nested.stuff import Magic
from hello_world.some.very_nested.other_stuff import MoreMagic
def test_magic_fact_works():
assert Magic().fact(3) == 6
# ...
Is there any way how to make the import lines shorter? They get too long in the real project.
For example, this would be nice, but it doesn't work :)
import hello_world.some.very_nested as vn
from vn.stuff import Magic
from vn.other_stuff import MoreMagic
I cannot use relative imports (I assume) beucase the tests are not inside the package. I could move them, but is it possible without changing project structure?
As #jonrsharpe says, you can aggregate your packages in a django style:
"""
Django validation and HTML form handling.
"""
from django.core.exceptions import ValidationError # NOQA
from django.forms.boundfield import * # NOQA
from django.forms.fields import * # NOQA
from django.forms.forms import * # NOQA
from django.forms.formsets import * # NOQA
from django.forms.models import * # NOQA
from django.forms.widgets import * # NOQA
And in your subpackage, eg.: django.forms.widgets add this:
__all__ = (
'Media', 'MediaDefiningClass', 'Widget', 'TextInput', 'NumberInput',
'EmailInput', 'URLInput', 'PasswordInput', 'HiddenInput',
'MultipleHiddenInput', 'FileInput', 'ClearableFileInput', 'Textarea',
'DateInput', 'DateTimeInput', 'TimeInput', 'CheckboxInput', 'Select',
'NullBooleanSelect', 'SelectMultiple', 'RadioSelect',
'CheckboxSelectMultiple', 'MultiWidget', 'SplitDateTimeWidget',
'SplitHiddenDateTimeWidget', 'SelectDateWidget',
)
To specify which items you want to import when using import *, this way you can organize your packages as deep as you want and keep them accessible at the same time.
In your case it would be something like:
hello_world/__init__.py
from hello_world.some.very_nested.stuff import *
from hello_world.some.very_nested.other_stuff import *
SO when importing your packages in tests for instance, you get this: from hello_world import Magic
Put into hello_world/__init__.py:
from __future__ import absolute_import
from .some import *
Into hello_world/some/__init__.py:
from __future__ import absolute_import
from .very_nested import *
Into hello_world/some/very_nested/__init__.py:
from __future__ import absolute_import
from .stuff import Magic
from .other_stuff import MoreMagic
So if hello_world/some/very_nested/stuff.py contains:
class Magic:
pass
And hello_world/some/very_nested/other_stuff.py contains:
class OtherMagic:
pass
Then you can easy import it. tests/test_stuff.py:
import pytest
from hello_world import Magic
from hello_world import MoreMagic
#pytest.mark.parametrize('cls', [Magic, MoreMagic])
def test_magic(cls):
assert cls()
Related
I have a folder structure as below
\root
|-collections
|-__init__.py
|-collection1.py => contains Collection1 class
|-collection2.py => contains Collection2 class
|-...so on
|-db.py
Inside of db.py, I need to use all of the Collection<n> classes. Is there any way to import them like the code below? Apparently __all__ variable in __init__.py only allows module names, not classes.
# db.py
from root.collections import *
a = Collection1()
b = Collection2()
...
Here's my trial and error
# collections\__init__.py
from collection1 import Collection1
from collection2 import Collection2
from collection3 import Collection3
__all__ = ['Collection1', 'Collection2', 'Collection3']
# db.py
def test_collections():
from root.collections import *
a = Collection1()
test_collections()
And this gives me...
SyntaxError: import * only allowed at module level
Import the classes in __init__.py:
__all__ = ['Collection1', 'Collection2']
from .collection1 import Collection1
from .collection2 import Collection2
I am running a python project like this:
project
Test.py
COMMON.py
SYSTEM.py
PTEST1
Hello.py
when run the code "Test.py" it will show NameError, I am not sure why?
But if I replaced the "from SYSTEM import *" with "from COMMON import *" in Test.py and PTEST1/Hello.py, it works as expect.
#Test.py is like this:
from SYSTEM import *
myvalue.Hello.printf()
# COMMON.py is like this:
myvalue = lambda: None
from PTEST1.Hello import Hello
myvalue.Hello = Hello
# SYSTEM.py is like this:
from COMMON import *
#PTEST1/Hello.py
from SYSTEM import *
class Hello():
#staticmethod
def printf():
print("Hello1")
print(vars(myvalue))
I expect there is no "NameError" by not changing import code. BTW, my python is 3.6+
Good practice is to give filenames in lowercase.
It looks like you are creating a Python project under project/. Any directory needs to have a file __init__.py in every directory in order for it to be discovered in Python.
You then need to refer to modules by their full name (not relative naming).
So the directory structure should be:
project/
__init__.py
test.py
common.py
system.py
ptest1/
__init__.py
hello.py
Every time you refer to file you should give the full path.
# import everything from hello.py
from project.ptest1.hello import *
# import everything from common.py
from project.common import *
# import everything from system.py
from project.system import *
I have the following structure for my views directory.
views
|--__init__.py
|--a_management.py
|--b_management.py
|--c_management.py
|--d_management.py
|--e_management.py
and __init__.py starts with the following:
from a_management import *
from b_management import *
from c_management import *
from d_management import *
from e_management import *
.........
...etc...
.........
In each of the files (a_management.py,b_management.py...) I need to write the same code importing modules:
import sys,os
from django.shortcuts import render, redirect
from django.http import HttpResponseRedirect
.........
...etc...
.........
Is it possible to perform all the module imports only in __init__.py?
When I tried, it still seems like the imports are not found and I get errors like: NameError: global name 'sys' is not defined
If each management module needs access to sys, then they all must import it. No way around that. (And really, if they all need it, then they all should import it. It's not a bad thing.)
You could save a bit of typing by having __init__ import sys, os and whtever else is needed, and then each management module can do from __init__ import *, thus "inheriting" all the imported modules from __init__.
Well, except you can't do it this way, because __init__ already imports stuff from the management modules, so the above suggestion would result in circular imports, which are a no-no.
I don't know the specifics of your application, but I have to believe that there's a better way to organize your modules to avoid so much repeated importing, and especially so much use of import *. Generally you want to use that as little as possible.
What about using a shared script to do all the system imports?
BTW, I agree that import * is not the greatest of idea. It makes sense in my use of importer, but I am not so sure in your general setup. Also, you need to careful about circular imports.
So, my answer is specifically only geared towards I need to write the same code importing modules:, not towards whether your setup as a whole makes sense.
Proof of concept, importer is what you really care about.:
├── pack
│ ├── __init__.py
│ ├── importer.py
│ ├── mgmt_1.py
│ ├── mgmt_2.py
└── test.py
test.py
import pack
pack.foo_1()
pack.foo_2()
init.py
from mgmt_1 import *
from mgmt_2 import *
mgmt_1.py
from .importer import *
print "sys", sys
print "os", os
def foo_1():
print "foo_1"
mgmt_2.py:
from .importer import *
print "sys", sys
print "os", os
def foo_2():
print "foo_2", dir(sys)[0:5]
importer.py
import sys
import os
output:
sys <module 'sys' (built-in)>
os <module 'os' from '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.pyc'>
sys <module 'sys' (built-in)>
os <module 'os' from '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.pyc'>
foo_1
foo_2:sys ['__displayhook__', '__doc__', '__excepthook__', '__name__', '__package__']
I place my frequently used modules in a subdirectory lib/ and expect to load all modules into main.py by: (refer to Python: how to import from all modules in dir?)
from lib import *
but encounter the issue TypeError: 'module' object is not callable. More specifically, in main.py:
#!/usr/bin/env python
from lib import * # cause: TypeError: 'module' object is not callable
#from lib.DominatingSets import * # it works
dominatingSets = DominatingSets()
The full exception traceback:
$ python main.py
Traceback (most recent call last):
File "main.py", line 6, in <module>
dominatingSets = DominatingSets()
TypeError: 'module' object is not callable
The directories in a tree-like format.
$ tree -P '*.py' .
.
├── __init__.py
├── lib
│ ├── AnalyzeGraph.py
│ ├── AutoVivification.py
│ ├── DominatingSets.py
│ ├── __init__.py
│ ├── Output.py
│ ├── PlotGraph.py
│ ├── ProcessDatasets.py
│ └── ReadGTFS.py
├── main.py
The contents of lib/__init__.py are as follows. (refer to Loading all modules in a folder in Python)
from os.path import dirname, basename, isfile
import glob
modules = glob.glob(dirname(__file__)+"/*.py")
__all__ = [ basename(f)[:-3] for f in modules if isfile(f) and not basename(f).startswith('__')] # exclude __init__.py
This confusion happened, in part, because your module names are the same as the names of the classes you want to load from them. (At least, that's what makes it more confusing.) Your code does correctly load the modules that your classes are in. However, it doesn't load the classes out of those modules, and this is what you actually wanted to do.
Because your class DominatingSets is in the module lib.DominatingSets, its full path from root is lib.DominatingSets.DominatingSets.
from lib import *
in your case will do the same thing as
from lib import DominatingSets
from lib import AnalyzeGraph
# ...
However,
from lib import DominatingSets
is equivalent to
import lib.DominatingSets
DominatingSets = lib.DominatingSets
but lib.DominatingSets is a module (lib/DominatingSets.py), not the class you want.
from lib.DominatingSets import DominatingSets
is equivalent to
import lib.DominatingSets
DominatingSets = lib.DominatingSets.DominatingSets
which is why it works: this is the class you want imported into the name DominatingSets.
If you want to have from lib import * import all the classes in the submodules, you need to import these classes into the lib module. For example, in lib/__init__.py:
from DominatingSets import *
from AnalyzeGraph import *
# ...
While you're making changes, I'd suggest (as others have) using normal Python naming conventions, and have your module names in lowercase: change DominatingSets.py to dominatingsets.py. Then this code would become
from dominatingsets import *
from analyzegraph import *
# ...
Looking at your Traceback, I think your problem might lie here:
Firstly, lets look at an example:
import datetime
d = datetime(2005, 23, 12)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'module' object is not callable
Basically, we've just imported the entire datetime module, and we're trying to call it like a class object within a module. Let's now do:
k = datetime.datetime(2005, 12, 22)
print k
2005-12-22 00:00:00
No problems this time, as we are referencing the datetime object type within the datetime module
If we do:
from datetime import datetime
datetime
<type 'datetime.datetime'>
Again we reach the desired object, as we are importing the datetime class within the datetime module.
Also, using *
from datetime import *
d = datetime(2005, 3, 12)
will also work, as you are importing all the classes within the datetime module.
Your code saying:
from lib import * # This imports all MODULES within lib, not the classes
#from lib.DominatingSets import * # it works because you import the classes within the DominatingSets Module
You could either use from lib.DominatingSets import DominatingSets which should solve your problem, or if you stick to from lib import *, change your code to dominatingsets = DominatingSets.DominatingSets()
Hope this helps!
I learnt a lot from the accepted answer here ... but I still had problems with what to put in the lib/__init__.py file, if this directory lib is not actually included in PYTHONPATH.
I found that in addition to adding the parent directory of lib in the caller file, i.e.
sys.path.append( '.../parent_dir_of_lib' )
I either 1) had to do this in addition in the caller file:
sys.path.append( '.../parent_dir_of_lib/lib' )
Or 2) had to make the lib directory "self-loading", by putting this in its __init__.py:
import sys
from pathlib import Path
parent_dir_str = str( Path(__file__).resolve().parent )
sys.path.append( parent_dir_str )
from analyse_graph import *
from auto_vivification import *
...
My project structure:
project1/
__init__.py
main/
__init__.py
{main}.py
file1.py
file2.py
subA/
__init__.py
{moduleA}.py
class {A_class}
file3.py
file4.py
subB/
__init__.py
{moduleB}.py
file5.py
file6.py
I want to import {muduleA}.py in {moduleB}.py. Why cannot do this attempt?
in {moduleB}.py...
from project1.subA import {moduleA}
(not work as well followings..)
from ..subA import {moduleA}
from ..subA.{moduleA} import {A_class}
from project1.subA.{moduleA} import {A_class}
import project1.subA.{moduleA}.{A_class}
solved as following
import sys,os
sys.path.append(os.path.abspath('../../..'))
import project1.subA.moduleA.A_class
from project1.subA.moduleA import A_class
It will be better to import module explicitly to avoid confusing in the future.