I have the following structure for my views directory.
views
|--__init__.py
|--a_management.py
|--b_management.py
|--c_management.py
|--d_management.py
|--e_management.py
and __init__.py starts with the following:
from a_management import *
from b_management import *
from c_management import *
from d_management import *
from e_management import *
.........
...etc...
.........
In each of the files (a_management.py,b_management.py...) I need to write the same code importing modules:
import sys,os
from django.shortcuts import render, redirect
from django.http import HttpResponseRedirect
.........
...etc...
.........
Is it possible to perform all the module imports only in __init__.py?
When I tried, it still seems like the imports are not found and I get errors like: NameError: global name 'sys' is not defined
If each management module needs access to sys, then they all must import it. No way around that. (And really, if they all need it, then they all should import it. It's not a bad thing.)
You could save a bit of typing by having __init__ import sys, os and whtever else is needed, and then each management module can do from __init__ import *, thus "inheriting" all the imported modules from __init__.
Well, except you can't do it this way, because __init__ already imports stuff from the management modules, so the above suggestion would result in circular imports, which are a no-no.
I don't know the specifics of your application, but I have to believe that there's a better way to organize your modules to avoid so much repeated importing, and especially so much use of import *. Generally you want to use that as little as possible.
What about using a shared script to do all the system imports?
BTW, I agree that import * is not the greatest of idea. It makes sense in my use of importer, but I am not so sure in your general setup. Also, you need to careful about circular imports.
So, my answer is specifically only geared towards I need to write the same code importing modules:, not towards whether your setup as a whole makes sense.
Proof of concept, importer is what you really care about.:
├── pack
│ ├── __init__.py
│ ├── importer.py
│ ├── mgmt_1.py
│ ├── mgmt_2.py
└── test.py
test.py
import pack
pack.foo_1()
pack.foo_2()
init.py
from mgmt_1 import *
from mgmt_2 import *
mgmt_1.py
from .importer import *
print "sys", sys
print "os", os
def foo_1():
print "foo_1"
mgmt_2.py:
from .importer import *
print "sys", sys
print "os", os
def foo_2():
print "foo_2", dir(sys)[0:5]
importer.py
import sys
import os
output:
sys <module 'sys' (built-in)>
os <module 'os' from '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.pyc'>
sys <module 'sys' (built-in)>
os <module 'os' from '/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/os.pyc'>
foo_1
foo_2:sys ['__displayhook__', '__doc__', '__excepthook__', '__name__', '__package__']
Related
I am running a python project like this:
project
Test.py
COMMON.py
SYSTEM.py
PTEST1
Hello.py
when run the code "Test.py" it will show NameError, I am not sure why?
But if I replaced the "from SYSTEM import *" with "from COMMON import *" in Test.py and PTEST1/Hello.py, it works as expect.
#Test.py is like this:
from SYSTEM import *
myvalue.Hello.printf()
# COMMON.py is like this:
myvalue = lambda: None
from PTEST1.Hello import Hello
myvalue.Hello = Hello
# SYSTEM.py is like this:
from COMMON import *
#PTEST1/Hello.py
from SYSTEM import *
class Hello():
#staticmethod
def printf():
print("Hello1")
print(vars(myvalue))
I expect there is no "NameError" by not changing import code. BTW, my python is 3.6+
Good practice is to give filenames in lowercase.
It looks like you are creating a Python project under project/. Any directory needs to have a file __init__.py in every directory in order for it to be discovered in Python.
You then need to refer to modules by their full name (not relative naming).
So the directory structure should be:
project/
__init__.py
test.py
common.py
system.py
ptest1/
__init__.py
hello.py
Every time you refer to file you should give the full path.
# import everything from hello.py
from project.ptest1.hello import *
# import everything from common.py
from project.common import *
# import everything from system.py
from project.system import *
I'm building a package and a noticed that when I import the submodules, they include all of the built-ins that I've imported as well. Is there a way to get around this so when I navigate the submodule with tab complete only the functions and objects from the script are present?
For example, when I import examplemodule.submodule to only see function_i_want when I'm navigating the package contents?
Directory structure
examplemodule
| __init__.py
| submodule
| __init__.py
| submodule.py
examplemodule | submodule | submodule.py
from collections import *
def function_i_want():
return True
Here's an example of what I can import from the module:
>>> import examplemodule
>>> from examplemodule import submodule
>>> submodule.
submodule.AsyncGenerator( submodule.MappingView(
submodule.AsyncIterable( submodule.MutableMapping(
submodule.AsyncIterator( submodule.MutableSequence(
submodule.Awaitable( submodule.MutableSet(
submodule.ByteString( submodule.OrderedDict(
submodule.Callable( submodule.Reversible(
submodule.ChainMap( submodule.Sequence(
submodule.Collection( submodule.Set(
submodule.Container( submodule.Sized(
submodule.Coroutine( submodule.UserDict(
submodule.Counter( submodule.UserList(
submodule.Generator( submodule.UserString(
submodule.Hashable( submodule.ValuesView(
submodule.ItemsView( submodule.defaultdict(
submodule.Iterable( submodule.deque(
submodule.Iterator( submodule.function_i_want(
submodule.KeysView( submodule.namedtuple(
submodule.Mapping( submodule.submodule
When you say from x.y import * you are importing everything defined in __all__ from the module y that resides in directory x.
If you'd only like to import a subset of y you can do the following:
Limit what you're importing from your script
from examplemodule.submodule import function_i_want
or
from examplemodule.submodule import (
function_i_want,
other_function_i_want
)
Define __all__ in your __init__.py
__all__ = ['function_i_want', 'other_function_i_want']
what you could do is import just the module e.g.
import examplemodule
examplemodule.submodule
what this does is only calls the function if you write the module first then .submodule. this is also good if you have more than one module
I place my frequently used modules in a subdirectory lib/ and expect to load all modules into main.py by: (refer to Python: how to import from all modules in dir?)
from lib import *
but encounter the issue TypeError: 'module' object is not callable. More specifically, in main.py:
#!/usr/bin/env python
from lib import * # cause: TypeError: 'module' object is not callable
#from lib.DominatingSets import * # it works
dominatingSets = DominatingSets()
The full exception traceback:
$ python main.py
Traceback (most recent call last):
File "main.py", line 6, in <module>
dominatingSets = DominatingSets()
TypeError: 'module' object is not callable
The directories in a tree-like format.
$ tree -P '*.py' .
.
├── __init__.py
├── lib
│ ├── AnalyzeGraph.py
│ ├── AutoVivification.py
│ ├── DominatingSets.py
│ ├── __init__.py
│ ├── Output.py
│ ├── PlotGraph.py
│ ├── ProcessDatasets.py
│ └── ReadGTFS.py
├── main.py
The contents of lib/__init__.py are as follows. (refer to Loading all modules in a folder in Python)
from os.path import dirname, basename, isfile
import glob
modules = glob.glob(dirname(__file__)+"/*.py")
__all__ = [ basename(f)[:-3] for f in modules if isfile(f) and not basename(f).startswith('__')] # exclude __init__.py
This confusion happened, in part, because your module names are the same as the names of the classes you want to load from them. (At least, that's what makes it more confusing.) Your code does correctly load the modules that your classes are in. However, it doesn't load the classes out of those modules, and this is what you actually wanted to do.
Because your class DominatingSets is in the module lib.DominatingSets, its full path from root is lib.DominatingSets.DominatingSets.
from lib import *
in your case will do the same thing as
from lib import DominatingSets
from lib import AnalyzeGraph
# ...
However,
from lib import DominatingSets
is equivalent to
import lib.DominatingSets
DominatingSets = lib.DominatingSets
but lib.DominatingSets is a module (lib/DominatingSets.py), not the class you want.
from lib.DominatingSets import DominatingSets
is equivalent to
import lib.DominatingSets
DominatingSets = lib.DominatingSets.DominatingSets
which is why it works: this is the class you want imported into the name DominatingSets.
If you want to have from lib import * import all the classes in the submodules, you need to import these classes into the lib module. For example, in lib/__init__.py:
from DominatingSets import *
from AnalyzeGraph import *
# ...
While you're making changes, I'd suggest (as others have) using normal Python naming conventions, and have your module names in lowercase: change DominatingSets.py to dominatingsets.py. Then this code would become
from dominatingsets import *
from analyzegraph import *
# ...
Looking at your Traceback, I think your problem might lie here:
Firstly, lets look at an example:
import datetime
d = datetime(2005, 23, 12)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'module' object is not callable
Basically, we've just imported the entire datetime module, and we're trying to call it like a class object within a module. Let's now do:
k = datetime.datetime(2005, 12, 22)
print k
2005-12-22 00:00:00
No problems this time, as we are referencing the datetime object type within the datetime module
If we do:
from datetime import datetime
datetime
<type 'datetime.datetime'>
Again we reach the desired object, as we are importing the datetime class within the datetime module.
Also, using *
from datetime import *
d = datetime(2005, 3, 12)
will also work, as you are importing all the classes within the datetime module.
Your code saying:
from lib import * # This imports all MODULES within lib, not the classes
#from lib.DominatingSets import * # it works because you import the classes within the DominatingSets Module
You could either use from lib.DominatingSets import DominatingSets which should solve your problem, or if you stick to from lib import *, change your code to dominatingsets = DominatingSets.DominatingSets()
Hope this helps!
I learnt a lot from the accepted answer here ... but I still had problems with what to put in the lib/__init__.py file, if this directory lib is not actually included in PYTHONPATH.
I found that in addition to adding the parent directory of lib in the caller file, i.e.
sys.path.append( '.../parent_dir_of_lib' )
I either 1) had to do this in addition in the caller file:
sys.path.append( '.../parent_dir_of_lib/lib' )
Or 2) had to make the lib directory "self-loading", by putting this in its __init__.py:
import sys
from pathlib import Path
parent_dir_str = str( Path(__file__).resolve().parent )
sys.path.append( parent_dir_str )
from analyse_graph import *
from auto_vivification import *
...
My project structure:
project1/
__init__.py
main/
__init__.py
{main}.py
file1.py
file2.py
subA/
__init__.py
{moduleA}.py
class {A_class}
file3.py
file4.py
subB/
__init__.py
{moduleB}.py
file5.py
file6.py
I want to import {muduleA}.py in {moduleB}.py. Why cannot do this attempt?
in {moduleB}.py...
from project1.subA import {moduleA}
(not work as well followings..)
from ..subA import {moduleA}
from ..subA.{moduleA} import {A_class}
from project1.subA.{moduleA} import {A_class}
import project1.subA.{moduleA}.{A_class}
solved as following
import sys,os
sys.path.append(os.path.abspath('../../..'))
import project1.subA.moduleA.A_class
from project1.subA.moduleA import A_class
It will be better to import module explicitly to avoid confusing in the future.
I have something like this:
# a.py
import os
class A:
...
# b.py
import a
class B(A):
...
In class B (b.py) I'd like to be able to use the modules imported in a.py (os in this case). Is it possible to achieve this behavior in Python or should I import the modules in both files?
Edit: I'm not worried about the import times, my problem is the visual clutter that the block of imports puts on the files. I end up having stuff like this in every controller (RequestHandler):
from django.utils import simplejson
from google.appengine.ext import webapp
from google.appengine.ext.webapp import template
from google.appengine.ext import db
That's what I'd like to avoid.
Yes you can use the imports from the other file by going a.os.
However, the pythonic way is to just import the exact modules you need without making a chain out of it (which can lead to circular references).
When you import a module, the code is compiled and inserted into a dictionary of names -> module objects. The dictionary is located at sys.modules.
import sys
sys.modules
>>> pprint.pprint(sys.modules)
{'UserDict': <module 'UserDict' from 'C:\python26\lib\UserDict.pyc'>,
'__builtin__': <module '__builtin__' (built-in)>,
'__main__': <module '__main__' (built-in)>,
'_abcoll': <module '_abcoll' from 'C:\python26\lib\_abcoll.pyc'>,
# the rest omitted for brevity
When you try to import the module again, Python will check the dictionary to see if its already there. If it is, it will return the already compiled module object to you. Otherwise, it will compile the code, and insert it in sys.modules.
Since dictionaries are implemented as hash tables, this lookup is very quick and takes up negligible time compared to the risk of creating circular references.
Edit: I'm not worried about the import
times, my problem is the visual
clutter that the block of imports puts
on the files.
If you only have about 4 or 5 imports like that, its not too cluttery. Remember, "Explicit is better than implicit". However if it really bothers you that much, do this:
<importheaders.py>
from django.utils import simplejson
from google.appengine.ext import webapp
from google.appengine.ext.webapp import template
from google.appengine.ext import db
<mycontroller.py>
from importheaders import *
Just import the modules again.
Importing a module in python is a very lightweight operation. The first time you import a module, python will load the module and execute the code in it. On any subsequent imports, you will just get a reference to the already-imported module.
You can verify this yourself, if you like:
# module_a.py
class A(object):
pass
print 'A imported'
# module_b.py
import module_a
class B(object):
pass
print 'B imported'
# at the interactive prompt
>>> import module_a
A imported
>>> import module_a # notice nothing prints out this time
>>> import module_b # notice we get the print from B, but not from A
B imported
>>>
You should import it separately. However, if you really need to forward some functionality, you can return a module from a function. Just:
import os
def x:
return os
But it seems like a plugin functionality - objects + inheritance would solve that case a bit better.
Sounds like you are wanting to use python packages. Look into those.
Yep. Once you import a module, that module becomes a property of the current module.
# a.py
class A(object):
...
# b.py
import a
class B(a.A):
...
In Django, for example, many of the packages simply import the contents of other modules. Classes and functions are defined in separate files just for the separation:
# django/db/models/fields/__init__.py
class Field(object):
...
class TextField(Field):
...
# django/db/models/__init__.py
from django.db.models.fields import *
# mydjangoproject/myapp/models.py
from django.db import models
class MyModel(models.Model):
myfield = models.TextField(...)
....
First you can shorten it to:
from django.utils import simplejson
from google.appengine.ext import webapp, db
from webapp import template
Secondly suppose you have those ^ imports in my_module.py
In my_module2.py you can do:
from my_module2.py import webapp, db, tempate
example:
In [5]: from my_module2 import MyMath2, MyMath
In [6]: m2 = MyMath2()
In [7]: m2.my_cos(3)
Out[7]: 0.94398413915231416
In [8]: m = MyMath()
In [9]: m.my_sin(3)
Out[9]: 0.32999082567378202
where my_module2 is:
from my_module import math, MyMath
class MyMath2(object):
the_meaning_of_life = 42
def my_cos(self, number):
return math.cos(number * 42)
and my_module1 is:
import math
class MyMath(object):
some_number = 42
def my_sin(self, num):
return math.sin(num * self.some_number)
Cheers,
Hope it helps
AleP