I'm trying to get Python to list all modules in a namespace package.
I have the following file structure:
cwd
|--a
| `--ns
| |--__init__.py
| `--amod.py
|--b
| `--ns
| |--__init__.py
| `--bmod.py
`--c
`--ns
|--__init__.py
`--cmod.py
Each __init__.py defines it's package as a namespace package by having the following line:
__import__('pkg_resources').declare_namespace(__name__)
The amod module contains a class A, bmod contains another class B and cmod contains the C class.
When having a clean environment the following happens:
>>> import inspect, sys, glob
>>> sys.path += glob.glob('*')
>>> import ns
>>> inspect.getmembers(ns, inspect.ismodule)
[]
As you can see, the modules are not listed.
Now when I import the modules manually and then call the inspect again:
>>> inspect.getmembers(ns, inspect.ismodule)
[('amod', <module 'ns.amod' from 'a/ns/amod.pyc'>), ('bmod', <module 'ns.bmod' from 'b/ns/bmod.pyc'>), ('cmod', <module 'ns.cmod' from 'c/ns/cmod.pyc'>)]
Now I'd like the inspect call to work without importing the modules manually, so how can I achieve this?
Related
My directory structure is
app.py
lib
__init__.py
_foo.py
Inside __init__.py I have written
from . import _foo as foo
Then inside app.py I try to make a call
from lib.foo import *
but it throws ModuleNotFoundError: No module named 'lib.foo' exception.
Basically I want to import everything from _foo.py, through the __init__.py script.
While I realize the code works if _foo.py is renamed into foo.py,
I still wonder if there is any way to make import work through __init__.py.
Not sure about hacking around the import statements, but you could get away with something less explicit like this:
lib/__init__.py
from . import _foo as foo
__all__ = ['foo']
lib/_foo.py
__all__ = [
'test'
]
test = 1
>>> from lib import *
>>> foo
<module 'lib._foo' from '/path/to/test/lib/_foo.py'>
>>> foo.test
1
>>>
EDIT: You could achieve something more explicit by updating sys.modules at runtime:
app.py
import sys
from lib import _foo
sys.modules['lib.foo'] = _foo
lib/_foo.py
test = 1
keep lib/__init__.py to make lib a module
After importing app lib.foo will be an available module
>>> import app
>>> from lib import foo
>>> foo
<module 'lib._foo' from '/path/to/test/lib/_foo.py'>
>>> foo.test
1
I'm building a package and a noticed that when I import the submodules, they include all of the built-ins that I've imported as well. Is there a way to get around this so when I navigate the submodule with tab complete only the functions and objects from the script are present?
For example, when I import examplemodule.submodule to only see function_i_want when I'm navigating the package contents?
Directory structure
examplemodule
| __init__.py
| submodule
| __init__.py
| submodule.py
examplemodule | submodule | submodule.py
from collections import *
def function_i_want():
return True
Here's an example of what I can import from the module:
>>> import examplemodule
>>> from examplemodule import submodule
>>> submodule.
submodule.AsyncGenerator( submodule.MappingView(
submodule.AsyncIterable( submodule.MutableMapping(
submodule.AsyncIterator( submodule.MutableSequence(
submodule.Awaitable( submodule.MutableSet(
submodule.ByteString( submodule.OrderedDict(
submodule.Callable( submodule.Reversible(
submodule.ChainMap( submodule.Sequence(
submodule.Collection( submodule.Set(
submodule.Container( submodule.Sized(
submodule.Coroutine( submodule.UserDict(
submodule.Counter( submodule.UserList(
submodule.Generator( submodule.UserString(
submodule.Hashable( submodule.ValuesView(
submodule.ItemsView( submodule.defaultdict(
submodule.Iterable( submodule.deque(
submodule.Iterator( submodule.function_i_want(
submodule.KeysView( submodule.namedtuple(
submodule.Mapping( submodule.submodule
When you say from x.y import * you are importing everything defined in __all__ from the module y that resides in directory x.
If you'd only like to import a subset of y you can do the following:
Limit what you're importing from your script
from examplemodule.submodule import function_i_want
or
from examplemodule.submodule import (
function_i_want,
other_function_i_want
)
Define __all__ in your __init__.py
__all__ = ['function_i_want', 'other_function_i_want']
what you could do is import just the module e.g.
import examplemodule
examplemodule.submodule
what this does is only calls the function if you write the module first then .submodule. this is also good if you have more than one module
Given a trivial Python package with an __init__.py:
$ ls -R foo/
foo/:
__init__.py bar.py
$ cat foo/bar.py
def do_stuff(): pass
$ cat foo/__init__.py
from .bar import *
I'm surprised that foo.bar is defined:
>>> import foo
>>> foo.bar
<module 'foo.bar' from 'foo/bar.pyc'>
My understanding of from x import * is that it doesn't define x in the current scope. For example:
>>> from abc import *
>>> abc
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'abc' is not defined
Why is foo.bar defined in my first example, even though I don't have import bar inside __init__.py?
When you reference it by foo.bar, it is not referencing to bar used in the import statement in __init__.py file, instead it is a reference to bar module/file itself. Even if you remove all the code in __init__.py file, import foo; foo.bar would still work.
If that weren't the case, you wouldn't have been able to do something like this
import foo.bar
Since foo is a package, as it contains __init__ file, hence it's internal files can be referenced directly.
Present situation:
I have a script: program.py.
I have a directory: /imports. Which contains an arbitrary amount of modules in subdirectories.
Python's package directory syntax is being used.
__init__.py
program.py
/imports
__init__.py
/module a
__init__.py
other_files.py
/module b
__init__.py
other
Intention:
program.py should import every subdirectory in /imports.
program.py needs object access to the modules. For example, imports['name'].Service().
Remark: I don't feel like illustrating my purpose or questioning the design choice. I'm using the above workaround.
The problem is described below.
The existing code:
import os
modules = []
IMPORT_PATH = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'imports')
to_import = [name for name in os.listdir(IMPORT_PATH) if os.path.isdir(os.path.join(IMPORT_PATH, name))]
modules = [__import__('imports.' + x) for x in to_import]
print modules
Output:
[<module 'imports' from '/code/test/imports/__init__.pyc'>, <module 'imports' from '/code/test/imports/__init__.pyc'>]
Desired output:
[<module 'B' from '/code/test/imports/B'>, <module 'C' from '/code/test/imports/C'>]
Problem identified: The fromlist=[] argument for __import__() is missing. See Why does Python's __import__ require fromlist? for elaboration.
Corrected example:
modules = {}
IMPORT_PATH = os.path.join(os.path.dirname(os.path.abspath(__file__)), 'imports')
to_import = [name for name in os.listdir(IMPORT_PATH) if os.path.isdir(os.path.join(IMPORT_PATH, name))]
for x in to_import:
modules[x] = __import__('imports.' + x, fromlist=['*'])
Output:
{'C': <module 'imports.C' from '/code/test/imports/C/__init__.pyc'>,
'B': <module 'imports.B' from '/code/test/imports/B/__init__.pyc'>}
Achieved result:
Now modules[dir_name].Service() can be used in program.py.
i have a 2.6 python script and library in the following directory structure:
+ bin
\- foo.py
+ lib
\+ foo
\- bar.py
i would like users to run bin/foo.py to instantiate the classes within lib/foo.py. to achieve this, in my bin/foo.py script i have the following code:
from __future__ import absolute_import
import foo
klass = foo.bar.Klass()
however, this results in:
AttributeError: 'module' object has no attribute 'bar'
ie it thinks that foo is itself rather than the library foo - renaming bin/foo.py to bin/foo-script.py works as expected.
is there a way i can keep the bin/foo.py script and import lib/foo.py?
The current directory is on the path by default, so you need to remove that before you import the other foo module:
import sys
sys.path = [dir for dir in sys.path if dir != '']
Alternatively, prepend the lib directory so that it takes precedence:
import sys
sys.path = ['../lib'] + sys.path
If you just write import foo, it will definitely load the foo module in the current scope. Assuming lib and foo as packages, won't you need to write something like this in order to make it work?
import lib.foo.bar as foobar
klass = foobar.Klass()