I am using gobject-introspection in python2.7 on ubuntu raring and I run into an import error while building some packages. I have isolated a minimal set of steps to replicate it:
Make a local directory structure:
gi:
__init__.py
overrides:
__init__.py
Put the standard boilerplate
from pkgutil import extend_path
__path__ = extend_path(__path__, __name__)
print __path__, __name__
in both __init__.py files.
From the directory containing your local copy of gi, run the following:
python -c "from gi import repository"
I get an error message that looks like:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/usr/lib/python2.7/dist-packages/gi/repository/__init__.py", line 25, in <module>
from ..importer import DynamicImporter
File "/usr/lib/python2.7/dist-packages/gi/importer.py", line 28, in <module>
from .module import DynamicModule
File "/usr/lib/python2.7/dist-packages/gi/module.py", line 37, in <module>
from .overrides import registry
ImportError: cannot import name registry
Any explanation? I can't find any decent documentation of the intended behavior because gobject-introspection seems like a very poorly documented project. Help is greatly appreciated!
From the Python documentation:
The __init__.py files are required to make Python treat the
directories as containing packages; this is done to prevent
directories with a common name, such as string, from unintentionally
hiding valid modules that occur later on the module search path.
Simply by having those __init__.py files accessible from the running directory, you are telling the interpreter that that is an implementation of the gi module. Any usage of the main gi module will not go correctly.
Now, why does it print the error as coming from /usr/lib ? Because gi was found in local/gi, but gi.repository was found in /usr/lib/python2.7/dist-packages/gi/repository . It is running /usr/lib/python2.7/dist-packages/gi/repository/__init__.py. From there, it imports some other submodules correctly, but when it tries to import overrides it finds your local stub in gi/overrides. Your stub does not define registry, so you have the fail.
Try and put registry='dumb_string' in gi/overrides/__init__.py and see that the error is gone.
Related
I've this package called classevivaAPI on PyPI (you can find it on GitHub here), of which I've for obvious reasons a git clone on my computer.
# main.py
from paths import paths
from variables.variables import *
In main.py I've the imports above, when main.py is the file marked as modified (with an M) below.
I'm testing this package with test.py, the gitignored file at the bottom of the screenshot, which has the following content:
#!\usr\bin\env python3
from classeviva import Session, Valutazioni, Note, Registro
from classeviva.variables import NoteSortBy
if __name__ == "__main__":
# Some code here
classeviva is the module included in the package classevivaAPI, and test.py is importing it via pip, I tested it and it gave me the same problem even if runned from a directory very far from src/ and in general src/classeviva.
When I ran it it gave me the following error:
Traceback (most recent call last):
File "d:\Python\Python\Classeviva\test.py", line 2, in <module>
from classeviva import Session, Valutazioni, Note, Registro
File "C:\Users\matti\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\classeviva\__init__.py", line 1, in <module>
from .main import *
File "C:\Users\matti\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\classeviva\main.py", line 3, in <module>
from paths import paths
ModuleNotFoundError: No module named 'paths'
I was quite confused since I had followed what explained here, here and expecially here.
So I had the idea to copy paste the if __name__ == '__main__' block of test.py at the bottom of main.py, and surprisingly it ran perfectly without giving me any error.
Does anybody know what is happening? Have you ever experienced this? Do you know how to fix it or what I'm doing wrong?
Edit
I searched for the module (classeviva) into the folder on my hard disk containing all the pip-installed modules, and I found it.
I opened its main.py, and the Visual Studio Code intellisense told me the same thing as CPython did before: there was no subfolder called path.
The setup.py file was missing the information about the submodules, I added them this way:
packages=[
'classeviva',
'classeviva.exceptions',
'classeviva.paths',
'classeviva.variables'
],
package_dir={
'': 'src',
'classeviva.exceptions': 'src/classeviva/exceptions',
'classeviva.paths': 'src/classeviva/paths',
'classeviva.variables': 'src/classeviva/variables',
},
I changed my main.py imports to the following:
# main.py
from .paths import paths
from .variables.variables import *
The .s were needed to access the subdirectories.
Now everything works fine.
I'm having trouble with a python package that uses separate modules to structure code. The package itself is working, however when imported from another environment fails with a ModuleNotFound error.
Here's the structure:
Project-root
|
|--src/
| __init__.py
| module_a.py
| module_b.py
| module_c.py
| module_d.py
|--tests
etc.
In module_a.py I have:
from module_a import function_a1,...
from module_b import function_b1,...
from module_c import function_c1,...
In module_c I import module_d like:
from module_d import function_d1,...
As mentioned above, executing module_a or module_c directly from the CLI work as expected, the unit tests I've created in the test directory also work (with the help of sys.path.insert), however if I create a new environment and import the package I get the following error:
>>> import module_a
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/<abs_path>/.venv/lib/python3.9/site-packages/module_a.py", line 22, in <module>
from module_c import function_c1, function_c2
File /<abs_path>/.venv/lib/python3.9/site-packages/module_c.py", line 9, in <module>
import module_d
ModuleNotFoundError: No module named 'module_d'
>>>
I've exhausted all ideas how to overcome this, besides combining the code of modules c and d in one file, which I'd hate to do, or rethink the flow so that all modules are imported from module_a.
Any suggestions how to approach this would be greatly appreciated.
Update: It turned out to be a typing mistake in the name of module_d in setup.py. For whatever reason python setup.py install was failing silently or I wasn't reading the logs carefully.
The problem comes down to understanding the basics of the import system and the PYTHONPATH.
When you try to import a module (import module_a), Python will search in order in every directory listed in sys.path. If a directory matches the name (module_a)1, then it runs the __init__.py file is such exist.
When you get an [https://docs.python.org/3/library/exceptions.html#ImportError], it means that there is no directory in sys.path containing a directory with the name asked.
You said for your tests you did something like sys.path.insert(0, "some/path/"), but it is not a solution, just a broken fix.
What you should do is set your PYTHONPATH environment variable to contain the directory where your modules are located, Project-root/src in your case. That way, no need to ever use sys.path.insert, or fiddle with relative/absolute paths in import statements.
When you create your new environment, just set your environment variable PYTHONPATH to include Project-root/src and you are done. This is how installing regular Python modules (libraries) work : they are all put into a directory in site-packages.
1: this changed since old Python versions, it used to be required for the directory to contain an __init__.py file
I need to run unit tests for my Flask app. In my test config file, I need to import the flask create_app function from a sibling directory to initialize the test app. I cannot figure it out without getting import errors.
I have tried putting __init__.py on virtually every folder without success. I have read that editing the sys path is not recommended so I would like a solution without.
Folder Structure
root/
----__init__.py
----server/
--------__init__.py
--------database.py
----tests/
--------__init__.py
--------config.py
config.py
from server import create_app
from server.database import db
Raises this error:
Traceback (most recent call last):
File "tests/config.py", line 2, in <module>
from server import create_app
ModuleNotFoundError: No module named 'server'
and:
from ..server import create_app
from ..server.database import db
Raises this error:
Traceback (most recent call last):
File "tests/config.py", line 2, in <module>
from ..server import create_app
ValueError: attempted relative import beyond top-level package
Can someone explain why it doesn't work and how python imports work in general? I have never been able to figure them out.
For an import statement to work correctly, names must be resolved in sys.path somehow. Creating a correct package structure and installing the package is usually the best way to get the names visible in sys.path.
Remove root/__init__.py and tests/__init__.py
Correct your import statements:
from ..server import create_app # no
from server import create_app # yes
Add root/setup.py with contents as described in setuptools basic guide.
Create/activate virtualenv (usually in root/.venv/ subdir, but doesn't really matter where)
From the project root, i.e. the directory containing setup.py, install your package:
pip install --editable .
I've read through a couple of similar questions, notably this one about imp.load_module which seems to be close to what I want, but I can't understand why I'm still getting ImportErrors. Here is my folder hierarchy:
program\
__init__.py
main.py
thirdparty\
__init__.py
css\
__init__.py
css.py
utils\
__init__.py
http.py
In main.py I have the following code. This is intended to search the thirdparty\ directory and load each module it finds. Each module is in its own separate directory.
import os
import imp
for root, dirs, files in os.walk("thirdparty"):
for source in (s for s in files if s.endswith(".py")):
name = os.path.splitext(os.path.basename(source))[0]
m = imp.load_module(name, *imp.find_module(name, [root]))
The problem is that css.py happens to use its own subfolder that it loads stuff off of, utils. It has a line in it that says:
from utils import http
And that is where it fails. I get this error when I run main.py.
Traceback (most recent call last):
File "main.py", line 7, in <module>
m = imp.load_module(name, *imp.find_module(name, [root]))
File "thirdparty/css/css.py", line 1, in <module>
from utils import http
ImportError: No module named utils
I'm stumped. css.py is self contained in its own folder, and when I run css.py separately it imports utils just fine. What is causing this?
Maybe you can solve this by changing the import to:
from .utils import http
Or by adding the folder you import to the Python Path:
sys.path.append(os.path.join(root, source))
When you import modules in thirdparty, the place Python looks for modules is still the main directory. The initial import works, because you give the right path to imp.find_module, but after that Python has no idea where to look for the modules.
This is my folder structure:
src/
__init__py
Lowlevel/
__init__.py
ModuleToCheck.Py
Test/
__init__.py
ModuleToCheck_test.py
(__init__.py are empty files)
Now I want to import ModuleToCheck.py in ModuleToCheck_test.py
How am I able to do this without appending anything to sys.path?
Update:
from ..Lowlevel import ModuleToCheck leads to:
src$ python Test/ModuleToCheck_test.py
Traceback (most recent call last):
File "Test/ModuleToCheck_test.py", line 6, in <module>
from ..Lowlevel import ModuleToCheck
ValueError: Attempted relative import in non-package
The following is from http://docs.python.org/tutorial/modules.html#intra-package-references
Note that both explicit and implicit
relative imports are based on the name
of the current module. Since the name
of the main module is always
"__main__", modules intended for use
as the main module of a Python
application should always use absolute
imports.
You're running your module ModuleToCheck_test.py as the main module, hence the exception.
One solution is to create a test.py module in your src directory containing the following:
import Test.ModuleToCheck_test
You can then run that module using python test.py