I've been wrestling most of the night trying to solve an import error.
This is a common issue, but no previous question quite answers my issue.
I am using PyDev (an Eclipse plugin), and the library Kivy (a Python library)
I have a file structure set up like this:
<code>
__init__.py
main.py
engine.py
main_menu_widget.py
"code" is held within the eclipse folder "MyProject" but it's not a package so I didn't include it.
The files look like this:
main.py
# main.py
from code.engine import Engine
class MotionApp(App):
# Ommited
engine.py
# engine.py
from code.main_menu_widget import MainMenuWidget
class Engine():
# Ommited
main_menu_widget.py
# main_menu_widget.py
from code.engine import Engine
class MainMenuWidget(Screen):
pass
The error I recieve, in full detail, is:
Traceback (most recent call last):
File "C:\MyProject\code\main.py", line 8, in <module>
from code.engine import Engine
File "C:\MyProject\code\engine.py", line 6, in <module>
from code.main_menu_widget import MainMenuWidget
File "C:\MyProject\code\main_menu_widget.py", line 3, in <module>
from code.engine import Engine
Any idea what I did wrong here? I just renamed my entire folder structure because I screwed up this module structure so bad, but I think i'm close to how it should look....
There seems to be a circular import.
from engine.py you are importing main_menu_widget while from main_menu_widgetyou are importing engine.
That is clearly a circular import which is not allowed by python.
it's in the same folder, use a relative package name (it's a good practice to do so anyway):
from .engine import Engine
Your code directory is a package. Ensure that the directory above it, i.e C:\MyProject judging from your error messages, is in your PYTHONPATH.
Open the context menu by selecting your project and clicking your mouse's right button, then select Properties. Select PyDev - PYTHONPATH and from there the Source folders tab.
Check that the directory mentioned above is present; if it isn't press Add source folder, select it from the dialogue and press OK.
Related
I am new to Azure Machine Learning and have been struggling with importing modules into my run script. I am using the AzureML SDK for Python. I think I somehow have to append the script location to PYTHONPATH, but have been unable to do so.
To illustrate the problem, assume I have the following project directory:
project/
src/
utilities.py
test.py
run.py
requirements.txt
I want to run test.py on a compute instance on AzureML and I submit the run via run.py.
A simple version of run.py looks as follows:
from azureml.core import Workspace, Experiment, ScriptRunConfig
from azureml.core.compute import ComputeInstance
ws = Workspace.get(...) # my credentials here
env = Environment.from_pip_requirements(name='test-env', file_path='requirements.txt')
instance = ComputeInstance(ws, '<instance-name>')
config = ScriptRunConfig(source_directory='./src', script='test.py', environment=env, compute_target=instance)
run = exp.submit(config)
run.wait_for_completion()
Now, test.py imports functions from utilities.py, e.g.:
from src.utilities import test_func
test_func()
Then, when I submit a run, I get the error:
Traceback (most recent call last):
File "src/test.py", line 13, in <module>
from src.utilities import test_func
ModuleNotFoundError: No module named 'src.utilities'; 'src' is not a package
This looks like a standard error where the directory is not appended to the Python path. I tried two things to get rid of it:
include an __init__.py file in src. This didn't work and I would also for various reasons prefer not to use __init__.py files anyways.
fiddle with the environment_variables passed to AzureML like so
env.environment_variables={'PYTHONPATH': f'./src:${{PYTHONPATH}}' but that didn't really work either and I assume that is simply not the correct way to append the PYTHONPATH
I would greatly appreciate any suggestions on extending PYTHONPATH or any other ways to import modules when running a script in AzureML.
The source diectory set in ScriptRunConfig will automaticaly add to the PYTHONPATH, that means remove the "src" directory from the import line.
from utilities import test_func
Hope that helps
I'm having trouble with a python package that uses separate modules to structure code. The package itself is working, however when imported from another environment fails with a ModuleNotFound error.
Here's the structure:
Project-root
|
|--src/
| __init__.py
| module_a.py
| module_b.py
| module_c.py
| module_d.py
|--tests
etc.
In module_a.py I have:
from module_a import function_a1,...
from module_b import function_b1,...
from module_c import function_c1,...
In module_c I import module_d like:
from module_d import function_d1,...
As mentioned above, executing module_a or module_c directly from the CLI work as expected, the unit tests I've created in the test directory also work (with the help of sys.path.insert), however if I create a new environment and import the package I get the following error:
>>> import module_a
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/<abs_path>/.venv/lib/python3.9/site-packages/module_a.py", line 22, in <module>
from module_c import function_c1, function_c2
File /<abs_path>/.venv/lib/python3.9/site-packages/module_c.py", line 9, in <module>
import module_d
ModuleNotFoundError: No module named 'module_d'
>>>
I've exhausted all ideas how to overcome this, besides combining the code of modules c and d in one file, which I'd hate to do, or rethink the flow so that all modules are imported from module_a.
Any suggestions how to approach this would be greatly appreciated.
Update: It turned out to be a typing mistake in the name of module_d in setup.py. For whatever reason python setup.py install was failing silently or I wasn't reading the logs carefully.
The problem comes down to understanding the basics of the import system and the PYTHONPATH.
When you try to import a module (import module_a), Python will search in order in every directory listed in sys.path. If a directory matches the name (module_a)1, then it runs the __init__.py file is such exist.
When you get an [https://docs.python.org/3/library/exceptions.html#ImportError], it means that there is no directory in sys.path containing a directory with the name asked.
You said for your tests you did something like sys.path.insert(0, "some/path/"), but it is not a solution, just a broken fix.
What you should do is set your PYTHONPATH environment variable to contain the directory where your modules are located, Project-root/src in your case. That way, no need to ever use sys.path.insert, or fiddle with relative/absolute paths in import statements.
When you create your new environment, just set your environment variable PYTHONPATH to include Project-root/src and you are done. This is how installing regular Python modules (libraries) work : they are all put into a directory in site-packages.
1: this changed since old Python versions, it used to be required for the directory to contain an __init__.py file
I have been creating an application that will run just like Microsoft Notepad. My program comprises of the following file and folder:
libs (Folder)
scripts (Folder)
test.py
The test.py is the file in which I am trying the following code:
from .libs import tkinter
from .libs import pyglet
from .libs import threading
test_window = Tk()
test_window.mainloop()
In the folder libs I have added the modules that are required in the application.
Example: Tkinter, Pyglet and Threading
When I am trying to import them into test.py using the code above and then running the program. I see the following Traceback:
Traceback (most recent call last):
File "c:\Users\Bhavyadeep\Desktop\NuclearPad\test.py", line 1, in <module>
from .libs import tkinter
ImportError: attempted relative import with no known parent package
I don't really know how I solve this problem. Is it because I added the module in the folder where Import in not available? Or is it because I did something wrong in this module folder copying-pasting to a folder not at Python's PATH?
from .libs import tkinter
Means "import tkinter from the package called lib which is located in the same directory as file (this script)"
so your first line should say something like
test_window = tkinter.Tk()
unless you use either of the following:
from .libs.tkinter import *
from .libs.tkinter import Tk
The error is likely due to you using relative import syntax from within the script you are trying to execute. Remove the leading dots in order to execute test.py, otherwise append its directory to sys.path or use site.addsitedir(path_to_directory_containing_test_py) and import it as follows import test.
With that said, you should not call the module's file test.py because there is already something with that name in python's default Lib directory or builtins. If you insist, there should be a hacky way to get it done without changing the name through importlib.
As for why this error occurs, I think it's because the interpreter is designed to execute scripts but not modules/packages whose contents are placed in .pyc files in __pycache__s when imported.
In order to import from scripts or libs, they must be packages. In order to turn them into packages you must add __init__.py files to them.
Lastly, It's a pretty terrible idea to place third-party libraries in your source like this. At the very least you should have installed them properly on your machine and then copied their sources from site-packages/the equivalent for venvs, which will fail if they use any executables and/or have any external dependencies
Documentation for distributing python packages
I was having some problem when trying to import method from another python class. I am following this guide. When I tried to run the attention.py, I am getting this error message:
File "D:\Desktop\tensorrec-master\examples\attention_example.py", line 8, in <module>
from test.datasets import get_movielens_100k
ModuleNotFoundError: No module named 'test.datasets'
I even tried to change the import statement to:
import test.datasets.py
test.datasets.get_movielens_100k(negative_value=0)
But I am still getting the same error message. Any ideas? Thanks!
Project structure needs to be known to your Python interpreter. And how to do it?
Set PYTHONPATH to base directory of your project
Add init.py file in each directory
Voila! You will be able to import from any directory under your base folder after that
From Import a file from a subdirectory? :
In short, you need to put a blank file named
__init__.py
in the "lib" directory.
I am using pycharm at one of my university projects and I wanted to integrated it with unittest module, but I have a problem with structuring my project
Part of this project involves generating abstract syntax trees, so I created AST directory and put __init__.py there, then I created expression module. I wanted to put my tests in test/ subdirectory, so it would look like this:
AST/
__init__.py
expression.py
test/
some_test.py
utils.py
now I have also module in my AST called symbol_table and module called utils, example test class looks like
import unittest
from ...AST import expression
from ...AST import utils
class ConstantExpressionTest(unittest.TestCase):
def testConstantExpressionCheck(self):
constantExpression = expression.ConstantExpression(17, 5, utils.TYPES.INT)
self.assertTrue(constantExpression.check())
when I right click on this file and select Run Unittest in ... I am getting errors:
/usr/bin/python2.7 /home/xubuntu/Downloads/pycharm-2.7.2/helpers/pycharm/utrunner.py /home/xubuntu/Przedmioty/VI/kompilatory/tk-projekt/src/AST/test/test_constant_expression.py true
Testing started at 12:06 PM ...
Traceback (most recent call last):
File "/home/xubuntu/Downloads/pycharm-2.7.2/helpers/pycharm/utrunner.py", line 110, in <module>
modules = [loadSource(a[0])]
File "/home/xubuntu/Downloads/pycharm-2.7.2/helpers/pycharm/utrunner.py", line 34, in loadSource
module = imp.load_source(moduleName, fileName)
File "/home/xubuntu/Przedmioty/VI/kompilatory/tk-projekt/src/AST/test/test_constant_expression.py", line 2, in <module>
from ...AST import utils
ValueError: Attempted relative import in non-package
Process finished with exit code 1
I have read about this problem and if I understand this right, this file is treated as it would be in top-level package so I can't use any relative imports.
But if that is the case, how can I run unit-tests from pycharm and also keep my current project strcture?
If I am not mistaken, putting tests in sub-package is pretty popular (http://as.ynchrono.us/2007/12/filesystem-structure-of-python-project_21.html) so there must be some kind of solution
Well, that is a bit silly, I found out that pycharm adds the root of the project to path so I can just use normal imports from the root of my project.
So for example I can write
from AST import expression in my some_test file