I've been reorganizing my project because there was an issue somewhere, but, as programming always goes, the problem is now 10 times worse and everything is broken.
My current file tree that I am satisfied with is:
Amazons AI
- .git
- Game_Code
- __pycache__
- game.py
- lib
- __pycache__
- __init__.py (empty)
- motion.py
- pieceManagement.py
- tests
- __pychache__
- test_game.py
- README.md
My issue is that in game.py (in the Game_Code folder, I need to import motion.py and pieceManagement.py (both in the lib directory).
I've tried multiple ways to go up a level in the directory, import lib and then everything from that, largely using the suggestions in Import a file from a subdirectory?, but nothing has worked. For reference, I am using Python 3.7.3.
I am not an expert But last weekend i wrote some python code with the similar structure and used from folder.file import reference to reflect the folder structure:
from lib.motion import classObject as ObjectName
from lib.pieceManagement import classMethod() as MethodName()
To access libs in parent directory of the current file, one can do this:
sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))) + "/libs")
It adds parent dictory + /libs to the sys path where python will know to look for it as described in Python - what is the libs subfolder for?. However, I don't like this solution as it leads to ugly code like this:
import os
import sys
sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))) + "/libs")
from pieceManagement import piece
import motion
So I'd still like to find a pythonic way to do it, maybe inline with the import statements. But I know this works (on my machine).
Related
So, this is the project structure:
<root directory>
- app
- - name
- - - module1
- - - module2
- - - module3
- - - - tests.py
- test_db.py
test_db.py contains an object called client which I need in all the tests.py files in different modules.
I could simply move test_db.file inside the app directory but test_db.py needs to import app from main.py which is again in the root directory and will cause the same issue again.
In every tests.py I need something like:
from ....test_db import client
but it just gets:
from ....test_db import client
E ImportError: attempted relative import beyond top-level package
So, I just can't figure out importing an object/package from the root directory.
P.S: I know one simple solution would be to change the directory structure a little, but It's a large project and changing the structure would be a LOT of work. So, ideally I need some way to add the root directory to the path inside every tests.py file. Something like this:
app/name/module3/tests.py:
import sys
#sys.path.something.. I am not really familiar with how to use this
from ....test_db import client #this should import client from test_db in root instead of throwing error
def test_create_todo():
pass
If you want to do a relative import without hacking around with sys.path or custom loaders/finders, you need:
the top-most directory you import relative to to contain an __init__.py
to specify said directory as a package in the path to your __main__ module (i.e. python -m root.stuff.main)
In short, this means you could add a directory project to contain the test_db.py and app, add an __init__.py, and call your program with python -m project.app.stuff.main.
If this doesn't work for you (you mention you don't want to change the project structure) you do have other options.
You could install each as its own package (app, and tests). Create a setup.py/setup.cfg/pyproject.toml, put test_db.py in a package tests, and pip install them as editable packages pip install -e .. This will allow you to import either without relative imports (just import app, and import tests.test_db, regardless of file). This is personally the route I would go, and recommend doing.
Otherwise, if you're just looking for a quick fix, there exists one more easy + hacky solution. You could add the path for test_db.py to sys.path (every path in this list is explored for the target module when importing), so you could import test_db from anywhere. Again, this is very hacky, and I don't recommend it beyond quick sanity checks or extremely urgent patches.
I've got this structure bellow,
I want to import data-parser/service.py classes from api/server.py how can I do it?
- project
__init__.py
main.py
- api
__init__.py
server.py
- data-parser
__init__.py
service.py
I'm confused by your file structure as a whole but from what I can see you have one project folder, your main script in one folder in the project folder, and your modules in a seperate folder in the project folder. If this is the case, I don't understand to why you don't just have your main.py file in the same directory as a folder with your modules in it (I only mention in case you can do this instead as it makes this a whole lot easier) but here is the solution if you're certain you need your file structure to be the same as it is currently:
import sys
from os import path
from pathlib import Path
sys.path.insert(0, path.join(Path(__file__).parent.absolute().parent.absolute(), "app"))
import submodule1
This, essentially, gets the path of the file, it's parent (the folder it's in), and then the parent's parent (the project folder) to add it to sys.path (you can use modules in the paths of this list).
I'm almost 100% certain there's a better, more efficient way of doing this, but this is what I had off the top of my head due to the uniqueness of the question. I hope it's somewhat helpful.
First off all: Sorry - I know that there already exist several questions which concern issues with the python import mechanism. However after hours of searching through SO I still couldn't find a sufficent answer for me.
Problem
I've got two independent Python projects hosted on my GitLab Server.
Let's call them foo and foo-Tester.
As the name implies, foo-Tester is a automatic Tester for the foo-Project.
foo has the following structure:
-foo
- __init__.py
- fooClass.py
-bar
- __init__.py
- barClass.py
-foobar
- __init__.py
- foobarClass.py
etc.
The classes import each other via relativ imports.
For sake of this example the foo-Tester Project just contains:
- foo-Tester
- __init__.py
- tester.py
While the tester.py just contains:
from foo.fooClass import fooClass
myfoo = fooClass()
Both Projects reside next to each other on the file-system:
- some-folder
- foo
- foo-Tester
So my question is: What is the best approach/best-practice to import the fooClass from foo into tester.py from the foo-Tester-Project?
What I've tried
Using setup.py
Some Answers (e.g. this suggest using setup.py. As far as I understand this approach, one would need to install the foo-Project each time it is changed - am I correct?
This wouldn't be a good solution for us as the Project is under heavy development with many changes a day.
Appending to system-path
This answer suggest adding the foo project path to the system path. And running the script with the -m Flag
Two concerns about this approach:
it seems a bit hacky to modify the systempaths
it seems like this doesn't work with the relative imports from the foo project as it always throws an relative imports not allowed-Error.
I am using Python 3.6.
So what would be the best approach for the described problem?
Thank you in advance!
From the thread out here: Importing modules from parent folder , this is the way how you can import modules from parent (or any other) folder.
import os,sys,inspect
currentdir = os.path.dirname(os.path.abspath(inspect.getfile(inspect.currentframe())))
parentdir = os.path.dirname(currentdir)
sys.path.insert(0,parentdir)
import mymodule
I have the following directory structure:
- src
- __init__.py
- scripts
- __init__.py
- preprocessing.py
- project1
- main.py
- project2
- main.py
I'm trying to access the script(s) inside the scripts folder from within both the main.py files.
I've tried adding in the __init__.py (blank) files, and importing with import scripts, from src import scripts, and from .. import scripts. None of these seem to work.
I either get: ValueError: Attempted relative import in non-package, or no module found.
Thanks in advance!
P.S. I assume that the directory structure will get deeper soon (e.g. multiple subdirectories within scripts and project1 / project2). So if these is an easy way to deal with this as well, that would be very much appreciated.
One way to deal with that (albeit not the cleanest one) is to manually add the root directory to the sys.path variable so that python can look for modules in it, for example, in your main.py you may add these lines at the top:
#/src/project1/main.py
import os
import sys
sys.path.append(os.getcwd() + '\\..\\') # this is the src directory
This will allow python to look for modules in the directory above the one running the script, and this will work:
import scripts.preprocessing
Keep in mind that python will only look for modules in the same directory or below as the script is running. If you start /src/project2/main.py, python doesn't know anything about /src/project1/ or /src/scripts/
I know a lot of questions have been posted about the intra-package importing. I want to know whether the below is the way for Python 2.7 too.
Source/
anomalyCheck/
__init__.py
DLthput.py
ULPowerStats.py
ULThput.py
config/
__init__.py
configure.py
parserTools/
__init__.py
logParser.py
utilities/
__init__.py
plotLogResults.py
__init__.py
lteDebugger.py
---- lteDebugger.py----
import parserTools.logParser
import anomalyCheck.DLthput
import utilities.plotLogResults
import configuration.TDDFDDconfiguration
import anomalyCheck.ULthput
### Work done after here ####
------DLThput.py------
from ..utilities.plotLogResults import *
from ..parserTools.logParser import *
### Work done after here ####
------ULThput.py-------
from ..parserTools.logParser import *
from ..utilities.plotLogResults import *
Error :
Upon running the lteDebugger.py file, the error is
ValueError: Attempted relative import beyond toplevel package
File "C:\Users\manojtut\Desktop\Project_LTE_SVN\Source\lteDebugger.py", line 2, in
import anomalyChecker.DLthput
File "C:\Users\manojtut\Desktop\Project_LTE_SVN\Source\anomalyChecker\DLthput.py", line 1, in
I've read almost all available docs and Guido's guide for intra-package importing. Also, I guess I've everything exactly in the right place. Am I missing something here? Please point out. Thanks a lot in advance. :) :)
Edit 1: The Problem mentioned is solved by Amber's answer. So, lteDebugger.py is working fine by importing all other modules. Now, another problem is that, I am unable to solve is that... when I want to compile/interpret(whatever u want to call) the DLThput.py/ULthput.py , it is showing the same error as above ... ValueError: Attempted relative import beyond toplevel package. Do I have any solution other adding paths to sys hacks? I really don't want to do that unless it's the only thing to do.
So, how can I dodge this?
You're running lteDebugger.py, which means that any "packages" must be at least one level lower in the directory tree - they need to be contained inside a folder for Python to recognize them as packages rather than modules (and thus for relative imports to work).
anomalyCheck is recognized as a package, but its parent directory is not (because that's where lteDebugger.py is), and thus you aren't allowed to use relative imports to go up to that parent directory.
One way you could fix this is by moving everything except lteDebugger.py into a subdirectory, e.g.:
Source/
debugger/
anomalyCheck/
__init__.py
DLthput.py
ULPowerStats.py
ULThput.py
config/
__init__.py
configure.py
parserTools/
__init__.py
logParser.py
utilities/
__init__.py
plotLogResults.py
__init__.py
lteDebugger.py
and then lteDebugger.py would do things like import debugger.anomalyCheck.DLthput.py.