Develop Reference

How do I import an object from main.py (in the root directory), into a module contained in a subdirectory?

So, this is the project structure:
<root directory>
- app
- - name
- - - module1
- - - module2
- - - module3
- - - - tests.py
- test_db.py
test_db.py contains an object called client which I need in all the tests.py files in different modules.
I could simply move test_db.file inside the app directory but test_db.py needs to import app from main.py which is again in the root directory and will cause the same issue again.
In every tests.py I need something like:
from ....test_db import client
but it just gets:
from ....test_db import client
E ImportError: attempted relative import beyond top-level package
So, I just can't figure out importing an object/package from the root directory.
P.S: I know one simple solution would be to change the directory structure a little, but It's a large project and changing the structure would be a LOT of work. So, ideally I need some way to add the root directory to the path inside every tests.py file. Something like this:
app/name/module3/tests.py:
import sys
#sys.path.something.. I am not really familiar with how to use this
from ....test_db import client #this should import client from test_db in root instead of throwing error
def test_create_todo():
pass

If you want to do a relative import without hacking around with sys.path or custom loaders/finders, you need:
the top-most directory you import relative to to contain an __init__.py
to specify said directory as a package in the path to your __main__ module (i.e. python -m root.stuff.main)
In short, this means you could add a directory project to contain the test_db.py and app, add an __init__.py, and call your program with python -m project.app.stuff.main.
If this doesn't work for you (you mention you don't want to change the project structure) you do have other options.
You could install each as its own package (app, and tests). Create a setup.py/setup.cfg/pyproject.toml, put test_db.py in a package tests, and pip install them as editable packages pip install -e .. This will allow you to import either without relative imports (just import app, and import tests.test_db, regardless of file). This is personally the route I would go, and recommend doing.
Otherwise, if you're just looking for a quick fix, there exists one more easy + hacky solution. You could add the path for test_db.py to sys.path (every path in this list is explored for the target module when importing), so you could import test_db from anywhere. Again, this is very hacky, and I don't recommend it beyond quick sanity checks or extremely urgent patches.

import from packages in other levels

I've got this structure bellow,
I want to import data-parser/service.py classes from api/server.py how can I do it?
- project
__init__.py
main.py
- api
__init__.py
server.py
- data-parser
__init__.py
service.py

I'm confused by your file structure as a whole but from what I can see you have one project folder, your main script in one folder in the project folder, and your modules in a seperate folder in the project folder. If this is the case, I don't understand to why you don't just have your main.py file in the same directory as a folder with your modules in it (I only mention in case you can do this instead as it makes this a whole lot easier) but here is the solution if you're certain you need your file structure to be the same as it is currently:
import sys
from os import path
from pathlib import Path
sys.path.insert(0, path.join(Path(__file__).parent.absolute().parent.absolute(), "app"))
import submodule1
This, essentially, gets the path of the file, it's parent (the folder it's in), and then the parent's parent (the project folder) to add it to sys.path (you can use modules in the paths of this list).
I'm almost 100% certain there's a better, more efficient way of doing this, but this is what I had off the top of my head due to the uniqueness of the question. I hope it's somewhat helpful.

unable to import python file module which is inside the another directory of the same level

I am grinding with an error that is also obvious but confusing related to python.
Attempted relative import beyond top-level packagepylint (relative-beyond-top-level)
The problem is:
there are some folders and some python files and I am getting an error(shown above)
-Main
- one.py
- two.py
- Root
- new.py
- class.py
I want to import a new.py which is in the root folder from one.py which is in the Main folder what should I do?
In the new.py
def func():
return 'I am Programmer'
and in the one.py
from ..Root.new import func
I am using this line to import but it gives me an error
Attempted relative import beyond top-level packagepylint (relative-beyond-top-level)
I tried too many lines to import the func which is a function from new.py which is inside the Root folder but what should I do?

Explanation
This is a problem I encouter a lot when using Python imports. First, let's take a look at sys.path when we run the program. As far as I know, this lists the folders your Python executable will look into when trying to import a module.
one.py
import sys
print(sys.path)
Now, let's run one.py from the root directory of your project (the folder containing root/ and main/). NB: I am using Windows, so the path formatting is a bit different from Unix systems.
console
$ python ./main/one.py
['C:\\Users\\spaceburger\\Downloads\\example\\main', ...]
Now we know Python knows the location of the main/ folder. However, it doesn't know it can go back to the example/ folder (even if we use from ..root import new, don't ask me why...). However you should be able to import other modules from the main/ folder quite easily. Let's check this out :
two.py
print("two")
one.py
import two
console
$ python ./main/one.py
two
You can also import from subfolders in of the main/ folder if you create one. E.g. after creating a file main/submain/three.py, you can use from submain import three kind of statements.
However, what we want is to import from a folder that is higher in the tree. To do so, I see two options :
Option 1: change your path
The idea is simple : since Python doesn't know the root/ folder, we just have to declare it. The idea is simple but this is not something I would recommend to do because it might make your code messy. But sometimes, this is the only way !
For that, you can add the following statements to one.py but the preferred method would be to create an __init__.py file in the main/ folder and to add the following :
import sys, os
rel_path = os.path.join(os.path.dirname(__file__), "..") # if we use just "..", the path will be based on cwd (which is my download/ folder but might also be anything else)
abs_path = os.path.abspath(rel_path)
sys.path.insert(1, abs_path)
print(sys.path)
Now you can re-write one.py
from root import new
Should work fine (note : from root and not from ..root). In fact, you can import from every folder in the example/ folder now.
If you replace os.path.join(os.path.dirname(__file__), "..") by os.path.join(os.path.dirname(__file__), "..", "root"), then you can directly import modules from the root/ folder as if your one.py was just next to your new.py. I.e. you should use import new in that situation.
Option 2: change the entry point of your program
The other way to add example/ or root/ to the Python path is to launch your program directly from there !
Create an entry.py file in the example/ folder:
entry.py
from main import one
one.do_something_with_new()
And in one.py you can assume the example/ folder is known to Python and that you can import from evey subfolder there.
one.py
from root import new
def do_something_with_new():
new.do_something()
As long as you always start your program with entry.py. Be careful with the name of your files, I would usually use app.py or main.py in stead of entry.py but it might cause confusion with your folder names (for python as well). I don't think new.py is recommended as well.
Conclusion
You can either set the entry point of your program in a folder that contains all the subfolders you wish to use and use import statements like from subfolder.subsubfolder import xxx, or you can modify your Python path and add the folders you wish to use in to the list of known locations that way you can use import statements such as import xxx directly.
However I do not recommend this last option because you might get conflicts with the name of your subfolders (if you have a main/A.py and a root/A.py, then import A will load the first one that is found, which depends on the order of your folders paths in your Python path variable). Also, this takes a few lines of code to write, whereas you can get a result just as nice just with some organization.
Disclaimer
I usually use one of the two options above because I do not know of any other. This is all based on my experience and reading StackOverflow posts, not on the documentation or anything like that, so please take my advice with a grain of salt.

I am not sure It will answer your Q but please check this post It may help you:
Importing files from different folder

Import file from lib directory?

I've been reorganizing my project because there was an issue somewhere, but, as programming always goes, the problem is now 10 times worse and everything is broken.
My current file tree that I am satisfied with is:
Amazons AI
- .git
- Game_Code
- __pycache__
- game.py
- lib
- __pycache__
- __init__.py (empty)
- motion.py
- pieceManagement.py
- tests
- __pychache__
- test_game.py
- README.md
My issue is that in game.py (in the Game_Code folder, I need to import motion.py and pieceManagement.py (both in the lib directory).
I've tried multiple ways to go up a level in the directory, import lib and then everything from that, largely using the suggestions in Import a file from a subdirectory?, but nothing has worked. For reference, I am using Python 3.7.3.

I am not an expert But last weekend i wrote some python code with the similar structure and used from folder.file import reference to reflect the folder structure:
from lib.motion import classObject as ObjectName
from lib.pieceManagement import classMethod() as MethodName()

To access libs in parent directory of the current file, one can do this:
sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))) + "/libs")
It adds parent dictory + /libs to the sys path where python will know to look for it as described in Python - what is the libs subfolder for?. However, I don't like this solution as it leads to ugly code like this:
import os
import sys
sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__))) + "/libs")
from pieceManagement import piece
import motion
So I'd still like to find a pythonic way to do it, maybe inline with the import statements. But I know this works (on my machine).

python how to run script in folder

This is my python path:
PYTHONPATH = D:\PythonPath
in the PythonPath folder I have MyTests folder that has a Script.py
in the PyThonPath folder I have ScrapyingProject folder
inside the Script.py I do this:
from ScrapyingProject.ScrapyingProject.spiders.XXXSpider import XXXSpider
I got this exception:
ImportError: No module named ScrapyingProjectScrapyingProject.spiders.XXXSpider
Edit:
the XXXSpider is in this location:
D:\PythonPath\ScrapyingProject2\ScrapyingProject2\spiders.py

Take a look at this to read more about Python modules and packages: http://docs.python.org/2/tutorial/modules.html
Turn your python-script-containing folder into a python package by adding __init__.py file to it. So, in your case, the directory structure should resemble this:
PYTHONPATH
- ScrapyingProject
- __init__.py
- script.py
Now, in this scheme, ScrappyProject becomes your python-package. Any .py file inside the folder becomes a python module. You can import a python module by dot-expanded python path starting PYTHONPATH. Something like,
from ScrapyingProject.script import XXXSpider
Same logic can be extended by nesting multiple packages inside each other. A nested package, for example looks like
PYTHONPATH
- ScrapyingProject2
- __init__.py
- ScrapyingProject2
- __init__.py
- script.py
Now, a package-nested script.py can be imported as
from ScrapyingProject2.ScrapyingProject2 import script
Or even
from ScrapyingProject2.ScrapyingProject2.script import XXXSpider
(Assuming you have defined class XXXSpider inside script.py)

For one you say that the directory is D:\PythonPath\ScrapyingProject2\ScrapyingProject2\spiders.py but you import from ScrapyingProject.ScrapyingProject (without the 2s).
If I understand you correctly, the wanted import should look something like this:
from ScrapyingProject2.ScrapyingProject2.spiders import XXXSpider
assuming the class XXXSpider is in the module spiders.py.
Remember to put __init__.py files in the folders you want to import from (turning them into 'packages'). This should include all folders from the PYTHONPATH to the one containing the *.py-files. See here from more details on packages.