Develop Reference

import from packages in other levels

I've got this structure bellow,
I want to import data-parser/service.py classes from api/server.py how can I do it?
- project
__init__.py
main.py
- api
__init__.py
server.py
- data-parser
__init__.py
service.py

I'm confused by your file structure as a whole but from what I can see you have one project folder, your main script in one folder in the project folder, and your modules in a seperate folder in the project folder. If this is the case, I don't understand to why you don't just have your main.py file in the same directory as a folder with your modules in it (I only mention in case you can do this instead as it makes this a whole lot easier) but here is the solution if you're certain you need your file structure to be the same as it is currently:
import sys
from os import path
from pathlib import Path
sys.path.insert(0, path.join(Path(__file__).parent.absolute().parent.absolute(), "app"))
import submodule1
This, essentially, gets the path of the file, it's parent (the folder it's in), and then the parent's parent (the project folder) to add it to sys.path (you can use modules in the paths of this list).
I'm almost 100% certain there's a better, more efficient way of doing this, but this is what I had off the top of my head due to the uniqueness of the question. I hope it's somewhat helpful.

ValueError: attempted relative import beyond top-level package python

I have a directory structure like this.
Chatbot/
utils/
abc.py
projects/
proj1/
utils/
__init__.py
data_process.py
components/
class1.py
I have two utils folders in my structure, one at the top level and one inside my projects folder.
Now I want to import data_process.py file inside class1.py. So I tried like this
from utils.data_process import DataProcess
but it is referencing the top-level utils folder and even VSCode is not recognizing it. I tried creating __init__.py file inside the utils folder but still did not work.
I tried with empty __init__.py and then placing this content
from . import data_process
__all__ = ['data_proces']
then
from .data_process import DataPreprocess
__all__ = ['DataPreprocess']
then I tried
from ..utils.data_process import DataProcess
VSCode is recognizing this but it is not working and throws the error
ValueError: attempted relative import beyond top-level package
Even I tried changing the name utils to some other name but still the same issue
How can I solve this?

A python module is defined by a special file called __init__.py and this file should be present in all the subdirectories. So, your file structure would be:
Chatbot/
__init__.py
utils/
__init__.py
abc.py
projects/
__init__.py
proj1/
__init__.py
utils/
__init__.py
data_process.py
components/
__init__.py
class1.py
class2.py
Now, you can do a relative import like:
Use . for importing something within the same directory. Example:
# file Chatbot/projects/proj1/components/class2.py
from .class1 import *
Similarly, use .. for two-level, and ... for three-level, and so on!
For instance:
# file Chatbot/projects/proj1/components/class2.py
from ..utils.data_process import * # import from data_process.py
# there will be four "." earlier I made a counting mistake
from ....utils.abc import * # import something from abc.py
Coding Convention
When writing a python module/package, you might want to follow PEP8 Convention.
Moreover, I believe you are trying to do different projects using your package Chatbot, is that correct? Then, it is a good practice that you set PYTHONPATH for Chatbot and do all the projects, and imports seperately.
EDIT: Uploading Dummy File
I'm able to work on this project seamlessly even with using two utils directories. Project Structure:
main file and its output:
# file main.py
from chatbot.utils.abc import hello as a1
from chatbot.projects.proj1.components.class1 import super_hello
from chatbot.projects.proj1.utils.data_process import hello as b1
print(a1(), b1())
print(super_hello())
Similarly, if you have chatbot under PYTHONPATH you can call the project from anywhere from your device.
Code details and files are added into my github account for details.

sys.path is where Python searches to find modules and packages and It does it in order.
So you can put ...../proj1/ at the beginning of the list, when python start searching it will find the utils folder in that path first !
import sys
sys.path.insert(0, r'...../proj1/')
But this cause another problem, python always find utils in that folder first, that's not what you want.
solution number 1
Absolute import:
from Chatbot.projects.proj1.utils.data_process import DataProcess
solution number 2
use relative imports which is mentioned by #Mr.Hobo.

unable to import python file module which is inside the another directory of the same level

I am grinding with an error that is also obvious but confusing related to python.
Attempted relative import beyond top-level packagepylint (relative-beyond-top-level)
The problem is:
there are some folders and some python files and I am getting an error(shown above)
-Main
- one.py
- two.py
- Root
- new.py
- class.py
I want to import a new.py which is in the root folder from one.py which is in the Main folder what should I do?
In the new.py
def func():
return 'I am Programmer'
and in the one.py
from ..Root.new import func
I am using this line to import but it gives me an error
Attempted relative import beyond top-level packagepylint (relative-beyond-top-level)
I tried too many lines to import the func which is a function from new.py which is inside the Root folder but what should I do?

Explanation
This is a problem I encouter a lot when using Python imports. First, let's take a look at sys.path when we run the program. As far as I know, this lists the folders your Python executable will look into when trying to import a module.
one.py
import sys
print(sys.path)
Now, let's run one.py from the root directory of your project (the folder containing root/ and main/). NB: I am using Windows, so the path formatting is a bit different from Unix systems.
console
$ python ./main/one.py
['C:\\Users\\spaceburger\\Downloads\\example\\main', ...]
Now we know Python knows the location of the main/ folder. However, it doesn't know it can go back to the example/ folder (even if we use from ..root import new, don't ask me why...). However you should be able to import other modules from the main/ folder quite easily. Let's check this out :
two.py
print("two")
one.py
import two
console
$ python ./main/one.py
two
You can also import from subfolders in of the main/ folder if you create one. E.g. after creating a file main/submain/three.py, you can use from submain import three kind of statements.
However, what we want is to import from a folder that is higher in the tree. To do so, I see two options :
Option 1: change your path
The idea is simple : since Python doesn't know the root/ folder, we just have to declare it. The idea is simple but this is not something I would recommend to do because it might make your code messy. But sometimes, this is the only way !
For that, you can add the following statements to one.py but the preferred method would be to create an __init__.py file in the main/ folder and to add the following :
import sys, os
rel_path = os.path.join(os.path.dirname(__file__), "..") # if we use just "..", the path will be based on cwd (which is my download/ folder but might also be anything else)
abs_path = os.path.abspath(rel_path)
sys.path.insert(1, abs_path)
print(sys.path)
Now you can re-write one.py
from root import new
Should work fine (note : from root and not from ..root). In fact, you can import from every folder in the example/ folder now.
If you replace os.path.join(os.path.dirname(__file__), "..") by os.path.join(os.path.dirname(__file__), "..", "root"), then you can directly import modules from the root/ folder as if your one.py was just next to your new.py. I.e. you should use import new in that situation.
Option 2: change the entry point of your program
The other way to add example/ or root/ to the Python path is to launch your program directly from there !
Create an entry.py file in the example/ folder:
entry.py
from main import one
one.do_something_with_new()
And in one.py you can assume the example/ folder is known to Python and that you can import from evey subfolder there.
one.py
from root import new
def do_something_with_new():
new.do_something()
As long as you always start your program with entry.py. Be careful with the name of your files, I would usually use app.py or main.py in stead of entry.py but it might cause confusion with your folder names (for python as well). I don't think new.py is recommended as well.
Conclusion
You can either set the entry point of your program in a folder that contains all the subfolders you wish to use and use import statements like from subfolder.subsubfolder import xxx, or you can modify your Python path and add the folders you wish to use in to the list of known locations that way you can use import statements such as import xxx directly.
However I do not recommend this last option because you might get conflicts with the name of your subfolders (if you have a main/A.py and a root/A.py, then import A will load the first one that is found, which depends on the order of your folders paths in your Python path variable). Also, this takes a few lines of code to write, whereas you can get a result just as nice just with some organization.
Disclaimer
I usually use one of the two options above because I do not know of any other. This is all based on my experience and reading StackOverflow posts, not on the documentation or anything like that, so please take my advice with a grain of salt.

I am not sure It will answer your Q but please check this post It may help you:
Importing files from different folder

Unable to import class even though I already have __init__.py files

I'm trying to import a class in a different directory to another file, but can't seem to get it to work. I know this question has been asked a lot and I have looked through multiple stackoverflow solutions and at https://docs.python.org/3/tutorial/modules.html#packages
1: Importing files from different folder
2: import python file in another directory failed
I want to try to just use the method containing just __init__.py file instead of doing an import sys
My directory structure is as follows:
django_vue/
__init__.py
devices/
__init__.py
models.py
lib/
__init__.py
my_file.py
I'm trying to import the class Device from /django_vue/devices/models.py to /django_vue/lib/my_file.py by:
from devices.models import Device
However when I do that I still get the error:
from devices.models import Device
ModuleNotFoundError: No module named 'devices'
I'm not sure what I'm dong wrong since I already have the __init__ file in both directories. Any help is appreciated. Also I'm running python 3.6.

This is the folder structure I'm working with.
.
└── django_vue
├── devices
│   └── models.py
└── lib
└── file.py
When you run
$ python file.py
python has no way of knowing what's outside the directory.
python can't go back and then into devices/ just like that.
The easiest way to solve this would be to add the folder devices/ to sys.path. When python imports a module, it searches for the module from sys.path. Adding the path to devices/ would make it available for imports.
Here are my files.
# models.py
Device = 'device'
# file.py
import sys
sys.path.append('..') # adds the parent dir (which is django-vue/) to path
# django-vue dir has devices/ so now this is available for imports
# importing this works now
from devices.models import Device
print(Device)
Output
django_vue/lib$ python3 file.py
device

Think about it your are inside my_file.py and import something called devices.
How can python know where the name devices has come from.
It won't search your entire Drive for that module/package
Relative Import
use a relative import instead. write from ..devices.models import Device. This is telling python to go up one directory to the parent directory and that's where it will find the devices package. Your lib module should now work as a module
If you however run the my_file.py package directly (as in python C:/django_vue/lib/my_file.py)
You will still get an error. Not the same error; the new error will be something like
ImportError: attempted relative import with no known parent package
This is happening because you are actually running my_file.py
If you think about it why would you want to run my_file.py by itself when it is clearly a part of a package. Maybe you are just testing to see if the import works when you use your package. The problem with this is that it makes it seem like your packages relative imports don't work even though this actually works.
Create a main.py in django_vue and write from lib import my_file. This will run your my_file.py and you will notice there is no error.
What's happening here
Have you heard of __package__?
if you put print(str(__package__)) in your my_file.py and run my_file.py directly you will see that it prints None.
However if you run main.py (that you just created) you will see that when It reaches my_file.py, __package__ will actually be defined to something.
Ahhh... you see now it all makes sense; The error you originally got said something about no known parent package. If __package__ is undefined that means there is no relative parent package because the file was obviously run directly instead of as part of a package.
Consider Absolute imports
you also might want to consider using absolute imports because if you are working on the package you might change it directory structure while developing. so you need to keep changing the import references on the affected files.
Although you can find IDE's with python extensions that automatically to this as you change your directory. I believe VS Code does this automatically.

Replace the __init__ files with __main__.

Import from another directory

I need to import .py file from another one in another directory (import app2 from app1)
So there is directory tree
app:
dir1:
app1.py
dir2:
app2.py
My problem is almost like Importing from another directory, but this solution doesnt work for me for some reason
Furthermore i've been trying to do like this (app1.py)
from ..dir2 import app2
The error is:
Attempted relative import beyond top-level package
How can i solve this?

Add your additional directories to the system path
import sys
sys.path.insert(0, "/path/to/app1/dir2")
sys.path.insert(0, "/path/to/app2/dir2")