Develop Reference

unable to import python file module which is inside the another directory of the same level

I am grinding with an error that is also obvious but confusing related to python.
Attempted relative import beyond top-level packagepylint (relative-beyond-top-level)
The problem is:
there are some folders and some python files and I am getting an error(shown above)
-Main
- one.py
- two.py
- Root
- new.py
- class.py
I want to import a new.py which is in the root folder from one.py which is in the Main folder what should I do?
In the new.py
def func():
return 'I am Programmer'
and in the one.py
from ..Root.new import func
I am using this line to import but it gives me an error
Attempted relative import beyond top-level packagepylint (relative-beyond-top-level)
I tried too many lines to import the func which is a function from new.py which is inside the Root folder but what should I do?

Explanation
This is a problem I encouter a lot when using Python imports. First, let's take a look at sys.path when we run the program. As far as I know, this lists the folders your Python executable will look into when trying to import a module.
one.py
import sys
print(sys.path)
Now, let's run one.py from the root directory of your project (the folder containing root/ and main/). NB: I am using Windows, so the path formatting is a bit different from Unix systems.
console
$ python ./main/one.py
['C:\\Users\\spaceburger\\Downloads\\example\\main', ...]
Now we know Python knows the location of the main/ folder. However, it doesn't know it can go back to the example/ folder (even if we use from ..root import new, don't ask me why...). However you should be able to import other modules from the main/ folder quite easily. Let's check this out :
two.py
print("two")
one.py
import two
console
$ python ./main/one.py
two
You can also import from subfolders in of the main/ folder if you create one. E.g. after creating a file main/submain/three.py, you can use from submain import three kind of statements.
However, what we want is to import from a folder that is higher in the tree. To do so, I see two options :
Option 1: change your path
The idea is simple : since Python doesn't know the root/ folder, we just have to declare it. The idea is simple but this is not something I would recommend to do because it might make your code messy. But sometimes, this is the only way !
For that, you can add the following statements to one.py but the preferred method would be to create an __init__.py file in the main/ folder and to add the following :
import sys, os
rel_path = os.path.join(os.path.dirname(__file__), "..") # if we use just "..", the path will be based on cwd (which is my download/ folder but might also be anything else)
abs_path = os.path.abspath(rel_path)
sys.path.insert(1, abs_path)
print(sys.path)
Now you can re-write one.py
from root import new
Should work fine (note : from root and not from ..root). In fact, you can import from every folder in the example/ folder now.
If you replace os.path.join(os.path.dirname(__file__), "..") by os.path.join(os.path.dirname(__file__), "..", "root"), then you can directly import modules from the root/ folder as if your one.py was just next to your new.py. I.e. you should use import new in that situation.
Option 2: change the entry point of your program
The other way to add example/ or root/ to the Python path is to launch your program directly from there !
Create an entry.py file in the example/ folder:
entry.py
from main import one
one.do_something_with_new()
And in one.py you can assume the example/ folder is known to Python and that you can import from evey subfolder there.
one.py
from root import new
def do_something_with_new():
new.do_something()
As long as you always start your program with entry.py. Be careful with the name of your files, I would usually use app.py or main.py in stead of entry.py but it might cause confusion with your folder names (for python as well). I don't think new.py is recommended as well.
Conclusion
You can either set the entry point of your program in a folder that contains all the subfolders you wish to use and use import statements like from subfolder.subsubfolder import xxx, or you can modify your Python path and add the folders you wish to use in to the list of known locations that way you can use import statements such as import xxx directly.
However I do not recommend this last option because you might get conflicts with the name of your subfolders (if you have a main/A.py and a root/A.py, then import A will load the first one that is found, which depends on the order of your folders paths in your Python path variable). Also, this takes a few lines of code to write, whereas you can get a result just as nice just with some organization.
Disclaimer
I usually use one of the two options above because I do not know of any other. This is all based on my experience and reading StackOverflow posts, not on the documentation or anything like that, so please take my advice with a grain of salt.

I am not sure It will answer your Q but please check this post It may help you:
Importing files from different folder

Do I need to add my project directory to the system path in every script to import a function from another directory?

I'm trying to keep a data science project well-organized so I've created a directory inside my src directory called utils that contains a file called helpers.py, which contains some helper functions that will be used in many scripts. What is the best practice for how I should import func_name from src/utils/helpers.py into a file in a totally different directory, such as src/processing/clean_data.py?
I see answers to this question, and I've implemented a solution that works, but this feels ugly:
sys.path.append(os.path.dirname(os.path.dirname(os.path.dirname(os.path.realpath(__file__))))))
Am I doing this right? Do I need to add this to every script that wants to import func_name, like train_model.py?
My current project folder structure:
myproject
/notebooks
notebook.ipynb
/src
/processing
clean_data.py
/utils
helpers.py
/models
train_model.py
__init__.py
Example files:
# clean_data.py
import os
import sys
sys.path.append(os.path.dirname(os.path.dirname(os.path.dirname(os.path.realpath(__file__))))))
from src.utils.helpers import func_name
func_name()
# helpers.py
def func_name():
print('I'm a helper function.')

The correct way to do it is to use __init__.py, setup.py and the setuptools Python package:
myPackage/
myPackage/
__init__.py
setup.py
This link has all the steps.

First of all, let me describe you the differences between a Python module & a Python package so that both of us are on the same page. ✌
A module is a single .py file (or files) that are imported under one import and used. ✔
import aModuleName
# Here 'aModuleName' is just a regular .py file.
Whereas, a package is a collection of modules in directories that give a package hierarchy. A package contains a distinct __init__.py file. ✔
from aPackageName import aModuleName
# Here 'aPackageName` is a folder with a `__init__.py` file
# and 'aModuleName', which is just a regular .py file.
Therefore, when we have a project directory named proj-dir of the following structure ⤵
proj-dir
--|--__init__.py
--package1
--|--__init__.py
--|--module1.py
--package2
--|--__init__.py
--|--module2.py
🔎 Notice that I've also added an empty __init__.py into the proj-dir itself which makes it a package too.
👍 Now, if you want to import any python object from module2 of package2 into module1 of package1, then the import statement in the file module1.py would be
from package2.module2 import object2
# if you were to import the entire module2 then,
from package2 import module2
I hope this simple explanation clarifies your doubts on Python imports' mechanism and solves the problem. If not then do comment here. 😊

First of all let me clarify you that importing an entire module, if you are going to use a part of it, then is not a good idea. Instead of that you can use from to import specific function under a library/package. By doing this, you make your program efficient in terms of memory and performance.
To know more refer these:
'import module' or 'from module import'
difference between import and from
Net let us look into the solution.
Before starting off with the solution, let me clarify you the use of __init__.py file. It just tells the python interpreter that the *.py files present there are importable which means they are modules and are/maybe a part of a package.
So, If you have N no of sub directories you have to put __init__.py file in all those sub directories such that they can also be imported. Inside __init__.py file you can also add some additional information like which path should be included, default functions,variables,scope,..etc. To know about these just google about __init__.py file or take some python library and go through the same __init__.py file to know about it. (Here lies the solution)
More Info:
modules
Be pythonic
So as stated by #Sushant Chaudhary your project structure should be like
proj-dir
--|--__init__.py
--package1
--|--__init__.py
--|--module1.py
--package2
--|--__init__.py
--|--module2.py
So now, If I put __init__.py file under my directory like above, Will
it be importable and work fine?
yes and no.
Yes :
If you are importing the modules within that project/package directory.
for example in your case
you are importing package1.module1 in pakage2.module2 as from package1 import module1.
Here you have to import the base dir inside the sub modules, Why? the project will run fine if you are running the module from the same place. i.e: inside package2 as python module2.py, But will throw ModuleNotFoundError If you run the module from some other directory. i.e: any other path except under package2 for example under proj-dir as python package2/module2.py. This is what happening in your case. You are running the module from project-dir.
So How to fix this?
1- You have to append basedir path to system path in module2.py as
from sys import path
dir_path = "/absolute/path/to/proj-dir"
sys.path.insert(0, dir_path)
So that module2 will be able to find package1 (and module1 inside it).
2- You have to add all the sub module paths in __init__.py file under proj-dir.
For example:
#__init__.py under lxml
# this is a package
def get_include():
"""
Returns a list of header include paths (for lxml itself, libxml2
and libxslt) needed to compile C code against lxml if it was built
with statically linked libraries.
"""
import os
lxml_path = __path__[0]
include_path = os.path.join(lxml_path, 'includes')
includes = [include_path, lxml_path]
for name in os.listdir(include_path):
path = os.path.join(include_path, name)
if os.path.isdir(path):
includes.append(path)
return includes
This is the __init__.py file of lxml (a python library for parsing html,xml data). You can refer any __init__.py file under any python libraries having sub modules.ex (os,sys). Here I've mentioned lxml because I thought it will be easy for you to understand. You can even check __init__.py file under other libraries/packages. Each will have it's own way of defining the path for submodules.
No
If you are trying to import modules outside the directory. Then you have to export the module path such that other modules can find them into environment variables. This can be done directly by appending absolute path of the base dir to PYTHONPATH or to PATH.
To know more:
PATH variables in OS
PYTHONPATH variable
So to solve your problem, include the paths to all the sub modules in __init__.py file under proj-dir and add the /absolute/path/to/proj-dir either to PYTHONPATH or PATH.
Hope the answer explains you about usage of __init__.py and solves your problem.

On Linux, you can just add the path to the parent folder of your src directory to ~/.local/lib/python3.6/site-packages/my_modules.pth. See
Using .pth files. You can then import modules in src from anywhere on your system.
NB1: Replace python3.6 by any version of Python you want to use.
NB2: If you use Python2.7 (don't know for other versions), you will need to create __init__.py (empty) files in src/ and src/utils.
NB3: Any name.pth file is ok for my_modules.pth.

Yes, you can only import code from installed packages or from files in you working directory or subdirectories.

the way I see it, your problem would be solved if you would have your module or package installed, like an yother package one installs and then imports (numpy, xml, json etc.)
I also have a package I constantly use in all my projects, ulitilies, and I know it's a pain with the importing.
here is a description on how to How to package a python application to make it pip-installable:
https://marthall.github.io/blog/how-to-package-a-python-app/

Navigate to your python installation folder
Navigate to lib
Navigate to site-packages
Make a new file called any_thing_you_want.pth
Type .../src/utils/helpers.py inside that file with your favorite text editor
Note: the ellipsis before scr/utils/helpers.py will look something like: C:/Users/blahblahblah/python_folders/scr... <- YOU DO NEED THIS!
This is a cheap way out but it keeps code clean, and is the least complicated. The downside is, for every folder your modules are in, example.pth will need them. Upside: works with Windows all the way up to Windows 10

Import Python file within a different parent directory

I have the following directory structure:
- src
- __init__.py
- scripts
- __init__.py
- preprocessing.py
- project1
- main.py
- project2
- main.py
I'm trying to access the script(s) inside the scripts folder from within both the main.py files.
I've tried adding in the __init__.py (blank) files, and importing with import scripts, from src import scripts, and from .. import scripts. None of these seem to work.
I either get: ValueError: Attempted relative import in non-package, or no module found.
Thanks in advance!
P.S. I assume that the directory structure will get deeper soon (e.g. multiple subdirectories within scripts and project1 / project2). So if these is an easy way to deal with this as well, that would be very much appreciated.

One way to deal with that (albeit not the cleanest one) is to manually add the root directory to the sys.path variable so that python can look for modules in it, for example, in your main.py you may add these lines at the top:
#/src/project1/main.py
import os
import sys
sys.path.append(os.getcwd() + '\\..\\') # this is the src directory
This will allow python to look for modules in the directory above the one running the script, and this will work:
import scripts.preprocessing
Keep in mind that python will only look for modules in the same directory or below as the script is running. If you start /src/project2/main.py, python doesn't know anything about /src/project1/ or /src/scripts/

Importing from another directory

I am structuring my Python application with the following folder architecture (roughly following the outline here):
myapp:
myapp_service1:
myapp_service1_start.py
...
myapp_service2:
myapp_service2_start.py
...
myapp_service3:
myapp_service3_start.py
...
common:
myapp_common1.py
myapp_common2.py
myapp_common3.py
...
scripts:
script1.py
script2.py
script3.py
...
tests:
...
docs:
...
LICENSE.txt
MANIFEST.in
README
This is ideal file/folder hierarchy for me, however, I am confused on how to reference modules from outside folders. For instance, myapp_service1_start.py needs to reference function in myapp_common1.py and myapp_common2.py.
I know I need to somehow add reference to the system path or pythonpath, but I'm not sure the best way to do this in code. Or rather where I would even do this in code.
How can I do this?
I've seen a lot of posts about creating a full Python package to be installed via pip, but this seems like overkill to me.

One way is to make each of your myapp_service*_start.py files add myapp/ directory to sys.path.
For example, drop a file called import_me.py into myapp_service1/ with code that appends the "one up" directory (relative to importing file) to sys.path:
import os
import sys
import inspect
this_dir = os.path.dirname(inspect.getfile(inspect.currentframe()))
src_dir = os.path.join(this_dir, '..')
sys.path.insert(0, src_dir)
Then, in your myapp_service1_start.py you can do something like:
import import_me
from common import myapp_common1
from common import myapp_common2
Of course, be sure to make common directory a Python package by dropping a (possibly empty) __init__.py file into it.

How to import using a top level path in Python?

I'm developing a python framework and want to do imports based on the top-level package (the project name). Users will use the framework by copying the entire framework and writing their own modules within.
My current structure looks like this:
myapp/
config.py
docs/
framework/
main.py
utils.py
file.py
lib/
some_module.py
unit_tests/
test_utils.py
I want to be able to use the following import in python files in lib and unit_tests in the following way:
from myapp.framework import utils
Is there a simple way to do this? I've tried doing sys.path.append() hacks but they don't really work. If there is a truly pythonic way to achieve this, I don't mind going extra lengths to get it working.
EDIT: Well I tried the sys.path.append() again and it actually works but it really is an in-elegant solution and I'd really like to hear if there's another way.

For short: You can't have two modules with the same name one inside the other. You have the myapp folder and the myapp.py file
That's because of the order of importing modules.
Inside the current directory
The current package
Global package folder
So, if you're trying to import myapp.lib.some_module in config.py the interpreter will first see that you have myapp.py in the same folder and literaly try to import lib.some_module from that file, which doesn't exist.
Would be something as trying to import myapp.myapp.lib.some_module from outside that module.
So, the best thing for you to do is to rename the myapp.py file