Develop Reference

Relative Imports & Directory Structure (Trillionth Time)

SUMMARY
I am fairly new to designing full-fledged python projects, and all my Python work earlier has been with Jupyter Notebooks. Now that I am designing some application with Python, I am having considerable difficulty making it 'run'.
I have visited the following sites -
Relative imports in Python
Ultimate answer to relative python imports
python relative import example code does not work
But none of them seem to solve my issue.
PROBLEM
Here's my repo structure -
my_app/
__init__.py
code/
__init__.py
module_1/
some_code_1.py
module_2/
some_code_2.py
module_3/
some_code_3.py
main.py
tests/
__init__.py
module_1/
test_some_code_1.py
module_2/
test_some_code_2.py
module_3/
test_some_code_3.py
resources/
__init__.py
config.json
data.csv
I am primarily using PyCharm and VS Code for development and testing.
The main.py file has the following imports -
from code.module_1.some_code_1 import class_1
from code.module_2.some_code_2 import class_2
from code.module_3.some_code_3 import class_3
In the PyCharm run configuration, I have the working directory set to `User/blah/blah/my_app/
Whenever I run the main.py from PyCharm, it runs perfectly.
But if I run the program from terminal like -
$ python code/main.py
Traceback (most recent call last):
File "code/main.py", line 5, in <module>
from code.module_1.some_code_1 import class_1
ModuleNotFoundError: No module named 'code.module_1.some_code_1'; 'code' is not a package
I get the same error if I run the main.py from VS Code.
Is there a way to make this work for PyCharm as well as terminal?
If I change the imports to -
from module_1.some_code_1 import class_1
from module_2.some_code_2 import class_2
from module_3.some_code_3 import class_3
This works on the terminal but doesn't work in PyCharm. The test cases fail too.
Is there something I am missing, or some configuration that can be done to make all this work seamlessly?
Can someone help me with this?
Thanks!

The problem is when you do python code/main.py it makes your current working directory code/, which makes all of your absolute imports incorrect since Python doesn't see above that directory unless you explicitly change a setting like the PYTHONPATH environment variable.
Your best option is to rename main.py to __main__.py and then use python -m code (although do note that package name clashes with a module in the stdlib).
You also don't need the __init__.py in my_app/ unless you're going to treat that entire directory as a package.
And I would consider using relative imports instead of absolute ones (and I would also advise importing to the module and not the object/class in a module in your import statements to avoid circular import issues). For instance, for the from code.module_1.some_code_1 import class_1 line in code.main, I would make it from .module_1 import some_code_1.

python import module fails

custom/
scripts/
__init__.py
file1.py
utils/
__init__.py
utilFile1.py
utilFile2.py
utilFile3.py
I have the above file structure I'm struggling trying to figure out how to import modules in python. I have gone through stackoverflow posts that have the same question but I still couldn't figure out why I can't get it to work.
I am trying to import these modules in maya and when I run
from utils import utilFile1 I get
ImportError: cannot import name utilFile1.
Running from custom.scripts.utils import utilFile1 gives me this error ImportError: no module named custom.scripts.utils. However if I run import file1 it imports without any errors
I have appended custom/scripts to sys.path and when that didn't work, tried appending custom/scripts/utils as well, but that didn't work either. Based off some of the posts on stackoverflow, I saw some people suggesting to run "python -m" but I'm not sure if I should run this or where to execute it from.
I'm really at a loss as to why I can't get it to work at all.

you have make it a python package by following steps,
in your root dir create setup.py,
in your case
custom/
setup.py
scripts/
...
...
in setup.py
from setuptools import setup, find_packages
setup(
name='your_package_name',
description='bla bla bla',
version='0.0.1-dev',
install_requires=[
],
)
and then hit
pip install -e .
for more info refer to this docs

Please update the directory structure as below.
Python might not be considering custom as a module as it doesn't have __init__.py file,
custom/
__init__.py
scripts/
__init__.py
file1.py
utils/
__init__.py
utilFile1.py
utilFile2.py
utilFile3.py

Yash's method works, I believe that's what companies use to setup their environments. An alternative way is to add your scripts path as a PYTHONPATH in your maya.env file instead of using sys.path.append. I'm not sure why there's a difference or if it's coz I'm trying this on windows, but oddly enough it worked in my case.

Python3 relative imports failing in package

I've been reading tons of questions related to this matter but none of the has help me so far. I'm currently using the Python click library to execute scripts as commands.
The current command that I'm trying to execute is placed inside a Python Package which has a __main__.py file, like the parent dir has. The current project structure is the following one.
/myproject
/foo_one
__init__.py
foo_one.py
/foo_two
__init__.py
foo_two.py
/foo_three
__init__.py
foo_three.py
/foo_four
__init__.py
foo_four.py
/foo_five
__init__.py
foo_five.py
/foo_six
__init__.py
foo_six.py
__init__.py
__main__.py
foo_seven.py
Whenever I try to run the __main__.py script located in the project folder, the following error comes up.
ModuleNotFoundError: No module named '__main__.foo_two'; '__main__' is not a package
However, if I try to execute that same script from a folder above with the -m option like this python3 myproject -m, the following is shown up.
ImportError: attempted relative import with no known parent package
The __main__.py has 2 imports like this... The __init__.py is empty.
from .foo_two.foo_two import AClass, AnotherClass, OtherClass
from .foo_three.foo_three import AnotherClassMore
UPDATE: Correcting the syntax error in a previous command, while calling python -m myproject gives me a ModuleNotFoundError because of a module that isn't my responsibility, which is basically a library that is used in the project.

Hopefully, this will be of value to someone out there - I went through half a dozen stackoverflow posts trying to figure out relative imports similar to whats posted above here. I set up everything as suggested but I was still hitting ModuleNotFoundError: No module named 'my_module_name'
Since I was just developing locally and playing around, I hadn't created/run a setup.py file. I also hadn't apparently set my PYTHONPATH.
I realized that when I ran my code as I had been when the tests were in the same directory as the module, I couldn't find my module:
$ python3 test/my_module/module_test.py 2.4.0
Traceback (most recent call last):
File "test/my_module/module_test.py", line 6, in <module>
from my_module.module import *
ModuleNotFoundError: No module named 'my_module'
However, when I explicitly specified the path things started to work:
$ PYTHONPATH=. python3 test/my_module/module_test.py 2.4.0
...........
----------------------------------------------------------------------
Ran 11 tests in 0.001s
OK
So, in the event that anyone has tried a few suggestions, believes their code is structured correctly and still finds themselves in a similar situation as myself try either of the following if you don't just add your export the current directory to your PYTHONPATH:
Run your code and explicitly include the path like so:
$ PYTHONPATH=. python3 test/my_module/module_test.py
To avoid calling PYTHONPATH=., create a setup.py file with contents like the following and run python setup.py development to add packages to the path:
# setup.py
from setuptools import setup, find_packages
setup(
name='sample',
packages=find_packages()
)

The correct syntax would be
python -m myproject
This should execute __main__ in the top-level package.

You need to have __init__.py in each sub folder with python code to tell the interpreter to treat the folder as a module
/myproject
/foo_one
__init__.py # add this
foo_one.py
/foo_two
__init__.py # add this
foo_two.py
/foo_three
__init__.py # add this
foo_three.py
/foo_four
__init__.py # add this
foo_four.py
/foo_five
__init__.py # add this
foo_five.py
/foo_six
__init__.py # add this
foo_six.py
__init__.py
__main__.py
foo_seven.py
the __init__.py is telling the interpreter to treat sub folders as python modules / packages and you should be able to import
The __init__.py file can be empty but needs to be present in the sub folders to be able to import that module / package

Testing with Nose. Why am I getting "ImportError: No module named bin.app"

I'm currently on the last exercise (EX52) of Learn Python The Hard Way and I'm using Nose to test out the code (the exercise is to expand Nose testing to test out more of the code).
This is my file structure
bin
app.py
gothonweb
__init__.py
map.py
sessions
tests
map_tests.py
app_tests.py
The sample code tests out the map.py file using the map_tests.py code.
from nose.tools import *
from gothonweb.map import *
def test_room():
central_corridor = Room("Central Corridor" ...
So I thought I'd expand this by creating a second test file, called app_tests.py that tested the app.py file. It contains this code
from nose.tools import *
from bin.app import *
def test():
pass
When I run nosetests I get this error: ImportError: No module named bin.app
What am I doing wrong?

From your directory structure you can see that bin is not a python package. It is a directory that contains a python module named app as referenced by app.py. Python packages have an __init__.py file inside of folders. I could explain in more detail but it would probably be better to post a link for you to reference.
http://docs.python-guide.org/en/latest/writing/structure/

Since you don't have a __init__.py in your bin directory, it's just a directory, not a package. Since it's not a package, you can't look into the package to find the bin module.
From your parent directory, do
touch bin/__init__.py
on Mac/Linux, or
type NUL>bin\__init__.py
on Windows

How do I correctly import packages with py.test?

I have the following layout:
/spamalot
/spam
__init__.py
spam.py
/spam_on_eggs
__init__.py
spam_on_eggs.py
/tests
test_spam.py
Spam just so happens to be a flask application.
Within spam.py I have
import spam_on_eggs.spam_on_eggs as eggs
# Other Flask setup & application code here.
And this works fine - from the spamalot directory I'm able to run python spam/spam.py
However, when I start to throw tests into the mix, it's not as awesome.
In my test_spam.py file I have:
import spam.spam
test_client = spam.spam.app.test_client()
def test_it_fails():
assert False
However, rather than failing where I would expect, it fails on the import line:
/spamalot/ $ py.test
# some output
E ImportError
I can fix this by putting __init__.py in my /tests folder, but then I get a different ImportError:
spam/spam.py:1: in <module>
> import spam_on_eggs.spam_on_eggs as eggs
E ImportError: No module named 'spam_on_eggs'
I can solve that one by changing the line to:
from spam.spam_on_eggs import spam_on_eggs
Which allows me to test but then I break my ability to run $ python spam/spam.py - because I get
ImportError: no module named 'spam'
Obviously I have a gap in my understanding of how module imports work and how py.test works with this system.
What am I missing?
Is it even possible to have the layout I've described - and be able to run both py.test and my server from the spamalot directory?

py.test will always use the shortest directory path with an __init__.py in it.
Put a __init__.py into spamalot and you can import spamalot.spam.spam.
Reference: choosing a test layout