Develop Reference

Relative path ImportError when trying to import a shared module from a subdirectory in a script

I am trying to import a util package one directory up from where my code is, but I get an ImportError which I don't understand.
I have a number of different variations on the import syntax in Python, none of which are working.
There are a number of similar questions on Stack Overflow, but none have helped me understand or fix this issue.
Of the top of my head, I have tried the following variations:
import util
import ..util
from .. import util
from ..util import parser
from AdventOfCode2022 import util
from ..AdventOfCode2022 import util
from ...AdventOfCode2022 import util
Most of these I guessed wouldn't work, but I tried them anyway to be sure.
Error message:
ImportError: attempted relative import with no known parent package
Directory structure:
.
├── day03
│ ├── input.txt
│ ├── part1.py
│ ├── part2.py
│ └── test_input.txt
└── util
├── __init__.py
└── parser.py
I just want to import my util package from any "day0*/" directory - not sure why Python makes it so hard!

Two options:
Add the full path to ./util/ to your PYTHONPATH environment variable.
For example on Bash, your ~/.bashrc might have export PYTHONPATH="${PYTHONPATH}:/Users/foobar/projects/advent-of-code/util/".
Add sys.path.append('/path/to/application/app/folder') before the import.
The other solutions don't work because:
day03 and the parent directory are not modules with their own __init__.py. Lines like from ..util import parser only work if everything involved is a module.
You are presumably running the code from within ./day03/.
View this as 'I have a bunch of independent Python projects (day01, day02 etc) that all want to share a common piece of code I have living in a different project (util) that lives somewhere else on my computer.'

How to import Python file from different folder? (Works in one project directory but not in another)

Problem description
I cannot run a python script located in one directory if it imports a file from another directory (see both Folder Structure sections below for more detail).
I can run python calculator.py from the calculator directory
I can run python arkham_horror.py from the arkham-horror directory
I cannot run python chaos_bag.py from the tools directory
I get the following error when I do:
Traceback (most recent call last):
File "C:\Users\alexa\Git\arkham-horror\tools\chaos_bag.py", line 7, in <module>
import components.custom_widgets as custom_widgets
ModuleNotFoundError: No module named 'components'
I can run python chaos_bag.py from the arkham-horror directory, if chaos_bag.py is located in the arkham-horror directory (instead of the tools directory)
GOAL: What can I do successfully run python chaos_bag.py from the tools directory?
Is there something I can do to see what Python is doing behind the scenes so I can see what is going right with the working project and what is going wrong with the other project?
A lot of solutions online suggest programmatically (or manually) adding the project path to PYTHONPATH and while I'm sure this works (see Workaround section) I know this is not the "best practices" solution and I know that it's not necessary since the other project works.
My worry is that the reality is I cannot achieve my goal because of the file/folder structure, i.e. because I'm trying to run a file that's in a sub-directory, and not in the parent directory it won't work unless I do the workaround (described below).
Workaround
If I add the path to my project, C:\Users\alexa\Git\arkham-horror, to my PYTHONPATH environment variable that solves the problem.
However, I have to add this path each time I open up a new terminal.
I don't want to have to permanently add my project path because I know that's not the "best practices" solution to this problem. Plus, the other project works without this workaround.
Environment details
Operation system = Windows 11
Terminal = PowerShell v7.3.0
Python version = 3.10.5
PYTHONPATH = "C:\Python310"
Using pyenv-win
The project that works
Github link: https://github.com/alexcwarren/calculator
Folder structure
calculator
├── components
│   └── calculator_controller.py
│   └── calculator_model.py
│   └── calculator_view.py
│   └── characters.py
└── calculator.py
Imports
calculator_controller.py
import components.calculator_model as calcmodel
import components.calculator_view as calcview
from components.characters import Character
calculator_model.py
import components.characters as chars
calculator_view.py
import abc
import tkinter as tk
from functools import partial
import components.characters as chars
calculator.py
import components.calculator_controller as calccontroller
import components.calculator_model as calcmodel
import components.calculator_view as calcview
The project that doesn't work
Github link: https://github.com/alexcwarren/arkham-horror
Folder structure
arkham-horror
├── components
│   └── __init__.py
│   └── custom_widgets.py
├── tools
│   └── __init__.py
│   └── chaos_bag.py
└── __init__.py
└── arkham_horror.py
Imports
custom_widgets.py
import tkinter as tk
chaos_bag.py
import json
import random
import tkinter as tk
from functools import partial
from tkinter import ttk
import components.custom_widgets as custom_widgets
arkham_horror.py
import json
import tkinter as tk
import components.custom_widgets as custom_widgets
What I've tried
As I mentioned in the Workaround section, if I add the path to my project, C:\Users\alexa\Git\arkham-horror, to my PYTHONPATH environment variable that solves the problem.
As I also mentioned in the Workaround section, I don't want to have to do this if it's not the ONLY solution.
I have also tried relative imports but none have worked for me.

The path from where you start your script is usually very important in Python. Try running python tools\chaos_bag.py from the arkham-horror directory.
You could also write a batch file to set PYTHONPATH and use the batch file as an entry point to your program.
You could also try my import library: ultraimport
It gives you more control over your imports. It would work without modifying PYTHONPATH. You could write in your chaos_bag.py something like:
import ultraimport
custom_widgets = ultraimport("__dir__\..\components\custom_widgets.py")
This would always work, no matter how you run your program or what is your current working directory.

from x import y works on Windows but not on Ubuntu

I'm trying to import a class from a different file but it keeps giving me a "ModuleNotFoundError: No module named 'x'" error.
I'm trying to make myself a little Telegram bot that can do a variety of things for me. I am using PyCharm on Windows to code. Everything works on Windows. But after I copied the code to my VPS it spits out "ModuleNotFoundError". I have tried to use relative imports as well as absolute imports but none of them seemed to work. Read about relative and absolute imports from here.
When using relative imports, I get a different error saying " __main __.fileName " is not a package, which I feel is a step backwards.
I also think that I am having the same issue as the person on this stackexchange post. I did what the answer there said and added an empty "init.py" file, but I had no changes in output. Then I saw that, correct me if I am wrong, Python 3.3 and after does not need an empty init.py file in each of the subdirectories. But I still have them. Just in case.
I've already gone through a bunch of stackoverflow questions on kind of the same issue. But none of the answers are really solutions to my issue in my opinion.
Here is my directory structure right now.
baivrau-bot/
├── env.py
├── imgurDownloader
│   ├── __init__.py
│   ├── downloader.py
│   ├── main.py
│   ├── readme.md
│   └── test.py
├── readme.md
├── requirements.txt
Here is the error I am getting. Line 10 is the culprit.
Traceback (most recent call last):
File "main.py", line 10, in <module>
from imgurDownloader.downloader import ImgurAlbumDownloader
ModuleNotFoundError: No module named 'imgurDownloader'
Here are lines 1-16 on main.py
import telepot
from telepot.namedtuple import InputMediaPhoto
import glob
import os
import re
import time
import sys
import shutil
from hashlib import md5
from imgurDownloader.downloader import ImgurAlbumDownloader
from env import bot_token
chat_id = sys.argv[1]
imgur_link = sys.argv[2]
bot = telepot.Bot(bot_token)
The file 'downloader' is from a Github repo.
I used PyCharm on my Windows computer and it works completely fine. I except the same when running on Ubuntu or any linux distro.

Sounds like you didn't set your PYTHONPATH variable where he should search for your packages.
For single test try in your command line:
export PYTHONPATH="$/pwd/path_to_dir"
before you start your script. But you should definitely set this variable permanently.

You're probably not accessing main.py from the same folder. Check your working directory. What edition of PyCharm are you using? Can you run it from the terminal? Also, did you try removing the directory prefix so it's just from downloader import ImgurAlbumDownloader?
My final tip would be to follow a conventional project structure where your tests are in a different folder: What is the best project structure for a Python application?

Python relative imports with sys.path

I have this weird Python import and I can't get it to work with any of the suggestions in other discussions.
I'm currently adding a small script to someone else's module, so I cannot change the file structure or any of the existing code, which makes things difficult.
Here is an example of the python script which contains a bunch of classes:
/path/to/root/directory/aaa/bbb/ccc/utils.py
The current developer imports this as follows:
from aaa.bbb.ccc import utils
utils.SomeClass.someMethod()
All directories in the tree have a __init__.py file
Now I want to call the module externally as follows:
import sys
sys.path.append('/path/to/root/directory')
from aaa.bbb.ccc import utils
utils.SomeClass.someMethod()
This does NOT work, and gives the the following error:
from aaa.bbb.ccc import utils
ImportError: No module named ccc
However, changing the import a little bit does work:
import sys
sys.path.append('/path/to/root/directory')
from aaa.bbb.ccc.utils import *
SomeClass.someMethod()
I do not understand why the 2nd one works and not the 1st. Again, I cannot change the existing code, so the following 2 statements must work with my sys.path.append and imports:
from aaa.bbb.ccc import utils
utils.SomeClass.someMethod()
I cannot remove the utils from utils.SomeClass
Does anyone know how I can achieve this?

This really isn't an answer, but there is no way I could fit this into a comment, let alone with any sane formatting. My apology, but it hopefully can still help us get somewhere.
I suspect there is still a bit of information missing somewhere in the question. I've setup the following tree under /tmp/so/xxx:
.
└── aaa
├── __init__.py
└── bbb
├── __init__.py
└── ccc
├── __init__.py
└── utils.py
__init__.py files are blank and utils.py says:
class SomeClass:
#staticmethod
def someMethod():
print("FOO")
Which if I read the description correctly should be enough to replicate the behavior. Now I try both snippets of yours:
import sys
sys.path.append("/tmp/so/xxx")
from aaa.bbb.ccc import utils
utils.SomeClass.someMethod()
And run:
$ python test1.py
FOO
And second one:
import sys
sys.path.append("/tmp/so/xxx")
from aaa.bbb.ccc.utils import *
SomeClass.someMethod()
Runs as well:
$ python test2.py
FOO
I've tried Python 3.6.5 and 2.7.15. Same result.
Based on your output, I presume you're running Python 2. The only way I could reproduce this problem (get the same error) was by removing __init__.py from ccc/, but the other syntax would not work either then.
Could you perhaps post your tree and content of __init__.py files? With the available data, I was unfortunately able to reproduce the problem.

relative paths for modules in python

I've attempted a few different techniques trying to do something that to me seems doable but I guess I am missing some gotchas about python (using 2.7 but would like this to work also for 3.* if possible).
I am not sure about terminology like package or module, but to me the following seems quite a "simple" doable scenario.
This is the directory structure:
.
├── job
│   └── the_script.py
└── modules
   ├── __init__.py
   └── print_module.py
The content of the_script.py:
# this does not work
import importlib
print_module = importlib.import_module('.print_module', '..modules')
# this also does not work
from ..modules import print_module
print_module.do_stuff()
The content of print_module:
def do_stuff():
print("This should be in stdout")
I would like to run all this "relative paths" stuff as:
/job$ python2 the_script.py
But the importlib.import_module gives various errors:
if I just use 1 input parameter ..modules.print_module, then I get: TypeError("relative imports require the 'package' argument")
if I use 2 input parameters (as in the example above), then I get: ValueError: Empty module name
On the other hand using the from ..modules syntax I get: ValueError: Attempted relative import in non-package.
I think the __init__.py empty file should be enough to qualify that code as "packages" (or modules? not sure about the terminology), but it seems there's something I am missing about how to manage relative paths.
I read that in the past people was hacking this using the path and other functions from import os and import sys, but according to the official docs (python 2.7 and 3.*) this should not be needed anymore.
What am I doing wrong and how could I achieve the result of printing the content modules/print_module.do_stuff calling it from a script in the "relative directory" job/?

If you follow the structure of this guide here: http://docs.python-guide.org/en/latest/writing/structure/#test-suite (highly recommend reading it all, it is very helpful) you will see this:
To give the individual tests import context, create a tests/context.py file:
import os
import sys
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
import sample
Then, within the individual test modules, import the module like so:
from .context import sample
This will always work as expected, regardless of installation method.
Translated in your case this means:
root_folder
├── job
│ ├── context.py <- create this file
│ └── the_script.py
└── modules
├── __init__.py
└── print_module.py
In the context.py file write the lines shown above, but import modules instead of import samples
Finally in your the_script.py: from .context import module and you will be set to go!
Good luck :)

I found a solution using sys and os.
The script the_script.py should be:
import sys
import os
lib_path = os.path.abspath(os.path.join(os.path.dirname(__file__), '../modules'))
sys.path.append(lib_path)
# commenting out the following shows the `modules` directory in the path
# print(sys.path)
import print_module
print_module.do_stuff()
Then I can run it via command line no matter where I am in the path e.g.:
/job$ python2 the_script.py
<...>/job$ python2 <...>/job/the_script.py

If you are not sure about terminology go to very nice tutorials:
http://docs.python-guide.org/en/latest/writing/structure/#modules
and
http://docs.python-guide.org/en/latest/writing/structure/#packages
But for your structure:
.
├── job
│ └── the_script.py
└── modules
├── __init__.py
└── print_module.py
just say in the the_script.py:
import sys
sys.append('..')
import modules.print_module
This will add parent directory to PYTHONPATH, and python will see directory 'parallel' to job directory and it will work.
I think that at the most basic level it is sufficent to know that:
package is any directory with __init__.py file
module is a file with .py, but when you are importing module you omit extension.