Develop Reference

From Import in Python is not importing my modules

I have several custom module scrips in my python project that are all in a folder that is one directory down named "modules". After about a day of use on every machine I touch, importing these modules starts spitting out this error:
ImportError: cannot import name 'letters' from 'modules' (C:\Program Files\Python39\lib\site-packages\modules.py)
The code for importing custom modules is here.
from modules import letters
from modules import sfx
from modules import bgm
from modules.colors import colors, hex2dec
from modules import lessons
from modules import sessions
from key import key
The project folder looks like this (folders and files unrelated to the issue excluded):
formapp
├───modules
│ ├───bgm.py
│ ├───bot_ir.py
│ ├───chat.py
│ ├───colors.py
│ ├───formbot.py
│ ├───ir.py
│ ├───lessons.py
│ ├───letters.py
│ ├───sessions.py
│ └───sfx.py
├───app.py
├───key.py
For the most part, sys.path.append(r"modules") had worked to solve this issue however hex2dec in colors.py still caused errors. Regardless, of the 15 people working on this project this error only appears for me so I should not have to append a new folder to search for modules. The only thing that separates me from the other 15 people is that I was using formbot.py
Here is the code for formbot.py.
import requests
import time
class FormBot():
def __init__(self, username, password, timeout=0, host='127.0.0.1', port=420) :
self.host = '192.168.10.69'
self.port = 420
self.username = username
self.password = password
self.loggedin = False
time.sleep(timeout)
self.login()
print('logged in as', username)
#change "userType": "login", to "usertype": "new" on first time use
def login(self):
loginAttempt = requests.post(url="http://"+self.host+":"+str(self.port)+"/login", data={"username": self.username, "password": self.password, "userType": "new", "bot": "True", "forward":"/"})
#forbius login
Forbius = FormBot('Forbius', 'Password#1')
I have also tried reinstalling Python entirely and that has not worked.
To test, I tried running a fresh copy of the main branch on a computer affected by this issue and got the same error. The same code works fine on other machines.

try adding an __init__.py file inside the modules folder
inside the __init__.py file put:
from .letters import *
from .sfx import *
from .bgm import *
.
.
.

You'll need to add a file called __init__.py to your modules folder. The file can be empty. Review this SO thread for more details: What is __init__.py for?

How to best deal with circular import ImportError due to import statement in __init__ file

So I'm working on a rather big Python 3 project where I'm writing unit tests for some of the files using the unittest library. In one unit test, the tested file imports a function from another python package whose __init__ file itself imports from the tested file. This leads to an ImportError during the unit test.
It is desired for the __init__.py to import from the tested file periphery\foo.py, so I would like to know if there is a possibility to make the unit test work without removing the import from __init__.py
Since the project contains a lot of files, I created a minimal example that illustrates the structure and in which the error can be reproduced. The project structure looks like this
├───core
│ bar.py
│ __init__.py
│
├───periphery
│ foo.py
│ __init__.py
│
└───tests
│ __init__.py
│
├───core
│ __init__.py
│
└───periphery
test_foo.py
__init__.py
The init file in core core/__init__.py contains the code
# --- periphery ---
from periphery.foo import Foo
while periphery/foo.py, which is the file to be tested, looks like
from core.bar import Bar
def Foo():
bar = Bar()
return bar
Finally, the unit test has the following structure
from unittest import TestCase
from periphery.foo import Foo
class Test(TestCase):
def test_foo(self):
""" Test that Foo() returns "bar" """
self.assertEqual(Foo(), "bar")
Running the unit test yields the following error:
ImportError: cannot import name 'Foo' from partially initialized module 'periphery.foo' (most likely due to a circular import)

Importing Python modules in large projects and "ModuleNotFoundError"

I have faced a rather famous issue while importing my python modules inside the project.
This code is written to replicate the existing situation:
multiply_func.py:
def multiplier(num_1, num_2):
return num_1 * num_2
power_func.py:
from math_tools import multiplier
def pow(num_1, num_2):
result = num_1
for _ in range(num_2 - 1):
result = multiplier(num_1, result)
return result
The project structure:
project/
│ main.py
│
└─── tools/
│ __init__.py
│ power_func.py
│
└─── math_tools/
│ __init__.py
│ multiply_func.py
I've added these lines to __init__ files to make the importing easier:
__init__.py (math_tools):
from .multiply_func import multiplier
__init__.py (tools):
from .power_func import pow
from .math_tools.multiply_func import multiplier
Here is my main file.
main.py:
from tools import pow
print(pow(2, 3))
Whenever I run it, there is this error:
>>> ModuleNotFoundError: No module named 'math_tools'
I tried manipulating the sys.path, but I had no luck eliminating this puzzling issue. I'd appreciate your kind help. Thank you in advance!

You messed it up in the "power_func.py" file.
You have to use . before math_tools to refer to the current directory module.
Update "power_func.py" as bellow, it works perfectly.
from .math_tools import multiplier

How to execute python file from other directory?

I've got this structure:
│
├ main.py
├ dir
| ├─ data.txt
| └─ other.py
Contents from other.py:
print(open('data.txt', 'utf-8').read())
I run main.py. It must start dir/other.py.
But other.py for works needs data.txt. Is there a way to start other.py from main.py, not editing other.py?
Note
User must be able to start other.py manualy without any errors

For this purpose you can use the import keyword. All you have to do is create an __init__.py script under the dir directory which will define the directory as a library. Then you can just use import others in the main script.
It is recommended to modify the others.py script with the below snippet
if __name__ == '__main__':
// do stuff
otherwise it will execute the library each time you import it
update
It is far more simple. You just have to change directory with the os.chdir("./dir") call. After that you can run a simple import and the script will be executed.
./dir/other.py:
print("Module starts")
print(open('data', 'r').read())
print("Module ends")
./main.py
print("Main start")
import os
os.chdir("./dir")
from others import other
print("Main end" )

You can import other in main file like from dir.other import *

How to get relative path for logging.ini and log file in python over multiple OS

I have created one wrapper logger class which actually wraps the functionality of logging module. I have created it so that every class in my application using logging doesn't need to add logging.config.fileConfig() and logging.getLogger() at the top every-time.
My wrapper class looks like this -
class MyFileLogger():
# The config file for logging formatter.
logging.config.fileConfig('logging.ini', disable_existing_loggers=False)
def __init__(self, name):
self.logger=logging.getLogger(name)
def log_debug(self, message):
self.logger.debug(message)
logging.ini file -
[loggers]
keys=root
[handlers]
keys=fileHandler
[formatters]
keys=myFormatter
[logger_root]
level=NOTSET
handlers=fileHandler
[handler_fileHandler]
class=handlers.TimedRotatingFileHandler
level=DEBUG
formatter=myFormatter
args=('testlog.log','midnight',)
[formatter_myFormatter]
format=%(asctime)s |%(name)-12s |%(levelname)-8s |%(message)s
datefmt=
class=logging.Formatter
MyFileLogger.py and logging.ini file resides on the same path. My intention to create testlog.log is also in same path as logging.ini file.
Implementation of the MyFileLogger.py would be like -
imp.py -
fileLogger = MyFileLogger(__name__)
message = "hey!"
fileLogger.log_debug(message)
Now this imp.py doesn't reside on the same path as MyFileLogger.py and logging.ini. That's why when I run it, this gives me below error cause the python is running at imp.py path.
File "C:\Python34\lib\logging\config.py", line 76, in fileConfig
formatters = _create_formatters(cp)
File "C:\Python34\lib\logging\config.py", line 109, in _create_formatters
flist = cp["formatters"]["keys"]
File "C:\Python34\lib\configparser.py", line 937, in __getitem__
raise KeyError(key)
KeyError: 'formatters'
I know if I just give an absolute path to logging.ini and testlog.log it will work. But I don't want to go there, since this application can be ran in Windows or in Linux. So there would be path problem.
Now even if I don't care about the Windows-Linux thing and only concentrate on the Linux. There is another problem. So if we assume the current directory structure as this ("ProjectDir/src" is added in PYTHONPATH and assuming there is __init__.py in every module folders)-
ProjectDir
├── src
│ └── sales
│ ├── common
│ │ ├── logger
│ │ │ ├── logging.ini
│ │ │ ├── MyFileLogger.py
│ │ │ └── testlog.log
│ │ └── implmnts
│ │ └── imp.py
│ └── unitTest
│ └── test_MyFileLogger.py
└── bar.txt
Let's assume:
imp.py -
fileLogger = MyFileLogger(__name__)
message = "From imp.py"
fileLogger.log_debug(message)
test_MyFileLogger.py -
fileLogger = MyFileLogger(__name__)
message = "from test_MyFileLogger.py"
fileLogger.log_debug(message)
Then if I run imp.py I need to change the logging.config.fileConfig('logging.ini', disable_existing_loggers=False) to logging.config.fileConfig('../logger/logging.ini', disable_existing_loggers=False)
After that it will run but when I try to run test_MyFileLogger.py I need to change again to logging.config.fileConfig('../common/logger/logging.ini', disable_existing_loggers=False)
So my code is dependent upon which file I am running which I want to get rid of. Is there any way I can dynamically get the path of logging.ini and testlog.log independent of which file(from other path) is invoking this? I don't want to hardcore the absolute path, since this needs to run in both Windows and Linux, any solution?