python import from different level directory getting import error
Directory structure.
# all __init__.py files are empty
import/
├── helpers
│ ├── __init__.py
│ ├── helper1
│ │ ├── __init__.py
│ │ ├── helper1_module1.py
│ │ └── helper1_module2.py
│ └── helper2
│ ├── __init__.py
│ ├── helper2_module1.py
│ └── helper2_module2.py
└── services
├── __init__.py
├── service1
│ ├── __init__.py
│ └── service1_api.py
└── service2
helper1_module1.py
class Helper1Module1():
def function1(self):
print("executed from Helper1Module1 Class function1")
def function2(self):
print("executed from Helper1Module1 Class function2")
service1_api.py
from helpers.helper1.helper1_module1 import Helper1Module1
h = Helper1Module1()
h.function1()
Error:
python3 services/service1/service1_api.py
Traceback (most recent call last):
File "services/service1/service1_api.py", line 1, in <module>
from helpers.helper1.helper1_module1 import Helper1Module1
ModuleNotFoundError: No module named 'helpers'
How to fix this?
Python: python3.6 and above
OS: Linux
You have to set the file path (PYTHONPATH) manually to use the files from other directories.
You can export Environment Variable like
export PYTHONPATH=’path/to/directory’
Or you can use sys module: Importing files from different folder
Related
SOS. I'm in the process of migrating my SQLite database to MySQL and I'd like to share my MySQL connection using a function in /db/connect.py with my app at /app/app.py. I'm using a temp file to test the connection at /app/test.py which contains the function import. The connection from /db/connect.py runs successfully by itself. However, even after following other guides to the letter, my function in /db/connect.py isn't importing correctly. Even after adding an empty __init__.py file in the /db/ directory.
What I've tried
Here's a small sampling of what I've tried here, here, here, and here (as well as all other questions in stack overflow on the topic, e.g., here, here, here, here, etc).
/db/test.py (temp file with function import)
Imports /db/connect.py > sql_db_connection().
from db.connect import sql_db_connection
import db
def get_all():
conn = sql_db_connection()
cursor = conn.cursor()
cursor.execute("select * from posts")
result = cursor.fetchall()
for x in result:
print(x)
Error (error from running test.py)
Traceback (most recent call last):
File "/Users/<uname>/<my-long-app-name>/app/test.py", line 1, in <module>
from db.connect import sql_db_connection
ModuleNotFoundError: No module named 'db'
/db/connect.py (function for mysql connection)
import sshtunnel
import MySQLdb
import os
from dotenv import load_dotenv
# Load all env vars
load_dotenv()
def sql_db_connection():
sshtunnel.SSH_TIMEOUT = 5.0
sshtunnel.TUNNEL_TIMEOUT = 5.0
with sshtunnel.SSHTunnelForwarder(
('ssh.pythonanywhere.com'),
ssh_username=os.getenv("REMOTE_DB_SSH_USERNAME"), ssh_password=os.getenv("REMOTE_SSH_PASSWORD"),
remote_bind_address=(
os.getenv("REMOTE_BIND_ADDRESS"), 3306)
) as tunnel:
conn = MySQLdb.connect(
user=os.getenv("REMOTE_DB_SSH_USERNAME"),
passwd=os.getenv("REMOTE_DB_PASSWORD"),
host='0.0.0.0', port=tunnel.local_bind_port,
db=os.getenv("REMOTE_DB_NAME"),
)
print('Connected!')
return conn
sql_db_connection()
Tree
.
├── Dockerfile
├── README.md
├── __pycache__
│ ├── app.cpython-310.pyc
│ ├── app.cpython-39.pyc
│ ├── config.cpython-39.pyc
│ ├── wsgi.cpython-38.pyc
│ └── wsgi.cpython-39.pyc
├── app
│ ├── Dockerfile
│ ├── __init__.py
│ ├── __pycache__
│ ├── app.py
│ ├── config.py
│ ├── public
│ ├── requirements.txt
│ ├── templates
│ ├── test.py
│ └── wsgi.py
├── db
│ ├── Dockerfile
│ ├── README.md
│ ├── __init__.py
│ ├── __pycache__
│ ├── connect.py
│ ├── database.db
│ ├── dump.sql
│ ├── init_db.py
│ ├── mysql_dump.sql
│ ├── schema.sql
│ └── schema_mysql.sql
├── docker-compose.yml
├── features.md
├── images
├── requirements.txt
├── setup.py
├── venv
│ ├── bin
│ ├── include
│ ├── lib
│ ├── man
│ └── pyvenv.cfg
└── wsgi.py
add these two lines as the first two lines in test.py
import sys
sys.path.append('/Users/<uname>/<my-long-app-name>')
I am trying to turn a project of mine into a package so I can deploy it as a wheel.
I have a project directory setup like this:
ProjectDir
├── setup.py
├── MyModule
│ ├── __init__.py
│ ├── Module1
│ │ ├── __init__.py
│ │ ├── main.py
│ ├── Module2
│ │ ├── file1.py
│ │ ├── __init__.py
│ │ ├── file2.py
│ │ ├── file3.py
│ └── Module3
│ ├── __init__.py
│ ├── Sub1
│ │ ├── file1.py
│ │ ├── file2.py
│ │ ├── main.py
│ └── Sub2
│ ├── file1.py
│ ├── main.py
└── test
├── test_Module_1
│ ├── __init__.py
│ └── test_main.py
├── test_Module_2
...
Top level __init__.py is empty
Module 1 __init__.py file
from main import Function1
Similar for other module __init__.py files
setup.py
from setuptools import setup, find_packages
import os
import pip
setup(name='MyModule',
description='Tool suite to run MyModule',
packages=['MyModule'])
I can import MyModule but when I attempt to access any submodule, I get the following
AttributeError: module 'MyModule' has no attribute 'Module1'
Or if I check attributes of my module, none are found.
import MyModule
dir(MyModule)
['builtins', 'cached', 'doc', 'file', 'loader', 'name', 'package', 'path', 'spec', 'os']
This is expected because by default submodules are not imported.
You should import them like so to use them:
import MyModule.Module1
To change this you have to tweak the MyModule/__init__.py file by adding:
import MyModule.Module1
This way, MyModule.Module1 will be available when you import MyModule, as the __init__.py file is executed.
I am very new to Python and I have the following structure for a project:
server/
├── config/
│ ├── __init__.py
│ ├── application.py
│ ├── dev.py
│ └── qa.py
├── lib/
│ ├── __init__.py
│ ├── redisdb.py
│ ├── logger.py
│ └── geo.py
└── scripts/
├── __init__.py
├── my_first_script.py
└── my_second_script.py
and in my_first_script.py file, I have the following code:
import pickle
from lib.redisdb import r
import re
import config.application as appconf
print( appconf.DOCUMENT_ENDPOINT )
partnerData = pickle.loads(r.get("partner_date_all"))
print( len(partnerData) )
When I run this code in the terminal using the command
python server/scripts/my_first_script.py
I am getting the following error:
Traceback (most recent call last):
File "my_first_script.py", line 3, in <module>
from lib.redisdb import r
ImportError: No module named lib.redisdb
I am using Python 2.7 version here. When I checked with Python 3 also, I have the same error. Now how can I execute this file? If my code doesn't have imports from the other local modules, everything works just fine.
Your modules are all siblings and you didn't not declare a parent package.
You could modify your structure this way so your modules can know each other.
server/ (assuming this is your project root)
server/ (assuming you want to call your package "server")
├── __init__.py
├── server.py (your package entry point)
├── config/
│ ├── __init__.py
│ ├── application.py
│ ├── dev.py
│ └── qa.py
├── lib/
│ ├── __init__.py
│ ├── redisdb.py
│ ├── logger.py
│ └── geo.py
└── scripts/
├── __init__.py
├── my_first_script.py
└── my_second_script.py
And now your imports can refer to the parent package:
for example:
# my_first_script.py
from server.config import application
This is my folder structure
src
├── __init__.py
└── foo
├── __init__.py
└── bar
├── __init__.py
├── events
│ ├── __init__.py
│ ├── execute
│ │ ├── __init__.py
│ │ ├── helloworld.py
│ │ ├── run.py
└── settings.py
$ cat src/foo/__init__.py outputs...
from pkgutil import extend_path
__path__ = extend_path(__path__, __name__)
in src/foo/bar/events/execute/run.py, I want to do something like this
from foo.bar.events.execute.helloworld import HelloWorld
I get the error
No module named 'foo.bar'
This is how I'm running my app
I understand it's not proper, but there's a reason I just simplified the question for brevity
$ python src/foo/bar/events/execute/run
How do I achieve importing this way from src/foo/bar/events/execute/run.py?
from foo.bar.events.execute.helloworld import HelloWorld
I have a project with the current structure, but some of my imports are not working when I think they should be. Shoudn't these imports work since the folders are properly marked as modules?
foo
├── app
│ ├── app.py
│ ├── folder1
│ │ ├── aaa.py
│ │ └── __init__.py
│ ├── folder2
│ │ ├── bbb.py
│ │ ├── __init__.py
│ ├── folder3
│ │ ├── ccc.py
│ │ ├── __init__.py
│ ├── __init__.py
│ └── main.py
├── README.md
└── .gitignore
WORKS
aaa.py
class X():
pass
main.py
from folder1.aaa import X
PWD: foo folder
CMD: python app/main.py
DOES NOT WORK
aaa.py
class X():
pass
main.py
from app.folder1.aaa import X
PWD: foo folder
CMD: python app/main.py
Traceback (most recent call last):
File "foo/app/main.py", line 1, in <module>
from app.folder1.aaa import X
ModuleNotFoundError: No module named 'app'
DOES NOT WORK
aaa.py
from app.folder2.bbb import Y
class X(Y):
pass
bbb.py
class Y():
pass
main.py
from folder1.aaa import X
PWD: foo folder
CMD: python app/main.py
File "foo/app/folder1/aaa.py", line 1, in <module>
from app.folder2.bbb import Y
ModuleNotFoundError: No module named 'app'
Python import works by searching the paths in sys.path.
check whether app is added to sys.path by running the below code
import sys
print(sys.path)
if it is not present in this list, append sys.path by including app directory.
import sys
import os
current_loc = os.path.realpath(__file__)
parent_dir = os.path.dirname(os.path.dirname(current_loc))
sys.path.append(parent_dir)