Related
I have this folder structure:
application
├── app
│ └── folder
│ └── file.py
└── app2
└── some_folder
└── some_file.py
How can I import a function from file.py, from within some_file.py? I tried:
from application.app.folder.file import func_name
but it doesn't work.
Note: This answer was intended for a very specific question. For most programmers coming here from a search engine, this is not the answer you are looking for. Typically you would structure your files into packages (see other answers) instead of modifying the search path.
By default, you can't. When importing a file, Python only searches the directory that the entry-point script is running from and sys.path which includes locations such as the package installation directory (it's actually a little more complex than this, but this covers most cases).
However, you can add to the Python path at runtime:
# some_file.py
import sys
# caution: path[0] is reserved for script path (or '' in REPL)
sys.path.insert(1, '/path/to/application/app/folder')
import file
Nothing wrong with:
from application.app.folder.file import func_name
Just make sure folder also contains an __init__.py, this allows it to be included as a package. Not sure why the other answers talk about PYTHONPATH.
When modules are in parallel locations, as in the question:
application/app2/some_folder/some_file.py
application/app2/another_folder/another_file.py
This shorthand makes one module visible to the other:
import sys
sys.path.append('../')
First import sys in name-file.py
import sys
Second append the folder path in name-file.py
sys.path.insert(0, '/the/folder/path/name-package/')
Third Make a blank file called __ init __.py in your subdirectory (this tells Python it is a package)
name-file.py
name-package
__ init __.py
name-module.py
Fourth import the module inside the folder in name-file.py
from name-package import name-module
I think an ad-hoc way would be to use the environment variable PYTHONPATH as described in the documentation: Python2, Python3
# Linux & OSX
export PYTHONPATH=$HOME/dirWithScripts/:$PYTHONPATH
# Windows
set PYTHONPATH=C:\path\to\dirWithScripts\;%PYTHONPATH%
Your problem is that Python is looking in the Python directory for this file and not finding it. You must specify that you are talking about the directory that you are in and not the Python one.
To do this you change this:
from application.app.folder.file import func_name
to this:
from .application.app.folder.file import func_name
By adding the dot you are saying look in this folder for the application folder instead of looking in the Python directory.
Try Python's relative imports:
from ...app.folder.file import func_name
Every leading dot is another higher level in the hierarchy beginning with the current directory.
Problems? If this isn't working for you then you probably are getting bit by the many gotcha's relative imports has.
Read answers and comments for more details:
How to fix "Attempted relative import in non-package" even with __init__.py
Hint: have __init__.py at every directory level. You might need python -m application.app2.some_folder.some_file (leaving off .py) which you run from the top level directory or have that top level directory in your PYTHONPATH. Phew!
The answers here are lacking in clarity, this is tested on Python 3.6
With this folder structure:
main.py
|
---- myfolder/myfile.py
Where myfile.py has the content:
def myfunc():
print('hello')
The import statement in main.py is:
from myfolder.myfile import myfunc
myfunc()
and this will print hello.
In Python 3.4 and later, you can import from a source file directly (link to documentation). This is not the simplest solution, but I'm including this answer for completeness.
Here is an example. First, the file to be imported, named foo.py:
def announce():
print("Imported!")
The code that imports the file above, inspired heavily by the example in the documentation:
import importlib.util
def module_from_file(module_name, file_path):
spec = importlib.util.spec_from_file_location(module_name, file_path)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
foo = module_from_file("foo", "/path/to/foo.py")
if __name__ == "__main__":
print(foo)
print(dir(foo))
foo.announce()
The output:
<module 'foo' from '/path/to/foo.py'>
['__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'announce']
Imported!
Note that the variable name, the module name, and the filename need not match. This code still works:
import importlib.util
def module_from_file(module_name, file_path):
spec = importlib.util.spec_from_file_location(module_name, file_path)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
baz = module_from_file("bar", "/path/to/foo.py")
if __name__ == "__main__":
print(baz)
print(dir(baz))
baz.announce()
The output:
<module 'bar' from '/path/to/foo.py'>
['__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'announce']
Imported!
Programmatically importing modules was introduced in Python 3.1 and gives you more control over how modules are imported. Refer to the documentation for more information.
From what I know, add an __init__.py file directly in the folder of the functions you want to import will do the job.
Using sys.path.append with an absolute path is not ideal when moving the application to other environments. Using a relative path won't always work because the current working directory depends on how the script was invoked.
Since the application folder structure is fixed, we can use os.path to get the full path of the module we wish to import. For example, if this is the structure:
/home/me/application/app2/some_folder/vanilla.py
/home/me/application/app2/another_folder/mango.py
And let's say that you want to import the mango module. You could do the following in vanilla.py:
import sys, os.path
mango_dir = (os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))
+ '/another_folder/')
sys.path.append(mango_dir)
import mango
Of course, you don't need the mango_dir variable.
To understand how this works look at this interactive session example:
>>> import os
>>> mydir = '/home/me/application/app2/some_folder'
>>> newdir = os.path.abspath(os.path.join(mydir, '..'))
>>> newdir
'/home/me/application/app2'
>>> newdir = os.path.abspath(os.path.join(mydir, '..')) + '/another_folder'
>>>
>>> newdir
'/home/me/application/app2/another_folder'
>>>
And check the os.path documentation.
Also worth noting that dealing with multiple folders is made easier when using packages, as one can use dotted module names.
I was faced with the same challenge, especially when importing multiple files, this is how I managed to overcome it.
import os, sys
from os.path import dirname, join, abspath
sys.path.insert(0, abspath(join(dirname(__file__), '..')))
from root_folder import file_name
Considering application as the root directory for your python project, create an empty __init__.py file in application, app and folder folders. Then in your some_file.py make changes as follows to get the definition of func_name:
import sys
sys.path.insert(0, r'/from/root/directory/application')
from application.app.folder.file import func_name ## You can also use '*' wildcard to import all the functions in file.py file.
func_name()
Worked for me in python3 on linux
import sys
sys.path.append(pathToFolderContainingScripts)
from scriptName import functionName #scriptName without .py extension
The best practice for creating a package can be running and accessing the other modules from a module like main_module.py at highest level directory.
This structure demonstrates you can use and access sub package, parent package, or same level packages and modules by using a top level directory file main_module.py.
Create and run these files and folders for testing:
package/
|
|----- __init__.py (Empty file)
|------- main_module.py (Contains: import subpackage_1.module_1)
|------- module_0.py (Contains: print('module_0 at parent directory, is imported'))
|
|
|------- subpackage_1/
| |
| |----- __init__.py (Empty file)
| |----- module_1.py (Contains: print('importing other modules from module_1...')
| | import module_0
| | import subpackage_2.module_2
| | import subpackage_1.sub_subpackage_3.module_3)
| |----- photo.png
| |
| |
| |----- sub_subpackage_3/
| |
| |----- __init__.py (Empty file)
| |----- module_3.py (Contains: print('module_3 at sub directory, is imported'))
|
|------- subpackage_2/
| |
| |----- __init__.py (Empty file)
| |----- module_2.py (Contains: print('module_2 at same level directory, is imported'))
Now run main_module.py
the output is
>>>'importing other modules from module_1...'
'module_0 at parent directory, is imported'
'module_2 at same level directory, is imported'
'module_3 at sub directory, is imported'
Opening pictures and files note:
In a package structure if you want to access a photo, use absolute directory from highest level directory.
let's Suppose you are running main_module.py and you want to open photo.png inside module_1.py.
what module_1.py must contain is:
Correct:
image_path = 'subpackage_1/photo.png'
cv2.imread(image_path)
Wrong:
image_path = 'photo.png'
cv2.imread(image_path)
although module_1.py and photo.png are at same directory.
├───root
│ ├───dir_a
│ │ ├───file_a.py
│ │ └───file_xx.py
│ ├───dir_b
│ │ ├───file_b.py
│ │ └───file_yy.py
│ ├───dir_c
│ └───dir_n
You can add the parent directory to PYTHONPATH, in order to achieve that, you can use OS depending path in the "module search path" which is listed in sys.path. So you can easily add the parent directory like following:
# file_b.py
import sys
sys.path.insert(0, '..')
from dir_a.file_a import func_name
This works for me on windows
# some_file.py on mainApp/app2
import sys
sys.path.insert(0, sys.path[0]+'\\app2')
import some_file
In my case I had a class to import. My file looked like this:
# /opt/path/to/code/log_helper.py
class LogHelper:
# stuff here
In my main file I included the code via:
import sys
sys.path.append("/opt/path/to/code/")
from log_helper import LogHelper
I bumped into the same question several times, so I would like to share my solution.
Python Version: 3.X
The following solution is for someone who develops your application in Python version 3.X because Python 2 is not supported since Jan/1/2020.
Project Structure
In python 3, you don't need __init__.py in your project subdirectory due to the Implicit Namespace Packages. See Is init.py not required for packages in Python 3.3+
Project
├── main.py
├── .gitignore
|
├── a
| └── file_a.py
|
└── b
└── file_b.py
Problem Statement
In file_b.py, I would like to import a class A in file_a.py under the folder a.
Solutions
#1 A quick but dirty way
Without installing the package like you are currently developing a new project
Using the try catch to check if the errors. Code example:
import sys
try:
# The insertion index should be 1 because index 0 is this file
sys.path.insert(1, '/absolute/path/to/folder/a') # the type of path is string
# because the system path already have the absolute path to folder a
# so it can recognize file_a.py while searching
from file_a import A
except (ModuleNotFoundError, ImportError) as e:
print("{} fileure".format(type(e)))
else:
print("Import succeeded")
#2 Install your package
Once you installed your application (in this post, the tutorial of installation is not included)
You can simply
try:
from __future__ import absolute_import
# now it can reach class A of file_a.py in folder a
# by relative import
from ..a.file_a import A
except (ModuleNotFoundError, ImportError) as e:
print("{} fileure".format(type(e)))
else:
print("Import succeeded")
Happy coding!
I'm quite special : I use Python with Windows !
I just complete information : for both Windows and Linux, both relative and absolute path work into sys.path (I need relative paths because I use my scripts on the several PCs and under different main directories).
And when using Windows both \ and / can be used as separator for file names and of course you must double \ into Python strings,
some valid examples :
sys.path.append('c:\\tools\\mydir')
sys.path.append('..\\mytools')
sys.path.append('c:/tools/mydir')
sys.path.append('../mytools')
(note : I think that / is more convenient than \, event if it is less 'Windows-native' because it is Linux-compatible and simpler to write and copy to Windows explorer)
If the purpose of loading a module from a specific path is to assist you during the development of a custom module, you can create a symbolic link in the same folder of the test script that points to the root of the custom module. This module reference will take precedence over any other modules installed of the same name for any script run in that folder.
I tested this on Linux but it should work in any modern OS that supports symbolic links.
One advantage to this approach is that you can you can point to a module that's sitting in your own local SVC branch working copy which can greatly simplify the development cycle time and reduce failure modes of managing different versions of the module.
I was working on project a that I wanted users to install via pip install a with the following file list:
.
├── setup.py
├── MANIFEST.in
└── a
├── __init__.py
├── a.py
└── b
├── __init__.py
└── b.py
setup.py
from setuptools import setup
setup (
name='a',
version='0.0.1',
packages=['a'],
package_data={
'a': ['b/*'],
},
)
MANIFEST.in
recursive-include b *.*
a/init.py
from __future__ import absolute_import
from a.a import cats
import a.b
a/a.py
cats = 0
a/b/init.py
from __future__ import absolute_import
from a.b.b import dogs
a/b/b.py
dogs = 1
I installed the module by running the following from the directory with MANIFEST.in:
python setup.py install
Then, from a totally different location on my filesystem /moustache/armwrestle I was able to run:
import a
dir(a)
Which confirmed that a.cats indeed equalled 0 and a.b.dogs indeed equalled 1, as intended.
Instead of just doing an import ..., do this :
from <MySubFolder> import <MyFile>
MyFile is inside the MySubFolder.
In case anyone still looking for a solution. This worked for me.
Python adds the folder containing the script you launch to the PYTHONPATH, so if you run
python application/app2/some_folder/some_file.py
Only the folder application/app2/some_folder is added to the path (not the base dir that you're executing the command in). Instead, run your file as a module and add a __init__.py in your some_folder directory.
python -m application.app2.some_folder.some_file
This will add the base dir to the python path, and then classes will be accessible via a non-relative import.
The code below imports the Python script given by it's path, no matter where it is located, in a Python version-safe way:
def import_module_by_path(path):
name = os.path.splitext(os.path.basename(path))[0]
if sys.version_info[0] == 2:
# Python 2
import imp
return imp.load_source(name, path)
elif sys.version_info[:2] <= (3, 4):
# Python 3, version <= 3.4
from importlib.machinery import SourceFileLoader
return SourceFileLoader(name, path).load_module()
else:
# Python 3, after 3.4
import importlib.util
spec = importlib.util.spec_from_file_location(name, path)
mod = importlib.util.module_from_spec(spec)
spec.loader.exec_module(mod)
return mod
I found this in the codebase of psutils, at line 1042 in psutils.test.__init__.py (most recent commit as of 09.10.2020).
Usage example:
script = "/home/username/Documents/some_script.py"
some_module = import_module_by_path(script)
print(some_module.foo())
Important caveat: The module will be treated as top-level; any relative imports from parent packages in it will fail.
Wow, I did not expect to spend so much time on this. The following worked for me:
OS: Windows 10
Python: v3.10.0
Note: Since I am Python v3.10.0, I am not using __init__.py files, which did not work for me anyway.
application
├── app
│ └── folder
│ └── file.py
└── app2
└── some_folder
└── some_file.py
WY Hsu's 1st solution worked for me. I have reposted it with an absolute file reference for clarity:
import sys
sys.path.insert(1, 'C:\\Users\\<Your Username>\\application')
import app2.some_folder.some_file
some_file.hello_world()
Alternative Solution: However, this also worked for me:
import sys
sys.path.append( '.' )
import app2.some_folder.some_file
some_file.hello_world()
Although, I do not understand why it works. I thought the dot is a reference to the current directory. However, when printing out the paths to the current folder, the current directory is already listed at the top:
for path in sys.path:
print(path)
Hopefully, someone can provide clarity as to why this works in the comments. Nevertheless, I also hope it helps someone.
This problem may be due Pycharm
I had the same problem while using Pycharm. I had this project structure
skylake\
backend\
apps\
example.py
configuration\
settings.py
frontend\
...some_stuff
and code from configuration import settings in example.py raised import error
the problem was that when I opened Pycharm, it considered that skylake is root path and ran this code
sys.path.extend(['D:\\projects\\skylake', 'D:/projects/skylake'])
To fix this I just marked backend directory as source root
And it's fixed my problem
My solution for people. who have all the necessary __init__.py in the package, but import still doesn't work.
import sys
import os
sys.path.insert(0, os.getcwd())
import application.app.folder.file as file
You can use importlib to import modules where you want to import a module from a folder using a string like so:
import importlib
scriptName = 'Snake'
script = importlib.import_module('Scripts\\.%s' % scriptName)
This example has a main.py which is the above code then a folder called Scripts and then you can call whatever you need from this folder by changing the scriptName variable. You can then use script to reference to this module. such as if I have a function called Hello() in the Snake module you can run this function by doing so:
script.Hello()
I have tested this in Python 3.6
I usually create a symlink to the module I want to import. The symlink makes sure Python interpreter can locate the module inside the current directory (the script you are importing the other module into); later on when your work is over, you can remove the symlink. Also, you should ignore symlinks in .gitignore, so that, you wouldn't accidentally commit symlinked modules to your repo. This approach lets you even successfully work with modules that are located parallel to the script you are executing.
ln -s ~/path/to/original/module/my_module ~/symlink/inside/the/destination/directory/my_module
I have this folder structure:
application
├── app
│ └── folder
│ └── file.py
└── app2
└── some_folder
└── some_file.py
How can I import a function from file.py, from within some_file.py? I tried:
from application.app.folder.file import func_name
but it doesn't work.
Note: This answer was intended for a very specific question. For most programmers coming here from a search engine, this is not the answer you are looking for. Typically you would structure your files into packages (see other answers) instead of modifying the search path.
By default, you can't. When importing a file, Python only searches the directory that the entry-point script is running from and sys.path which includes locations such as the package installation directory (it's actually a little more complex than this, but this covers most cases).
However, you can add to the Python path at runtime:
# some_file.py
import sys
# caution: path[0] is reserved for script path (or '' in REPL)
sys.path.insert(1, '/path/to/application/app/folder')
import file
Nothing wrong with:
from application.app.folder.file import func_name
Just make sure folder also contains an __init__.py, this allows it to be included as a package. Not sure why the other answers talk about PYTHONPATH.
When modules are in parallel locations, as in the question:
application/app2/some_folder/some_file.py
application/app2/another_folder/another_file.py
This shorthand makes one module visible to the other:
import sys
sys.path.append('../')
First import sys in name-file.py
import sys
Second append the folder path in name-file.py
sys.path.insert(0, '/the/folder/path/name-package/')
Third Make a blank file called __ init __.py in your subdirectory (this tells Python it is a package)
name-file.py
name-package
__ init __.py
name-module.py
Fourth import the module inside the folder in name-file.py
from name-package import name-module
I think an ad-hoc way would be to use the environment variable PYTHONPATH as described in the documentation: Python2, Python3
# Linux & OSX
export PYTHONPATH=$HOME/dirWithScripts/:$PYTHONPATH
# Windows
set PYTHONPATH=C:\path\to\dirWithScripts\;%PYTHONPATH%
Your problem is that Python is looking in the Python directory for this file and not finding it. You must specify that you are talking about the directory that you are in and not the Python one.
To do this you change this:
from application.app.folder.file import func_name
to this:
from .application.app.folder.file import func_name
By adding the dot you are saying look in this folder for the application folder instead of looking in the Python directory.
Try Python's relative imports:
from ...app.folder.file import func_name
Every leading dot is another higher level in the hierarchy beginning with the current directory.
Problems? If this isn't working for you then you probably are getting bit by the many gotcha's relative imports has.
Read answers and comments for more details:
How to fix "Attempted relative import in non-package" even with __init__.py
Hint: have __init__.py at every directory level. You might need python -m application.app2.some_folder.some_file (leaving off .py) which you run from the top level directory or have that top level directory in your PYTHONPATH. Phew!
The answers here are lacking in clarity, this is tested on Python 3.6
With this folder structure:
main.py
|
---- myfolder/myfile.py
Where myfile.py has the content:
def myfunc():
print('hello')
The import statement in main.py is:
from myfolder.myfile import myfunc
myfunc()
and this will print hello.
In Python 3.4 and later, you can import from a source file directly (link to documentation). This is not the simplest solution, but I'm including this answer for completeness.
Here is an example. First, the file to be imported, named foo.py:
def announce():
print("Imported!")
The code that imports the file above, inspired heavily by the example in the documentation:
import importlib.util
def module_from_file(module_name, file_path):
spec = importlib.util.spec_from_file_location(module_name, file_path)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
foo = module_from_file("foo", "/path/to/foo.py")
if __name__ == "__main__":
print(foo)
print(dir(foo))
foo.announce()
The output:
<module 'foo' from '/path/to/foo.py'>
['__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'announce']
Imported!
Note that the variable name, the module name, and the filename need not match. This code still works:
import importlib.util
def module_from_file(module_name, file_path):
spec = importlib.util.spec_from_file_location(module_name, file_path)
module = importlib.util.module_from_spec(spec)
spec.loader.exec_module(module)
return module
baz = module_from_file("bar", "/path/to/foo.py")
if __name__ == "__main__":
print(baz)
print(dir(baz))
baz.announce()
The output:
<module 'bar' from '/path/to/foo.py'>
['__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', 'announce']
Imported!
Programmatically importing modules was introduced in Python 3.1 and gives you more control over how modules are imported. Refer to the documentation for more information.
From what I know, add an __init__.py file directly in the folder of the functions you want to import will do the job.
Using sys.path.append with an absolute path is not ideal when moving the application to other environments. Using a relative path won't always work because the current working directory depends on how the script was invoked.
Since the application folder structure is fixed, we can use os.path to get the full path of the module we wish to import. For example, if this is the structure:
/home/me/application/app2/some_folder/vanilla.py
/home/me/application/app2/another_folder/mango.py
And let's say that you want to import the mango module. You could do the following in vanilla.py:
import sys, os.path
mango_dir = (os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))
+ '/another_folder/')
sys.path.append(mango_dir)
import mango
Of course, you don't need the mango_dir variable.
To understand how this works look at this interactive session example:
>>> import os
>>> mydir = '/home/me/application/app2/some_folder'
>>> newdir = os.path.abspath(os.path.join(mydir, '..'))
>>> newdir
'/home/me/application/app2'
>>> newdir = os.path.abspath(os.path.join(mydir, '..')) + '/another_folder'
>>>
>>> newdir
'/home/me/application/app2/another_folder'
>>>
And check the os.path documentation.
Also worth noting that dealing with multiple folders is made easier when using packages, as one can use dotted module names.
I was faced with the same challenge, especially when importing multiple files, this is how I managed to overcome it.
import os, sys
from os.path import dirname, join, abspath
sys.path.insert(0, abspath(join(dirname(__file__), '..')))
from root_folder import file_name
Considering application as the root directory for your python project, create an empty __init__.py file in application, app and folder folders. Then in your some_file.py make changes as follows to get the definition of func_name:
import sys
sys.path.insert(0, r'/from/root/directory/application')
from application.app.folder.file import func_name ## You can also use '*' wildcard to import all the functions in file.py file.
func_name()
Worked for me in python3 on linux
import sys
sys.path.append(pathToFolderContainingScripts)
from scriptName import functionName #scriptName without .py extension
The best practice for creating a package can be running and accessing the other modules from a module like main_module.py at highest level directory.
This structure demonstrates you can use and access sub package, parent package, or same level packages and modules by using a top level directory file main_module.py.
Create and run these files and folders for testing:
package/
|
|----- __init__.py (Empty file)
|------- main_module.py (Contains: import subpackage_1.module_1)
|------- module_0.py (Contains: print('module_0 at parent directory, is imported'))
|
|
|------- subpackage_1/
| |
| |----- __init__.py (Empty file)
| |----- module_1.py (Contains: print('importing other modules from module_1...')
| | import module_0
| | import subpackage_2.module_2
| | import subpackage_1.sub_subpackage_3.module_3)
| |----- photo.png
| |
| |
| |----- sub_subpackage_3/
| |
| |----- __init__.py (Empty file)
| |----- module_3.py (Contains: print('module_3 at sub directory, is imported'))
|
|------- subpackage_2/
| |
| |----- __init__.py (Empty file)
| |----- module_2.py (Contains: print('module_2 at same level directory, is imported'))
Now run main_module.py
the output is
>>>'importing other modules from module_1...'
'module_0 at parent directory, is imported'
'module_2 at same level directory, is imported'
'module_3 at sub directory, is imported'
Opening pictures and files note:
In a package structure if you want to access a photo, use absolute directory from highest level directory.
let's Suppose you are running main_module.py and you want to open photo.png inside module_1.py.
what module_1.py must contain is:
Correct:
image_path = 'subpackage_1/photo.png'
cv2.imread(image_path)
Wrong:
image_path = 'photo.png'
cv2.imread(image_path)
although module_1.py and photo.png are at same directory.
├───root
│ ├───dir_a
│ │ ├───file_a.py
│ │ └───file_xx.py
│ ├───dir_b
│ │ ├───file_b.py
│ │ └───file_yy.py
│ ├───dir_c
│ └───dir_n
You can add the parent directory to PYTHONPATH, in order to achieve that, you can use OS depending path in the "module search path" which is listed in sys.path. So you can easily add the parent directory like following:
# file_b.py
import sys
sys.path.insert(0, '..')
from dir_a.file_a import func_name
This works for me on windows
# some_file.py on mainApp/app2
import sys
sys.path.insert(0, sys.path[0]+'\\app2')
import some_file
In my case I had a class to import. My file looked like this:
# /opt/path/to/code/log_helper.py
class LogHelper:
# stuff here
In my main file I included the code via:
import sys
sys.path.append("/opt/path/to/code/")
from log_helper import LogHelper
I bumped into the same question several times, so I would like to share my solution.
Python Version: 3.X
The following solution is for someone who develops your application in Python version 3.X because Python 2 is not supported since Jan/1/2020.
Project Structure
In python 3, you don't need __init__.py in your project subdirectory due to the Implicit Namespace Packages. See Is init.py not required for packages in Python 3.3+
Project
├── main.py
├── .gitignore
|
├── a
| └── file_a.py
|
└── b
└── file_b.py
Problem Statement
In file_b.py, I would like to import a class A in file_a.py under the folder a.
Solutions
#1 A quick but dirty way
Without installing the package like you are currently developing a new project
Using the try catch to check if the errors. Code example:
import sys
try:
# The insertion index should be 1 because index 0 is this file
sys.path.insert(1, '/absolute/path/to/folder/a') # the type of path is string
# because the system path already have the absolute path to folder a
# so it can recognize file_a.py while searching
from file_a import A
except (ModuleNotFoundError, ImportError) as e:
print("{} fileure".format(type(e)))
else:
print("Import succeeded")
#2 Install your package
Once you installed your application (in this post, the tutorial of installation is not included)
You can simply
try:
from __future__ import absolute_import
# now it can reach class A of file_a.py in folder a
# by relative import
from ..a.file_a import A
except (ModuleNotFoundError, ImportError) as e:
print("{} fileure".format(type(e)))
else:
print("Import succeeded")
Happy coding!
I'm quite special : I use Python with Windows !
I just complete information : for both Windows and Linux, both relative and absolute path work into sys.path (I need relative paths because I use my scripts on the several PCs and under different main directories).
And when using Windows both \ and / can be used as separator for file names and of course you must double \ into Python strings,
some valid examples :
sys.path.append('c:\\tools\\mydir')
sys.path.append('..\\mytools')
sys.path.append('c:/tools/mydir')
sys.path.append('../mytools')
(note : I think that / is more convenient than \, event if it is less 'Windows-native' because it is Linux-compatible and simpler to write and copy to Windows explorer)
If the purpose of loading a module from a specific path is to assist you during the development of a custom module, you can create a symbolic link in the same folder of the test script that points to the root of the custom module. This module reference will take precedence over any other modules installed of the same name for any script run in that folder.
I tested this on Linux but it should work in any modern OS that supports symbolic links.
One advantage to this approach is that you can you can point to a module that's sitting in your own local SVC branch working copy which can greatly simplify the development cycle time and reduce failure modes of managing different versions of the module.
I was working on project a that I wanted users to install via pip install a with the following file list:
.
├── setup.py
├── MANIFEST.in
└── a
├── __init__.py
├── a.py
└── b
├── __init__.py
└── b.py
setup.py
from setuptools import setup
setup (
name='a',
version='0.0.1',
packages=['a'],
package_data={
'a': ['b/*'],
},
)
MANIFEST.in
recursive-include b *.*
a/init.py
from __future__ import absolute_import
from a.a import cats
import a.b
a/a.py
cats = 0
a/b/init.py
from __future__ import absolute_import
from a.b.b import dogs
a/b/b.py
dogs = 1
I installed the module by running the following from the directory with MANIFEST.in:
python setup.py install
Then, from a totally different location on my filesystem /moustache/armwrestle I was able to run:
import a
dir(a)
Which confirmed that a.cats indeed equalled 0 and a.b.dogs indeed equalled 1, as intended.
Instead of just doing an import ..., do this :
from <MySubFolder> import <MyFile>
MyFile is inside the MySubFolder.
In case anyone still looking for a solution. This worked for me.
Python adds the folder containing the script you launch to the PYTHONPATH, so if you run
python application/app2/some_folder/some_file.py
Only the folder application/app2/some_folder is added to the path (not the base dir that you're executing the command in). Instead, run your file as a module and add a __init__.py in your some_folder directory.
python -m application.app2.some_folder.some_file
This will add the base dir to the python path, and then classes will be accessible via a non-relative import.
The code below imports the Python script given by it's path, no matter where it is located, in a Python version-safe way:
def import_module_by_path(path):
name = os.path.splitext(os.path.basename(path))[0]
if sys.version_info[0] == 2:
# Python 2
import imp
return imp.load_source(name, path)
elif sys.version_info[:2] <= (3, 4):
# Python 3, version <= 3.4
from importlib.machinery import SourceFileLoader
return SourceFileLoader(name, path).load_module()
else:
# Python 3, after 3.4
import importlib.util
spec = importlib.util.spec_from_file_location(name, path)
mod = importlib.util.module_from_spec(spec)
spec.loader.exec_module(mod)
return mod
I found this in the codebase of psutils, at line 1042 in psutils.test.__init__.py (most recent commit as of 09.10.2020).
Usage example:
script = "/home/username/Documents/some_script.py"
some_module = import_module_by_path(script)
print(some_module.foo())
Important caveat: The module will be treated as top-level; any relative imports from parent packages in it will fail.
Wow, I did not expect to spend so much time on this. The following worked for me:
OS: Windows 10
Python: v3.10.0
Note: Since I am Python v3.10.0, I am not using __init__.py files, which did not work for me anyway.
application
├── app
│ └── folder
│ └── file.py
└── app2
└── some_folder
└── some_file.py
WY Hsu's 1st solution worked for me. I have reposted it with an absolute file reference for clarity:
import sys
sys.path.insert(1, 'C:\\Users\\<Your Username>\\application')
import app2.some_folder.some_file
some_file.hello_world()
Alternative Solution: However, this also worked for me:
import sys
sys.path.append( '.' )
import app2.some_folder.some_file
some_file.hello_world()
Although, I do not understand why it works. I thought the dot is a reference to the current directory. However, when printing out the paths to the current folder, the current directory is already listed at the top:
for path in sys.path:
print(path)
Hopefully, someone can provide clarity as to why this works in the comments. Nevertheless, I also hope it helps someone.
This problem may be due Pycharm
I had the same problem while using Pycharm. I had this project structure
skylake\
backend\
apps\
example.py
configuration\
settings.py
frontend\
...some_stuff
and code from configuration import settings in example.py raised import error
the problem was that when I opened Pycharm, it considered that skylake is root path and ran this code
sys.path.extend(['D:\\projects\\skylake', 'D:/projects/skylake'])
To fix this I just marked backend directory as source root
And it's fixed my problem
My solution for people. who have all the necessary __init__.py in the package, but import still doesn't work.
import sys
import os
sys.path.insert(0, os.getcwd())
import application.app.folder.file as file
You can use importlib to import modules where you want to import a module from a folder using a string like so:
import importlib
scriptName = 'Snake'
script = importlib.import_module('Scripts\\.%s' % scriptName)
This example has a main.py which is the above code then a folder called Scripts and then you can call whatever you need from this folder by changing the scriptName variable. You can then use script to reference to this module. such as if I have a function called Hello() in the Snake module you can run this function by doing so:
script.Hello()
I have tested this in Python 3.6
I usually create a symlink to the module I want to import. The symlink makes sure Python interpreter can locate the module inside the current directory (the script you are importing the other module into); later on when your work is over, you can remove the symlink. Also, you should ignore symlinks in .gitignore, so that, you wouldn't accidentally commit symlinked modules to your repo. This approach lets you even successfully work with modules that are located parallel to the script you are executing.
ln -s ~/path/to/original/module/my_module ~/symlink/inside/the/destination/directory/my_module
Suppose I have a project in the following structure
projectfoo/
|- mymodule/
|--|- __init__.py
|--|- library.py
|- preprocessor.py
and in the __init__.py in mymodule looks like this
from . import library #library itself has other functions
def some_function():
blar blar blar...
and the preprocessor.py would look like follows
import mymodule
def main():
something()
def something():
mymodule.some_function() # calls the function defined in __init__.py
if __name__ == '__main__':
main()
Then I started projectbar, which is using a lot of common code from projectfoo. So instead of copying and pasting code between projects, I wish to import projectfoo into project bar, as follows.
projectbar/
|- projectfoo/
|--|- mymodule/
|--|--|- __init__.py
|--|--|- library.py
|--|- preprocessor.py
|- index.py
So I am trying to import preprocessor in my index.py as follows
from projectfoo import preprocessor
However I am getting an error saying preprocessor.py is now unable to import mymodule.
ImportError: No module named 'mymodule'
Am I doing this correctly? I am using python3.4 running in ubuntu 14.04 in my setup.
EDIT: I also tried adding __init__.py to projectfoo, but I am still getting the same error
You are getting this error because you did not added the path of preprocessor as a library package
from sys import path as pylib #im naming it as pylib so that we won't get confused between os.path and sys.path
import os
pylib += [os.path.abspath(r'/projectfoo')]
from projectfoo import preprocessor
FYI: os.path will return the absolute path. but sys.path will return the path env. variable in system settings.
Hope it helps.
Try to add empty __init__.py to projectfoo folder.
You have to add an __init__.py file (can be empty) in your projectfoo/ folder to make it a valid module.
Then use relative imports to explicitly specify you're requesting the current module's submodule mymodule like this:
from .projectfoo import preprocessor
The . stands for the current module in which the file containing the import statement is located. Its parent module would be denoted as .., its "grandparent" would be ... etc.
I have several python modules in the project and I put them in different folders, for example,
pythonProject\folderA\modulex.py
pythonProject\folderB\moduley.py
pythonProject\commonModule\module1.py
I have __init__.py in each folder.
In this situation, how can I import module1 into modulex?
Use relatively import
# in modulex
from ..commonModule import module1
Whenever you have python packages (those folders that contain __init__.py files), you can import the modules like below
modulex.py
----------
from pythonproject.commonModule import module1
Try this, If the pythonproject is not defined by the tool, then you could use the relative addressing like below
from ..commonModule import module1
The best if all modules are in the same directory. In case any of them in different possible use of os.chdir(path). With os.chdir(path) method (https://docs.python.org/3.2/library/os.html) possible change working directory in your program.
import os
import modulex
#assume working directory is "pythonProject\folderA\"
os.chdir(r'pythonProject\commonModule\')
#now working directory is "pythonProject\commonModule\"
import module1
I have a module with __init__.py located in a specific path. Then I:
import sys
sys.path.append('C:/Users/ME/Dropbox/Python/Library/ModuleDir')
But when I from ModuleDir.x import classA, it doesn't recognize the statement meaning that it wasn't able to find the module. What am I missing?
Under ModuleDir:
ModuleDir
__init__.py
X.py
Subdir
__init__.py
y.py
You need to add the directory containing the package subdir to your path:
sys.path.append('C:/Users/ME/Dropbox/Python/Library')
See docs.
Try sys.path.insert()
import sys
sys.path.insert(1, 'C:/Users/ME/Dropbox/Python/Library/')
from ModuleDir.x import classA