This question already has answers here:
Import a module from a relative path
(22 answers)
Closed 3 years ago.
I'm trying to make a relative import in python. but I can't understand the syntax and everytime I search for it here in SO, I can't get an answer:
Here is my folder structure:
Root
libraries
mylibrary
__init__
projects
project
myproject.py
and I want to import 'mylibrary' using a relative path, what's the syntax for doing it?
You have to add the directory to your python path.
import sys
sys.path.append("/libraries")
But I think it might be better to either put the libraries in the folders of the projects that need it or just install them to one of the standard places already in sys.path.
I don't think it can be done with a simple import statement. What I would do is to append the relative path to your library folder to sys.path like this:
import sys
sys.path.append('../../')
from libraries import mylibrary
Note that this works only if you start the python interpreter from the projects/project directory.
There is an unfortunate source of confusion with relative imports. When you first learn about them, you think that they allow you to use generally relative file/directory paths to refer to individual files that will be imported. (Or at least, I thought so.) In fact, they only allow you to use relative paths within a package. This means that certain modules within a package can use relative-import syntax when they need to import other modules from within the same package.
In your example, myproject.py is not in the same package as mylibrary, and in fact is not in any package, so there is no way to use relative imports from inside myproject.py . Relative imports just don't apply in that situation.
There are a couple things you can do to get the effect you want. One is to put your libraries in subdirectories of the system site-packages directory. Another is to put .PTH files in the system site-package directory, with those .PTH files containing the paths to the places where your libraries are stored. Another is to use PYTHONPATH to point to directories where you store your libraries.
Related
I wanted to ask if there is a way to import modules/functions from a folder in a project that is adjacent to another folder.
For example lets say I have two files:
project/src/training.py
project/lib/functions.py
Now both these folders have the __init__.py file in them. If I wanted to import functions.py into training.py, it doesn't seem to detect. I'm trying to use from lib.functions import * .I know this works from the upper level of the folder structure, where I can call both files from a script, but is there a way to do it files in above/sideways folders?
Fundamentally, the best way of doing this depends on exactly how the two modules are related. There's two possibilities:
The modules are part of one cohesive unit that is intended to be used as a single whole, rather than as separate pieces. (This is the most common situation, and it should be your default if you're not sure.)
The functions.py module is a dependency for training.py but is intended to be used separately.
Cohesive unit
If the modules are one cohesive unit, this is not the standard way of structuring a project in Python.
If you need multiple modules in the same project, the standard way of structuring the folders is to include all the modules in a single package, like so:
project/
trainingproject/
__init__.py
training.py
functions.py
other/
...
project/
...
folders/
...
The __init__.py file causes Python to recognize the trainproject/ directory as a single unit called a package. Using a package enables to use of relative imports:
training.py
from . import functions
# The rest of training.py code
Assuming your current directory is project, you can then invoke training.py as a module:
python -m trainingproject.training
Separate units
If your modules are actually separate packages, then the simplest idiomatic solutions during development is to modify the PYTHONPATH environment variable:
sh-derviative syntax:
# All the extra PYTHONPATH references on the right are to append if it already has values
# Using $PWD ensures that the path in the environment variable is absolute.
PYTHONPATH=$PYTHONPATH${PYTHONPATH:+:}$PWD/lib/
python ./src/training.py
PowerShell syntax:
$env:PYTHONPATH = $(if($env:PYTHONPATH) {$env:PYTHONPATH + ';'}) + (Resolve-Path ./lib)
python ./src/training.py
(This is possible in Command Prompt, too, but I'm omitting that since PowerShell is preferred.)
In your module, you would just do a normal import statement:
training.py
import functions
# Rest of training.py code
Doing this will work when you deploy your code to production as well if you copy all the files over and set up the correct paths, but you might want to consider putting functions.py in a wheel and then installing it with pip. That will eliminate the need to set up PYTHONPATH by installing functions.py to site-packages, which will make the import statement just work out of the box. That will also make it easier to distribute functions.py for use with other scripts independent of training.py. I'm not going to cover how to create a wheel here since that is beyond the scope of this question, but here's an introduction.
Yes, it’s as simple as writing the entire path from the working directory:
from project.src.training import *
Or
from project.lib.functions import *
I agree with what polymath stated above. If you were also wondering how to run these specific scripts or functions once they are imported, use: your_function_name(parameters), and to run a script that you have imported from the same directory, etc, use: exec(‘script_name.py). I would recommend making functions instead of using the exec command however, because it can be a bit hard to use correctly.
I have multiple python projects which should use a number of shared files but I can not figure out how to do this in python.
If I just copy the file into the pyhton working directory it works fine with:
from HBonds import calculateHBondsForSeveralReplicas, compareSameShapeHBMatrices, calculateHBonds
But I do not want to copy it. I want to include it from: /home/b/data/pythonWorkspace/util/HBonds
For me it would make sense to do it like this (but it does not work):
from /home/b/data/pythonWorkspace/util/HBonds/HBonds.py import calculateHBondsForSeveralReplicas, compareSameShapeHBMatrices, calculateHBonds
How can I do this?
You have to make sure the PYTHONPATH includes the path to that directory as it was pointed out in previous answer.
Or you can use a nastier way: make it available at runtime with piece of code like this.
import os
import sys
folder = os.path.dirname('/home/b/data/pythonWorkspace/util/')
if dossier not in sys.path:
sys.path.append(folder)
from HBonds import HBonds
For 3rd-party libraries, it's best to install them the stock way - either to the system's site-packages or into a virtualenv.
For project(s) you're actively developing at the machine where it's running, a maintainable solution is to add their root directory(ies) to PYTHONPATH so that you can import <top_level_module>.<submodule>.<etc> from anywhere. That's what we did at my previous occupation. The main plus here is trivial code base update and switch.
Another way is to use relative imports, but it's intended for intra-package references, so that you don't have to reiterate the package's name everywhere. If many otherwise unrelated parts of the code use the same module(s), it's probably more convenient to make the shared part a separate package which would be a dependency for all of them.
Hey I'm working on a project that has a set hierarchical modules with a folder structure set up like so:
module_name/
__init__.py
constants.py
utils.py
class_that_all_submodules_need_access_to.py
sub_module_a/
__init__.py
sub_module_a_class_a.py
sub_module_a_class_b.py
useful_for_sub_module_a/
__init__.py
useful_class_a.py
useful_class_b.py
sub_module_b/
__init__.py
sub_module_b_class_a.py
sub_module_b_class_b.py
etc etc etc...
The problem is, I can't figure out how to set up the imports in the init.py's so that I can access class_that_all_submodules_need_access_to.py from sub_module_a/useful_for_sub_module_a/useful_class_a.py.
I've tried looking this up on Google/StackOverflow to exhaustion and I've come up short. The peculiar thing is that PyCharm has the paths set up in such a way that I don't encounter this bug when working on the project in PyCharm, but only from other environments.
So here's one particularly inelegant solution that I've come up with. My sub_module_a/useful_for_sub_module_a/init.py looks like:
import sys
import os
sys.path.insert(0, os.path.abspath(os.path.join(os.path.dirname(__file__), '..', '..', '..')))
import module_name
This is similar in sub_module_*/, where instead of 3 '..'s it's just two (i.e. '..', '..' instead of '..', '..', '..' for the sys.path.insert line above). And then in sub_module_a/useful_for_sub_module_a/useful_class_a.py, I have to import module_name/constants.py (and others) like this:
import module_name.constants
from module_name.class_that_all_submodules_need_access_to import ImportantClass
While this solution works, I was wondering if there is a better/more elegant way to set up the imports and/or folder hierarchy? I'm concerned about messing with the python system path for users of this module. Is that even a valid concern?
There are two kinds of Python import: absolute and relative, see Python import. But the first import thing is that you need to understand how python finds your package? If you just put your package under your home folder, then Python knows nothing about it. You can check this blog [Python search] (https://leemendelowitz.github.io/blog/how-does-python-find-packages.html).
Thus to import modules, the first thing is that let Python know your package. After knowing the locations where Python would search packages, there are often two ways to accomplish this goal:
(1) PYTHONPATH environment variable. Set this variable in your environment configuration file, e.g., .bash_profile. This is also the simplest way.
(2) Use setuptools, which could help you distribute your package. This is a long story. I would not suggest you choose it, unless you would like to distribute your package in the future. However, it worth to know it.
If you set your path correctly, the Python import is just straight forward. If you would like use the absolute import, try
from module_name import class_that_all_submodules_need_access_to
If you would like to use the relative import, it depends on which module you are now. Suppose you are writing the module module_name.sub_module_a.sub_moduel_a_class_a, then try
from .class_that_all_submodules_need_access_to import XX_you_want
Note that relative import supports from .xx import xx format only.
Thanks.
I'm trying to import a module I wrote in python that just prints out a list containing numbers. The issue I'm having is that I want to be able to import it from a separate directory but the answers I have read so far don't seem to be working for my situation.
For example, given I want to import printnumbers.py from a directory in my documents folder I am supposed to implement the following:
import sys
sys.path.append('/home/jake/Documents')
import printnumbers.py
This snipit of code results in a "Import error" telling me that the specified module does not exist. I'm not exactly sure where to proceed from here, I have checked multiple times to make sure it's the right spelling for the path as well as for the module name. I'm still trying to understand exactly what appending to the "sys.path" does. From what I understand it's telling the program to search for modules in that directory?
Thanks for anyone who answers my rather novice question. I'm just looking for a better understanding of it that the python documentation isn't providing for my frame of mind.
When the file is printnumbers.py, the module is called printnumbers (without the .py). Therefore use
import printnumbers
import sys
sys.path.append('/home/jake/Documents')
appends '/home/jake/Documents' to the end of sys.path. The directories listed in sys.path are searched (in the order listed) whenever an import statement causes Python to search for a module. (Already imported modules are cached in sys.modules, so Python does not always need to search sys.path directories to import a module...)
So if you have a file /home/jake/Documents/printnumbers.py, then import printnumbers will cause Python to import it provided there is no other file named printnumbers.py in a directory listed in sys.path ahead of /home/jake/Documents/.
Note that injecting directories into sys.path is not the usual way to set up Python to search for modules. Usually it is preferable to add /home/jake/Documents to your PYTHONPATH environment variable. sys.path will automatically include the directories listed in the PYTHONPATH environment variable.
and one more thing, use an empty __ init __.py file in you directory
to make it as a python package (only then Python will know that this
directory is a Python package directory other than an ordinary
directory). Thus you can import modules from that package from
different directory.
I am creating a Python package with multiple modules. I want to make sure that when I import modules within the package that they are importing only from the package and not something outside the package that has the same name.
Is the correct way of doing this is to use relative imports? Will this interfere when I move my package to a different location on my machine (or gets installed wherever on a customer's machine)?
Modern relative imports (here's a reference) are package-relative and package-specific, so as long as the internal structure of your package does not change you can move the package as a whole around wherever you want.
While Joran Beasley's answer should work as well (though does not seem necessary in those older versions of Python where absolute imports aren't the default, as the old style of importing checked within the package's directory first), I personally don't really like modifying the import path like that when you don't have to, especially if you need to load some of those other packages or modules that your modules or packages now shadow.
A warning, however: these do require that the module in question is loaded as part of a package, or at least have their __name__ set to indicate a location in a package. Relative imports won't work for a module when __name__ == '__main__', so if you're writing a simple/casual script that utilizes another module in the same directory as it (and want to make sure the script will refer to the proper directory, things won't work right if the current working directory is not set to the script's), you could do something like import os, sys; sys.path.insert(0, os.path.dirname(os.path.abspath(__file__))) (with thanks to https://stackoverflow.com/a/1432949/138772 for the idea). As noted in S.Lott's answer to the same question, this probably isn't something you'd want to do professionally or as part of a team project, but for something personal where you're just doing some menial task automation or the like it should be fine.
the sys.path tells python where to look for imports
add
import sys
sys.path.insert(0,".")
to the top of your main python script this will ensure local packages are imported BEFORE builtin packages (although tbh I think this happens automagically)
if you really want to import only packages in your folder do
import sys
sys.path = ["."]
however I do not recommend this at all as it will probably break lots of your stuff ...
most IDE's (eclipse/pycharm/etc) provide mechanisms to set up the environment a project uses including its paths
really the best option is not to name packages the same as builtin packages or 3rd party modules that are installed on your system
also the best option is to distribute it via a correctly bundled package, this should more than suffice