Develop Reference

Import file from subdirectory into file in another subdirectory

I have a python project a folder structure like this:
main_directory
main.py
drivers
__init__.py
xyz.py
utils
__init__.py
connect.py
I want to import connect.py into xyz.py and here's my code:
from utils import connect as dc
But I keep getting this error no matter what I do, please help:
ModuleNotFoundError: No module named 'utils'
Update: People are telling me to set the path or directory, I don't understand why I need to do this only for importing file. This is something that should work automatically.

In your utils folder __init__.py should be blank. If this doesn't work, try adding from __future__ import absolute_import in your xyz.py file.

Check your current directory. It must be main_directory.
import os
print("Current working directory is: ", os.getcwd()
if not, you can change using
os.chdir("path/to/main_directory")
also works with relative path
os.chdir('..')

I was also facing same problem when you use row python script so i use the following code at the beginning of the file
import os
import sys
current_dir = os.path.dirname(os.path.realpath(__file__))
parent_dir = os.path.dirname(current_dir)
sys.path.append(parent_dir)
This way it will search the require file at parent directory.
Hope this will work for you also..

You could move your utils folder into drivers, making the path to the to-be imported file a subdirectory of your executing file, like:
main_directory/drivers/utils/connect.py
Alternatively, you could try
from ..utils import connect as dc
This will move up a directory before import.
Lasty, you could add the directory to Path in your script via
import sys
sys.path.insert(0,'/path/to/mod_directory')
For this method, see also this question

sys.path.append which main.py will be imported

If I have a project with two files main.py and main2.py. In main2.py I do import sys; sys.path.append(path), and I do import main, which main.py will be imported? The one in my project or the one in path (the question poses itself only if there are two main.py obviously)?
Can I force Python to import a specific one?

The one that is in your project.
According to python docs, the import order is:
Built-in python modules. You can see the list in the variable sys.modules
The sys.path entries
The installation-dependent default locations
Your added directory to sys.path, so it depends on the order of entries in sys.path.
You can check the order of imports by running this code:
import sys
print(sys.path)
As you will observe, the first entry of sys.path is your module's directory ("project directory") and the last is the one you appended.
You can force python to use the outer directory by adding it in the beginning of your sys.path:
sys.path.insert(0,path)
Although I would recommend against having same names for the modules as it is often hard to manage.
Another thing to keep in mind is that you can't import relative paths to your sys.path. Check this answer for more information on how to work around that: https://stackoverflow.com/a/35259170/6599912

When two paths contain the same module name, using sys.path.append(path) isn't going to change the priority of the imports, since append puts path at the end of the python path list
I'd use
sys.path.insert(0,path)
so modules from path come first
Example: my module foo.py imports main and there's one main.py in the same directory and one main.py in the sub directory
foo.py
main.py
sub/main.py
in foo.py if I do:
import sys
sys.path.append("sub")
import main
print(main.__file__)
I get the main.py file path of the current directory because sys.path always starts by the main script directory.
So I could remove that directory from sys.path but it's a bad idea because I could need other modules from there.
Now if I use insert instead:
import sys
sys.path.insert(0,"sub")
import main
print(main.__file__)
I get sub/main.py as sub is the first directory python looks into when searching modules.

How to change jupyter notebooks working directory configuration

Id like the working directory of all notebooks to be the location where I run the command from, not the location the notebook is at.
As an example:
root_folder/
notebooks/
subject_A/
notebook.ipynb
src/
module.py
# notebook.ipynb
from src import module
I want to run jupyter notebook (or lab) from root_folder and import modules from the working directory. I dont consider os.chdir a good solution because moving the notebook to another folder will break the import. I'd also rather not add absolute paths inside the notebooks, this solution would not be exportable: my absolute path is not the same as yours.
I know I could add a setup.py to make src installable, but was looking for a more direct approach (I want to change the working directory, not install packages).
I also want an approach that is independent of the notebook location, it is the same regardless if it is at notebooks/A or in notebook/A/B.
Is there a way to do this?

I would try with sys and then import module. It does not change your cwd but you will be able to import your modules in src
import sys
sys.path.append("absolute/path/to/src")
from module import *

Find root_folder like this:
import os
# Get current working directory, in your case cwd is ../root_folder/notebooks/subject_A
cwd = os.getcwd()
root_folder = os.path.dirname(os.path.dirname(cwd))
After finding root_folder:
path_to_src = os.path.join(root_folder, 'src')
And finally, append path_to_src to sys.path :
import sys
sys.path.append(path_to_src)
Now Python can import any modules from src. If the root_folder is moved to other system or directory the above-mentioned procedure will be able to import any modules from src.

How to import Python file?

Sorry, this is definitely a duplicate, but I can't find the answer. I'm working in Python 3 and this is the structure of my app:
/home
common.py
australia/
new-south-wales/
fetch.py
I am in the home/ directory, running fetch.py. How can I import functions from common.py in that script?
I've set up fetch.py as follows:
from common import writeFile
But I get the following error:
File "australia/new-south-wales/fetch.py", line 8, in <module>
from common import writeFile
ModuleNotFoundError: No module named 'common'
If I just do python -c "from common import writeFile" I don't see an error.
Shouldn't the interpreter look in the current directory for modules?

before import your directories that need to be imported must have file __init__.py in that folder
#solution 1 (import in runtime)
To import a specific Python file at 'runtime' with a known name:
import os
import sys
script_dir = "/path/to/your/code/directory"
# Add the absolute directory path containing your
# module to the Python path
sys.path.append(os.path.abspath(script_dir))
import filename
#solution 2(add files to one of python libraries)
also as you have a common library for you can run
>>> import sys
>>> print sys.path
and see what directories you can put your code and use in every project.you can move your common package to one of this directories and treat it like a normal package.for example for common.py if you put it in one root directory of one of this directory you can import like import common
#solution 3(use relative import)
# from two parent above current directory import common
# every dot for one parent directory
from ... import common
and then go to parent directory and run
python -m home.australia.new-south-wales.fetch

From the description I'm assuming you're not running this as complete python package, just as separate files.
What you can do is use complete modules. This means adding empty __init__.py to directories with your code. You'll also have to change the name of new-south-wales to new_south_wales, since it needs to be a valid module name.
Assuming home is the name of your app, you should end up with:
home/
__init__.py
common.py
australia/
__init__.py
new_south_wales/
__init__.py
fetch.py
Next, you'll need a startup script for your app - this means either something simple like:
#!/usr/bin/env python
from australia.new_south_wales import fetch
fetch.your_main_function()
Or you can add a setup.py with a full package description. If you specify entry points and the script will be automatically created.
Now that you're starting your code in context of a package, your fetch.py can do:
from ..common import writeFile

Using relative imports in python with mutliple parent and children folders

I had a small problem with importing a script that was in a parent folder, but I managed to resolve it using:
import sys
sys.path.append("../")
My directory is like this:
Data
|->->code
|->->script1.py
|->->->->subfolder
|->->->script2.py
When I run script2 (which imports script1) from the subfolder directory, the script runs without problems. But if I try to run script2 from the code directory using:
:~ ./subfolder/script2.py
I get an error :
ImportError: No module named script1
I tried using relative imports but because my code is not structured in packages it doesn't work. Is there a way I can run script2 from both directories (the parent and the child) and still be able to import script1 everytime?
Thank you in advance,
Georgi Nikolov
EDIT: Ok, after I read through all the suggestions, I did a "simple" hack which is quite ugly in my opinion but works quite well:
import sys
parent_folder = sys.path[0].split("/subfolder")[0]
sys.path.append(parent_folder)
import script1
Now I can even call script2 from the root and it will manage to import script1

To import a module that is up a level, you can use this.
import os, sys
sys.path.append(os.path.join(os.path.dirname(__file__), ".."))
An Explanation
__file__ # The full path to your running file.
os.path.dirname # See below. (1)
os.path.join # See below. (2)
sys.path.append # See below. (3)
".." # Universal for "up a level".
os.path.dirname(path) -
Return the directory name of pathname path.
os.path.join(path, *paths) -
Join one or more path components intelligently.
sys.path -
A list of strings that specifies the search path for modules.
You can add a string with the append method.

You should use absolute paths:
import sys, os
HERE = os.path.abspath(os.path.dirname(__file__))
sys.path.append(os.path.join(HERE, ".."))
But better do not do this: Guido views running scripts within a package as an anti-pattern
You should put the standalone scripts in the root folder of the project. A script should not be used as a module and as the main script.
If you cannot move the script to the root, make another bootstrap script at the root which will import your service script running a dedicated function in it.