I am trying to import python files which have been developed. Initially, when all files were in the same directory then I there was no problems with them. However, I decided to split projects to dedicated directories. After that, I am unable to import files anymore.
My file structure is similar to the following:
dirFoo\
mainFoo1.py
dirFoo\
mainFoo2.py
dirFooCommon\
commonFoo.py
commonFoo2.py
Initially, I was trying to change the import path of my mainFoo1.py and do:
from dirFooCommon import commonFoo. However, that approach gave me the following error:
ModuleNotFoundError: No module named 'common'.
Apart from that, I tried to use imp.load_source which seems that is working. Unfortunately, this is generating so additional problems. For example what if some of the commonFoo.py libraries must include others? In this case, each of them needs to use an absolute project path, which will be a problem when I will try to use common libraries in any other place.
Do you have an idea what should I do to include my common libraries to my main projects?
You can use this folder structure that will allow you to call each package and module properly.
dirFoo\ <=== This is a package know
__init__.py
mainFoo1.py
dirFoo2\ <==== Change the name or you will have namespace issue
__init__.py
mainFoo2.py
dirFooCommon\ <=== This is a package know
__init__.py
commonFoo.py
commonFoo2.py
So in mainFoo1.pyyou can call commonFoo.pylike this
import sys
sys.path.append("/path/to/dirFooCommon")
from dirFooCommon import commonFoo
To replace sys.path.appendyou can also add the folder in your PYTHONPATH.
Related
This question already has answers here:
Relative imports for the billionth time
(12 answers)
Closed last year.
So basicly I want to acces a created module from a folder on the parent directory from the folder I am
Currently I'm at Twitter.py and I want to access /utils/magic_eden.py
On the __init__.py file I've got:
from .magic_eden import MagicEden
from .open_sea import OpenSea
from .tools import Tools
Now inside the Twitter.py file im trying to import these classes of the module by doing:
from utils import MagicEden
But im getting ModuleNotFoundError: No module named 'utils'.
I've tried so many sort of things but none worked. What should I do?
(btw if I execute the __init__.py file I get ImportError: attempted relative import with no known parent package)
From what I see, it seems that utils is a utility package logically separate from the code in Twitter.py. Given this, what you want is to cause your utils package to be on your Python search path (sys.path) so that you can import that package as a separate entity (no relative paths). If you don't want to configure your environment to place utils on the Python search path, you can do it in your Twitter.py file. Here's how that looks:
import os
import sys
here = os.path.dirname(__file__)
sys.path.append(os.path.join(here, '..'))
import utils
utils.MagicEden.hello()
utils.OpenSea.hello()
utils.Tools.hello()
The first line of code computes the full path to the directory containing Twitter.py. The second line then computes the path to the parent directory of that directory and adds that path to sys.path, the Python search path. After doing this, import utils will work, giving you access to that package and everything imported in that package's __init__.py file.
I created three small files for magic_eden.py, open_sea.py, and tools.py, each containing something that looks like this:
class MagicEden:
#staticmethod
def hello():
print("Hello from MagicEden!")
I can then run Twitter.py, and I get the following result with no additional configuration:
Hello from MagicEden!
Hello from OpenSea!
Hello from Tools!
There's nothing wrong with using the above solution during early development. But you will likely at some point want to remove the code that is hacking sys.path and instead install your module in a more official way. There's a way to do this from the start so that you never have to change code on either side when you want to install your module in the official way...
What I do in situations like this is create a setup.py file in my package that lets me build it as an installable package. There are many docs and tutorials on the net that explain how to do this. Here's just one of them: https://python-packaging-tutorial.readthedocs.io/en/latest/setup_py.html
Once you've done this, what you can do is install your package in "development mode" (pip install -e <package file>). What this does is install the package so that your system can find it (adds it to sys.path as a package), but installs links to the original sources rather than installing copies of them. In this mode, when you make changes to that package's source files, the changes take immediate effect in the installed module. So now you have the best of both worlds. Your package is installed in the official way such that you don't need to do anything special in the code that imports that package (no need to modify sys.path), and yet you can still make changes directly to the sources for that package and not have to keep reinstalling the package to see those changes take affect.
When you're done messing with a package in this way, you can reinstall it in the regular mode (pip install <package file>) and isolate changes to the sources for that package from uses of the package elsewhere on your system. At this point, you need to rebuild and reinstall the package to see any changes you've made to its sources.
You're attempting to import from a "sibling" folder.
To do this, you'll need to add __init__.py to your Twitter/ and parent src/ folders.
Then you'll need to import the path as Twitter.py doesn't know about any parent structure in this setup. You can then import from the utils module which is in the path.
import sys
sys.path.append('..')
from utils.magic_eden import MagicEden
assuming MagicEden is a class in your magic_eden.py file.
I am working on a python project which consist of several packages and modules. Hence I need to import modules from different packages. However, when importing these modules I am getting import errors. The folder structure is as follows:
Image of folder structure
Within the module Stage0.py I use the relative import: "from ..data.Datapipe import DataFactory" to import the class DataFactory. However when I execute the script I get an error message: "ModuleNotFoundError: No module named 'MT'"
I would appreciate any feedback as I'm becoming desperate
I'm not 100% sure I've understood this correctly, but by the looks of it when you run the Stage0.py it will set the root python path to the app directory, the problem is that python's use of relative imports only allows to find files within that root directory, this site explains it a bit better to be honest
See here
Note that for relative imports, the dots . can go up only up to (but not including) the directory containing the script run from the command line
Anyway, it to fix it you could create a __main__.py file within the mt folder which calls the Stage0.py script, this will set that mt folder as the root of the project giving access to the other directories.
As a simple solution, I recommend that you put Stage0.py in the same location as the requirements.txt and execute it from there
as follows:
-data/
-__init__.py
-Datapipe.py
-graph/
-__init__.py
-s0.py
-stage0.py
stage0.py
from data.Datapipe import DataFactory
I'm trying to import a class in a different directory to another file, but can't seem to get it to work. I know this question has been asked a lot and I have looked through multiple stackoverflow solutions and at https://docs.python.org/3/tutorial/modules.html#packages
1: Importing files from different folder
2: import python file in another directory failed
I want to try to just use the method containing just __init__.py file instead of doing an import sys
My directory structure is as follows:
django_vue/
__init__.py
devices/
__init__.py
models.py
lib/
__init__.py
my_file.py
I'm trying to import the class Device from /django_vue/devices/models.py to /django_vue/lib/my_file.py by:
from devices.models import Device
However when I do that I still get the error:
from devices.models import Device
ModuleNotFoundError: No module named 'devices'
I'm not sure what I'm dong wrong since I already have the __init__ file in both directories. Any help is appreciated. Also I'm running python 3.6.
This is the folder structure I'm working with.
.
└── django_vue
├── devices
│ └── models.py
└── lib
└── file.py
When you run
$ python file.py
python has no way of knowing what's outside the directory.
python can't go back and then into devices/ just like that.
The easiest way to solve this would be to add the folder devices/ to sys.path. When python imports a module, it searches for the module from sys.path. Adding the path to devices/ would make it available for imports.
Here are my files.
# models.py
Device = 'device'
# file.py
import sys
sys.path.append('..') # adds the parent dir (which is django-vue/) to path
# django-vue dir has devices/ so now this is available for imports
# importing this works now
from devices.models import Device
print(Device)
Output
django_vue/lib$ python3 file.py
device
Think about it your are inside my_file.py and import something called devices.
How can python know where the name devices has come from.
It won't search your entire Drive for that module/package
Relative Import
use a relative import instead. write from ..devices.models import Device. This is telling python to go up one directory to the parent directory and that's where it will find the devices package. Your lib module should now work as a module
If you however run the my_file.py package directly (as in python C:/django_vue/lib/my_file.py)
You will still get an error. Not the same error; the new error will be something like
ImportError: attempted relative import with no known parent package
This is happening because you are actually running my_file.py
If you think about it why would you want to run my_file.py by itself when it is clearly a part of a package. Maybe you are just testing to see if the import works when you use your package. The problem with this is that it makes it seem like your packages relative imports don't work even though this actually works.
Create a main.py in django_vue and write from lib import my_file. This will run your my_file.py and you will notice there is no error.
What's happening here
Have you heard of __package__?
if you put print(str(__package__)) in your my_file.py and run my_file.py directly you will see that it prints None.
However if you run main.py (that you just created) you will see that when It reaches my_file.py, __package__ will actually be defined to something.
Ahhh... you see now it all makes sense; The error you originally got said something about no known parent package. If __package__ is undefined that means there is no relative parent package because the file was obviously run directly instead of as part of a package.
Consider Absolute imports
you also might want to consider using absolute imports because if you are working on the package you might change it directory structure while developing. so you need to keep changing the import references on the affected files.
Although you can find IDE's with python extensions that automatically to this as you change your directory. I believe VS Code does this automatically.
Replace the __init__ files with __main__.
I have trouble importing package.
My file structure is like this:
filelib/
__init__.py
converters/
__init__.py
cmp2locus.py
modelmaker/
__init__.py
command_file.py
In module command_file.py I have a class named CommandFile which i want to call in the cmp2locus.py module.
I have tried the following in cmp2locus.py module:
import filelib.modelmaker.command_file
import modelmaker.command_file
from filelib.modelmaker.command_file import CommandFile
All these options return ImportError: No modules named ...
Appreciate any hint on solving this. I do not understand why this import does not work.
To perform these imports you have 3 options, I'll list them in the order I'd prefer. (For all of these options I will be assuming python 3)
Relative imports
Your file structure looks like a proper package file structure so this should work however anyone else trying this option should note that it requires you to be in a package; this won't work for some random script.
You'll also need to run the script doing the importing from outside the package, for example by importing it and running it from there rather than just running the cmp2locus.py script directly
Then you'll need to change your imports to be relative by using ..
So:
import filelib.modelmaker.command_file
becomes
from ..modelmaker import command_file
The .. refers to the parent folder (like the hidden file in file systems).
Also note you have to use the from import syntax because names starting with .. aren't valid identifiers in python. However you can of course import it as whatever you'd like using from import as.
See also the PEP
Absolute imports
If you place your package in site-packages (the directories returned by site.getsitepackages()) you will be able to use the format of imports that you were trying to use in the question. Note that this requires any users of your package to install it there too so this isn't ideal (although they probably would, relying on it is bad).
Modifying the python path
As Meera answered you can also directly modify the python path by using sys.
I dislike this option personally as it feels very 'hacky' but I've been told it can be useful as it gives you precise control of what you can import.
To import from another folder, you have to append that path of the folder to sys.path:
import sys
sys.path.append('path/filelib/modelmaker')
import command_file
So I am working on a Python project that was here before me in an SVN repo. When I first pulled it, the structure was a bit odd due to the fact that it was similar to:
Proj\
src\
tags\
trunk\
And then everything is inside src\ are the python module files except src\ turns out to just be a logical folder with no overall package inside. There isn't a __init__.py in the project anywhere. So I want to restructure it at least so I can use relative imports through my project. I also want to set it up so it looks more like this.
Proj\
src\
model\
controller\
view\
test\
tags\
trunk\
However, I tried setting this up and no matter what I seem to do, it cannot resolve the relative import the moment I have to traverse packages. I placed a __init__.py file in each level package including one inside the src\ folder with all of them having __all__ defined. However, when I try to make a unit test in my test\ package and do an import saying:
from ..model.foo import Foo
to attempt to import the Foo class from module foo.py located inside of the model package, it doesn't resolve. Just in case it was a problem specifically with unit tests, I also tried this with a module in the controller package that was dependent on a class in the model package and vice versa. None of them worked. How do I resolve this?
Have you added the root folder to your system path?
import sys
sys.path.append(<place the Proj dir here>)
then you could import as follows:
from src.model.somefile import Something
If you don't know the absolute path for Proj, you can always use combinations such as
os.path.dirname(os.getcwd())