This question already has answers here:
Relative imports for the billionth time
(12 answers)
Closed last year.
So basicly I want to acces a created module from a folder on the parent directory from the folder I am
Currently I'm at Twitter.py and I want to access /utils/magic_eden.py
On the __init__.py file I've got:
from .magic_eden import MagicEden
from .open_sea import OpenSea
from .tools import Tools
Now inside the Twitter.py file im trying to import these classes of the module by doing:
from utils import MagicEden
But im getting ModuleNotFoundError: No module named 'utils'.
I've tried so many sort of things but none worked. What should I do?
(btw if I execute the __init__.py file I get ImportError: attempted relative import with no known parent package)
From what I see, it seems that utils is a utility package logically separate from the code in Twitter.py. Given this, what you want is to cause your utils package to be on your Python search path (sys.path) so that you can import that package as a separate entity (no relative paths). If you don't want to configure your environment to place utils on the Python search path, you can do it in your Twitter.py file. Here's how that looks:
import os
import sys
here = os.path.dirname(__file__)
sys.path.append(os.path.join(here, '..'))
import utils
utils.MagicEden.hello()
utils.OpenSea.hello()
utils.Tools.hello()
The first line of code computes the full path to the directory containing Twitter.py. The second line then computes the path to the parent directory of that directory and adds that path to sys.path, the Python search path. After doing this, import utils will work, giving you access to that package and everything imported in that package's __init__.py file.
I created three small files for magic_eden.py, open_sea.py, and tools.py, each containing something that looks like this:
class MagicEden:
#staticmethod
def hello():
print("Hello from MagicEden!")
I can then run Twitter.py, and I get the following result with no additional configuration:
Hello from MagicEden!
Hello from OpenSea!
Hello from Tools!
There's nothing wrong with using the above solution during early development. But you will likely at some point want to remove the code that is hacking sys.path and instead install your module in a more official way. There's a way to do this from the start so that you never have to change code on either side when you want to install your module in the official way...
What I do in situations like this is create a setup.py file in my package that lets me build it as an installable package. There are many docs and tutorials on the net that explain how to do this. Here's just one of them: https://python-packaging-tutorial.readthedocs.io/en/latest/setup_py.html
Once you've done this, what you can do is install your package in "development mode" (pip install -e <package file>). What this does is install the package so that your system can find it (adds it to sys.path as a package), but installs links to the original sources rather than installing copies of them. In this mode, when you make changes to that package's source files, the changes take immediate effect in the installed module. So now you have the best of both worlds. Your package is installed in the official way such that you don't need to do anything special in the code that imports that package (no need to modify sys.path), and yet you can still make changes directly to the sources for that package and not have to keep reinstalling the package to see those changes take affect.
When you're done messing with a package in this way, you can reinstall it in the regular mode (pip install <package file>) and isolate changes to the sources for that package from uses of the package elsewhere on your system. At this point, you need to rebuild and reinstall the package to see any changes you've made to its sources.
You're attempting to import from a "sibling" folder.
To do this, you'll need to add __init__.py to your Twitter/ and parent src/ folders.
Then you'll need to import the path as Twitter.py doesn't know about any parent structure in this setup. You can then import from the utils module which is in the path.
import sys
sys.path.append('..')
from utils.magic_eden import MagicEden
assuming MagicEden is a class in your magic_eden.py file.
Related
I have three scripts:
C:\code\voiceTerm\master.py:
from voice_terminal_module.voice_terminal import VoiceTerminal
vterm = VoiceTerminal()
C:\code\voiceTerm\voice_terminal_module\voice_terminal.py:
from chatbot_module.chatbot_module import Chatbot
class VoiceTerminal:
print("INITIALIZING VOICE TERMINAL")
cb = Chatbot()
C:\code\voiceTerm\voice_terminal_module\chatbot_module\chatbot_module.py:
class Chatbot:
print("CHATBOT INITIALIZED")
Here is the wierd thing: When I run chatbot_module.py it works, and if I run voice_terminal.py it works. For some reason however, master.py errors out with the following message:
Traceback (most recent call last):
File "c:\code\voiceTerm\master.py", line 1, in <module>
from voice_terminal_module.voice_terminal import VoiceTerminal
File "c:\code\voiceTerm\voice_terminal_module\voice_terminal.py", line 1, in <module>
from chatbot_module.chatbot_module import Chatbot
ModuleNotFoundError: No module named 'chatbot_module'
Why does it work sometimes, but sometimes not?
You'll need to restructure your project in a way that you have either: (1) two separate and installed packages containing each module or; (2) one package containing all modules. The simplest way forward is (2) and it looks like:
master.py
src/ # Name it anything you wish
__init__.py
voice_terminal.py
chatbot_module.py
Then use relative imports in voice_terminal.py
from .chatbot_module import Chatbot
Rule: Packages does not know others exists unless the other package is installed, added to sys.path, a subpackage, or part of a parent package.
Python general importing guide (for your own code)
I'm sure there should be a duplicate for this by now, but I can't find it, so I'm writing a full essay.
To solve the problem properly, you must take care of two things:
Make sure that sys.path - the path that Python searches for modules - includes the path to the project root.
Make sure that the import statements in the code are written to work with that path.
The search path
For the first part, you must understand how sys.path works. It is a list of folders where Python will look, that is set up automatically at startup (even if you don't import sys - just like sys.argv is).
Normally, it will be a list which contains, in order:
A path to the entry point for execution (the details of this will depend on how Python is started; typically this is '' when running a .py file directly from the same directory, a path the another directory if you do e.g. python somewhere/else/file.py, and an explicit path when using the -m switch)
Paths to system libraries
Paths to virtual environment libraries, if a virtual environment is active
Paths to things that were explicitly installed by the user
You can modify this list, with the expected impact on future import statements; but you ordinarily should not.
To ensure your project root is on the path, normally you should just install the project - ideally into a virtual environment. Please see the Python packaging guide for details. Failing that, make sure to start the project from just outside the folder containing your "top-level" code. In OP's case, that means C:\code. This will ensure that C:\code (or something equivalent) is on sys.path, which we can then rely upon for the second step.
The imports
We can fundamentally do this in two ways: By absolute imports specifying the path from the root, or by relative imports specifying the path from the current source file. Read more on Stack Overflow about relative imports here and here.
Absolute imports
In either case, we want to treat the root folder for our project (here, C:\code\voiceTerm) as a package. For absolute imports, this means we will always mention the root folder name in our import path.
Thus:
In C:\code\voiceTerm\master.py: from voiceTerm.voice_terminal_module.voice_terminal import VoiceTerminal
In C:\code\voiceTerm\voice_terminal_module\voice_terminal.py: from voiceTerm.voice_terminal_module.chatbot_module.chatbot_module import Chatbot
(You don't really want to have _module in your folder or file names. It doesn't really add information, makes this considerably harder to type, and is actually a bit misleading.)
We can also import entire modules: import voiceTerm.master, import voiceTerm.voice_terminal_module.voice_terminal; import voiceTerm.voice_terminal_module.chatbot_module.chatbot_module. Additionally, by naming a file __init__.py, we become able to import the folder as a package: import voiceTerm; import voiceTerm.voice_terminal_module; import voiceTerm.voice_terminal_module.chatbot_module. We can also import a module from one of those packages: from voiceTerm import master; from voiceTerm.voice_terminal_module import voice_terminal; from voiceTerm.voice_terminal_module.chatbot_module import chatbot_module.
Relative imports
With relative imports, we must use the from syntax, and we can only import things from within our own package hierarchy. However, we have the advantage that we don't have to specify full paths, and we can rename packages without having to edit the code. My personal recommendation is to use relative imports where possible; this also makes a strong visual distinction with imports from system libraries or other third-party packages.
For a relative import, first we specify the package or sub-package that we're importing from, using a relative import path. By starting with a single ., we start from the current directory, looking for another module or package within the same folder.
Thus:
In C:\code\voiceTerm\master.py: from .voice_terminal_module.voice_terminal import VoiceTerminal to import the class; from .voice_terminal_module import voice_terminal to import the module.
In C:\code\voiceTerm\voice_terminal_module\voice_terminal.py: from .chatbot_module.chatbot_module import Chatbot to import the class; from .chatbot_module import chatbot_module to import the module.
Additional .s navigate up the package hierarchy (but we cannot go beyond the root this way). For example, from ... import master to import the top-level master.py from the lower-down chatbot_module.py. (Of course, this wouldn't actually work in this case, because it would be a circular import.) To import another source file (as a module) from the same directory, simply from . import other_file. You get the idea.
I solved it. You can appearantly use importlib for this. I replaced the import code in my voice_terminal.py script with this:
chatbot_module = SourceFileLoader("chatbot_module", "C:/code/voiceTerm/voice_terminal_module/chatbot_module/chatbot_module.py").load_module()
Can I import a python module from a distant folder? (Possible duplicate of How to import a Python module from a sibling folder?)
In general, the git repo should have a requirements.txt if it is for general use. If it has one then you can run pip install requirements.txt
It is also fairly easy for the repo owner to generate this file. At the root of their project they can run pip freeze > requirements.txt. pip freeze lists all current dependencies and so this command outputs the result to requirements.txt.
As for your second point, it really depends on the package structure. If they want to expose the code in the package they may have imported it in one of the top level __init__.py files. Otherwise, you can always directly import by following the paths.
For example if your structure is:
project
folder
subfolder
module
And module has a function called foo then you can import foo like so: from project.folder.subfolder.module import foo. Of course this assumes each of these directories has its own __init__.py file
I could suggest places to look for packages using
import sys
sys.path.append('/path/to/your/module/address/')
Source: Import a file from another directory
I have a large repository with some fixed structure and I have extended it by some folders and python scripts to add extra functionality to it as a whole. The structure looks as follows:
toplevelfolder
featureA
someModuleA.py
__ init __.py
featureB
someModuleB.py
__ init __.py
application
__ init __.py
app.py
Now someModuleA.py and someModuleB.py can be invoked via app.py but at the same time also have be able to be invoked directly, however this invocation must come from the toplevelfolder for the relative paths in the file to resolve correctly, i.e. via python ./featureA/someModuleA.py.
This all works well, but now I need some function definitions from someModuleB in someModuleA and hence I want to import this module. I have tried both absolute and relative imports, but both fail with different errors, the absolute import with
from toplevelfolder.featureA import someModuleA as A
# ModuleNotFoundError: No module named 'toplevelfolder'
and the relative import with
from toplevelfolder.featureA import someModuleA as A
# ImportError: attempted relative import with no known parent package
Now I can see that the relative import would cause problems when python is invoked from the toplevelfolder, as .. would represent the latter's parent directory, rather than the parent directory of featureA. However, I cannot get a hold of the first error message, especially since toplevelfolder should not be a module but a package.
Is there another way to import in Python that I'm not aware of, if possibly without modifying PYTHONPATH or sys.path or something like that?
Not 100% sure on what the goal is here. My advice would be:
Identify clearly what you want your top level modules and packages to be.
Make all imports absolute.
Either:
make your project a real installable project, so that those top level modules and packages are installed in the environment's site-packages directory;
or make sure that the current working directory is the one containing the top level modules and packages.
Make sure to call your code via the executable module or package method instead of the script method, if the "entry point" you want to execute is part of a package
DO (executable module or package):
path/to/pythonX.Y -m toplevelpackage.module
path/to/pythonX.Y -m toplevelpackage.subpackage (assuming there is a toplevelpackage/subpackage/__main__.py file)
DON'T (script within a package):
path/to/pythonX.Y toplevelpackage/module.py
(Optional) Later on, once it all works well and everything is under control, you might decide to change some or all imports to relative. (If things are done right, I believe it could be possible to make it so that it is possible to call the executable modules from any level within the directory structure as the current working directory.)
References:
Old reference, possibly outdated, but assuming I interpreted it right, it says that running scripts that live in a package is an anti pattern, and one should use python -m package.module instead: https://mail.python.org/pipermail/python-3000/2007-April/006793.html -- https://www.python.org/dev/peps/pep-3122/
Don't include the top level directory. In featureB.someModuleB:
from featureA.someModuleA import two
Sample directory.
Try pasting this above your import:
import os,sys,inspect
currentdir = os.path.dirname(os.path.abspath(inspect.getfile(inspect.currentframe())))
parentdir = os.path.dirname(currentdir)
sys.path.insert(0,parentdir)
Then you should be able to import a file from the parent-folder.
I can't figure out how to import modules from sibling directories in Python 3 using absolute imports.
modify the sys.path.
turn the directory into a pip installable package via __init__.py and setup.py.
For option 1. I figured out how to import modules from sibling directories by modifying the sys.path, but this method seems a little hackey to me. Also, I've read that it is not preferred. Why? Is there something inherently wrong or dangerous about modifying the sys.path?
For option 2. What exactly do I need to do make my package pip installable? I've alreay created my __init__.py file, but it seems that I need to create and configure a setup.py script to prepare my package for distribution? I'm still in the development mode, so is this really the best/pythonic method? If it is, then do I just type python setup.py install into my terminal after creating the setup.py?
Edit: I'm now trying to figure this out using absolute imports as python 3 does support relative imports.
From what I've read, Python 3 does not support relative imports
It does.
To import myproject/foo/__init__.py from myproject/bar/baz.py, you can use this:
from .. import foo
Or if you want to import an object/module in foo:
from ..foo import object
This requires myproject to be a package, so myproject/__init__.py has to exist.
I'm trying to build an app with py2app. I can build the app but when I run it I get ImportError in the console, the import error is that there is No module named PythonApp which is the folder all my source is in, the location of which is in my PYTHONPATH.
I have been importing local files like from PythonApp import file instead of just import file to help avoid namespace issues.
I have tried to build it with the following flag python setup.py py2app --use-pythonpath but this hasn't appeared to make any difference.
Should I just change the import statements throughout to import file?
How can I make py2app realise my PYTHONPATH?
If I understood your problem well, you have the following layout:
PythonApp/
__init__.py
file.py
foo.py
And you try to import the module file from foo.py by from PythonApp import file. If this the only reason you want to set PYTHONPATH, there is a simpler solution: use relative imports:
from . import file
You can use from .. import file in a sub-package of PythonApp, and so on. This way you can avoid name collisions with standard modules.
If you need to hack the import path for some other reasons, you can also set the sys.path variable in the startup script (probably py2app has some options for that, too). Keep in mind though that if you add external directories into the import path, it will be harder to distribute the app bundle.
Also, a more trivial explanation for the ImportError is that py2app did not copy your package into the app bundle. Make sure you have all your packages listed in the packages parameter of setup().