I have been creating an application that will run just like Microsoft Notepad. My program comprises of the following file and folder:
libs (Folder)
scripts (Folder)
test.py
The test.py is the file in which I am trying the following code:
from .libs import tkinter
from .libs import pyglet
from .libs import threading
test_window = Tk()
test_window.mainloop()
In the folder libs I have added the modules that are required in the application.
Example: Tkinter, Pyglet and Threading
When I am trying to import them into test.py using the code above and then running the program. I see the following Traceback:
Traceback (most recent call last):
File "c:\Users\Bhavyadeep\Desktop\NuclearPad\test.py", line 1, in <module>
from .libs import tkinter
ImportError: attempted relative import with no known parent package
I don't really know how I solve this problem. Is it because I added the module in the folder where Import in not available? Or is it because I did something wrong in this module folder copying-pasting to a folder not at Python's PATH?
from .libs import tkinter
Means "import tkinter from the package called lib which is located in the same directory as file (this script)"
so your first line should say something like
test_window = tkinter.Tk()
unless you use either of the following:
from .libs.tkinter import *
from .libs.tkinter import Tk
The error is likely due to you using relative import syntax from within the script you are trying to execute. Remove the leading dots in order to execute test.py, otherwise append its directory to sys.path or use site.addsitedir(path_to_directory_containing_test_py) and import it as follows import test.
With that said, you should not call the module's file test.py because there is already something with that name in python's default Lib directory or builtins. If you insist, there should be a hacky way to get it done without changing the name through importlib.
As for why this error occurs, I think it's because the interpreter is designed to execute scripts but not modules/packages whose contents are placed in .pyc files in __pycache__s when imported.
In order to import from scripts or libs, they must be packages. In order to turn them into packages you must add __init__.py files to them.
Lastly, It's a pretty terrible idea to place third-party libraries in your source like this. At the very least you should have installed them properly on your machine and then copied their sources from site-packages/the equivalent for venvs, which will fail if they use any executables and/or have any external dependencies
Documentation for distributing python packages
Related
I have three scripts:
C:\code\voiceTerm\master.py:
from voice_terminal_module.voice_terminal import VoiceTerminal
vterm = VoiceTerminal()
C:\code\voiceTerm\voice_terminal_module\voice_terminal.py:
from chatbot_module.chatbot_module import Chatbot
class VoiceTerminal:
print("INITIALIZING VOICE TERMINAL")
cb = Chatbot()
C:\code\voiceTerm\voice_terminal_module\chatbot_module\chatbot_module.py:
class Chatbot:
print("CHATBOT INITIALIZED")
Here is the wierd thing: When I run chatbot_module.py it works, and if I run voice_terminal.py it works. For some reason however, master.py errors out with the following message:
Traceback (most recent call last):
File "c:\code\voiceTerm\master.py", line 1, in <module>
from voice_terminal_module.voice_terminal import VoiceTerminal
File "c:\code\voiceTerm\voice_terminal_module\voice_terminal.py", line 1, in <module>
from chatbot_module.chatbot_module import Chatbot
ModuleNotFoundError: No module named 'chatbot_module'
Why does it work sometimes, but sometimes not?
You'll need to restructure your project in a way that you have either: (1) two separate and installed packages containing each module or; (2) one package containing all modules. The simplest way forward is (2) and it looks like:
master.py
src/ # Name it anything you wish
__init__.py
voice_terminal.py
chatbot_module.py
Then use relative imports in voice_terminal.py
from .chatbot_module import Chatbot
Rule: Packages does not know others exists unless the other package is installed, added to sys.path, a subpackage, or part of a parent package.
Python general importing guide (for your own code)
I'm sure there should be a duplicate for this by now, but I can't find it, so I'm writing a full essay.
To solve the problem properly, you must take care of two things:
Make sure that sys.path - the path that Python searches for modules - includes the path to the project root.
Make sure that the import statements in the code are written to work with that path.
The search path
For the first part, you must understand how sys.path works. It is a list of folders where Python will look, that is set up automatically at startup (even if you don't import sys - just like sys.argv is).
Normally, it will be a list which contains, in order:
A path to the entry point for execution (the details of this will depend on how Python is started; typically this is '' when running a .py file directly from the same directory, a path the another directory if you do e.g. python somewhere/else/file.py, and an explicit path when using the -m switch)
Paths to system libraries
Paths to virtual environment libraries, if a virtual environment is active
Paths to things that were explicitly installed by the user
You can modify this list, with the expected impact on future import statements; but you ordinarily should not.
To ensure your project root is on the path, normally you should just install the project - ideally into a virtual environment. Please see the Python packaging guide for details. Failing that, make sure to start the project from just outside the folder containing your "top-level" code. In OP's case, that means C:\code. This will ensure that C:\code (or something equivalent) is on sys.path, which we can then rely upon for the second step.
The imports
We can fundamentally do this in two ways: By absolute imports specifying the path from the root, or by relative imports specifying the path from the current source file. Read more on Stack Overflow about relative imports here and here.
Absolute imports
In either case, we want to treat the root folder for our project (here, C:\code\voiceTerm) as a package. For absolute imports, this means we will always mention the root folder name in our import path.
Thus:
In C:\code\voiceTerm\master.py: from voiceTerm.voice_terminal_module.voice_terminal import VoiceTerminal
In C:\code\voiceTerm\voice_terminal_module\voice_terminal.py: from voiceTerm.voice_terminal_module.chatbot_module.chatbot_module import Chatbot
(You don't really want to have _module in your folder or file names. It doesn't really add information, makes this considerably harder to type, and is actually a bit misleading.)
We can also import entire modules: import voiceTerm.master, import voiceTerm.voice_terminal_module.voice_terminal; import voiceTerm.voice_terminal_module.chatbot_module.chatbot_module. Additionally, by naming a file __init__.py, we become able to import the folder as a package: import voiceTerm; import voiceTerm.voice_terminal_module; import voiceTerm.voice_terminal_module.chatbot_module. We can also import a module from one of those packages: from voiceTerm import master; from voiceTerm.voice_terminal_module import voice_terminal; from voiceTerm.voice_terminal_module.chatbot_module import chatbot_module.
Relative imports
With relative imports, we must use the from syntax, and we can only import things from within our own package hierarchy. However, we have the advantage that we don't have to specify full paths, and we can rename packages without having to edit the code. My personal recommendation is to use relative imports where possible; this also makes a strong visual distinction with imports from system libraries or other third-party packages.
For a relative import, first we specify the package or sub-package that we're importing from, using a relative import path. By starting with a single ., we start from the current directory, looking for another module or package within the same folder.
Thus:
In C:\code\voiceTerm\master.py: from .voice_terminal_module.voice_terminal import VoiceTerminal to import the class; from .voice_terminal_module import voice_terminal to import the module.
In C:\code\voiceTerm\voice_terminal_module\voice_terminal.py: from .chatbot_module.chatbot_module import Chatbot to import the class; from .chatbot_module import chatbot_module to import the module.
Additional .s navigate up the package hierarchy (but we cannot go beyond the root this way). For example, from ... import master to import the top-level master.py from the lower-down chatbot_module.py. (Of course, this wouldn't actually work in this case, because it would be a circular import.) To import another source file (as a module) from the same directory, simply from . import other_file. You get the idea.
I solved it. You can appearantly use importlib for this. I replaced the import code in my voice_terminal.py script with this:
chatbot_module = SourceFileLoader("chatbot_module", "C:/code/voiceTerm/voice_terminal_module/chatbot_module/chatbot_module.py").load_module()
I'm having trouble with a python package that uses separate modules to structure code. The package itself is working, however when imported from another environment fails with a ModuleNotFound error.
Here's the structure:
Project-root
|
|--src/
| __init__.py
| module_a.py
| module_b.py
| module_c.py
| module_d.py
|--tests
etc.
In module_a.py I have:
from module_a import function_a1,...
from module_b import function_b1,...
from module_c import function_c1,...
In module_c I import module_d like:
from module_d import function_d1,...
As mentioned above, executing module_a or module_c directly from the CLI work as expected, the unit tests I've created in the test directory also work (with the help of sys.path.insert), however if I create a new environment and import the package I get the following error:
>>> import module_a
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/<abs_path>/.venv/lib/python3.9/site-packages/module_a.py", line 22, in <module>
from module_c import function_c1, function_c2
File /<abs_path>/.venv/lib/python3.9/site-packages/module_c.py", line 9, in <module>
import module_d
ModuleNotFoundError: No module named 'module_d'
>>>
I've exhausted all ideas how to overcome this, besides combining the code of modules c and d in one file, which I'd hate to do, or rethink the flow so that all modules are imported from module_a.
Any suggestions how to approach this would be greatly appreciated.
Update: It turned out to be a typing mistake in the name of module_d in setup.py. For whatever reason python setup.py install was failing silently or I wasn't reading the logs carefully.
The problem comes down to understanding the basics of the import system and the PYTHONPATH.
When you try to import a module (import module_a), Python will search in order in every directory listed in sys.path. If a directory matches the name (module_a)1, then it runs the __init__.py file is such exist.
When you get an [https://docs.python.org/3/library/exceptions.html#ImportError], it means that there is no directory in sys.path containing a directory with the name asked.
You said for your tests you did something like sys.path.insert(0, "some/path/"), but it is not a solution, just a broken fix.
What you should do is set your PYTHONPATH environment variable to contain the directory where your modules are located, Project-root/src in your case. That way, no need to ever use sys.path.insert, or fiddle with relative/absolute paths in import statements.
When you create your new environment, just set your environment variable PYTHONPATH to include Project-root/src and you are done. This is how installing regular Python modules (libraries) work : they are all put into a directory in site-packages.
1: this changed since old Python versions, it used to be required for the directory to contain an __init__.py file
As per screen print, import shows error in Python 3.7 version, earlier it was working fine in version Python 2.7 and I am using IntelliJ Idea.
If you see, EOC related .py files are in the same folder and have classes which are being called in Main_EOC.py by passing objects which are inter-related. It's amazing to see the red line while importing files from same folder.
Please help me why it's showing such error
"This inspection detects names that should resolve but don't. Due to dynamic dispatch and duck typing, this is possible in a limited but useful number of cases. Top-level and class-level items are supported better than instance items.`"
Also, if you see the line which have full path, is not showing error
from EOC_Module.eoc.script.config import Config
Please help me if there is a way to add this full path on top of the code or other option.
The behavior of import path search changed between python2 and python3. The import path always includes the directory from which the main module was loaded, but it no longer includes directories from which modules were imported.
You need to change your import statement syntax as follows, if you want to import a module that lives in the same directory as the module in which you do the import:
# old way, import works if the named module is in this module's directory
import x
# new (Python3) way:
from . import x
For the second part: adding a path so all code can import from a certain directory: if that directory is (and will always be) relative to your main: you can add a few lines in the main module to make it available. Something like this:
import sys # if you haven't imported it already
import os.path
home = os.path.dirname(sys.argv[0])
sys.path.append( os.path.join(home, "EOC_Module/eoc/script") )
# now, you can import straight from the script directory
import EOC_Intraction
When using pycharm the root directory for your python executable is the same as the root directory of your project, this means that python will start looking for files in the root directory with this files:
.idea/
EOC_module/
logs/
reports/
sql/
This is the reason of why: from EOC_Module.eoc.script.config import Config works.
If you execute your code from the terminal with: python3 Main_EOC.py (not pycharm) the root directory for your python will be the same as the one containing the file, all the other imports will work but from EOC_Module.eoc.script.config import Config not.
So you need to make your imports from project directory if you are using pycharm.
This question already has an answer here:
ImportError on python 3, worked fine on python 2.7
(1 answer)
Closed 5 years ago.
I need assistance on how to organize source in a python package - I've already followed several tutorials on the web (especially this one) on how to do so, but it does not work as explained and how I imagined it.
I want to create a python package named binaryio. It should offer two classes named BinaryReader and BinaryWriter which I want users to be able to import with
from binaryio import BinaryReader
from binaryio import BinaryWriter
Thus I have created my repository and package directory structure as follows:
binaryio (repository root)
binaryio (package root)
__init__.py (s. below)
binaryreader.py (contains the BinaryReader class)
binarywriter.py (contains the BinaryWriter class)
setup.py (contains the setuptools.setup call)
.gitignore, README.md, LICENSE, ...
As you can see, the classes are in separate files as I'm used to this (coming from a C# background). I'm not sure if this is a good idea due to modules being the "unit" in Python - but otherwise cramping all classes into one huge file did not seem logical to me.
__init__.py looks as follows to import those classes, making (as I understood it) the from binaryio import BinaryReader imports possible for users later on:
from binaryreader import BinaryReader
from binarywriter import BinaryWriter
However, when I install the package locally (which seems to work fine) and try to import binaryio, I get the following error:
>>> import binaryio
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "D:\Projects\Git\binaryio\binaryio\__init__.py", line 1, in <module>
from binaryreader import BinaryReader
ModuleNotFoundError: No module named 'binaryreader'
Apparently, something is wrong with my __init__.py file. I don't understand this, as a binaryreader.py file aka module exists in the same folder as you can see above. Funnily enough, my IDE (PyCharm, having set the package root as source folder) does not complain about the statements in it and can resolve all references.
What am I doing wrong here? According to the tutorial linked above, putting a file into xyz.py with a class named Abc and then write from xyz import Abc into __init__.py should work, but apparently it doesn't for me.
Your code would work for Python 2.x, but not 3.x because of different relative imports syntax: without dot, Python 2.x would look for modules in module root and current package, and Python 3.x will look only in module root.
Import statements you want to use are these:
from binaryio.binaryreader import BinaryReader
from binaryio.binarywriter import BinaryWriter
Works in both Python 2.x and 3.x without "futures"
I think you need to add a dot in your import statements :
from .binaryreader import BinaryReader
from .binarywriter import BinaryWriter
I'm having trouble understanding how objects declared inside '__init__.py' are/should be imported to other files.
I have a directory structure like so
top/
|
|_lib/
|_ __init__.py
|_ one.py
File contents are as follows
lib/__init__.py
a=object()
lib/one.py
from lib import a
Here is the problem. If I fire a python shell from top directory, the following command runs well
>>> from lib.one import a
However if I change directory to top/lib and fire a similar command in a new python shell, I get error.
>>> from one import a
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "one.py", line 1, in <module>
from lib import a
ImportError: No module named lib
Ofcourse, I can change one.py like so, that will make everything work.
from __init__ import a
But I'm really trying to understand, why import command works from top directory and not from top/lib.
Thanks.
Generally speaking, I think it's best practice to have the data funnel up to __init__.py from the modules/subpackages rather than needing to rely on data from __init__.py in the surrounding modules. In other words, __init__.py can use one.py, but one.py shouldn't use data/functions in __init__.py.
Now, to your question...
It works in top because python does a relative import (which is gone in python3.x IIRC, so don't depend on it ;-). In other words, python looks in the current directory for a module or package name lib and it imports it. That's all fine so far. running from lib.one import a first imports lib (__init__.py) which works fine. Then it imports one -- lib still imports ok from one because it's relative to your current working directory -- Not relative to the source file.
When you move into the lib directory, python can no longer find lib in the current directory making it not importable. Note that with most packages, this is fixed by installing the package which puts it someplace that python can find it without it needing to be in the current directory.