custom/
scripts/
__init__.py
file1.py
utils/
__init__.py
utilFile1.py
utilFile2.py
utilFile3.py
I have the above file structure I'm struggling trying to figure out how to import modules in python. I have gone through stackoverflow posts that have the same question but I still couldn't figure out why I can't get it to work.
I am trying to import these modules in maya and when I run
from utils import utilFile1 I get
ImportError: cannot import name utilFile1.
Running from custom.scripts.utils import utilFile1 gives me this error ImportError: no module named custom.scripts.utils. However if I run import file1 it imports without any errors
I have appended custom/scripts to sys.path and when that didn't work, tried appending custom/scripts/utils as well, but that didn't work either. Based off some of the posts on stackoverflow, I saw some people suggesting to run "python -m" but I'm not sure if I should run this or where to execute it from.
I'm really at a loss as to why I can't get it to work at all.
you have make it a python package by following steps,
in your root dir create setup.py,
in your case
custom/
setup.py
scripts/
...
...
in setup.py
from setuptools import setup, find_packages
setup(
name='your_package_name',
description='bla bla bla',
version='0.0.1-dev',
install_requires=[
],
)
and then hit
pip install -e .
for more info refer to this docs
Please update the directory structure as below.
Python might not be considering custom as a module as it doesn't have __init__.py file,
custom/
__init__.py
scripts/
__init__.py
file1.py
utils/
__init__.py
utilFile1.py
utilFile2.py
utilFile3.py
Yash's method works, I believe that's what companies use to setup their environments. An alternative way is to add your scripts path as a PYTHONPATH in your maya.env file instead of using sys.path.append. I'm not sure why there's a difference or if it's coz I'm trying this on windows, but oddly enough it worked in my case.
Related
I'm working on a package that has the following directory structure:
setup.py
README.md
src/
__init__.py
file.py
subpackage/
...
tests/
...
I want to import file.py in __init__.py
This import statement works when I directly run __init__.py:
import file
.
.
.
object = file.class()
But it fails when I install the package using pip by using python -m pip install . (the . refers to the root package directory) and call __init__.py using an entry point defined in setup.py
I get the following error for the same import statement:
ModuleNotFoundError: No module named 'file'
I observed that the import succeeds in the entry script if I do:
from . import file
But it fails when I run init.py the regular way with this error:
ImportError: attempted relative import with no known parent package
For reference, here's the entry script defined in setup.py:
entry_points={
'gui_scripts': [
'runpack =packagename.__init__:main',
],
}
Any ideas as to what I'm doing wrong?
Thank you.
First, your file directory structure does not seem correct. I would recommend you to rename the src directory to packagename (or something like that), which will be your "top-level importable package".
I also recommend to only ever use absolute imports, which means that all imports should start from the top-level importable packages. So to import from your own project the import statements should start from packagename. For example import packagename.file or from packagename import file.
It seems to me like your entry point should be runpack=packagename:main, the __init__ is not needed here.
I would also add a packagename/__main__.py file with a content similar to this:
import packagename
def my_main():
return packagename.main()
if __name__ == '__main__':
my_main()
Finally you could run your code as either python -m packagename or runpack but you definitely should not run your code as python packagename/__init__.py or python packagename/__main__.py.
Say I have a project structure as such:
/project_name
setup.cfg
pyproject.toml
/src
/package1
__init__.py
/package2
__init__.py
module.py
I want to import and use module.py in package1/__init__.py and I can do so with the two following imports (relative or absolute):
from ..package2 import module
import src.package2.module
This works ok for developing the library and testing it, but my issue arises when the end user installs and uses the package downstream. Inside my setup.cfg file I have:
...
[options]
package_dir =
= src
packages = find:
...
[options.packages.find]
where = src
I've been using py -m build to build the .tar.gz and -py3-none-any.whl files and uploading them to a private repository. Users can then install it using:
pip install --extra-index-url http://mysite/private/repo project_name --trusted-host mysite
The install goes ok, but when when they go to try and use package1 with import package1 that is where things get messy as they get one of these errors:
"ValueError: attempted relative import beyond top-level package"
This happens when I use from ..package2 import module as described above
"ModuleNotFoundError: No module named 'src'"
This happens when I use import src.package2.module as described above
Update
I found that if I restructure my project as such I get it working for the end user:
/project_name
setup.cfg
pyproject.toml
/src
/project_name
__init__.py
/package1
__init__.py
/package2
__init__.py
module.py
The end user can use the modules using from project_name import package1 which I was hoping to keep it down to just import package1 but whatever that is fine. What I really have a problem with is that in package1 I HAVE to use a relative import:
from ..package2 import module
The reason this bothers me is because I've read in several places that it is better to use absolute imports as the end user can modify the path and screw things up, but if I do it this way:
import src.project_name.package2.module
It breaks the end user running the code because they don't see src!!! So I guess the heart of my question is: how does python determine what the root package is? Why do I have to use src in the import statement (which DOESN'T have a __init__.py) instead of just project_name.package2.module. I've looked all over and can't seem to answer this. Very confused.
The way you setup your build/install, everything would be installed following the tree hierarchy below src (not including src). And you can do:
import project_name
import project_name.package1
Note that in this setup you cannot do import package1. It seems like in your development environment you have modified PYTHONPATH so you can do import package1, and this is what causes your issues. In this case python treats package1 as the top-level package, and you cannot do a relative import that goes above the top-level. It seems you have also set your development environment so you can do import src.project_name.package1, which is inconsistent with the actual install location.
You cannot do relative imports between two separate packages. They need to be within a larger package, like your project_name, as you found out. If you want relative imports, you definitely need this top-level package.
Bottom-line, if in your development environment you set your PYTHONPATH to search in the src directory, you can do import project_name.package1 and either a relative import
from ..package2 import module
or absolute import (note I do not include src)
import project_name.package2.module
My file structure is
project/
__init__.py
features/
__init__.py
CompareText.py
tests/
test.py
in test.py I am trying to import CompareText
from project.features import CompareText
I get an error of:
ModuleNotFoundError: No module named 'features'`
I checked the documentation and I think my import statement is correct. How can I fix it?
Add an __init__ file in test. Your project directory should look like this:
project/
__init__.py
features/
__init__.py
CompareText.py
tests/
__init__.py
test.py
Then in project/tests/test.py the following import statement will work:
from ..features import CompareText
Oh, and this will still raise an error if you try to run it directly. In the question you said you tried to import it like this:
from project.features import CompareText
This will only work if the parent directory of project is in Python's module search path. So, if you want to run the tests directly then modify the module search path as needed (See: sys.path).
Your import statement is supposed to look like this :
(But make sure your working directory is the same directory as your project folder is located during execution)
from project.features import CompareText
This is supposed to work if your current path while executing the script has the project folder
If you execute it while inside project folder you can use:
from .features import CompareText
Hope this helps!
I assume you are running test.py as a script. test.py needs to find the project package and two ways to do that are to make your project installable or to hack sys.path.
Installable
First, change your directory structure a bit so that project is a subdirectory of some anonymous directory you happen to be using for development. If you are checking this stuff into source control, it needs to be written so that it can be checked out anywhere. Move tests down one directory.
mydevdir/
setup.py
project/
__init__.py
features/
__init__.py
CompareText.py
tests/
test.py
How write a setup.py. This can get quite complicated. You can read Building and Distributing Packages with Setuptools and lookup other resources on the net, but a minimalist setup.py is
#!/usr/bin/env python
from setuptools import setup, find_packages
setup(name='project',
version='0.1',
description='This is project: project',
packages=find_packages(),
)
Now, while in mydevdir do python setup.py develop. Or you can actually produce an install package and put it in a virtual env for test.
Hack sys.path
It may be easier to hack paths in test.py. Note that this will need to be undone if you make project installable later. Just add to the top of test.py
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).absolute().parents[2]))
This puts the parent directory in the python path and now project will be found on import. It runs the risk that a .py file in the same directory as project can mask an installed module. If you have a local csv.py and you import csv, you'd get the local.
I have 2 identical files:
a.py and b.py .
they both contain the line
from mypackage.utils import common
I also have a package:
mypackage/
__init__.py
mymodule.py
utils/
__init__.py
common.py
myutils1.py
b.py
data_classes/
__init__.py
command_file.py
myclass.py
I preliminarly do a
pip install mypackage
Then I run:
python a.py
which succeeds
and then I run
python b.py (note that b.py is a file within mypackage)
b.py fails with no module named utils
Any ideas?
(note: I am fairly new to stackoverflow, this is not really related to
Python modules import fails , although somehow a followup question, so if I am doing something wrong by asking on a new question please let me know :)
edit:
I install mypackage (and specifically the file b within mypackage) in:
$HOME\github\mypackage\mypackage\utils\b.py
I copy b.py to a.py in:
$HOME\github\mypackage
this works
I copy b.py to a.py in:
$HOME\github\
this doesn't work
so this somewhat explains this... although I still don't know how to fix it!
edit:
I edited setup.py from the following line:
packages=['mypackage'],
to
packages=['mypackage','mypackage.data_classes','mypackage.utils']
this seems to fix my problem: now everything works.
is there any good reason why I should not do this?
i have found a couple of solutions to this:
import common
or
import sys
sys.path.append("..\\..") # relative path to mypackage, check the slash if you're on linux
from mypackage.utils import common
some questions:
why are you moving b.py?
have you installed your package?
The issue is that Python doesn't know where to find your package in the first place. In the first explain, it looks at the current directory and is able to understand that it is a module, thanks to your __init__.py file. So it is able to search for your utils.
In the 2nd example, there is no __init__.py file in the current directory and so Python doesn't know where to search for the modules.
The PYTHONPATH variable tells Python the directories/modules to search for accessed methods/packages.
So in your case, if you can add your current directory to the PYTHONPATH like export PYTHONPATH=$(pwd) while in "$HOME\github\", that should help resolve your issue.
P.S: The OP wants to use the installed package instead of the local module. This solution is applicable when your local modules are not visible for the Python interpreter. For more details, check the comments below.
I edited setup.py from the following line:
packages=['mypackage'],
to
packages=['mypackage','mypackage.data_classes','mypackage.utils']
this seems to fix my problem: now everything works.
is there any good reason why I should not do this?
I've been reading tons of questions related to this matter but none of the has help me so far. I'm currently using the Python click library to execute scripts as commands.
The current command that I'm trying to execute is placed inside a Python Package which has a __main__.py file, like the parent dir has. The current project structure is the following one.
/myproject
/foo_one
__init__.py
foo_one.py
/foo_two
__init__.py
foo_two.py
/foo_three
__init__.py
foo_three.py
/foo_four
__init__.py
foo_four.py
/foo_five
__init__.py
foo_five.py
/foo_six
__init__.py
foo_six.py
__init__.py
__main__.py
foo_seven.py
Whenever I try to run the __main__.py script located in the project folder, the following error comes up.
ModuleNotFoundError: No module named '__main__.foo_two'; '__main__' is not a package
However, if I try to execute that same script from a folder above with the -m option like this python3 myproject -m, the following is shown up.
ImportError: attempted relative import with no known parent package
The __main__.py has 2 imports like this... The __init__.py is empty.
from .foo_two.foo_two import AClass, AnotherClass, OtherClass
from .foo_three.foo_three import AnotherClassMore
UPDATE: Correcting the syntax error in a previous command, while calling python -m myproject gives me a ModuleNotFoundError because of a module that isn't my responsibility, which is basically a library that is used in the project.
Hopefully, this will be of value to someone out there - I went through half a dozen stackoverflow posts trying to figure out relative imports similar to whats posted above here. I set up everything as suggested but I was still hitting ModuleNotFoundError: No module named 'my_module_name'
Since I was just developing locally and playing around, I hadn't created/run a setup.py file. I also hadn't apparently set my PYTHONPATH.
I realized that when I ran my code as I had been when the tests were in the same directory as the module, I couldn't find my module:
$ python3 test/my_module/module_test.py 2.4.0
Traceback (most recent call last):
File "test/my_module/module_test.py", line 6, in <module>
from my_module.module import *
ModuleNotFoundError: No module named 'my_module'
However, when I explicitly specified the path things started to work:
$ PYTHONPATH=. python3 test/my_module/module_test.py 2.4.0
...........
----------------------------------------------------------------------
Ran 11 tests in 0.001s
OK
So, in the event that anyone has tried a few suggestions, believes their code is structured correctly and still finds themselves in a similar situation as myself try either of the following if you don't just add your export the current directory to your PYTHONPATH:
Run your code and explicitly include the path like so:
$ PYTHONPATH=. python3 test/my_module/module_test.py
To avoid calling PYTHONPATH=., create a setup.py file with contents like the following and run python setup.py development to add packages to the path:
# setup.py
from setuptools import setup, find_packages
setup(
name='sample',
packages=find_packages()
)
The correct syntax would be
python -m myproject
This should execute __main__ in the top-level package.
You need to have __init__.py in each sub folder with python code to tell the interpreter to treat the folder as a module
/myproject
/foo_one
__init__.py # add this
foo_one.py
/foo_two
__init__.py # add this
foo_two.py
/foo_three
__init__.py # add this
foo_three.py
/foo_four
__init__.py # add this
foo_four.py
/foo_five
__init__.py # add this
foo_five.py
/foo_six
__init__.py # add this
foo_six.py
__init__.py
__main__.py
foo_seven.py
the __init__.py is telling the interpreter to treat sub folders as python modules / packages and you should be able to import
The __init__.py file can be empty but needs to be present in the sub folders to be able to import that module / package