Basically, I have two python projects, one is located under myapp/screening and the other myapp/server. I'm currently developing the server module and want to import functions from screening using myapp.screening.
My folder structure is as shown bellow:
myapp/
screening/
screening-env/
myapp/
__init__.py
screening/
__init__.py
screening_task.py
submodule1/
# __init__.py and ub module files
submodule2/
# __init__.py and sub module files
submodule3/
# __init__.py and sub module files
tests/
# __init__.py and test scripts
setup.py
server/
server-env/
myapp/
__init__.py
server/
__init__.py
server_task.py
tests/
__init__.py
server_test.py
I structured my project following this answer.
My setup.py is basically as bellow:
from setuptools import setup
setup(
name='myapp-screening',
version='0.1.0',
packages=[
'myapp.screening',
'myapp.screening.submodule1',
'myapp.screening.submodule2',
'myapp.screening.submodule3'
],
)
I activated my server-env and installed the screening project by navigating to myapp/screening/ (same directory as setup.py) and ran python setup.py develop.
Finally, both server_test.py and server_task are such as bellow:
from myapp.screening.screening_test import foo
foo()
When I run python -m myapp.server.server_task or python -m test.server_test I get:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'myapp.screening'
This error makes sense if I'm running python -m myapp.server.server_task because local myapp existis and might be overwriting the installed myapp that contains the screening modules.
Is there a way to import stuff from screening using from myapp.screening.screening_task import foo?!
So, after some more research I found this similar (in a way) question that leads to import python modules with the same name and How do I create a namespace package in Python?.
The answer for "importing modules with same name" is not useful since it says to rename one module or turn the project directory into a package.
The other question is exactly what I want. It basically talks about the pkgutil with which you can 'append' modules to a given namespace.
I understand and share some opinions against this technique for some cases (such as this) but since it's presented here sometimes you want separated structures so you don't patch everything togheter even when you don't want everything
Related
I created a project that runs fine locally, but fails when run from PyPi deploy/install, with ModuleNotFound. The link contains a very small sample, and run / deploy instructions. My original thought was regarding inheritance, but it seems more basic.
Traceback (most recent call last):
File "/Users/val/python/vscode/inheritance/venv/bin/inheritance-run", line 5, in <module>
from inheritance.run import start
ModuleNotFoundError: No module named 'inheritance'
Your setup.py has:
packages=find_packages(),
This will find modules (directories with __init__.py files) in the same directory as your setup.py file, but you don't have any. Instead, you have a base.py file.
If you want to be able to import inheritance, you should move this into an __init__.py file in new directory named inheritance, so you have something like:
.
├── inheritance
│ └── __init__.py
└── setup.py
Then setuptools will find this module and include it as an importable package in your project.
FYI, It's usually preferable to use a src-based layout instead, however. See https://github.com/pypa/sampleproject/ for a full example.
My file structure is
project/
__init__.py
features/
__init__.py
CompareText.py
tests/
test.py
in test.py I am trying to import CompareText
from project.features import CompareText
I get an error of:
ModuleNotFoundError: No module named 'features'`
I checked the documentation and I think my import statement is correct. How can I fix it?
Add an __init__ file in test. Your project directory should look like this:
project/
__init__.py
features/
__init__.py
CompareText.py
tests/
__init__.py
test.py
Then in project/tests/test.py the following import statement will work:
from ..features import CompareText
Oh, and this will still raise an error if you try to run it directly. In the question you said you tried to import it like this:
from project.features import CompareText
This will only work if the parent directory of project is in Python's module search path. So, if you want to run the tests directly then modify the module search path as needed (See: sys.path).
Your import statement is supposed to look like this :
(But make sure your working directory is the same directory as your project folder is located during execution)
from project.features import CompareText
This is supposed to work if your current path while executing the script has the project folder
If you execute it while inside project folder you can use:
from .features import CompareText
Hope this helps!
I assume you are running test.py as a script. test.py needs to find the project package and two ways to do that are to make your project installable or to hack sys.path.
Installable
First, change your directory structure a bit so that project is a subdirectory of some anonymous directory you happen to be using for development. If you are checking this stuff into source control, it needs to be written so that it can be checked out anywhere. Move tests down one directory.
mydevdir/
setup.py
project/
__init__.py
features/
__init__.py
CompareText.py
tests/
test.py
How write a setup.py. This can get quite complicated. You can read Building and Distributing Packages with Setuptools and lookup other resources on the net, but a minimalist setup.py is
#!/usr/bin/env python
from setuptools import setup, find_packages
setup(name='project',
version='0.1',
description='This is project: project',
packages=find_packages(),
)
Now, while in mydevdir do python setup.py develop. Or you can actually produce an install package and put it in a virtual env for test.
Hack sys.path
It may be easier to hack paths in test.py. Note that this will need to be undone if you make project installable later. Just add to the top of test.py
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).absolute().parents[2]))
This puts the parent directory in the python path and now project will be found on import. It runs the risk that a .py file in the same directory as project can mask an installed module. If you have a local csv.py and you import csv, you'd get the local.
custom/
scripts/
__init__.py
file1.py
utils/
__init__.py
utilFile1.py
utilFile2.py
utilFile3.py
I have the above file structure I'm struggling trying to figure out how to import modules in python. I have gone through stackoverflow posts that have the same question but I still couldn't figure out why I can't get it to work.
I am trying to import these modules in maya and when I run
from utils import utilFile1 I get
ImportError: cannot import name utilFile1.
Running from custom.scripts.utils import utilFile1 gives me this error ImportError: no module named custom.scripts.utils. However if I run import file1 it imports without any errors
I have appended custom/scripts to sys.path and when that didn't work, tried appending custom/scripts/utils as well, but that didn't work either. Based off some of the posts on stackoverflow, I saw some people suggesting to run "python -m" but I'm not sure if I should run this or where to execute it from.
I'm really at a loss as to why I can't get it to work at all.
you have make it a python package by following steps,
in your root dir create setup.py,
in your case
custom/
setup.py
scripts/
...
...
in setup.py
from setuptools import setup, find_packages
setup(
name='your_package_name',
description='bla bla bla',
version='0.0.1-dev',
install_requires=[
],
)
and then hit
pip install -e .
for more info refer to this docs
Please update the directory structure as below.
Python might not be considering custom as a module as it doesn't have __init__.py file,
custom/
__init__.py
scripts/
__init__.py
file1.py
utils/
__init__.py
utilFile1.py
utilFile2.py
utilFile3.py
Yash's method works, I believe that's what companies use to setup their environments. An alternative way is to add your scripts path as a PYTHONPATH in your maya.env file instead of using sys.path.append. I'm not sure why there's a difference or if it's coz I'm trying this on windows, but oddly enough it worked in my case.
I've been reading tons of questions related to this matter but none of the has help me so far. I'm currently using the Python click library to execute scripts as commands.
The current command that I'm trying to execute is placed inside a Python Package which has a __main__.py file, like the parent dir has. The current project structure is the following one.
/myproject
/foo_one
__init__.py
foo_one.py
/foo_two
__init__.py
foo_two.py
/foo_three
__init__.py
foo_three.py
/foo_four
__init__.py
foo_four.py
/foo_five
__init__.py
foo_five.py
/foo_six
__init__.py
foo_six.py
__init__.py
__main__.py
foo_seven.py
Whenever I try to run the __main__.py script located in the project folder, the following error comes up.
ModuleNotFoundError: No module named '__main__.foo_two'; '__main__' is not a package
However, if I try to execute that same script from a folder above with the -m option like this python3 myproject -m, the following is shown up.
ImportError: attempted relative import with no known parent package
The __main__.py has 2 imports like this... The __init__.py is empty.
from .foo_two.foo_two import AClass, AnotherClass, OtherClass
from .foo_three.foo_three import AnotherClassMore
UPDATE: Correcting the syntax error in a previous command, while calling python -m myproject gives me a ModuleNotFoundError because of a module that isn't my responsibility, which is basically a library that is used in the project.
Hopefully, this will be of value to someone out there - I went through half a dozen stackoverflow posts trying to figure out relative imports similar to whats posted above here. I set up everything as suggested but I was still hitting ModuleNotFoundError: No module named 'my_module_name'
Since I was just developing locally and playing around, I hadn't created/run a setup.py file. I also hadn't apparently set my PYTHONPATH.
I realized that when I ran my code as I had been when the tests were in the same directory as the module, I couldn't find my module:
$ python3 test/my_module/module_test.py 2.4.0
Traceback (most recent call last):
File "test/my_module/module_test.py", line 6, in <module>
from my_module.module import *
ModuleNotFoundError: No module named 'my_module'
However, when I explicitly specified the path things started to work:
$ PYTHONPATH=. python3 test/my_module/module_test.py 2.4.0
...........
----------------------------------------------------------------------
Ran 11 tests in 0.001s
OK
So, in the event that anyone has tried a few suggestions, believes their code is structured correctly and still finds themselves in a similar situation as myself try either of the following if you don't just add your export the current directory to your PYTHONPATH:
Run your code and explicitly include the path like so:
$ PYTHONPATH=. python3 test/my_module/module_test.py
To avoid calling PYTHONPATH=., create a setup.py file with contents like the following and run python setup.py development to add packages to the path:
# setup.py
from setuptools import setup, find_packages
setup(
name='sample',
packages=find_packages()
)
The correct syntax would be
python -m myproject
This should execute __main__ in the top-level package.
You need to have __init__.py in each sub folder with python code to tell the interpreter to treat the folder as a module
/myproject
/foo_one
__init__.py # add this
foo_one.py
/foo_two
__init__.py # add this
foo_two.py
/foo_three
__init__.py # add this
foo_three.py
/foo_four
__init__.py # add this
foo_four.py
/foo_five
__init__.py # add this
foo_five.py
/foo_six
__init__.py # add this
foo_six.py
__init__.py
__main__.py
foo_seven.py
the __init__.py is telling the interpreter to treat sub folders as python modules / packages and you should be able to import
The __init__.py file can be empty but needs to be present in the sub folders to be able to import that module / package
I can't make this work..
My structure is:
program_name/
__init__.py
setup.py
src/
__init__.py
Process/
__init__.py
thefile.py
tests/
__init__.py
thetest.py
thetest.py:
from ..src.Process.thefile.py import sth
Running: pytest ./tests/thetest.py from program_name gives :
ValueError: attempted relative import beyond top-level package
I tried also other approaches but i am receiving various errors.
But I would expect for the above to work.
ValueError: Attempted relative import in non-package
States that you're trying to use relative import in the module, which are to be used for packages i.e. to make it a package add __init__.py and call the thetest.py from some file outside the package.
Directly running thetest.py from interpreter won't work.
Relative imports require that the module which uses them is being
imported itself either as package module.
Suggestion 1:
The current tests directory has a __init__.py file but that doesn't allow you to run it as a module (via shell) - to make your current (relative) import work, you need to import it in an external (to package) file/module - let's create a main.py (can name it anything you like):
main.py
program_name/
__init__.py
setup.py
src/
__init__.py
Process/
__init__.py
thefile.py
tests/
__init__.py
thetest.py
src/Process/thefile.py:
s = 'Hello world'
tests/thetest.py:
from ..src.Process.thefile import s
print s
main.py:
from program_name.tests.thetest import s
Executing main.py:
[nahmed#localhost ~]$ python main.py
Hello world
Suggestion 2:
Execute the file just above root dir i.e. one level up the program_name/ , in the following fashion:
[nahmed#localhost ~]$ python -m program_name.tests.thetest
Hell World
P.S. relative imports are for packages, not modules.
Just solved a similar problem with a lot of googling.
Here's two solutions without changing the existing file structor:
1
The way to import module from parent folder from ..src.Process.thefile.py import sth is called "relative import".
It's only supported when launching as a package from the top-level package. In your case, that is launching command line from the directory which contains program_name/ and type (for win environment)
python -m program_name.tests.thetest
or simply (useful for many pytest files):
python -m pytest
2
Otherwise -- when trying to run a script alone or from a non top-level package --
you could manually add directory to the PYTHONPATH at run time.
import sys
from os import path
sys.path.append(path.dirname(path.dirname(path.abspath(__file__))))
from src.Process.thefile import s
Try the first one first see if it's compatiable with the pytest framework. Otherwise the second one should always solve the problem.
Reference (How to fix "Attempted relative import in non-package" even with __init__.py)
When importing a file, Python only searches the current directory, the directory that the entry-point script is running from.
you can use sys.path to include different locations
import sys
sys.path.insert(0, '/path/to/application/app/folder')
import thefile