I have a folder structure like this:
setup.py
core/
__init__.py
interpreter.py
tests/
__init__.py
test_ingest.py
If I try to import core in test_ingest.py and run it, I get an ImportError saying that the core module can't be found. However, I can import core in setup.py without an issue. My IDE doesn't freak out, so why is this error occurring?
When you import your package, Python searches the directories on sys.path until it finds one of these: a file called "core.py", or a directory called "core" containing a file called __init__.py. Python then imports your package.
You are able to successfully import core from setup.py because the path to the core directory is found in sys.path. You can see this yourself by running this snippet from your file:
import sys
for line in sys.path:
print line
If you want to import core from a different file in your folder structure, you can append the path to the directory where core is found to sys.path in your file:
import sys
sys.path.append("/path/to/your/module")
Related
My file structure is
project/
__init__.py
features/
__init__.py
CompareText.py
tests/
test.py
in test.py I am trying to import CompareText
from project.features import CompareText
I get an error of:
ModuleNotFoundError: No module named 'features'`
I checked the documentation and I think my import statement is correct. How can I fix it?
Add an __init__ file in test. Your project directory should look like this:
project/
__init__.py
features/
__init__.py
CompareText.py
tests/
__init__.py
test.py
Then in project/tests/test.py the following import statement will work:
from ..features import CompareText
Oh, and this will still raise an error if you try to run it directly. In the question you said you tried to import it like this:
from project.features import CompareText
This will only work if the parent directory of project is in Python's module search path. So, if you want to run the tests directly then modify the module search path as needed (See: sys.path).
Your import statement is supposed to look like this :
(But make sure your working directory is the same directory as your project folder is located during execution)
from project.features import CompareText
This is supposed to work if your current path while executing the script has the project folder
If you execute it while inside project folder you can use:
from .features import CompareText
Hope this helps!
I assume you are running test.py as a script. test.py needs to find the project package and two ways to do that are to make your project installable or to hack sys.path.
Installable
First, change your directory structure a bit so that project is a subdirectory of some anonymous directory you happen to be using for development. If you are checking this stuff into source control, it needs to be written so that it can be checked out anywhere. Move tests down one directory.
mydevdir/
setup.py
project/
__init__.py
features/
__init__.py
CompareText.py
tests/
test.py
How write a setup.py. This can get quite complicated. You can read Building and Distributing Packages with Setuptools and lookup other resources on the net, but a minimalist setup.py is
#!/usr/bin/env python
from setuptools import setup, find_packages
setup(name='project',
version='0.1',
description='This is project: project',
packages=find_packages(),
)
Now, while in mydevdir do python setup.py develop. Or you can actually produce an install package and put it in a virtual env for test.
Hack sys.path
It may be easier to hack paths in test.py. Note that this will need to be undone if you make project installable later. Just add to the top of test.py
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).absolute().parents[2]))
This puts the parent directory in the python path and now project will be found on import. It runs the risk that a .py file in the same directory as project can mask an installed module. If you have a local csv.py and you import csv, you'd get the local.
Sorry, this is definitely a duplicate, but I can't find the answer. I'm working in Python 3 and this is the structure of my app:
/home
common.py
australia/
new-south-wales/
fetch.py
I am in the home/ directory, running fetch.py. How can I import functions from common.py in that script?
I've set up fetch.py as follows:
from common import writeFile
But I get the following error:
File "australia/new-south-wales/fetch.py", line 8, in <module>
from common import writeFile
ModuleNotFoundError: No module named 'common'
If I just do python -c "from common import writeFile" I don't see an error.
Shouldn't the interpreter look in the current directory for modules?
before import your directories that need to be imported must have file __init__.py in that folder
#solution 1 (import in runtime)
To import a specific Python file at 'runtime' with a known name:
import os
import sys
script_dir = "/path/to/your/code/directory"
# Add the absolute directory path containing your
# module to the Python path
sys.path.append(os.path.abspath(script_dir))
import filename
#solution 2(add files to one of python libraries)
also as you have a common library for you can run
>>> import sys
>>> print sys.path
and see what directories you can put your code and use in every project.you can move your common package to one of this directories and treat it like a normal package.for example for common.py if you put it in one root directory of one of this directory you can import like import common
#solution 3(use relative import)
# from two parent above current directory import common
# every dot for one parent directory
from ... import common
and then go to parent directory and run
python -m home.australia.new-south-wales.fetch
From the description I'm assuming you're not running this as complete python package, just as separate files.
What you can do is use complete modules. This means adding empty __init__.py to directories with your code. You'll also have to change the name of new-south-wales to new_south_wales, since it needs to be a valid module name.
Assuming home is the name of your app, you should end up with:
home/
__init__.py
common.py
australia/
__init__.py
new_south_wales/
__init__.py
fetch.py
Next, you'll need a startup script for your app - this means either something simple like:
#!/usr/bin/env python
from australia.new_south_wales import fetch
fetch.your_main_function()
Or you can add a setup.py with a full package description. If you specify entry points and the script will be automatically created.
Now that you're starting your code in context of a package, your fetch.py can do:
from ..common import writeFile
I can't make this work..
My structure is:
program_name/
__init__.py
setup.py
src/
__init__.py
Process/
__init__.py
thefile.py
tests/
__init__.py
thetest.py
thetest.py:
from ..src.Process.thefile.py import sth
Running: pytest ./tests/thetest.py from program_name gives :
ValueError: attempted relative import beyond top-level package
I tried also other approaches but i am receiving various errors.
But I would expect for the above to work.
ValueError: Attempted relative import in non-package
States that you're trying to use relative import in the module, which are to be used for packages i.e. to make it a package add __init__.py and call the thetest.py from some file outside the package.
Directly running thetest.py from interpreter won't work.
Relative imports require that the module which uses them is being
imported itself either as package module.
Suggestion 1:
The current tests directory has a __init__.py file but that doesn't allow you to run it as a module (via shell) - to make your current (relative) import work, you need to import it in an external (to package) file/module - let's create a main.py (can name it anything you like):
main.py
program_name/
__init__.py
setup.py
src/
__init__.py
Process/
__init__.py
thefile.py
tests/
__init__.py
thetest.py
src/Process/thefile.py:
s = 'Hello world'
tests/thetest.py:
from ..src.Process.thefile import s
print s
main.py:
from program_name.tests.thetest import s
Executing main.py:
[nahmed#localhost ~]$ python main.py
Hello world
Suggestion 2:
Execute the file just above root dir i.e. one level up the program_name/ , in the following fashion:
[nahmed#localhost ~]$ python -m program_name.tests.thetest
Hell World
P.S. relative imports are for packages, not modules.
Just solved a similar problem with a lot of googling.
Here's two solutions without changing the existing file structor:
1
The way to import module from parent folder from ..src.Process.thefile.py import sth is called "relative import".
It's only supported when launching as a package from the top-level package. In your case, that is launching command line from the directory which contains program_name/ and type (for win environment)
python -m program_name.tests.thetest
or simply (useful for many pytest files):
python -m pytest
2
Otherwise -- when trying to run a script alone or from a non top-level package --
you could manually add directory to the PYTHONPATH at run time.
import sys
from os import path
sys.path.append(path.dirname(path.dirname(path.abspath(__file__))))
from src.Process.thefile import s
Try the first one first see if it's compatiable with the pytest framework. Otherwise the second one should always solve the problem.
Reference (How to fix "Attempted relative import in non-package" even with __init__.py)
When importing a file, Python only searches the current directory, the directory that the entry-point script is running from.
you can use sys.path to include different locations
import sys
sys.path.insert(0, '/path/to/application/app/folder')
import thefile
My package structure is:
main.py
mapp/
__init__.py
core/
__init__.py
tobeimported.py
test/
__init__.py
(test modules)
utils/
__init__.py
blasttofasta.py
The file blasttofasta.py is executed as script.
blasttofasta.py looks like:
import mapp.core.tobeimported
def somefunc():
pass
if __name__ == '__main__':
pass
But Exception occurs:
Traceback (most recent call last):
File "utils/blasttofasta.py", line 5, in <module>
import mapp.core.tobeimported
ImportError: No module named mapp.core.analyzers
How to import tobeimported module? I run the blasttofasta.py from top directory (where main.py is)
EDIT: Maybe better question is: How to get mapp package to the sys.path? Because script file only see its own directory but not the package directory.
Thank you
If I want to including blasttofasta.py or run it as script simultaneously mos important is to have directory containing mapp package in sys.path.
This worked for me:
Before importing mapp (or other module from this package) I wrote into blasttofasta.py:
import os
os.sys.path.append(os.path.dirname(os.path.realpath(__file__))+ '/../../')
This append mapp package path and I can run it as script. On the other side is no problem with is included in another package.
Follow the absolute structure to import.
To import blasttofasta.py in tobeimport.py
ToBeimport contents
from myapp.utils import blasttofasta
Your structure is good.
Two things need to happen:
Your map directory needs a __init__.py file.
You can simply do this (naively):
$ touch /path/to/map/__init__.py
/path/to/map needs to be in sys.path
Please read: http://docs.python.org/2/tutorial/modules.html for more details.
I am running into pathing issues with some scripts that I wrote to test parsers I've written. It would appear that Python (27 and 3) both do not act like IronPython when it comes to using the current working directory as part of sys.path. As a result my inner package pathings do not work.
This is my package layout.
MyPackage
__init__.py
_test
__init__.py
common.py
parsers
__init__.py
_test
my_test.py
I am attempting to call the scripts from within the MyPackage directory.
Command Line Statements:
python ./parsers/_test/my_test.py
ipy ./parsers/_test/my_test.py
The import statement located in my my_test.py file is.
from _test.common import TestClass
In the python scenario I get ONLY the MyPackage/parsers/_test directory appended to sys.path so as a result MyPackage cannot be found. In the IronPython scenario both the MyPackage/parsers/_test directory AND MyPackage/ is in sys.path. I could get the same bad reference if I called the test from within the _test directory (it would have no idea about MyPackage). Even if i consolidated the test files into the _test directory I would still have this pathing issue.
I should note, that I just tested and if i did something like
import sys
import os
sys.path.append(os.getcwd())
It will load correctly. But I would have to do this for every single file.
Is there anyway to fix this kind of pathing issue without doing the following.
A. Appending a relative pathing for sys.path in every file.
B. Appending PATH to include MyPackage (I do not want my package as part of the python "global" system pathing)
Thanks alot for any tips!
Two options spring to mind:
Use the environment variable PYTHONPATH and include .
Add it to your sys.path at the beginning of your program
import sys.path
sys.path.append('.')
If you need to import a relative path dynamically you can always do something like
import somemodule
import sys
dirname = os.path.dirname(os.path.abspath(somemodule.__file__))
sys.path.append(dirname)