I'm having a hard time understanding how module importing works in Python (I've never done it in any other language before either).
Let's say I have:
myapp/__init__.py
myapp/myapp/myapp.py
myapp/myapp/SomeObject.py
myapp/tests/TestCase.py
Now I'm trying to get something like this:
myapp.py
===================
from myapp import SomeObject
# stuff ...
TestCase.py
===================
from myapp import SomeObject
# some tests on SomeObject
However, I'm definitely doing something wrong as Python can't see that myapp is a module:
ImportError: No module named myapp
In your particular case it looks like you're trying to import SomeObject from the myapp.py and TestCase.py scripts. From myapp.py, do
import SomeObject
since it is in the same folder. For TestCase.py, do
from ..myapp import SomeObject
However, this will work only if you are importing TestCase from the package. If you want to directly run python TestCase.py, you would have to mess with your path. This can be done within Python:
import sys
sys.path.append("..")
from myapp import SomeObject
though that is generally not recommended.
In general, if you want other people to use your Python package, you should use distutils to create a setup script. That way, anyone can install your package easily using a command like python setup.py install and it will be available everywhere on their machine. If you're serious about the package, you could even add it to the Python Package Index, PyPI.
The function import looks for files into your PYTHONPATH env. variable and your local directory. So you can either put all your files in the same directory, or export the path typing into a terminal::
export PYTHONPATH="$PYTHONPATH:/path_to_myapp/myapp/myapp/"
exporting path is a good way. Another way is to add a .pth to your site-packages location.
On my mac my python keeps site-packages in /Library/Python shown below
/Library/Python/2.7/site-packages
I created a file called awesome.pth at /Library/Python/2.7/site-packages/awesome.pth and in the file put the following path that references my awesome modules
/opt/awesome/custom_python_modules
You can try
from myapp.myapp import SomeObject
because your project name is the same as the myapp.py which makes it search the project document first
You need to have
__init__.py
in all the folders that have code you need to interact with.
You also need to specify the top folder name of your project in every import even if the file you tried to import is at the same level.
In your first myapp directory ,u can add a setup.py file and add two python code in setup.py
from setuptools import setup
setup(name='myapp')
in your first myapp directory in commandline , use pip install -e . to install the package
pip install on Windows 10 defaults to installing in 'Program Files/PythonXX/Lib/site-packages' which is a directory that requires administrative privileges. So I fixed my issue by running pip install as Administrator (you have to open command prompt as administrator even if you are logged in with an admin account). Also, it is safer to call pip from python.
e.g.
python -m pip install <package-name>
instead of
pip install <package-name>
In my case it was Windows vs Python surprise, despite Windows filenames are not case sensitive, Python import is. So if you have Stuff.py file you need to import this name as-is.
let's say i write a module
import os
my_home_dir=os.environ['HOME'] // in windows 'HOMEPATH'
file_abs_path=os.path.join(my_home_dir,"my_module.py")
with open(file_abs_path,"w") as f:
f.write("print('I am loaded successfully')")
import importlib
importlib.util.find_spec('my_module') ==> cannot find
we have to tell python where to look for the module. we have to add our path to the sys.path
import sys
sys.path.append(file_abs_path)
now importlib.util.find_spec('my_module') returns:
ModuleSpec(name='my_module', loader=<_frozen_importlib_external.SourceFileLoader object at 0x7fa40143e8e0>, origin='/Users/name/my_module.py')
we created our module, we informed python its path, now we should be able to import it
import my_module
//I am loaded successfully
This worked for me:
from .myapp import SomeObject
The . signifies that it will search any local modules from the parent module.
Short Answer:
python -m ParentPackage.Submodule
Executing the required file via module flag worked for me. Lets say we got a typical directory structure as below:
my_project:
| Core
->myScript.py
| Utils
->helpers.py
configs.py
Now if you want to run a file inside a directory, that has imports from other modules, all you need to do is like below:
python -m Core.myscript
PS: You gotta use dot notation to refer the submodules(Files/scripts you want to execute). Also I used python3.9+. So I didnt require neither any init.py nor any sys path append statements.
Hope that helps! Happy Coding!
If you use Anaconda you can do:
conda develop /Path/To/Your/Modules
from the Shell and it will write your path into a conda.pth file into the standard directory for 3rd party modules (site-packages in my case).
If you are using the IPython Console, make sure your IDE (e.g., spyder) is pointing to the right working directory (i.e., your project folder)
Besides the suggested solutions like the accepted answer, I had the same problem in Pycharm, and I didn't want to modify imports like the relative addressing suggested above.
I finally found out that if I mark my src/ (root directory of my python codes) as the source in Interpreter settings, the issue will be resolved.
Related
This question already has answers here:
Relative imports for the billionth time
(12 answers)
Closed last year.
So basicly I want to acces a created module from a folder on the parent directory from the folder I am
Currently I'm at Twitter.py and I want to access /utils/magic_eden.py
On the __init__.py file I've got:
from .magic_eden import MagicEden
from .open_sea import OpenSea
from .tools import Tools
Now inside the Twitter.py file im trying to import these classes of the module by doing:
from utils import MagicEden
But im getting ModuleNotFoundError: No module named 'utils'.
I've tried so many sort of things but none worked. What should I do?
(btw if I execute the __init__.py file I get ImportError: attempted relative import with no known parent package)
From what I see, it seems that utils is a utility package logically separate from the code in Twitter.py. Given this, what you want is to cause your utils package to be on your Python search path (sys.path) so that you can import that package as a separate entity (no relative paths). If you don't want to configure your environment to place utils on the Python search path, you can do it in your Twitter.py file. Here's how that looks:
import os
import sys
here = os.path.dirname(__file__)
sys.path.append(os.path.join(here, '..'))
import utils
utils.MagicEden.hello()
utils.OpenSea.hello()
utils.Tools.hello()
The first line of code computes the full path to the directory containing Twitter.py. The second line then computes the path to the parent directory of that directory and adds that path to sys.path, the Python search path. After doing this, import utils will work, giving you access to that package and everything imported in that package's __init__.py file.
I created three small files for magic_eden.py, open_sea.py, and tools.py, each containing something that looks like this:
class MagicEden:
#staticmethod
def hello():
print("Hello from MagicEden!")
I can then run Twitter.py, and I get the following result with no additional configuration:
Hello from MagicEden!
Hello from OpenSea!
Hello from Tools!
There's nothing wrong with using the above solution during early development. But you will likely at some point want to remove the code that is hacking sys.path and instead install your module in a more official way. There's a way to do this from the start so that you never have to change code on either side when you want to install your module in the official way...
What I do in situations like this is create a setup.py file in my package that lets me build it as an installable package. There are many docs and tutorials on the net that explain how to do this. Here's just one of them: https://python-packaging-tutorial.readthedocs.io/en/latest/setup_py.html
Once you've done this, what you can do is install your package in "development mode" (pip install -e <package file>). What this does is install the package so that your system can find it (adds it to sys.path as a package), but installs links to the original sources rather than installing copies of them. In this mode, when you make changes to that package's source files, the changes take immediate effect in the installed module. So now you have the best of both worlds. Your package is installed in the official way such that you don't need to do anything special in the code that imports that package (no need to modify sys.path), and yet you can still make changes directly to the sources for that package and not have to keep reinstalling the package to see those changes take affect.
When you're done messing with a package in this way, you can reinstall it in the regular mode (pip install <package file>) and isolate changes to the sources for that package from uses of the package elsewhere on your system. At this point, you need to rebuild and reinstall the package to see any changes you've made to its sources.
You're attempting to import from a "sibling" folder.
To do this, you'll need to add __init__.py to your Twitter/ and parent src/ folders.
Then you'll need to import the path as Twitter.py doesn't know about any parent structure in this setup. You can then import from the utils module which is in the path.
import sys
sys.path.append('..')
from utils.magic_eden import MagicEden
assuming MagicEden is a class in your magic_eden.py file.
Let's say my project structure looks like this:
app/
main.py
modules/
__init__.py
validation.py
configuration.py
modules package contains reusable code.
main.py executes main application logic.
When I try this in main.py
from modules import validation
I get an error which says that import inside of the validation failed. Validation tries to import configuration and I get 'no module named configuration'
I am using Anaconda distribution on windows.
What is the best way of handling PYTHONPATH during development of the package ?
Is there a way to utilize virtualenv (or conda env) in order to get package, that is in development,on the PYTHONPATH without changing sys.path from the code ?
What is the preferred practice when developing a package ?
I've also tried adding modules (folder) package to the lib/site-packages but it still didn't work.
Change your import in validation.py to:
from . import configuration
This is needed for Python 3 but also works with Python 2.
What is the correct way to fix this ImportError error?
I have the following directory structure:
/home/bodacydo
/home/bodacydo/work
/home/bodacydo/work/project
/home/bodacydo/work/project/programs
/home/bodacydo/work/project/foo
And I am in the directory
/home/bodacydo/work/project
Now if I type
python ./programs/my_python_program.py
I instantly get
ImportError: No module named foo.tasks
The ./programs/my_python_program.py contains the following line:
from foo.tasks import my_function
I can't understand why python won't find ./foo/tasks.py - it's there.
If I do it from the Python shell, then it works:
python
>>> from foo.tasks import my_function
It only doesn't work if I call it via python ./programs/my_python_program.py script.
Python does not add the current directory to sys.path, but rather the directory that the script is in. Add /home/bodacydo/work/project to either sys.path or $PYTHONPATH.
Do you have a file called __init__.py in the foo directory? If not then python won't recognise foo as a python package.
See the section on packages in the python tutorial for more information.
A better fix than setting PYTHONPATH is to use python -m module.path
This will correctly set sys.path[0] and is a more reliable way to execute modules.
I have a quick writeup about this problem, as other answerers have mentioned the reason for this is python path/to/file.py puts path/to on the beginning of the PYTHONPATH (sys.path).
Here is a step-by-step solution:
Add a script called run.py in /home/bodacydo/work/project and edit it like this:
import programs.my_python_program
programs.my_python_program.main()
(replace main() with your equivalent method in my_python_program.)
Go to /home/bodacydo/work/project
Run run.py
Explanation:
Since python appends to PYTHONPATH the path of the script from which it runs, running run.py will append /home/bodacydo/work/project. And voilà, import foo.tasks will be found.
Example solution for adding the library to your PYTHONPATH.
Add the following line into your ~/.bashrc or just run it directly:
export PYTHONPATH="$PYTHONPATH:$HOME/.python"
Then link your required library into your ~/.python folder, e.g.
ln -s /home/user/work/project/foo ~/.python/
In my mind I have to consider that the foo folder is a stand-alone library. I might want to consider moving it to the Lib\site-packages folder within a python installation. I might want to consider adding a foo.pth file there.
I know it's a library since the ./programs/my_python_program.py contains the following line:
from foo.tasks import my_function
So it doesn't matter that ./programs is a sibling folder to ./foo. It's the fact that my_python_program.py is run as a script like this:
python ./programs/my_python_program.py
If you have this problem when using an instaled version, when using setup.py, make sure your module is included inside packages
setup(name='Your program',
version='0.7.0',
description='Your desccription',
packages=['foo', 'foo.bar'], # add `foo.bar` here
I have some problems in structuring my python project. Currently it is a bunch of files in the same folder. I have tried to structure it like
proj/
__init__.py
foo.py
...
bar/
__init__.py
foobar.py
...
tests/
foo_test.py
foobar_test.py
...
The problem is that I'm not able, from inner directories, to import modules from the outer directories. This is particularly annoying with tests.
I have read PEP 328 about relative imports and PEP 366 about relative imports from the main module. But both these methods require the base package to be in my PYTHONPATH. Indeed I obtain the following error
ValueError: Attempted relative import in non-package.
So I added the following boilerplate code on top of the test files
import os, sys
sys.path.append(os.path.join(os.getcwd(), os.path.pardir))
Still I get the same error. What is the correct way to
structure a package, complete with tests, and
add the base directory to the path to allow imports?
EDIT As requested in the comment, I add an example import that fails (in the file foo_test.py)
import os, sys
sys.path.append(os.path.join(os.getcwd(), os.path.pardir))
from ..foo import Foo
When you use the -m switch to run code, the current directory is added to sys.path. So the easiest way to run your tests is from the parent directory of proj, using the command:
python -m proj.tests.foo_test
To make that work, you will need to include an __init__.py file in your tests directory so that the tests are correctly recognised as part of the package.
I like to import modules using the full proj.NAME package prefix whenever possible. This is the approach the Google Python styleguide recommends.
One option to allow you to keep your package structure, use full package paths, and still move forward with development would be to use a virtualenv and put your project in develop mode. Your project's setup.py will need to use setuptools instead of distutils, to get the develop command.
This will let you avoid the sys.path.append stuff above:
% virtualenv ~/virt
% . ~/virt/bin/activate
(virt)~% cd ~/myproject
(virt)~/myproject% python setup.py develop
(virt)~/myproject% python tests/foo_test.py
Where foo_test.py uses:
from proj.foo import Foo
Now when you run python from within your virtualenv your PYTHONPATH will point to all of the packages in your project. You can create a shorter shell alias to enter your virtualenv without having to type . ~/virt/bin/activate every time.
I'm developing/testing a package in my local directory. I want to import it in the interpreter (v2.5), but sys.path does not include the current directory. Right now I type in sys.path.insert(0,'.'). Is there a better way?
Also,
from . import mypackage
fails with this error:
ValueError: Attempted relative import in non-package
You can use relative imports only from in a module that was in turn imported as part of a package -- your script or interactive interpreter wasn't, so of course from . import (which means "import from the same package I got imported from") doesn't work. import mypackage will be fine once you ensure the parent directory of mypackage is in sys.path (how you managed to get your current directory away from sys.path I don't know -- do you have something strange in site.py, or...?)
To get your current directory back into sys.path there is in fact no better way than putting it there.
Keep it simple:
try:
from . import mymodule # "myapp" case
except:
import mymodule # "__main__" case
See the documentation for sys.path:
http://docs.python.org/library/sys.html#sys.path
To quote:
If the script directory is not available (e.g. if the interpreter is invoked interactively or if the script is read from standard input), path[0] is the empty string, which directs Python to search modules in the current directory first.
So, there's no need to monkey with sys.path if you're starting the python interpreter from the directory containing your module.
Also, to import your package, just do:
import mypackage
Since the directory containing the package is already in sys.path, it should work fine.
If you want to run an unmodified python script so it imports libraries from a specific local directory you can set the PYTHONPATH environment variable - e.g. in bash:
export PYTHONPATH=/home/user/my_libs
python myscript.py
If you just want it to import from the current working directory use the . notation:
export PYTHONPATH=.
python myscript.py
Inside a package if there is setup.py, then better to install it
pip install -e .
A simple way to make it work is to run your script from the parent directory using python's -m flag, e.g. python -m packagename.scriptname. Obviously in this situation you need an __init__.py file to turn your directory into a package.
Using sys.path should include current directory already.
Try:
import .
or:
from . import sth
however it may be not a good practice, so why not just use:
import mypackage
A bit late to the party, but this is what worked for me:
>>> import sys
>>> sys.path.insert(0, '')
Apparently, if there is an empty string, Python knows that it should look in the current directory. I did not have the empty string in sys.path, which caused this error.
Speaking for python3.. I wanted to use an improved version of a library that's installed in my environment. There are some extra print statements it makes to show that it and not the original lib is being used.
I placed the lib's folder next to the python script. Ran the script.. it ran with the local lib with the modifications.
Removed the folder and ran it again - this time it ran with the installed lib.
So, solution was simple : place the lib's folder (with same name as in your import statement) in your project folder. That does the job, at least at my end.
This is on a standard Linux Mint 20.04 system, with a python 3.8 virutal environment activated (so "(py3.8)" appears in my terminal when I'm in the virtual env)
You can import package_name if the package is a module: this needs you have init.py under the package and things that you want to use are needed to import in the init.py
Or if you want to import class under the package, you can use from package_name import class_name