My projects are generally structured like this:
projectname/
__init__.py
python/
mymodule.py
other_stuff/
more_stuff/
where __init__.py contains the following code
import os
mypath = os.path.dirname(os.path.realpath(os.path.abspath(__file__)))
__path__ = [mypath, mypath+"/python"]
This "skips" the python directory when importing to allow python code in the form from projectname import mymodule rather than from projectname.python import mymodule.
This appears to break pylint however, being unable to import any modules in the project despite $PYTHONPATH being set correctly. Creating a softlink projectname -> python in the projectname fixes things but isn't a suitable solution.
Any suggestions on how to fix this without altering the directory structure?
I think you're kind of stuck. Pylint doesn't process your __init__.py file so unless you can find another way of getting that information into pylint, I don't think it's going to work. Good luck.
Related
I'm newish to imports so it's possible my approach is patchy but i'm really struggling to piece together how this is supposed to be done.
So I have a github repo I am working on that currently looks something like this:
Github-repo-name
/src
/repo-name
__init__.py
/packages
__init__.py
mypackage1/
__init__.py
p1_module1.py
mypackage2/
__init__.py
p2_module1.py
main.py
/scripts
do_something.py
/datasets
I'm trying to import mypackage1 into do_something.py in order to generate some data to put in datasets/ but was having lots of trouble when I ran the do_something.py script in the src directory.
I had this line in do_something.py:
import repo-name.packages.mypackage1
and was hitting: ModuleNotFoundError: No module named repo-name
I tried relative imports to no more success
from ..repo-name.packages import mypackage1
ImportError: attempted relative import with no known parent package
I couldn't make head or tail of this issue so I instead tried adding my packages to the path in do_something.py
file_path = pathlib.Path(__file__)
path = file_path.resolve().parents[2]
sys.path.append(str(path))
import repo-name.packages.mypackage1
Success!
Except now i'm wondering if this really what I should be doing and if this isn't a bit hacky. The idea of needing to add the path at runtime seems a bit messy so i'd rather have it all the time. I've read I can use PYTHONPATH for this purpose which would be great if it were just me using this repo, but I intend for this to be shared and reproduceable. I thought about maybe writing a shell script to set the PYTHONPATH but i'm not sure that's much better.
So my question is what is the right/better/a good way of doing what i'm trying to do?
Oliver.
I don't know what is the best way, but I'm trying to show you how our team works.
When developing,
i) register PYTHONPATH in ~/.bashrc
ii) sys.path.append(path/to/root/repo) on the top of the code
Our team usually does it as an initial way.
To distinguish whether they receive the authority of repo or not, you can check the below.
# Suppose you created a setup.py(to make `.whl` file) on `repo-name`.
if __name__ == '__main__':
from repo-name.packages.mypackage1 import *
else:
from ..repo-name.packages.mypackage1 import *
I'm trying to import a class in a different directory to another file, but can't seem to get it to work. I know this question has been asked a lot and I have looked through multiple stackoverflow solutions and at https://docs.python.org/3/tutorial/modules.html#packages
1: Importing files from different folder
2: import python file in another directory failed
I want to try to just use the method containing just __init__.py file instead of doing an import sys
My directory structure is as follows:
django_vue/
__init__.py
devices/
__init__.py
models.py
lib/
__init__.py
my_file.py
I'm trying to import the class Device from /django_vue/devices/models.py to /django_vue/lib/my_file.py by:
from devices.models import Device
However when I do that I still get the error:
from devices.models import Device
ModuleNotFoundError: No module named 'devices'
I'm not sure what I'm dong wrong since I already have the __init__ file in both directories. Any help is appreciated. Also I'm running python 3.6.
This is the folder structure I'm working with.
.
└── django_vue
├── devices
│ └── models.py
└── lib
└── file.py
When you run
$ python file.py
python has no way of knowing what's outside the directory.
python can't go back and then into devices/ just like that.
The easiest way to solve this would be to add the folder devices/ to sys.path. When python imports a module, it searches for the module from sys.path. Adding the path to devices/ would make it available for imports.
Here are my files.
# models.py
Device = 'device'
# file.py
import sys
sys.path.append('..') # adds the parent dir (which is django-vue/) to path
# django-vue dir has devices/ so now this is available for imports
# importing this works now
from devices.models import Device
print(Device)
Output
django_vue/lib$ python3 file.py
device
Think about it your are inside my_file.py and import something called devices.
How can python know where the name devices has come from.
It won't search your entire Drive for that module/package
Relative Import
use a relative import instead. write from ..devices.models import Device. This is telling python to go up one directory to the parent directory and that's where it will find the devices package. Your lib module should now work as a module
If you however run the my_file.py package directly (as in python C:/django_vue/lib/my_file.py)
You will still get an error. Not the same error; the new error will be something like
ImportError: attempted relative import with no known parent package
This is happening because you are actually running my_file.py
If you think about it why would you want to run my_file.py by itself when it is clearly a part of a package. Maybe you are just testing to see if the import works when you use your package. The problem with this is that it makes it seem like your packages relative imports don't work even though this actually works.
Create a main.py in django_vue and write from lib import my_file. This will run your my_file.py and you will notice there is no error.
What's happening here
Have you heard of __package__?
if you put print(str(__package__)) in your my_file.py and run my_file.py directly you will see that it prints None.
However if you run main.py (that you just created) you will see that when It reaches my_file.py, __package__ will actually be defined to something.
Ahhh... you see now it all makes sense; The error you originally got said something about no known parent package. If __package__ is undefined that means there is no relative parent package because the file was obviously run directly instead of as part of a package.
Consider Absolute imports
you also might want to consider using absolute imports because if you are working on the package you might change it directory structure while developing. so you need to keep changing the import references on the affected files.
Although you can find IDE's with python extensions that automatically to this as you change your directory. I believe VS Code does this automatically.
Replace the __init__ files with __main__.
I'm developing a system in Python that includes a calculation engine and a front end. I've split them up into two projects as the calculation engine can be used for other front ends as well.
I'm using Eclipse and PyDev. Everything works perfectly in Eclipse/PyDev, but as soon as I try to run it outside of PyDev (from command line) I get importing errors. I've done quite a bit of research to find the problem, but I just don't see a solution that works nicely. I believe that PyDev modifies the Python path.
In my project layout below I have two packages (package1 and tests) within one project (Calculations). I can't seem to import anything from package1 in tests. I also have another project (Frontend). Here I also can't import anything from package1.
What I want to understand is the proper way of calling my script/tests files from the command line? Both for two separate projects and two packages in the same project. I assume it would be similar to how PyDev does it. So far I think I have the following options:
Create python code to append to sys.path (seems hacky/not good practice)
Modify the PYTHONPATH when I call the test_some_calc.py like this: PYTHONPATH= python test_some_calc.py (I think this is how PyDev does it, but it seems a bit long - there must be a simpler way?
Make a install package (eventually I might go this method, but not yet.)
I have the following project layout.
CodeSolution/
Calculations/
package1/
__init__.py
subpackage/
__init__.py
some_calc.py
subpackage2/
__init__.py
another_calc.py
tests/
__init__.py
subpackage/
__init__.py
test_some_calc.py # Unable to import from package1
subpackage2/
__init__.py
test_another_calc.py # Unable to import from package1
Frontend/
some_script.py # Unable to import from package1
Comments on my project layout will also be appreciated.
A clean, quick and modular way to include certain python from anywhere on your system is to make a file named mymodule.pth and put it inside the path of site-packages
mymodule.pth should have the path of your project. Project folder must have an __init__.py file.
for example put:
for Linux:
/home/user/myproject
inside
/usr/lib/python2.7/site-packages/mymodule.pth
or
for Windows
C:\\Users\myUsername\My Documents\myproject
inside
C:\PythonXY\Lib\site-packages\mymodule.pth
I wrote a script to load PYTHONPATHs from PyDev's project properties. It's allow you to run your code from console without problems like "ModuleNotFoundError: No module named ...".
import sys
from xml.dom import minidom
import os
print(sys.path)
def loadPathsFromPyDev():
sys_path = sys.path[0]
# Load XML
xmldoc = minidom.parse(sys_path+'\.pydevproject')
# Get paths
xmlpaths = xmldoc.getElementsByTagName('path')
# Get paths' values
paths = list()
for xmlpath in xmlpaths:
paths.append(xmlpath.firstChild.data)
# set path variable
for path in paths:
# Set backslashes to forwardslashes
path = os.path.normpath(path)
# Set string's sys_path
path = path.replace("\${PROJECT_DIR_NAME}", sys_path)
if path not in sys.path:
# Add to system path
sys.path.insert(1,path)
loadPathsFromPyDev()
print(sys.path)
I hope it will help You :)
So I am working on a Python project that was here before me in an SVN repo. When I first pulled it, the structure was a bit odd due to the fact that it was similar to:
Proj\
src\
tags\
trunk\
And then everything is inside src\ are the python module files except src\ turns out to just be a logical folder with no overall package inside. There isn't a __init__.py in the project anywhere. So I want to restructure it at least so I can use relative imports through my project. I also want to set it up so it looks more like this.
Proj\
src\
model\
controller\
view\
test\
tags\
trunk\
However, I tried setting this up and no matter what I seem to do, it cannot resolve the relative import the moment I have to traverse packages. I placed a __init__.py file in each level package including one inside the src\ folder with all of them having __all__ defined. However, when I try to make a unit test in my test\ package and do an import saying:
from ..model.foo import Foo
to attempt to import the Foo class from module foo.py located inside of the model package, it doesn't resolve. Just in case it was a problem specifically with unit tests, I also tried this with a module in the controller package that was dependent on a class in the model package and vice versa. None of them worked. How do I resolve this?
Have you added the root folder to your system path?
import sys
sys.path.append(<place the Proj dir here>)
then you could import as follows:
from src.model.somefile import Something
If you don't know the absolute path for Proj, you can always use combinations such as
os.path.dirname(os.getcwd())
I'm developing a python framework and want to do imports based on the top-level package (the project name). Users will use the framework by copying the entire framework and writing their own modules within.
My current structure looks like this:
myapp/
config.py
docs/
framework/
main.py
utils.py
file.py
lib/
some_module.py
unit_tests/
test_utils.py
I want to be able to use the following import in python files in lib and unit_tests in the following way:
from myapp.framework import utils
Is there a simple way to do this? I've tried doing sys.path.append() hacks but they don't really work. If there is a truly pythonic way to achieve this, I don't mind going extra lengths to get it working.
EDIT: Well I tried the sys.path.append() again and it actually works but it really is an in-elegant solution and I'd really like to hear if there's another way.
For short: You can't have two modules with the same name one inside the other. You have the myapp folder and the myapp.py file
That's because of the order of importing modules.
Inside the current directory
The current package
Global package folder
So, if you're trying to import myapp.lib.some_module in config.py the interpreter will first see that you have myapp.py in the same folder and literaly try to import lib.some_module from that file, which doesn't exist.
Would be something as trying to import myapp.myapp.lib.some_module from outside that module.
So, the best thing for you to do is to rename the myapp.py file