I'm developing a python framework and want to do imports based on the top-level package (the project name). Users will use the framework by copying the entire framework and writing their own modules within.
My current structure looks like this:
myapp/
config.py
docs/
framework/
main.py
utils.py
file.py
lib/
some_module.py
unit_tests/
test_utils.py
I want to be able to use the following import in python files in lib and unit_tests in the following way:
from myapp.framework import utils
Is there a simple way to do this? I've tried doing sys.path.append() hacks but they don't really work. If there is a truly pythonic way to achieve this, I don't mind going extra lengths to get it working.
EDIT: Well I tried the sys.path.append() again and it actually works but it really is an in-elegant solution and I'd really like to hear if there's another way.
For short: You can't have two modules with the same name one inside the other. You have the myapp folder and the myapp.py file
That's because of the order of importing modules.
Inside the current directory
The current package
Global package folder
So, if you're trying to import myapp.lib.some_module in config.py the interpreter will first see that you have myapp.py in the same folder and literaly try to import lib.some_module from that file, which doesn't exist.
Would be something as trying to import myapp.myapp.lib.some_module from outside that module.
So, the best thing for you to do is to rename the myapp.py file
Related
I've got this structure bellow,
I want to import data-parser/service.py classes from api/server.py how can I do it?
- project
__init__.py
main.py
- api
__init__.py
server.py
- data-parser
__init__.py
service.py
I'm confused by your file structure as a whole but from what I can see you have one project folder, your main script in one folder in the project folder, and your modules in a seperate folder in the project folder. If this is the case, I don't understand to why you don't just have your main.py file in the same directory as a folder with your modules in it (I only mention in case you can do this instead as it makes this a whole lot easier) but here is the solution if you're certain you need your file structure to be the same as it is currently:
import sys
from os import path
from pathlib import Path
sys.path.insert(0, path.join(Path(__file__).parent.absolute().parent.absolute(), "app"))
import submodule1
This, essentially, gets the path of the file, it's parent (the folder it's in), and then the parent's parent (the project folder) to add it to sys.path (you can use modules in the paths of this list).
I'm almost 100% certain there's a better, more efficient way of doing this, but this is what I had off the top of my head due to the uniqueness of the question. I hope it's somewhat helpful.
I'm trying to import a class in a different directory to another file, but can't seem to get it to work. I know this question has been asked a lot and I have looked through multiple stackoverflow solutions and at https://docs.python.org/3/tutorial/modules.html#packages
1: Importing files from different folder
2: import python file in another directory failed
I want to try to just use the method containing just __init__.py file instead of doing an import sys
My directory structure is as follows:
django_vue/
__init__.py
devices/
__init__.py
models.py
lib/
__init__.py
my_file.py
I'm trying to import the class Device from /django_vue/devices/models.py to /django_vue/lib/my_file.py by:
from devices.models import Device
However when I do that I still get the error:
from devices.models import Device
ModuleNotFoundError: No module named 'devices'
I'm not sure what I'm dong wrong since I already have the __init__ file in both directories. Any help is appreciated. Also I'm running python 3.6.
This is the folder structure I'm working with.
.
└── django_vue
├── devices
│ └── models.py
└── lib
└── file.py
When you run
$ python file.py
python has no way of knowing what's outside the directory.
python can't go back and then into devices/ just like that.
The easiest way to solve this would be to add the folder devices/ to sys.path. When python imports a module, it searches for the module from sys.path. Adding the path to devices/ would make it available for imports.
Here are my files.
# models.py
Device = 'device'
# file.py
import sys
sys.path.append('..') # adds the parent dir (which is django-vue/) to path
# django-vue dir has devices/ so now this is available for imports
# importing this works now
from devices.models import Device
print(Device)
Output
django_vue/lib$ python3 file.py
device
Think about it your are inside my_file.py and import something called devices.
How can python know where the name devices has come from.
It won't search your entire Drive for that module/package
Relative Import
use a relative import instead. write from ..devices.models import Device. This is telling python to go up one directory to the parent directory and that's where it will find the devices package. Your lib module should now work as a module
If you however run the my_file.py package directly (as in python C:/django_vue/lib/my_file.py)
You will still get an error. Not the same error; the new error will be something like
ImportError: attempted relative import with no known parent package
This is happening because you are actually running my_file.py
If you think about it why would you want to run my_file.py by itself when it is clearly a part of a package. Maybe you are just testing to see if the import works when you use your package. The problem with this is that it makes it seem like your packages relative imports don't work even though this actually works.
Create a main.py in django_vue and write from lib import my_file. This will run your my_file.py and you will notice there is no error.
What's happening here
Have you heard of __package__?
if you put print(str(__package__)) in your my_file.py and run my_file.py directly you will see that it prints None.
However if you run main.py (that you just created) you will see that when It reaches my_file.py, __package__ will actually be defined to something.
Ahhh... you see now it all makes sense; The error you originally got said something about no known parent package. If __package__ is undefined that means there is no relative parent package because the file was obviously run directly instead of as part of a package.
Consider Absolute imports
you also might want to consider using absolute imports because if you are working on the package you might change it directory structure while developing. so you need to keep changing the import references on the affected files.
Although you can find IDE's with python extensions that automatically to this as you change your directory. I believe VS Code does this automatically.
Replace the __init__ files with __main__.
I am currently working in the following project, and running into some difficulty with imports. I previously came from a Ruby background before Python, so I suspect I'm just missing something.
-src
--project
---actions
----some .py files
---config
----some .py files
---db
----some .py files
-tests
--some .py files
-run.py
Some of the actions I'd like to do are:
import src/project/config/file.py from run.py
import between second level folder in project (ie. file in actions imports something from config)
import any file into a test
Would anyone have any advice on how to accomplish this?
So it's quite easy to reference something going deeper into the directory . For example, from run.py, you would be able to use something such as:
from src.project.config.file import foo
It's easy to access something in the same directory as well, say you're in src/file1.py trying to access src/file2.py, this would just be:
from file2 import foo
This works too. However if you were trying to import upward say from file.py to run.py and tried running something like this:
from ... import run.foo
You would get the following error:
ValueError: attempted relative import beyond top-level package
The problem is that Python is assuming that your top-level package is wherever the file being run is located. That's why you can import everything on the same level and deeper but can't get at anything above.
An easy hack to get past this is to just append the path to your file to your sys.path so:
import sys
sys.append('path/to/run')
from run import foo
So that's the answer to you question.
Based on the way your project is setup though it seems like you want to make a module and that's a bit more hands on, I suggest reading the documentation it's pretty good.
I am developing python functions in different .py files (example DisplayTools.py, CollectionTools.py...) in order to import them as tools in a more general file Start.py. It works fine if all the files are in the same directory. I can say in Start.py "import DisplayTools" ...
But how to organize those in a more project way and more user-friendly (where they only have to work on the Start.py file). For example having such an file organization :
Project/
Start.py
Tools/
DisplayTools.py
CollectionTools.py
I've read the use of __init__ files but how they works, where to put those files and what are they containing ?
Please if you have some help to give me in that way to organize my project.
Many thanks
I'd refactor your code organization just a bit and give your toplevel directory a more descriptive name. Today, I pick happy_bananas. So let's say you organize your files like this:
happy_bananas
start.py
DisplayTools.py
CollectionTools.py
then all you need to do is add an empty __init__.py file and you can use it just like any other package, e.g.:
happy_bananas
__init__.py
start.py
DisplayTools.py
CollectionTools.py
And now if you can do:
from happy_bananas import DisplayTools
just like you would have before.
Now, to get this into your system, you need to package it and use an install script. You can do this using distutils or setuptools but perhaps the simplest existing description of how to do this is in Zed Shaw's Learn Python The Hard Way Exercise 46. You really can just copy/paste those files as described there and end up with a directory structure like this:
happy_bananas
setup.py
tests
test_happy_bananas.py
happy_bananas
__init__.py
start.py
DisplayTools.py
.
.
Then, when you have your setup script written, you can go into your folder and run python setup.py install (or python setup.py develop) and be able to import happy_bananas in any file.
On a separate note, the naming convention in python is to use snakecase for file and function names. So instead of DisplayTools.py, it would be better to rename it display_tools.py. CamelCase is usually reserved for class names only.
Well for starters i would simply change my files to have a set of functions and some main code since the files could also be executed.
For example:
if __name__ == "__main__":
dosomething()
Then in the main, you simply import the other scripts and you can use the functions used there without actually running the script.
My projects are generally structured like this:
projectname/
__init__.py
python/
mymodule.py
other_stuff/
more_stuff/
where __init__.py contains the following code
import os
mypath = os.path.dirname(os.path.realpath(os.path.abspath(__file__)))
__path__ = [mypath, mypath+"/python"]
This "skips" the python directory when importing to allow python code in the form from projectname import mymodule rather than from projectname.python import mymodule.
This appears to break pylint however, being unable to import any modules in the project despite $PYTHONPATH being set correctly. Creating a softlink projectname -> python in the projectname fixes things but isn't a suitable solution.
Any suggestions on how to fix this without altering the directory structure?
I think you're kind of stuck. Pylint doesn't process your __init__.py file so unless you can find another way of getting that information into pylint, I don't think it's going to work. Good luck.