Let's say that we have python packages package_a and package_b. As a user, I want to be able to say:
from package_a.package_b import some_functionality
My question is: is there a way to support this without literally copy-pasting the code of package_b to package_a?
EDIT 1:
To clarify, I have shadow naming (my python package is named the same way as my folder), and I am wondering if there was a way to make the python package available somewhere else.
And yes, I'm looking into ways how to rename the package/folder in a way it still makes sense in the domain.
This is an abuse of Python's module system, and I would urge you to reconsider this design, but if you really must, you can do it by messing with sys.modules:
In package_a/__init__.py:
import sys
import package_b
sys.modules[__name__ + '.package_b'] = package_b
You can see something like this in the standard lib, os.path is either thentpath or posixpath module depending on platform, neither of which is actually part of os (it couldn't be, os is a module, not a package).
https://github.com/python/cpython/blob/master/Lib/os.py
sys.modules['os.path'] = path
With setuptools you can do something like this:
Project
├── setup.py
└── src
├── package_a
│ └── __init__.py
└── package_b
└── __init__.py
setup.py
#!/usr/bin/env python3
import setuptools
setuptools.setup(
name='Project',
version='0.0.7',
packages=[
'package_a',
'package_b',
'package_a.package_b',
],
package_dir={
'package_a': 'src/package_a',
'package_b': 'src/package_b',
'package_a.package_b': 'src/package_b',
},
)
This will effectively install two copies of package_b, one as a top level package and another one as a sub-package of package_a. Which could be a problem since if for example you have a global variables in package_b then it won't have the same value as the same global variable in package_a.package_b. But it might as well be a positive side effect, depending on the use cases.
Related
Imagine a package with this structure:
└── main_package
├── __init__.py
├── subpackage1
│ ├── __init__.py
│ └── module1.py
└── subpackage2
├── __init__.py
└── module2.py
What is the best way of importing a function in module1 to module2?
Should the __init__.py files contain something?
The best solution I have found is to include in module2
import sys
sys.path.append('./')
from main_package.subpackage1.module1 import fun1
And when running module2 from the directory in which main_package is it works.
But I feel like this does not take advantage of the fact that they are packages. And also it would be nice to be able to run this from wherever I want if possible.
The most reliable is to always use absolute imports. In this case for module 1:
from MainPackage.module2 import some_function
In module1_1 that would be:
from MainPackage.Subpackage2.module2_1 import some_function
Note that the package naming convention is to have all lower case letters and in between words, an underscore (_) can be used. In your specific example, MainPackage would become main_package and so on.
For your first question:
What is the best way of importing a function in module1 to module2?
You may either use relative or absolute imports. Personally, I prefer to use relative imports only within the context of a package I'm working on - to me this highlights when internal packages are related. In addition (and this is environment specific), certain environments will not recognize main_package until you install the package (i.e. with pip install -e . during development).
Absolute import in module2: from main_package.subpackage1.module1 import my_function
Relative import in module2: from ..subpackage1.module1 import my_function
For your second question:
Should the init.py files contain something?
They can, and that may provide you with shortcuts for imports. It is common practice to expose general and/or first-class functions/classes on the package level, which essentially is just importing them in __init__.py.
For example, in subpackage1\__init__.py you could:
from .module1 import my_function
And then in subpackage2\module2.py, you could simply use from ..subpackage1 import my_function (relative import) or from main_package.subpackage1 import my_function.
You may also be interested in the topic from the original Python documentation - import/submodules, Importing * From a Package (for additional reading on the import system, followed by intra-package imports), PEP328 (the rationale for relative imports) and Guido's thoughts regardings relative imports.
In Maple importing a module in the main file of your module will automatically share it with the other files and submodules of your module. You can also call the functions that you defined in different files of the same module as far as you have linked them by just including the address to the files of the module in the main file. So you only type the addresses in one file and you do not need to write them in other files of your module. In my opinion that makes life very easier.
In Python I couldn't do the same as in Maple and here is what I do in my current practice. Let me explain on a toy example. There is a folder containing a python file main.py and the package structure as you showed in your question. The content of each file is as follows.
main.py
import main_package
main_package.fun2()
The __init__.py of main_package.
from .subpackage1.module1 import fun1
from .subpackage2.module2 import fun2
The two init files of subpackages are empty.
module1.py
import sympy
x = sympy.Symbol('x')
def fun1(polynomial):
return(sympy.degree(polynomial, gen = x))
module2.py
import sympy
from ..subpackage1.module1 import fun1
def fun2():
polynomial = sympy.sympify(input("Type a univariate polynomial with the variable x and integer coefficients:"))
print(f"Degree of your polynomial in x is {fun1(polynomial)}.")
Now, instead of running the __init__.py of main_package, I run main.py in my VSCode. Note that my main.py file is next to main_package folder, not inside of it.
Again, I think it is sad that I have to write import sympy in each of the two files module1.py and module2.py.
I am very interested if someone someday shows a way so that one can write import sympy and x = sympy.Symbol('x') only in the top parent __init__.py and avoid writing it in the two files module1.py and module2.py.
For reference, this is the opposite of this question. That question asks how to get pytest to use your local module. I want to avoid pytest using the local module.
I need to test my module's installation procedure. We should all be testing our modules' installation procedures. Therefore, I want my test suite to pretend like it's any other python module trying to import my module, which I have installed (whether or not it's up to date with my latest edits and/or an editable install is my business).
My project layout looks like this:
.
├── my_package
│ ├── __init__.py
│ └── my_module.py
├── setup.py
├── pyproject.toml
└── tests
├── conftest.py
├── __init__.py
└── test_my_package.py
If I
pip install .
cd tests
python -c "import my_package"
it all works. However, if I
pip install .
pytest
it does not. This is because pytest automatically adds the calling directory to the PYTHONPATH, making it impossible to test that pip install has worked. I want it to not do that.
I need to do this because I am using setuptools-scm (which has different behaviour in editable and non-editable installs) and setuptools.find_packages, which makes it easy to ignore subpackages. However, to reiterate, my issue is with pytest's discovery, not with the use of these two utilities.
See docs for pytest import mechanisms and sys.path/PYTHONPATH, which describe three import modes, which control this behavior. For instance, try this:
$ pytest --import-mode=importlib
which uses importlib to import test modules, rather than manipulating sys.path or PYTHONPATH.
A workaround is to manually edit the PYTHONPATH by changing tests/conftest.py to include
import sys
sys.path.pop(0)
before the first time my_module is imported, but it's not pretty and makes the assumption about where in the PYTHONPATH that item is going to show up. Of course, more code could be added to check explicitly, but that's really ugly:
import sys
from pathlib import Path
project_dir = str(Path(__file__).resolve().parent.parent)
sys.path = [p for p in sys.path if not p.startswith(project_dir)]
I have the following directory structure:
package
├── __init__.py
└── core.py
where the contents of core.py are:
def my_super_cool_func():
crufty_helper_func()
pass
def crufty_helper_func():
pass
and the contents of __init__.py are:
from .core import my_super_cool_func
Now, if I go into the Python REPL, I can import package. Moreover, I can do package.my_super_cool_func() and I can't do package.crufty_helper_func() (as expected). But what I can also do is package.core.crufty_helper_func(), which to me, is undesirable.
Is there any way of excluding modules in a Python package? Ideally, in the setup above, I'd prefer for the core module not to be visible in the package package.
do use __all__in __init__.py as
__all__ = ['my_super_cool_func']
then you can use it as from package import my_super_cool_func or
import package
x= package.my_super_cool_func
learn __all__
I am designing a Python package. Please see below the project structure-
android_py
├── README.md
├── setup.py
└── android_py
├── __init__.py
├── options.py
└── android.py
Below is the content of setup.py-
from setuptools import setup, find_packages
setup(name='android_py',
version='0.1',
description='The description goes here',
url='http://github.com/example_user/android_py',
author='Bob',
author_email='abc#example.com',
license='MIT',
packages=find_packages(),
zip_safe=False,
)
The above package can be installed successfully by using python setup.py. However, in order to use this package, I need to write long import statements as shown below-
from android_py.android import Android
from android_py.options import Power
my_robot = Android()
my_robot.set_power(Power.On)
As you can see, there are following two issues-
The first import, i.e., from android_py.android import Android is way too long and not user-friendly as it is difficult to remember. I think something shorter such as import android is much nicer.
The second import, i.e., from android_py.options import Power is troublesome. It should be imported automatically by the first import.
Can you please suggest me how to reconfigure this package in order to overcome with above-mentioned issues? Please note that I am using Python 2.7 (if that matters)!
Additional to my comments I will try to give a short example. Say you have a power.py:
class Power:
On = True
and in the same package a android.py:
from . import power
class Android:
Power = power.Power
In the android_py package __init__.py:
from .android import Android
Now, from outside in your app.py, main.py or whatever you can:
from android_py import Android
my_robot = Android()
my_robot.set_power(my_robot.Power.On)
BTW: I'am not very happy with the package name android_py. Name it also android, it is no problem to have an android.py in a package android. Or explainted in path names: it is no problem to have android/android.py. With the relative import . used in the android.py and __init__.py in the example above it should work.
Is this what you're looking for? https://python-packaging.readthedocs.io/en/latest/everything.html
Edit: Using the code in the link, I was able to do this:
import funniest.funniest
import types
print(dir(funniest))
print([getattr(funniest, a) for a in dir(funniest)
if isinstance(getattr(funniest, a), types.FunctionType)])
print(funniest.funniest.joke())
print("works")
This calls joke() in ./python-packaging-master/funniest/funniest/init.py
Just change the folder structure in the example and you can plainly call import funniest
I'm sure you can apply the same thing to your package.
This question already has answers here:
Python3 correct way to import relative or absolute?
(2 answers)
Closed 2 years ago.
I have the following directory:
mydirectory
├── __init__.py
├── file1.py
└── file2.py
I have a function f defined in file1.py.
If, in file2.py, I do
from .file1 import f
I get the following error:
SystemError: Parent module '' not loaded, cannot perform relative
import
Why? And how to make it work?
Launching modules inside a package as executables is a bad practice.
When you develop something you either build a library, which is intended to be imported by other programs and thus it doesn't make much sense to allow executing its submodules directly, or you build an executable in which case there's no reason to make it part of a package.
This is why in setup.py you distinguish between packages and scripts. The packages will go under site-packages while the scripts will be installed under /usr/bin (or similar location depending on the OS).
My recommendation is thus to use the following layout:
/
├── mydirectory
| ├── __init__.py
| ├── file1.py
└── file2.py
Where file2.py imports file1.py as any other code that wants to use the library mydirectory, with an absolute import:
from mydirectory.file1 import f
When you write a setup.py script for the project you simply list mydirectory as a package and file2.py as a script and everything will work. No need to fiddle with sys.path.
If you ever, for some reason, really want to actually run a submodule of a package, the proper way to do it is to use the -m switch:
python -m mydirectory.file1
This loads the whole package and then executes the module as a script, allowing the relative import to succeed.
I'd personally avoid doing this. Also because a lot of people don't even know you can do this and will end up getting the same error as you and think that the package is broken.
Regarding the currently accepted answer, which says that you should just use an implicit relative import from file1 import f because it will work since they are in the same directory:
This is wrong!
It will not work in python3 where implicit relative imports are disallowed and will surely break if you happen to have installed a file1 module (since it will be imported instead of your module!).
Even if it works the file1 will not be seen as part of the mydirectory package. This can matter.
For example if file1 uses pickle, the name of the package is important for proper loading/unloading of data.
When launching a python source file, it is forbidden to import another file, that is in the current package, using relative import.
In documentation it is said:
Note that relative imports are based on the name of the current module. Since the name of the main module is always "__main__", modules intended for use as the main module of a Python application must always use absolute imports.
So, as #mrKelley said, you need to use absolute import in such situation.
since file1 and file2 are in the same directory, you don't even need to have an __init__.py file. If you're going to be scaling up, then leave it there.
To import something in a file in the same directory, just do like this
from file1 import f
i.e., you don't need to do the relative path .file1 because they are in the same directory.
If your main function, script, or whatever, that will be running the whole application is in another directory, then you will have to make everything relative to wherever that is being executed.
myproject/
mypackage
├── __init__.py
├── file1.py
├── file2.py
└── file3.py
mymainscript.py
Example to import from one file to another
#file1.py
from myproject import file2
from myproject.file3 import MyClass
Import the package example to the mainscript
#mymainscript.py
import mypackage
https://docs.python.org/3/tutorial/modules.html#packages
https://docs.python.org/3/reference/import.html#regular-packages
https://docs.python.org/3/reference/simple_stmts.html#the-import-statement
https://docs.python.org/3/glossary.html#term-import-path
The variable sys.path is a list of strings that determines the interpreter’s search path for modules. It is initialized to a default path taken from the environment variable PYTHONPATH, or from a built-in default if PYTHONPATH is not set. You can modify it using standard list operations:
import sys
sys.path.append('/ufs/guido/lib/python')
sys.path.insert(0, '/ufs/guido/myhaxxlib/python')
Inserting it at the beginning has the benefit of guaranteeing that the path is searched before others (even built-in ones) in the case of naming conflicts.