it is possible to install some special sub-package from package?
For example, I want to create package with slack, datadog, sentry plugins (wrappers). But I want to allow user what he wants to install.
Like:
pip install super_plugins --plugins slack, datadog
Can it be done without separating all plugins to different packages?
Actually, It is quite simple. This is called Packaging namespace packages.
https://packaging.python.org/guides/packaging-namespace-packages/
All you need is to separate all packages to sub - packages and after install it with a namespace.
# for all packages
pip install super_plugins
# for specific
pip install super_plugins.slack super_plugins.datadog
Related
Similar to how Node.js automatically adds dependencies to package-lock.json, is there a way I can automatically add requirements to my requirements.txt file for Python?
Since you mentioned Node.js specifically, the Python project that comes closest to what you're looking for is probably Pipenv.
Blurb from the Pipenv documentation:
Pipenv is a dependency manager for Python projects. If you're familiar with Node.js's npm or Ruby’s bundler, it is similar in spirit to those tools. While pip can install Python packages, Pipenv is recommended as it’s a higher-level tool that simplifies dependency management for common use cases.
It's quite a popular package among developers as the many stars on GitHub attest.
Alternatively, you can use a "virtual environment" in which you only install the external dependencies that your project needs. You can either use the venv module from the standard library or the Virtualenv package from PyPI, which offers certain additional features (that you may or may not need). With either of those, you can then use Python's (standard) package manager Pip to update the requirements file:
pip freeze >requirements.txt
This is the "semi-automatic" way, so to speak. Personally, I prefer to do this manually. That's because in a typical development environment ("virtual" or not), you also install packages that are only required for development tasks, such as running tests or building the documentation. They don't need to be installed along with your package on end-user machines, so shouldn't be in requirements.txt. Popular packaging tools such as Flit and Poetry manage these "extra dependencies" separately, as does Pip.
If you are using Linux you can create an alias like this:
alias req='pip3 freeze > ~/requerments.txt'
And then when you want to install new package use this command:
pip3 install <package> | req
I think, to-requirements.txt is what you need:
pip install to-requirements.txt
requirements-txt setup
After that installed packages will be appended to requirements.txt. And uninstalled packages will be removed.
It might require root access if you install it on system-wide Python interpreter. Add sudo if it failes.
I would like to know if there's a way to package a simple Python project and have it perform installation over the internet, just like when you install a module with pip.
Sure there is. This is how all the 3rd party packages we are all using did.
The formal pypa explain how to do it here.
Basically you need to package your project to a wheel file and upload it to the pypi repository. To do this you need to declare (mainly in setup.py), what is your package name, version, which sub-packages you want to pack to the wheel etc..
If your packages are required for a particular project, it is straightforward to contain them in the Git repository. You can put them in the directory named wheelhouse, which comes from the name of the previous default directory created by pip wheel.
If you put the private package foo in the wheelhouse, you can install as follows:
pip install foo -f wheelhouse
I see more and more commands like this:
$ pip install "splinter[django]"
What do these square brackets do?
The syntax that you are using is:
pip install "project[extra]"
In your case, you are installing the splinter package which has the added support for django. The square brackets ([]) are not specific syntax, just convention. Really, you are installing the package named: "splinter[django]".
An explanation from #chetner:
The command pip install splinter django would install two packages named splinter and django. splinter[django], on the other hand, installs a variant of the splinter package which contains support for django. Note that it has nothing to do with the django package itself, but is just a string defined by the splinter package for a particular feature set that gets enabled.
Brackets [optional] in PIP signify optional dependencies
Just in case another developer comes along looking to implement this pattern in their own Python package deployment, here's further explanation of the brackets [] in pip.
For Example: Apache Airflow
To install airflow from pip we use this command:
pip install 'apache-airflow'
You can install optional components of airflow with:
pip install 'apache-airflow[aws]'
# [optional] -----------^
When we search pypi for apache-airflow note that the optional packages do not show up:
pip search 'apache-airflow'
apache-airflow (1.10.9) - Programmatically author, schedule and monitor data pipelines
pylint-airflow (0.1.0a1) - A Pylint plugin to lint Apache Airflow code.
swe-airflow-tools (0.0.3) - Tools for Apache Airflow Application
airflow (0.6) - Placeholder for the old Airflow package
...
Implementation via setup.py
You can see how this was accomplished in the setup.py script
On the left in setup.py - extras_require is defined.
On the right are the correlated installation commands for these optional sub-packages.
Pretty sure these are setuptools extras:
https://setuptools.readthedocs.io/en/latest/setuptools.html#declaring-extras-optional-features-with-their-own-dependencies
Sometimes a project has “recommended” dependencies, that are not required for all uses of the project. For example, a project might offer optional PDF output if ReportLab is installed, and reStructuredText support if docutils is installed. These optional features are called “extras” ...
Maybe worthwhile to know that this optional package syntax admits multiple extras (separated by comma within the brackets) as in:
python -m pip install SomePackage[PDF,EPUB] # multiple extras
As per the pip manual
TLDR
The square bracket contains the 'extra' option's information defined in setup.py that pip will use to install additional dependencies.
pip install "splinter[django]"
To be specific, the above line will install first the 'splinter' package, then install the extra dependencies the 'splinter' project requires with the 'django' option specified in a setup.py of 'splinter' project.
Explanation
pip install "splinter[django]"
pip install "splinter" "Django>=2.0.6" "lxml>=4.2.4" "cssselect"
As of splinter==0.16.0, with python==3.9.2, the above two commands are equivalent.
Both pip install will result in the following packages given a clean virtual enviroment.
The reason why the two pip install commands achieve same is because this is literally what has been run in the background based on the setup.py of the splinter package
The '[django]' is the 'extra' option for the 'splinter' package. Pip will look into the setup.py of splinter package, and find what needs to be installed with the '[django]' option specified. In this case, it is these 3 packages: ["Django>=2.0.6", "lxml>=4.2.4", "cssselect"]
This is exactly the list from the setup.py file for the project in question:
"django": ["Django>=1.7.11;python_version<'3.0'", "Django>=2.0.6;python_version>'3.3'", "lxml>=2.3.6", "cssselect", "six"],
I see more and more commands like this:
$ pip install "splinter[django]"
What do these square brackets do?
The syntax that you are using is:
pip install "project[extra]"
In your case, you are installing the splinter package which has the added support for django. The square brackets ([]) are not specific syntax, just convention. Really, you are installing the package named: "splinter[django]".
An explanation from #chetner:
The command pip install splinter django would install two packages named splinter and django. splinter[django], on the other hand, installs a variant of the splinter package which contains support for django. Note that it has nothing to do with the django package itself, but is just a string defined by the splinter package for a particular feature set that gets enabled.
Brackets [optional] in PIP signify optional dependencies
Just in case another developer comes along looking to implement this pattern in their own Python package deployment, here's further explanation of the brackets [] in pip.
For Example: Apache Airflow
To install airflow from pip we use this command:
pip install 'apache-airflow'
You can install optional components of airflow with:
pip install 'apache-airflow[aws]'
# [optional] -----------^
When we search pypi for apache-airflow note that the optional packages do not show up:
pip search 'apache-airflow'
apache-airflow (1.10.9) - Programmatically author, schedule and monitor data pipelines
pylint-airflow (0.1.0a1) - A Pylint plugin to lint Apache Airflow code.
swe-airflow-tools (0.0.3) - Tools for Apache Airflow Application
airflow (0.6) - Placeholder for the old Airflow package
...
Implementation via setup.py
You can see how this was accomplished in the setup.py script
On the left in setup.py - extras_require is defined.
On the right are the correlated installation commands for these optional sub-packages.
Pretty sure these are setuptools extras:
https://setuptools.readthedocs.io/en/latest/setuptools.html#declaring-extras-optional-features-with-their-own-dependencies
Sometimes a project has “recommended” dependencies, that are not required for all uses of the project. For example, a project might offer optional PDF output if ReportLab is installed, and reStructuredText support if docutils is installed. These optional features are called “extras” ...
Maybe worthwhile to know that this optional package syntax admits multiple extras (separated by comma within the brackets) as in:
python -m pip install SomePackage[PDF,EPUB] # multiple extras
As per the pip manual
TLDR
The square bracket contains the 'extra' option's information defined in setup.py that pip will use to install additional dependencies.
pip install "splinter[django]"
To be specific, the above line will install first the 'splinter' package, then install the extra dependencies the 'splinter' project requires with the 'django' option specified in a setup.py of 'splinter' project.
Explanation
pip install "splinter[django]"
pip install "splinter" "Django>=2.0.6" "lxml>=4.2.4" "cssselect"
As of splinter==0.16.0, with python==3.9.2, the above two commands are equivalent.
Both pip install will result in the following packages given a clean virtual enviroment.
The reason why the two pip install commands achieve same is because this is literally what has been run in the background based on the setup.py of the splinter package
The '[django]' is the 'extra' option for the 'splinter' package. Pip will look into the setup.py of splinter package, and find what needs to be installed with the '[django]' option specified. In this case, it is these 3 packages: ["Django>=2.0.6", "lxml>=4.2.4", "cssselect"]
This is exactly the list from the setup.py file for the project in question:
"django": ["Django>=1.7.11;python_version<'3.0'", "Django>=2.0.6;python_version>'3.3'", "lxml>=2.3.6", "cssselect", "six"],
I'm wondering if there's a way to "install" single-file python modules using pip (i.e. just have pip download the specified version of the file and copy it to site-packages).
I have a Django project that uses several 3rd-party modules which aren't proper distributions (django-thumbs and a couple others) and I want to pip freeze everything so the project can be easily installed elsewhere. I've tried just doing
pip install git+https://github.com/path/to/file.git
(and tried with the -e tag too) but pip complains that there's no setup.py file.
Edit: I should have mentioned - the reason I want to do this is so I can include the required module in a requirements.txt file, to make setting up the project on a new machine or new virtualenv easier.
pip requires a valid setup.py to install a python package. By definition every python package has a setup.py... What you are trying to install isn't a package but rather a single file module... what's wrong with doing something like:
git clone git+https://github.com/path/to/file.git /path/to/python/install/lib
I don't quite understand the logic behind wanting to install something that isn't a package with a package manager...