I am working on a setup.py file for a Python package. I want to include the package "rdkit" among the "install_requires" dependencies of my package. However, that does not work, as rdkit cannot be directly pip-installed. My preferred method in this case is to use conda (https://anaconda.org/rdkit/rdkit).
Is there a way to automate the installation (or upgrade) if required of a package using conda in a setup.py file, similarly to what install_requires does for pip-installable dependencies?
Thank you very much for your help
setup(
...
install_requires=[
'numpy >= 1.8.0',
'scipy >= 1.6.1',
],
)
Is there a way to automate the installation (or upgrade) if required of a package using conda in a setup.py file…?
Nop, no way. setup.py is intended for python setup.py install or pip install and it doesn't know anything about conda.
Perhaps it should be solved the other way — starting from conda that then calls pip install to install pip-installable packages.
Related
We've made a library which depends on other libraries. But there are necessary (e.g. for server batch processing) and optional dependencies (e.g. for clients with GUI).
Is something like this possible:
pip install mylib.tar.gz # automatically downloads and installs with the minimal set of dependencies
pip install mylib.tar.gz --install-option="complete" # automatically installs with all dependencies
I've found the extra_require flag, but how can I tell pip to use them? The setup.py looks like this:
from setuptools import setup
# ...
# Hard library depencencies:
requires = [
"numpy>=1.4.1",
"scipy>=0.7.2",
"traits>=3.4.0"
]
# Soft library dependencies:
recommended = {
"mpl": ["matplotlib>=0.99.3"],
"bn": ["bottleneck>=0.6"]
}
# ...
# Installer parameters:
setup(
name = "mylib",
#...
install_requires = requires,
extras_require = recommended
)
You can install the packages in extras_require by appending the name of the recommended dependency in square brackets (i.e. [mpl] or [bn] in your case) to the package name in pip.
So to install 'mylib' with the additional requirements, you would call pip like this:
pip install 'mylib[mpl]'
pip install 'mylib[bn]'
This will first download and install the extra dependencies, and then mylib's core dependencies.
This is anologous to how you declare those dependencies with setuptools: http://pythonhosted.org/setuptools/setuptools.html#declaring-extras-optional-features-with-their-own-dependencies (see the install_requires value in the third example)
So pip is actually quite picky about installing libraries with extra requirements
pip install -e ".[extra,requirements]" # works with file paths
pip install "package[extra,requirements]" # works when downloading packages
pip install ".[extra,requirments]" # DOES NOT WORK
I think this is down to how the RequirementsSpec parser works, and pip does some extra magic with the -e flag. Anyhow after much head banging, here's a mildly ugly workaround
pip install "file:///path/to/your/python_code#egg=SomeName[extra,requirements]"
The egg=SomeName part is basically ignored, but pip correctly picks up the extra requirements
Caveats
Tested with pip 1.5.6 so make sure you're using a current version of pip.
As far as I can tell, the file:/// syntax is undocumented in pip, so I'm not sure if it'll change in the future. It looks a bit like the VCS Support syntax but I was a bit surprised it worked.
You could also get around this by running your own pypi server, but that's a bit out of scope.
This weekend I've been reading up on conda and the python packaging user guide because I have a simple pure python project that depends on numpy. It seemed to me that distributing/installing this project via conda was better than pip due to this dependency.
One thing on which I'm still not clear: conda will install a python package from a recipe in build.sh, but it seems like build.sh just ends up calling python setup.py install for most python packages.
So even if I want to distribute/install my python package with conda, I still end up depending on setuptools (or distutils) for the actual installation, correct? I was unable to find a conda utility analogous to setuptools; am I missing something?
FWIW, I posted this question on the conda issue tracker.
Thanks!
Typically you will still be using distutils (or setuptools if the library requires it) to install things, yes. It is not technically required. The build.sh can be anything. If you wanted to, you could just copy the code into site-packages. Using setup.py install is recommended, though, as libraries will already have setup.py working, it will install metadata that can be read by pip, and it will compile any extension modules and install any data files.
We've made a library which depends on other libraries. But there are necessary (e.g. for server batch processing) and optional dependencies (e.g. for clients with GUI).
Is something like this possible:
pip install mylib.tar.gz # automatically downloads and installs with the minimal set of dependencies
pip install mylib.tar.gz --install-option="complete" # automatically installs with all dependencies
I've found the extra_require flag, but how can I tell pip to use them? The setup.py looks like this:
from setuptools import setup
# ...
# Hard library depencencies:
requires = [
"numpy>=1.4.1",
"scipy>=0.7.2",
"traits>=3.4.0"
]
# Soft library dependencies:
recommended = {
"mpl": ["matplotlib>=0.99.3"],
"bn": ["bottleneck>=0.6"]
}
# ...
# Installer parameters:
setup(
name = "mylib",
#...
install_requires = requires,
extras_require = recommended
)
You can install the packages in extras_require by appending the name of the recommended dependency in square brackets (i.e. [mpl] or [bn] in your case) to the package name in pip.
So to install 'mylib' with the additional requirements, you would call pip like this:
pip install 'mylib[mpl]'
pip install 'mylib[bn]'
This will first download and install the extra dependencies, and then mylib's core dependencies.
This is anologous to how you declare those dependencies with setuptools: http://pythonhosted.org/setuptools/setuptools.html#declaring-extras-optional-features-with-their-own-dependencies (see the install_requires value in the third example)
So pip is actually quite picky about installing libraries with extra requirements
pip install -e ".[extra,requirements]" # works with file paths
pip install "package[extra,requirements]" # works when downloading packages
pip install ".[extra,requirments]" # DOES NOT WORK
I think this is down to how the RequirementsSpec parser works, and pip does some extra magic with the -e flag. Anyhow after much head banging, here's a mildly ugly workaround
pip install "file:///path/to/your/python_code#egg=SomeName[extra,requirements]"
The egg=SomeName part is basically ignored, but pip correctly picks up the extra requirements
Caveats
Tested with pip 1.5.6 so make sure you're using a current version of pip.
As far as I can tell, the file:/// syntax is undocumented in pip, so I'm not sure if it'll change in the future. It looks a bit like the VCS Support syntax but I was a bit surprised it worked.
You could also get around this by running your own pypi server, but that's a bit out of scope.
I'm working on creating a Python package that is somewhat modular in nature and I was wondering what's the best way to go about handling multiple installation configurations?
Take the following setup.py for the package of a simple document parser:
setup(
name = 'my_document_parser',
...
entry_points = {
'my_document_parser.parsers': [
'markdown = my_document_parser.parsers.misaka:Parser [misaka]',
'textile = my_document_parser.parsers.textile:Parser [textile]'
]
},
install_requires = [
'some_required_package'
],
extras_require = {
'misaka' = ['misaka'],
'textile' = ['textile']
},
...
)
I'd like to make something like the following available:
A default install
python setup.py install or pip install my_document_parser
installs my_document_parser, some_required_package, misaka
A bare install
something like python setup.py install --bare
installs my_document_parser, some_required_package
A way to enable other dependencies
something like python setup.py install --bare --with-textile
installs my_document_parser, some_required_package, textile
This is my first time messing around with Python packages from a developer standpoint, so I'm open to anything anyone has to offer if I'm going about things the wrong way or such.
Thanks.
The default installation will install everything in your install_requires and packages sections. Packages under extras_require will never be installed by your setup.py script; only if you create some logic for this kind of installation.
By default, setuptools and distutils have no way to explictly say "install using these sections of my setup call". You would need to implement by yourself.
pip always installs packages using no parameter to setup.py but --single-version-externally-managed, to tell setuptools to never create a .egg file.
Conclusion: if you want people to install your package by using pip, forget --bare and --with-textile. The best you can do is a full installation including misaka and textile as dependencies.
Maybe you should read The Hitchhiker's Guide To Packaging, specifically Creating a Package section and setuptool's specification -- it is a pretty good one, it covers everything setuptools can do.
I have made a distribution of my python package with the following setup.py
#!/usr/bin/env python
from setuptools import setup
setup(name='mypackagename',
version='0.1',
description='Tool ....',
author='Peter Smit',
author_email='lala#lala.com',
packages=['mypackagename'],
package_dir={'': 'src'},
install_requires=['boto'],
entry_points = dict(console_scripts=[
'mypackagenamescript = mypackagename.launcher:run',
])
)
I created an egg of this with python setup.py bdist_egg.
Trying to install it now with pip gives the following error:
bin/pip install mypackagename-0.1-py2.6.egg
Downloading/unpacking mypackagename-0.1-py2.6.egg
Could not find any downloads that satisfy the requirement mypackagename-0.1- py2.6.egg
No distributions at all found for mypackagename-0.1-py2.6.egg
Storing complete log in /home/peter/.pip/pip.log
The mentioned log files showed that it tries to download the package from pypi, where it obviously does not exist.
What did I do wrong? How can I install this egg of mine plus it's dependencies?
why not using setuptools easy_install?
easy_install mypackagename-0.1-py2.6.egg
If you want to work with eggs that's the way.
pip cannot install from eggs.
If you want your package to be available on PyPI, you need to register and account there and upload it. You can then simply say pip install myproject. It will search PyPI, find it, download and install it.
If you have your setup.py ready and want to install your application locally, all you need to do is to say python setup.py install. You don't need to use pip or easy_install.
The hitchhikers guide to packaging contains details on all these things. It should make things clear.
Pip cannot install eggs. IMHO that is a serious lack. I would suggest you to try out Pyg. Just download the get-pyg.py script and execute it:
$ curl -O https://raw.github.com/rubik/pyg/master/get-pyg.py
$ python get-pyg.py
Retrieving archive from ... etc.
Note: As an alternative, you can install it via easy_install or pip.
Then you can use it:
$ pyg install mypackagename-0.1-py2.6.egg
Pyg supports virtualenv too.
rubik