Assume that someone wants to package a Python (Cython) library that depends on the C++ boost library.
What is the best way to configure the setup.py so that the user is properly informed that it is required to install the boost library (i.e., apt-get install libboost-dev in Ubuntu, etc in other OSes)? Or is it a better practice to include the boost library in the python package distribution?
The question is better asked as
What is the best way to distribute a Python extension including
an external library dependency.
This is better dealt with binary wheel packages.
User does not need to know anything about setup.py, which is used for building and installing source code. User just needs to download and install a binary wheel package.
Including just the header files does not solve the problem of needing the library to build with and link to. It also opens up issues with version incompatibilities.
So setup.py need not have anything special about any of this, it just needs to know where to find headers which will be a sub-dir in your project if the library is included and which libraries to link with.
The documentation should include instructions on how to build from source, for which more than just boost is needed (python header files, appropriate compilers etc).
Tools like auditwheel then take care of bundling external library dependencies into the binary wheel, so end-users need not have the library installed to use your package.
See also manylinux for distributing binary Python extensions and this demo project.
Related
I created a python library using setuptools that contains .so files. When I try to pip install the library the .so files aren't being installed into my virtual environment.
Your question, as it stands, is somewhat vague so I can't be sure if I'm answering it. However, from the setup.py which you've pasted as a comment, I think you've not specified how to build the extension locally. Putting the .so directly as part of your package is not very wise since it's not cross platform. The opposite approach is to compile it on the target machine but they'll need a compiler toolchain locally installed.
Please refer to the example here which describes how to include C extensions in your project.The official python packaging page on this topic is incomplete and there's an issue describing this over here. You might find something useful there as well.
VTK library cannot be installed via pip.
Though, it can be compiled and installed from sources.
My Python project depends on VTK.
I want it to install VTK automatically by calling pip install . from root directory of the project.
In this case setup.py file should be able to
download VTK sources of needed version from GitHub
call cmake in order to prepare build
compile sources and create Python bindings
install needed files into currently used site-packages (e. g., it should not be installed into /usr/local/lib/python3/site-packages if I use virtualenv, pipenv or pyenv)
Is it possible?
If yes, how can I do this?
In principle, you can include any executable code in the setup file. However, nowhere in the setuptools documentation could I find information that would solve the problem here.
Also, the installation procedure for vtk is a bit complex which is why kitware uses cmake in the first place.
So, the short answer would be "no" or "don't do that".
Further, the problems you will encounter:
Users will expect a transparent install. But achieving a cross-platform build process on the basis of the cmake build instructions for vtk will prevent you from setting the customization (path to vtk, path to the Python interpreter, platform-specific C flags).
The install process will be harder to debug. Users will come to you for VTK build problems.
Kitware themselve do not propose vtk on pypi. This suggests that it is too much time intensive, impossible or too fragile to maintain to achieve this goal.
If you wish to see a popular Python project that relies on vtk, there is mayavi. The installation instructions request to install vtk beforehand.
Looks like VTK has presented their official binding via pypi, so I can use it in setup.py file simply by appending it to install_requires list.
Works well in my project, doesn't need compilation anymore.
Though, there is a caution in the mayavi documentation
The latest VTK wheels are available on all the major platforms
(Windows, MacOS, and Linux), but only for 64 bit machines. Python 3.x
is fully supported on all these operating systems and Python 2.7.x on
MacOS and Linux. If you are out of luck, and your platform is not
supported then you will need to install VTK yourself using your
particular distribution as discussed in the General Build and
Installation instructions
I'm writing a python packages which I would like to distribute as a wheel, so it can be easily installed via pip install.
As part of the functionality of the package itself I compile C++ code. For that, I distribute with the package some set of header files for the C++ code to include. Now, in order to speed up those compilation operations I'd like to provide a precompiled-header as part of the package.
I am able to do this if the package is installed via python setup.py install because I can add a step after the installation that generates the precompiled header in the installation directory (some-virtualenv/lib/python3.5/site-packages/...) directly.
But now I can't figure out how to do this when I distribute a wheel. It seems to me that the installation process of a wheel is supposed to be a simple unpack and copy and provides me no way of performing some extra configuration on the installed package (that would generate that precompiled header for example).
As part of my search of how to do this I stumbled across this, but no solution is offered there.
Is there any way around this or am I forced to use a source distribution for my package?
I'm an author of a pure Python library that aims to be also convenient to use from a command line. For Windows users it would be nice just installing the package from an .exe or .msi package.
However I cannot get the installer to install package dependencies (especially the dependency on setuptools itself, so that running the software fails with an import error on pkg_resources). I don't believe that providing an easy .exe installer makes much sense, if the user then needs to manually install setuptools and other libraries on top. I'd rather tell them how to add easy_install to their PATH and go through this way (http://stackoverflow.com/questions/1449494/how-do-i-install-python-packages-on-windows).
I've build .exe packages in the past, but don't remember if that ever worked the way I'd preferred it to.
It is quite common to distribute packages that have dependencies, especially those as you have, but I understand your wish to make installation as simple as possible.
Have a look at deployment bootstrapper, a tool dedicated to solving the problem of delivering software including its prerequisites.
Regardless of what packaging method you eventually choose, maintain your sanity by staying away from including MSIs in other MSI in any way. That just does not work because of transactional installation requirements and locking of the Windows Installer database.
I have read the documentation but I don't understand.
Why do I have to use distutils to install python modules ?
Why do I just can't save the modules in python path ?
You don't have to use distutils. You can install modules manually, just like you can compile a C++ library manually (compile every implementation file, then link the .obj files) or install an application manually (compile, put into its own directory, add a shortcut for launching). It just gets tedious and error-prone, as every repetive task done manually.
Moreover, the manual steps I listed for the examples are pretty optimistic - often, you want to do more. For example, PyQt adds the .ui-to-.py-compiler to the path so you can invoke it via command line.
So you end up with a stack of work that could be automated. This alone is a good argument.
Also, the devs would have to write installing instructions. With distutils etc, you only have to specify what your project consists of (and fancy extras if and only if you need it) - for example, you don't need to tell it to put everything in a new folder in site-packages, because it already knows this.
So in the end, it's easier for developers and for users.
what python modules ? for installing python package if they exist in pypi you should do :
pip install <name_of_package>
if not, you should download them .tar.gz or what so ever and see if you find a setup.py and run it like this :
python setup.py install
or if you want to install it in development mode (you can change in package and see the result without installing it again ) :
python setup.py develop
this is the usual way to distribute python package (the setup.py); and this setup.py is the one that call disutils.
to summarize this distutils is a python package that help developer create a python package installer that will build and install a given package by just running the command setup.py install.
so basically what disutils does (i will sit only important stuff):
it search dependencies of the package (install dependencies automatically).
it copy the package modules in site-packages or just create a sym link if it's in develop mode
you can create an egg of you package.
it can also run test over your package.
you can use it to upload your package to pypi.
if you want more detail see this http://docs.python.org/library/distutils.html
You don't have to use distutils to get your own modules working on your own machine; saving them in your python path is sufficient.
When you decide to publish your modules for other people to use, distutils provides a standard way for them to install your modules on their machines. (The "dist" in "distutils" means distribution, as in distributing your software to others.)