I am about to build a new python lib and I was seeking information concerning packaging in Python.
I understand that "setup.py" is the script that controls everything. I wonder how to deal with it when there are external libraries in svn for instance.
How to download automatically a given version from the repository using "setup.py" ?
docs for this are at the cheese shop
use the requires keyword
I may not have understood the problem correctly.
For any additional dependencies, you mention them in setup.py as
install_requires=['module1 >= 1.3', 'module2 >=1.8.2']
When you use setuptools, easy_install oo pip, these external dependencies will get installed during setup, if required. These should also be available in package repositories for download.
Related
I created a python library using setuptools that contains .so files. When I try to pip install the library the .so files aren't being installed into my virtual environment.
Your question, as it stands, is somewhat vague so I can't be sure if I'm answering it. However, from the setup.py which you've pasted as a comment, I think you've not specified how to build the extension locally. Putting the .so directly as part of your package is not very wise since it's not cross platform. The opposite approach is to compile it on the target machine but they'll need a compiler toolchain locally installed.
Please refer to the example here which describes how to include C extensions in your project.The official python packaging page on this topic is incomplete and there's an issue describing this over here. You might find something useful there as well.
I followed the AWS guide to prepare a deployment package for my lambda function. The generated zip file is around 9 - 10MB which includes pip, setuptools, pylint. Are they really required?
Here are the commands.
virtualenv v-env
source v-env/bin/activate
pip install xmltodict
pip install requests
deactivate
cd v-env/lib/python3.7/site-packages/
zip -r9 ../../../../function.zip .
Edit: Remove installing boto as it is provided by AWS already
Well, as you may see the guides provide standars and, obviously, guidance for a clean and nice coding or project deployment.
Pylint has a lot of features that help you out while using Python as your programming language, such as checking coding standars, error detection, refactoring help in order to prevend duplicated code, among other tools.
Setuptools is really useful too. Is a development process library designed to facilitate packaging Python projects by enhancing the Python standard library distribution utilities, I encourage you to use it in order to wrap your processes and models in order to have a strong modular project.
And pip is a Package Manager for python packages or modules. You can add, download, delete and a lot more things with it by simple using few words on a line of code. This package manager is useful, you can download wheels, zip's and modules from the internet and easily install them by just using
pip install <module or library name>
So, by answering your question, if you downloaded and installed a package for AWS supported in Python and it installed those libraries I must think those are being used across the modules you want to use.
You can always check the source code in order to be sure.
If the libraries aren't really being used, they aren't necessary since there are several libraries and packages that do what those libraries do.
Hope it helps, happy coding.
I understand there is already a question about packaging into pip, but this is more generic. On what mechanism does pip identify packages? To which central server should I add the name so that when someone types in
pip install <mypackagename>
how does pip know, where to look for the package. What should I do to add mine to that name resolution directory?
Pip pulls from the Python Package Index. It is very easy to submit a package, assuming you have a configured setup.py to build the package.
You'll need to register an account on PyPi, have certain metadata defined in setup.py (license, etc), and a setup.cfg if you're using markdown-formatted readme (as on Github). Then it's just a shell command to register the package :
Register:
python setup.py register -r pypi
Submit:
python setup.py sdist upload -r pypi
Python's crowdsourced package repository, PyPI, aka the Python Package Index.
You will want to start with a tutorial on how to package your code for, then submit to, PyPI. This is one. There is a learning curve, but it is most worthwhile.
It helps to look at packages already on PyPI, then follow the links back to their source code repositories to see all of the files and configurations that were used. For example, my intspan package is hosted at bitbucket. As many PyPI packages are hosted at either Bitbucket or Github, so there are many examples available from which to learn.
I have a Python application that I'm distributing with the standard setup.py. I need to package the dependencies too because the audience I'm distributing do not have network access. I have downloaded the packages from PyPi but am stuck where to place them so when someone does "python setup.py install", they will get installed.
Any ideas?
EDIT: Forgot to mention, I'm trying to achieve this on a Unix platform and without using other tools such as pip.
I have read the documentation but I don't understand.
Why do I have to use distutils to install python modules ?
Why do I just can't save the modules in python path ?
You don't have to use distutils. You can install modules manually, just like you can compile a C++ library manually (compile every implementation file, then link the .obj files) or install an application manually (compile, put into its own directory, add a shortcut for launching). It just gets tedious and error-prone, as every repetive task done manually.
Moreover, the manual steps I listed for the examples are pretty optimistic - often, you want to do more. For example, PyQt adds the .ui-to-.py-compiler to the path so you can invoke it via command line.
So you end up with a stack of work that could be automated. This alone is a good argument.
Also, the devs would have to write installing instructions. With distutils etc, you only have to specify what your project consists of (and fancy extras if and only if you need it) - for example, you don't need to tell it to put everything in a new folder in site-packages, because it already knows this.
So in the end, it's easier for developers and for users.
what python modules ? for installing python package if they exist in pypi you should do :
pip install <name_of_package>
if not, you should download them .tar.gz or what so ever and see if you find a setup.py and run it like this :
python setup.py install
or if you want to install it in development mode (you can change in package and see the result without installing it again ) :
python setup.py develop
this is the usual way to distribute python package (the setup.py); and this setup.py is the one that call disutils.
to summarize this distutils is a python package that help developer create a python package installer that will build and install a given package by just running the command setup.py install.
so basically what disutils does (i will sit only important stuff):
it search dependencies of the package (install dependencies automatically).
it copy the package modules in site-packages or just create a sym link if it's in develop mode
you can create an egg of you package.
it can also run test over your package.
you can use it to upload your package to pypi.
if you want more detail see this http://docs.python.org/library/distutils.html
You don't have to use distutils to get your own modules working on your own machine; saving them in your python path is sufficient.
When you decide to publish your modules for other people to use, distutils provides a standard way for them to install your modules on their machines. (The "dist" in "distutils" means distribution, as in distributing your software to others.)