Develop Reference

Is there a way to package a simple Python project and have it perform installation over the internet

I would like to know if there's a way to package a simple Python project and have it perform installation over the internet, just like when you install a module with pip.

Sure there is. This is how all the 3rd party packages we are all using did.
The formal pypa explain how to do it here.
Basically you need to package your project to a wheel file and upload it to the pypi repository. To do this you need to declare (mainly in setup.py), what is your package name, version, which sub-packages you want to pack to the wheel etc..

If your packages are required for a particular project, it is straightforward to contain them in the Git repository. You can put them in the directory named wheelhouse, which comes from the name of the previous default directory created by pip wheel.
If you put the private package foo in the wheelhouse, you can install as follows:
pip install foo -f wheelhouse

Python install sub-package from package

it is possible to install some special sub-package from package?
For example, I want to create package with slack, datadog, sentry plugins (wrappers). But I want to allow user what he wants to install.
Like:
pip install super_plugins --plugins slack, datadog
Can it be done without separating all plugins to different packages?

Actually, It is quite simple. This is called Packaging namespace packages.
https://packaging.python.org/guides/packaging-namespace-packages/
All you need is to separate all packages to sub - packages and after install it with a namespace.
# for all packages
pip install super_plugins
# for specific
pip install super_plugins.slack super_plugins.datadog

example of python distutils package code that depends on other modules?

can someone point out a simple example of making a Python package using distutils that depends on other python modules being installed, e.g. numpy (a particular version) and scipy? I found very simple examples online but could not find an example that depends on a known package. I want to find the easiest system where such dependencies are installed for the user when the user installs the package I am defining using setup.py install. thanks.

Distutils by itself does not install dependencies. You need to use an add-on to Distutils, like pip or plain Distribute/setuptools.

What exactly does distutils do?

I have read the documentation but I don't understand.
Why do I have to use distutils to install python modules ?
Why do I just can't save the modules in python path ?

You don't have to use distutils. You can install modules manually, just like you can compile a C++ library manually (compile every implementation file, then link the .obj files) or install an application manually (compile, put into its own directory, add a shortcut for launching). It just gets tedious and error-prone, as every repetive task done manually.
Moreover, the manual steps I listed for the examples are pretty optimistic - often, you want to do more. For example, PyQt adds the .ui-to-.py-compiler to the path so you can invoke it via command line.
So you end up with a stack of work that could be automated. This alone is a good argument.
Also, the devs would have to write installing instructions. With distutils etc, you only have to specify what your project consists of (and fancy extras if and only if you need it) - for example, you don't need to tell it to put everything in a new folder in site-packages, because it already knows this.
So in the end, it's easier for developers and for users.

what python modules ? for installing python package if they exist in pypi you should do :
pip install <name_of_package>
if not, you should download them .tar.gz or what so ever and see if you find a setup.py and run it like this :
python setup.py install
or if you want to install it in development mode (you can change in package and see the result without installing it again ) :
python setup.py develop
this is the usual way to distribute python package (the setup.py); and this setup.py is the one that call disutils.
to summarize this distutils is a python package that help developer create a python package installer that will build and install a given package by just running the command setup.py install.
so basically what disutils does (i will sit only important stuff):
it search dependencies of the package (install dependencies automatically).
it copy the package modules in site-packages or just create a sym link if it's in develop mode
you can create an egg of you package.
it can also run test over your package.
you can use it to upload your package to pypi.
if you want more detail see this http://docs.python.org/library/distutils.html

You don't have to use distutils to get your own modules working on your own machine; saving them in your python path is sufficient.
When you decide to publish your modules for other people to use, distutils provides a standard way for them to install your modules on their machines. (The "dist" in "distutils" means distribution, as in distributing your software to others.)

How to install distutils packages using distutils api or setuptools api

I'm working on a buildout script that needs to install a distutils package on remote server.
On PyPi there are 2 recipes for doing this
collective.recipe.distutils 0.1 and zerokspot.recipe.distutils 0.1.1.
The later module a derivative of the former, and is a little more convenient then the first, but the both suffer from the same problem, which I will describe now.
When bootstrap.py is executed, it downloads zc.buildout package and puts it into buildout's eggs directory. This gives ./bin/buildout access to zc.buildout code, but /usr/local/python does not know anything about zc.buildout at this point.
Buildout attepts to install the package by running 'python setup.py install' inside of a subprocess. This produces an ImportError because zc.buildout is not installed for /usr/local/python.
So, I have several solutions.
Install zc.buildout using easy_install on the remote server. I don't like this option at all, it makes a special case for a module that is very insignificant.
Modify zerokspot.recipe.distutils to put try block around 'import zc.buildout' this way, it will install even if zc.buildout is not installed. It's an ok solution, but somewhat hackish.
Replace subprocess with code that will install the package using distutils api or setuptools api. This would be the best solution in my opinion.
The question is how would i do #3?
Thank you,
Taras
PS: I solved the problem by creating another package that does not have dependancy on zc.buildout. My package is called taras.recipe.distutils and it's available on pypi.

You can call a command line program within your Python program using the subprocess module:
import subprocess
subprocess.call('python setup.py install')
However, how much control do you have over the environment that this install will be run? If it is a package that you are distributing, you will likely have issues no matter what solution people propose. How will you handle cases of needing root access (e.g. sudo python setup.py install)?
You may consider looking into Paver since it provides an API that is in some ways an extension of setuptools.

zerokspot.recipe.distutils is fundamentally broken in that it adds a dependency on zc.buildout in it's setup.py, as follows:
setup.py imports get_version from zerokspot.recipe.distutils
All of zerokspot.recipe.distutils is defined in it's __init__.py, including get_version
__init__.py in zerokspot.recipe.distutils imports zc.buildout
Why the author defines get_version is a mystery to me; best practice keeps a simple version string in setup.py itself and lets setuptools deal with dev versions (through setup.cfg), and distutils for version metadata extraction.
Generally it is not a good idea to import the whole package in setup.py as that would require all the package dependencies to be present at install time. Obviously the author of the package has zc.buildout installed as a site-wide package and didn't notice his oversight.
Your best bet is to fork the package on github, remove the get_version dependency, and propose the change to the original author while you use your fork instead.

Are you sure you don't want to just generate a bdist?