Develop Reference

Can pip install editable avoid recompiling from sratch?

I'm developing a python module with a long pybind11 C++ extension compilation step invoked via cmake in setup.py.
When making changes to a C++ file, cmake invoked via python setup.py develop will avoid recompiling units whose dependent files have not changed. However, invoking setup.py directly ignores the settings in my pyproject.toml and I understand that the modern way to do a developmental build is with python -m pip install -e .
While pip install -e successfully builds, it unfortunately starts from scratch inside a clean temporary directory every invocation. Is there a way to instruct pip install -e to maintain my CMakeCache.txt and compilation dependency tracking?
(And/or does this somehow indicate that I misunderstand pip install -e or am using it incorrectly?)
This previous unanswered question is quite similar sounding. Perhaps, I have the added detail that my python setup.py develop is working in this regard.

compile translation files when calling setup.py install

I'm developing a Flask application using Babel. Thanks to Distutils/Setuptools Integration, all the parameters of compile/extract/... functions are stored in setup.cfg and compiling the i18n files is as easy as
./setup.py compile_catalog
Great. Now I would like this to be done automatically when running
./setup.py install
In make's words, that would be letting install target depend on compile_catalog target.
The context
We store only translation (.po) files in the code repository. .gitignore excludes .mo and .pot files from being tracked.
When a developer pulls a new revision of the code, he runs
pip install -r requirements.txt
to update dependencies and install the project in development mode. Then, using above command line, he compiles the translation binary (.mo) files.
Is there a simple and recommended way to modify setup.py to do both operations in one step? Or am I trying to misuse setuptools?
Using a script that like this would work for development purposes:
#!/bin/sh
./setup.py compile_catalog
pip install -r requirements.txt
but I would like a solution that also works when the package is installed with usual setup.py install instructions, like if installed from PyPi.
Should I understand that setuptools are not meant to be used like this, and people distributing software compile their translation files either manually or using custom scripts when creating their archives, rather than relying on setup.py to compile them at installation time?
I didn't find many posts on the Internet addressing this. The ones I found involved running pybabel command line interface from a function in setup.py, which sounds like a shame as it misses the point of setuptools integration.

I think your demand is totally valid and I'm quite surprised that there seems to be no official guideline on how to accomplish this.
The project I working on now also went multilingual and this is what I did:
In setup.cfg, make appropriate entries so that compile_catalog can be run without options.
In setup.py, subclass the install command from setuptools:
setup.py:
from setuptools import setup
from setuptools.command.install import install
class InstallWithCompile(install):
def run(self):
from babel.messages.frontend import compile_catalog
compiler = compile_catalog(self.distribution)
option_dict = self.distribution.get_option_dict('compile_catalog')
compiler.domain = [option_dict['domain'][1]]
compiler.directory = option_dict['directory'][1]
compiler.run()
super().run()
Then, when calling setup(), register our InstallWithCompile command with the name "install" and make sure that the *.mo files will be included in the package:
setup(
...
cmdclass={
'install': InstallWithCompile,
},
...
package_data={'': ['locale/*/*/*.mo', 'locale/*/*/*.po']},
)
Since babel is used during the setup, you should add it as a setup dependency:
setup_requires=[
'babel',
],
Note that a package (here, babel) appearing in both setup_requires and install_requires won't be installed correctly using python setup.py install due to an issue in setuptools, but it works fine with pip install.

Installing Python Module to non-root directory

First time posting on this site.
I'm trying to install a python module named Pygments into my local directory at work (non-root). I am using C Shell, so i changed default path using
setenv PYTHONPATH "~/usr/lib/python2.4/site-packages:${PYTHONPATH}"
(with usr/lib/... being self-made empty directories made to mimic the system's hierarchy at my boss's suggestion). I attempted to run
python setup.py install
however, I got the following error.
/usr/lib64/python2.4/distutils/dist.py:236: UserWarning: Unknown distribution option: 'zip_safe'
warnings.warn(msg)
/usr/lib64/python2.4/distutils/dist.py:236: UserWarning: Unknown distribution option: 'include_package_data'
warnings.warn(msg)
running install
running build
running build_py
running build_scripts
running install_lib
creating /usr/local/lib64
error: could not create '/usr/local/lib64': Read-only file system
I want the module to install to my local directory and not to any root directory. I've been stuck on this for a couple of days. Any help would be greatly appreciated.
Colten
Extra Information: I have setup.py in ~/Pygments-1.6, and I only have write access to directories within ~/.

PYTHONPATH is where python looks for modules, not where it chooses to install modules. You need to specify that on the setup.py line:
python setup.py install --prefix=${HOME}/usr/
or something similar. Another thing that you'll see in these cases is:
python setup.py install --user
which will put it in: '${HOME}/.local/lib/pythonX.Y/site-packages' (And I believe that this path should be searched by python for modules by default.)

Setup tools install command differences

What are the differences between the below commands
python setup.py install develop
Doesn't work for me error No such file or directory: 'build/bdist.macosx-10.7-intel/egg/test-easy-install-37886.pth'
python setup.py develop
Works for me appears to make an .egg link file
python setup.py install
Works for me appears to make a .egg file which in .zip file format

Develop is a setuptools / distribute feature that allows you to add a project
to your Python environment without installing it - so you can continue
its "development"
In other words, when you call "python setup.py develop", setuptools will
compile the metadata and hook your project into Python's site-package,
but the packages and modules that will be used are the one in the
directory where you've run that command.
This is useful to continue working on your code and testing it without
having to run "python setup.py install" on every run
With develop, Python 'pseudo-installs' a package by running the setup.py script instead of install. The difference is a modification of the environment (it doesn't with develop), so a package can be imported from it's current location instead of a site-package directory. The advantage of this is you can develop packages that are being used by other packages, and you can modify source code in place with develop.
As far as "setup.py install develop", I've never seen anyone use that before, sorry.
source
source
source

python setup.py install develop
Is a wrong command.
When you use develop you use the current code when you run your application.
When you use install and then modify you code, your modifications will not be taken in account while running your app. until you rerun install or develop.

Can not install hcluster from pypi under python 2.6 on xp

I am using the setup.py file supplied with hcluster with the following lines added:
sys.path.append("c:\\Program Files\\Python26\\Lib\\site-packages\\hcluster-0.2.0")
sys.path.append("c:\\Program Files\\Python26\\Lib\\site-packages\\hcluster-0.2.0\\hcluster")
Then used setup.py as follows:
"c:\program files\python26\python.exe" "c:\Program Files\Python26\Lib\site-packages\hcluster-0.2.0\setup.py" install
I get the following error messages:
running install
running build
running build_py
error: package directory 'hcluster' does not exist
Don't know if it trying to read or write hcluster.
Any help appreciated

You don't need to add packages in site-packages in sys.path.
Did you copy the hcluster in site-package manually? It is not the correct way to do it.
2.1 You should have the hcluster outside the site-packages say in your home directory and then run "python setup.py install"
2.2 This will put the package after build into site-package directory. This is where all external package reside by default after they are installed.
Remove the folders related to hcluster from site-packages and install with instruction 2.
Read the following to understand your error: http://docs.python.org/install/index.html