I found that some applications developed with python drop their files in this path, what is the use of this path, and what files should I put in it ?
That directory includes architecture-independent python modules that can be shared by multiple python version. Do not manipulate that directory.
See Debian Python Policy Chapter 1 - Python Packaging
Take a look at the Debian python policy.
1.5 Module Path
By default, Python modules are searched in the directories listed in
the PYTHONPATH environment variable and in the sys.path Python
variable. Since python2.4 version 2.4.5-3, python2.5 version 2.5.2-7,
python2.6 version 2.6.2-1, and in all python2.7 versions, sys.path
does not include a /usr/lib/pythonXY.zip entry anymore. Directories
with private Python modules must be absent from the sys.path. Public
Python modules not handled by python-central or python-support must be
installed in the system Python modules directory,
/usr/lib/pythonX.Y/dist-packages for python2.6 and later, and
/usr/lib/pythonX.Y/site-packages for python2.5 and earlier. Public
Python 3 modules must be installed in /usr/lib/python3/dist-packages.
Modules managed by python-support are installed in another directory
which is added to the sys.path using the .pth mechanism. The .pth
mechanism is documented in the Python documentation of the site
module. A special directory is dedicated to public Python modules
installed by the local administrator, /usr/lib/python3/dist-packages
for all python3 versions, /usr/local/lib/python2.Y/dist-packages for
python2.6 and later, and /usr/local/lib/python2.Y/site-packages for
python2.5 and earlier. For a local installation by the administrator
of python2.6 and later, a special directory is reserved to Python
modules which should only be available to this Python,
/usr/local/lib/python2.Y/site-packages (and
/usr/local/lib/python3/site-packages for all python3 versions).
Unfortunately, for python2.5 and earlier this directory is also
visible to the system Python. Additional information on appending
site-specific paths to the module search path is available in the
official documentation of the site module.
When binary packages ship identical source code for multiple Python
versions, for instance /usr/lib/python2.6/dist-packages/foo.py and
/usr/lib/python2.5/site-packages/foo.py, these should point to a
common file. Version specific directories for identical source code
are not required for python3 and must not be used for this. A common
location to share, across Python versions, arch-independent files
which would otherwise go to the directory of system public modules is
/usr/share/pyshared. For python3, a special location is not required,
use /usr/lib/python3/dist-packages
1.6 Hooks for updates to installed runtimes
The python binary package has special hooks to allow other packages to
act upon updates to the installed runtimes. This mechanism is required
to handle changes of the default Python runtime in some packages and
to enable the Python packaging helpers. There are three supported hook
types which come in the form of scripts which are invoked from the
maintainer scripts of the Python runtime packages when specific
installations, removals, or upgrades occur.
/usr/share/python/runtime.d/*.rtinstall: these are called when a
runtime is installed or becomes supported. The first argument is
"rtinstall", the second argument is the affected runtime (for example
pythonX.Y) and the third and fourth argument are the old and new
version of this packaged runtime if this runtime was already installed
but unsupported.
/usr/share/python/runtime.d/*.rtremove: these are called when a
runtime is removed or stops being supported. The first argument is
"rtremove", and the second argument is the affected runtime (for
example pythonX.Y).
/usr/share/python/runtime.d/*.rtupdate: these are called when the
default runtime changes. The first argument is either "pre-rtupdate",
called before changing the default runtime, or "rtupdate", called when
changing the default runtime, or "post-rtupdate", called immediately
afterwards. The second argument is the old default runtime (for
example pythonX.Y), and the third argument is the new default runtime
(for example pythonX.Z).
Related
Using the c-python api I try to embed python 3.6 into a c++ application.
However instead of using the system install i'd like to use a virtual environment.
I didn't find any documentation on the way to do that.
Some related documentation mention
py_SetProgramName
or
py_SetPythonHome
Also when reading c-python code i can see the use of pvenv.cfg or ._pth files, but none of these solution seem to work.
Any idea what's the right way to use virtual environment from c api?
EDIT
Let's take a concrete example. I have python installed in
c:\python36
For my c++ application I created a virtual env using the command python -m venv c:\my_cpp_app\python_venv\ in:
c:\my_cpp_app\python_venv\
Using the c-python api I'd like to make my cpp application use the virtual environment located in python_venv instead of c:\python36\
As stated in comments, embedded python 3.6 and virtual environment created with venv seem incompatible (bugs.python.org/issue22213)
I managed to make it work using virtualenv instead and by calling Py_SetPythonHome prior Py_Initialize.
See more details on the python startup sequence
Locating Python and the standard library
The location of the Python
binary and the standard library is influenced by several elements. The
algorithm used to perform the calculation is not documented anywhere
other than in the source code. Even that description is
incomplete, as it failed to be updated for the virtual environment
support added in Python 3.3 (detailed in PEP 405).
These calculations
are affected by the following function calls (made prior to calling
Py_Initialize()) and environment variables:
Py_SetPythonHome()
Py_SetProgramName()
PYTHONHOME
The filesystem is also inspected for
pyvenv.cfg files (see PEP 405) or, failing that, a lib/os.py (Windows)
or lib/python$VERSION/os.py file.
The build time settings for PREFIX
and EXEC_PREFIX are also relevant, as are some registry settings on
Windows. The hardcoded fallbacks are based on the layout of the
CPython source tree and build output when working in a source
checkout.
The implementation of pep 587 in later versions should facilitate all this!
The venv module (shipped with Python 3.3 or later), and virtualenv, still widely in use, allow to install a project's dependencies not to the system-wide Python installation, but to a directory specific to that project.
One of the subdirectories of such a "virtual environment" contains a copy of the Python interpreter as well as "activate" and "deactivate" scripts - but this subdirectory is called Scripts on Windows, and bin on all other systems.
This is somewhat surprising. Why did they special-case Windows?
(Neither PEP 405, nor the venv or virtualenv sources (or docs) contain any explanation - a commit message in virtualenv refers to "convention")
"I think the commit message is the best you'll get. Everything else will be pure speculation." (Bryan's comment, refering to the commit message in virtualenv)
Most ms-windows programs have a GUI which is started by an icon or menu-entry. So there is no need for a standardized location for binaries (which is then put in the $PATH) such as UNIX has. Also, the name bin wouldn't mean anything like it does to UNIX users.
Additionaly, ms-windows only has a very primitive package management (if you can even call it package management), so applications tend to be installed in their own directory tree where they won't interfere with each other.
As described in: http://bugs.python.org/issue22139, the python27.dll is installed in the windows systems (in my case C:\Windows\Systems32) folder.
But I would like to know why? Why is it not installed next to the python.exe, for example in C:\Python27\?
Reason I ask: I've made a mercurial hook in python that our developers need to use to check if the commit message is valid. It checks a.o. for a valid JIRA issue number. To prevent all our developers to install python themselves and install the required modules manually (a lot of work and errorprone), I zipped the python installation and asked the developers to unzip it locally. But they can't run it, because the python27.dll is missing, or worse, they already have another minor version of python installed, and the hook will fail due to the wrong python27.dll used. Confusing.
If I just add the python27.dll (the correct version) to the zip file, it all seems to work great. So, why is it not installed in that location in the first place? What is the advantage of installing it in C:\Windows\System32?
Hope someone can explain this to me!
Thanks in advance,
Tallandtree.
I use the Anaconda Python distribution from http://continuum.io. They put python27.dll into c:\anaconda right next to its python.exe. This distribution is also superior in that you can have multiple python environments with precisely the packages you need and switch between them easily (http://conda.pydata.org/docs/using/envs.html). You can also get the package list of one of your environments and distribute it to others.
I recommend this Python distribution over the one from python.org and Enthought, because of this issue.
.dlls are quite windows-specific files. I imagine you will have shared object (.so) files for LINUX/UNIX-specific Python stuff? You said your developer's couldn't run it, because they didn't have the correct DLL (i.e. the one relevant to their Python installation).
Also, the advantage of installing it to System32 is that it's in the default PATH. Additionally, if any other application is internally using Python and require access to the .dll file, and also NOT reference your Python directory, they will probably be looking for a location that "Actually" exists (I wanted to say guaranteed to exist, but......never mind). That location would be `C:/windows/Systems32'.
I found it to work just fine to put python27.dll in the Python directory (c:\Python27 or wherever). As long as it's in the PATH, it seems to work. I did this for a "relocatable" installation of Python. I can copy the installation directory to a Windows machine that has no Python installed, set the PATH to include that directory, and run Python, including all the libraries I had installed with pip install on the original machine.
I have a system python installation and a personal python installation in my home directory. My personal python comes in ahead in my $PATH and I usually run that. But the system python installation has some modules that I want to use with my personal python installation.
So, basically, I want to append the sys.path of the system python installation to the sys.path of the personal python installation.
I read up on the docs and source of site module and saw that I could use the sitecustomize or usercustomize modules to do this. But where I am stuck is how do I get the sys.path of the system python to be appended to the personal python's sys.path.
The gensitepackages function in the site modules seems to calculate the paths to be added to sys.path but it is using the PREFIXES global variable instead of taking it as an argument, so for all I know, I can't use it. Adding system python's prefixes to PREFIXES is also not an option as by the time the customize module(s) are loaded, the PREFIXES is already used to build the path.
Any ideas on how to go about this?
Also, I'm not sure if I should ask this on askubuntu/unix&linux. Comments?
Edit: Guess I wasn't clear on this part. I want the system python's path to be appended so that when I try to use modules that are not present in my personal python, it will automatically fallback to the system python's modules.
Either set the PYTHONPATH environment variable or, in your personal Python installation's site-packages directory, add one or more path configuration files (.pth files) that point to the directories you want to add from the system Python installation.
UPDATE: In general, it's not a good idea to mix modules from different Python instances. For instance, there could be differences in the configure parameters used to build the two Python that could produce incompatibilities in C extension modules built for the existing system Python but used with your personal Python. But, if you really want to go down this path, you could manipulate the path with .pth files or modify the site module. Or what might be a more-straightforward and easier-to-manage approach is to use virtualenv to create a private virtual environment using the system Python but then replace the interpreter in the environment; see, for example, the section Using Virtualenv without bin/python.
python is gonna check if there is a $PYTHONPATH environment variable set. Use that for the path of your other modules.
use export PYTHONPATH="path:locations"
I've embedded python on a mobile device successfully, but now how do I include a python library such as urllib?
Additionally, how can I include my own python scripts without a PYTHONPATH?
(please note: python is not installed on this system)
The easiest way is to create a .zip file containing all the python code you need and add this to your process's PYTHONPATH environment variable (via setenv()) prior to initializing the embedded Python interpreter. Usage of .pyd libraries can be done similarly by adding them to the same directory as the .zip and including the directory in the PYTHONPATH as well.
Usage of the setenv() call can cause trouble on Windows if you're mixing c-runtime versions. I spent many aggrivating hours learing that setenv() only sets the environment variables for the version of the c-runtime your compiler ships with. So if, for example, Python was built with VC++ 2005 and your compiler is VC++ 2008, you'll need to use an alternative mechanism. Browsing the sources for py2exe and/or PyInstaller may provide you with a better solution (since you're doing essentially the same thing as these tools) but a simple alternative is to "cheat" by using PyRun_SimpleString() to set the module search path from within Python itself.
snprintf(buff, "import sys\nsys.path.append("%s")\n", py_zip_filename)
PyRun_SimpleString(buff)