I've embedded python on a mobile device successfully, but now how do I include a python library such as urllib?
Additionally, how can I include my own python scripts without a PYTHONPATH?
(please note: python is not installed on this system)
The easiest way is to create a .zip file containing all the python code you need and add this to your process's PYTHONPATH environment variable (via setenv()) prior to initializing the embedded Python interpreter. Usage of .pyd libraries can be done similarly by adding them to the same directory as the .zip and including the directory in the PYTHONPATH as well.
Usage of the setenv() call can cause trouble on Windows if you're mixing c-runtime versions. I spent many aggrivating hours learing that setenv() only sets the environment variables for the version of the c-runtime your compiler ships with. So if, for example, Python was built with VC++ 2005 and your compiler is VC++ 2008, you'll need to use an alternative mechanism. Browsing the sources for py2exe and/or PyInstaller may provide you with a better solution (since you're doing essentially the same thing as these tools) but a simple alternative is to "cheat" by using PyRun_SimpleString() to set the module search path from within Python itself.
snprintf(buff, "import sys\nsys.path.append("%s")\n", py_zip_filename)
PyRun_SimpleString(buff)
Related
Using the c-python api I try to embed python 3.6 into a c++ application.
However instead of using the system install i'd like to use a virtual environment.
I didn't find any documentation on the way to do that.
Some related documentation mention
py_SetProgramName
or
py_SetPythonHome
Also when reading c-python code i can see the use of pvenv.cfg or ._pth files, but none of these solution seem to work.
Any idea what's the right way to use virtual environment from c api?
EDIT
Let's take a concrete example. I have python installed in
c:\python36
For my c++ application I created a virtual env using the command python -m venv c:\my_cpp_app\python_venv\ in:
c:\my_cpp_app\python_venv\
Using the c-python api I'd like to make my cpp application use the virtual environment located in python_venv instead of c:\python36\
As stated in comments, embedded python 3.6 and virtual environment created with venv seem incompatible (bugs.python.org/issue22213)
I managed to make it work using virtualenv instead and by calling Py_SetPythonHome prior Py_Initialize.
See more details on the python startup sequence
Locating Python and the standard library
The location of the Python
binary and the standard library is influenced by several elements. The
algorithm used to perform the calculation is not documented anywhere
other than in the source code. Even that description is
incomplete, as it failed to be updated for the virtual environment
support added in Python 3.3 (detailed in PEP 405).
These calculations
are affected by the following function calls (made prior to calling
Py_Initialize()) and environment variables:
Py_SetPythonHome()
Py_SetProgramName()
PYTHONHOME
The filesystem is also inspected for
pyvenv.cfg files (see PEP 405) or, failing that, a lib/os.py (Windows)
or lib/python$VERSION/os.py file.
The build time settings for PREFIX
and EXEC_PREFIX are also relevant, as are some registry settings on
Windows. The hardcoded fallbacks are based on the layout of the
CPython source tree and build output when working in a source
checkout.
The implementation of pep 587 in later versions should facilitate all this!
I am working with the 3.6.4 source release of Python. I have no trouble building it with Visual Studio as a dynamic library (/MDd) I can link the Python .dll to my own code and verify its operation.
But when I build it (and my code) with (/MTd) it soon runs off the rails when I try to open a file with a Python program. A Debug assertion fails in read.cpp ("Expression: _osfile(fh) & FOPEN"). What I believe is happening is the Python .dll is linking with improper system libraries. What I can't figure out is how to get it to link with the correct ones (static libraries).
This is what I needed to do to build and use python statically embedded in another application.
To build the static python library (e.g., python36_d.lib, python36.lib)
Convert ALL projects in the python solution (pcbuild.sln) to static. This is about 40 projects, so it may take awhile. This includes setting library products to be build as 'static lib', and setting all /MD and /MDd build options to /MT and /MTd.
For at least the pythoncore project alter the Preprocess define to be Py_NO_ENABLE_SHARED. This tells the project it will be looking for calls from static libraries.
By hook or crook, find yourself a pyconfig.h file and put it in the Include area of your Python build. It is unclear how this file is built from Windows tools, but one seems to be able to snag one from other sources and it works ok. One could probably grab the pyconfig.h from the Pre-compiled version of the code you are building. [By the way, the Python I built was 3.6.5 and was built with Windows 2015, update 3.]
Hopefully, this should enable you to build both python36.lib and python36_d.lib. Now you need to make changes to your application project(s) to enable it to link with the python library. You need to do this:
Add the Python Include directory to the General->Include Directories list.
Add the Python Library directories to the General->Library Directories lists.
This will be ..\PCBuild\win32 and ..\PCBuild\amd64.
Add the define Py_NO_ENABLE_SHARED to the C/C++ -> Preprocessor area.
For Linker->input add (for releases) python36.lib;shlwapi.lib;version.lib and (for debugs) python36_d.lib;shlwapi.lib;version.lib.
And that should be it. It should run and work. But one more thing. In order to be able to function, the executable needs to access the Lib directory of the python build. So a copy of that needs to be moved to wherever the executable (containing the embedded python) resides. Or you can add the Lib area to the execution PATH for windows. That should work as well.
That's about all of it.
Is there a way to distribute a python library with an application, so that it can be run with out any installation? The app is primarily going to be used in a computer lab, where users do not have permission to install global libraries. Ideally, users would simply be able to unzip a folder and run the app. The following can be assumed:
The python interpreter is present
Linux operating system
The specific library I need is matplotlib, but I would like to find a generic solution. I've looked at programs like PyInstaller, but they create very large programs that are slow to start. They also include a python interpreter, which is unnecessary.
Firstly, p2exe is Windows only.
In principle you can put all of you libraries into the ZIP file so they get expanded in place with thie application. At most you may need to adjust the PYTHONPATH variable to point at the lib's location.
There is no technical difference between modules installed on the path and in the system's python installation.
Have you looked at cxfreeze?
I can't seem to import any of the basic modules located in the "lib-dynload" directory. They are all there, but I get the error: "ImportError: No module named X" when trying to import them.
I checked my sys.path and it includes the directory where all of these modules are located and my PYTHONHOME environment variable is set correctly. I'm at a bit of a loss as to what the problem could be. Some background info: This is cross-compiled from Python 2.6.6 source and installed onto an ARM embedded Linux board with Angstrom.
It did have python on there before, I had tried to bit-bake it into the image but it was missing a lot of stuff. I ended up doing my best to clean the directory tree of anything to do with the previous python before loading on my cross compiled version.
An strace of a simple script that just attempts to import math: http://pastebin.com/3XgJ3nPR
I see no checks in that trace for filenames like math.so or mathmodule.so which might indicate that shared-object modules are turned off entirely — that the version of Python you have compiled cannot load binary modules dynamically.
More: looking over the config.out from my most recent Python build, I see several lines where Python is investigating whether the platform will let it dynamically load binary modules that end in .so:
checking for dlopen... yes
checking DYNLOADFILE... dynload_shlib.o
checking MACHDEP_OBJS... MACHDEP_OBJS
What do these lines say on your cross-compile?
I have recently run across a similar issue building Python 2.7.13, and I believe it is this bug which is being fixed for Python 3 but not ported back to 2. The build process (setup.py) generates a list of modules to build, and then subtracts the list of built-in modules (sys.builtin_module_names); however, setup.py is run (from the Makefile) using python2.7 which in my case picked up the system (Ubuntu) binary rather than the one built, so it subtracts off modules that are built-in for the system python (including operator and collections) but not for the one being built, so they are neither built-in nor built as external modules.
I was able to use a suggestion from the bug and prepend the built python, in the source directory, to the path (and add a symlink from python2.7 -> python). This worked because I was building an x86 python on a multi-arch x64 machine; if you are building for another system like ARM you may need to apply the patch from the bug to get the list of built-in modules from earlier in the build process rather than the host python.
I'm trying to use PyUNO as a method to convert different document formats (doc, wordperfect, html, etc) to PDF from within my Django server. I'm having a heck of a time getting import uno to work. It seems to fail when doing import pyuno, with a message of ImportError: DLL Load Failed: The specified module could not be found.
The only way I can get this to work is to use the Python 2.6 that came with OpenOffice, but I really want to use my other 2.6 installation. The docs for PyUNO are all for Python 2.2, and reputed to be out-of-date.
I'm guessing that some (or all) of the following files need to be copied from the OpenOffice directory to my site-packages directory (or some subdirectory thereof):
pythonloader.py
pythonloader.uno.ull
pythonloader.uno.ini
pythonscript.py
pyuno.pyd
Has anyone had any success getting this to work?
This is on Windows.
For simply conversions, you needn't reinvent the wheel. Look at unoconv: http://dag.wieers.com/home-made/unoconv/
'Import uno' will automatically work IF the python interpreter was bundled with OpenOffice, or in some Linux systems where the packagers have done a lot of work for you already.
Alternative 1: For other Python installs on Win32 systems, you need to import three environment variables and add one item to your Pythonpath. The detailed tutorial is at http://user.services.openoffice.org/en/forum/viewtopic.php?f=45&t=36370&p=166783
The three environment variables you must get FROM your OO-installed-Python and add TO your other install of Python are:
(Using Python 2.6 and OO 3.1.2)
os.environ['URE_BOOTSTRAP'] = 'vnd.sun.star.pathname:c:\Program Files\OpenOffice.org 3\program\fundamental.ini'
os.environ['UNO_PATH'] = 'c:\Program Files\OpenOffice.org 3\program\'
os.environ['PATH'].append('c:\Program Files\OpenOffice.org 3\URE\bin;c:\Program Files\OpenOffice.org 3\Basis\program;')
The pythonpath item you must add TO your other install of Python is the location of the uno module:
sys.path.append('C:\Program Files\OpenOffice.org 3\Basis\program')
Now you can simply 'import uno'.
Pyuno is only compatible with a similar version of Python. Since OO 3.1 bundles Python 2.6.1, that pyuno is only compatible with another Python 2.6. Attempting to import uno into a different version of Python will cause a runtime error. But there's a way around that in Alternative 2.
Alternative 2: For other Python installs on WIN32 systems, you can ignore the Python-UNO bridge completely and use the Python-COM bridge instead. You must install one new module, and the API has a few differences, but you can use ANY version of Python, including Python3.
Install the pywin32module to get COM access: http://sourceforge.net/projects/pywin32/
Discussion on the API differences: http://user.services.openoffice.org/en/forum/viewtopic.php?f=45&t=36608&p=168179
I also need an answer to this. I'm using windows XP and have a system where I extract data from a SQL server DB and insert it into template excel files. I DON'T want to add macro's etc to the OpenOffice. but I want to use the service to produce these files.
However, the only way I seem to be able to import the uno modude is to use the python.exe from the openoffice directory. even then I can't associate this exec with the py files in my system for some reason, thus I have to put in full path names each and every time.
I know that in Linux, the Pyuno module is an addon and can be used in normal python, but in windows this doesn't seem to be the case. I have listed all the relevant path details from the python exec under openoffice and duplicated them in python 2.7, but pyuno still fails with unable to find DDL with no reference to what DDL.
I think the answer is "this is not possible." From other reading on the web, it appears that the stdlib used to compile/link the python executable from python.org is different from the stdlib used to compile/link the python.exe distributed with OpenOffice.org. I don't know why, and I'm still confused by the fact that both pythons give me the same startup messages. So I could be completely on the wrong track here.