I found issues on installing a cross compiler on my Ubuntu19.04 64 bit machine. I would like to cross compile python code into executable for my raspberry pi 3 model b+ running Debian Stretch.
I followed many guides, none of them worked. I am actually following https://github.com/Yadoms/yadoms/wiki/Cross-compile-for-raspberry-PI
I followed step of the above written guide:
- Setup Environment
- Install cross compiler
- Boost 1.64
- Python
On the last part (Python) it fails to execute the last instruction.
$ CC=arm-linux-gnueabihf-gcc CXX=arm-linux-gnueabihf-g++ AR=arm-linux-gnueabihf-ar RANLIB=arm-linux-gnueabihf-ranlib ./configure --host=arm-linux-gnueabihf --target=arm-linux-gnueabihf --build=x86_64-linux-gnu --prefix=$HOME/Desktop/rapsberry/depsBuild/python --disable-ipv6 ac_cv_file__dev_ptmx=no ac_cv_file__dev_ptc=no ac_cv_have_long_long_format=yes --enable-shared
output:
checking build system type... x86_64-pc-linux-gnu
checking host system type... arm-unknown-linux-gnueabihf
checking for python3.7... python3.7
checking for python interpreter for cross build... python3.7
checking for --enable-universalsdk... no
checking for --with-universal-archs... no
checking MACHDEP... checking for --without-gcc... no
checking for --with-icc... no
checking for arm-linux-gnueabihf-gcc... arm-linux-gnueabihf-gcc
checking whether the C compiler works... no
configure: error: in `/home/slr/Desktop/raspberry/boost_1_64_0/Python-3.7.5':
configure: error: C compiler cannot create executables
See `config.log' for more details
and then:
$ make HOSTPYTHON=$HOME/Desktop/raspberry/depsBuild/pythonhost/python HOSTPGEN=$HOME/Desktop/raspberry/depsBuild/pythonhost/Parser/pgen BLDSHARED="arm-linux-gnueabihf-gcc -shared" CROSS-COMPILE=arm-linux-gnueabihf- CROSS_COMPILE_TARGET=yes HOSTARCH=arm-linux BUILDARCH=arm-linux-gnueabihf
output:
make: *** No targets specified and no makefile found. Stop.
I need python3 for my purpose.
I am really stuck in this problem, could someone have some idea? I tried also with QEMU and Docker (https://raspberrypi.stackexchange.com/questions/109488/building-a-virtual-machine-with-the-img-file-of-the-raspberry-pi-stretch) and both of them failed in compiling my target code:
gcc: internal compiler error
My code is pretty long ( Some thousand of lines ), while small codes successfully works. Thanks in advice.
There seems to be something wrong with the toolchain you're using, or with the way you're invoking the Python configure script.
Either way, it's impossible to debug without seeing your exact setup, so I'll start from scratch here.
I'm in the process of documenting a similar project here: Raspberry Pi C++ development.
Toolchain
The toolchains in the raspberrypi/tools repository are pretty old. I usually just build a new one using Crosstool-NG (which is what the raspberrypi/tools toolchains were built with as well, IIRC).
I used the armv8-rpi3-linux-gnueabihf sample.
You can of course build it yourself, but this can take quite some time, so you can also download the one I built from Docker Hub (see later).
You can find more information about how it was built here: Detailed information about configuring and building toolchains
.
Compiling Python for the build system
In order to cross-compile a Python module, you need the same version of Python twice: once for your build system (the computer you're building on) and once for the host system (the Raspberry Pi you're building for).
Both will be compiled from source, to ensure that they are exactly the same version.
The build Python will just be used for cross-compiling the host Python, so we won't enable optimizations and optional modules.
This is the script I used: python-build.sh
You need OpenSSL, Zlib and libffi for pip to work. I built them from source as well, but you could of course install them using your package manager (you need the -dev versions).
Again, you can find the install scripts I used here.
Cross-compiling Python for the host system
Before you can cross-compile Python for the Raspberry Pi, you'll have to cross-compile its dependencies. A detailed explanation can be found here: Cross-compiling the dependencies.
You can find the scripts in the same folder on GitHub I linked to earlier, for instance, python.sh.
There are some caveats when cross-compiling, you need a pkg-config with the right prefix that looks for the required libraries in the sysroot of your cross-compilation, instead of in your build system's library folders. You also have to specify the include directories and library folders when calling the configure script.
All of this is handled by this Dockerfile and the scripts in the same folder.
Crossenv
The easiest way to cross-compile Python modules is to use Crossenv. The instructions can be found in the README of the GitHub page.
When everything is set up, you can run python setup.py bdist_wheel.
Example
As an example, you can follow these steps to compile a simple Python module using Cython:
1. Pull the toolchain and the cross-compiled Python from Docker hub
This is an image that contains the cross-compilation toolchain, native and cross-compiled Python, as well as Crossenv.
docker pull tttapa/rpi3-armv8-python-opencv-cross
2. Create the Python file to compile and setup.py to build it
Create these two files:
helloworld.py
print("Hello, World")
setup.py
from setuptools import setup
from Cython.Build import cythonize
from sys import version_info
setup(
name='helloworld',
version='0.0.1',
ext_modules = cythonize("helloworld.py", language_level=version_info[0])
)
3. Create a script that builds the Python module
This script will be run inside of the Docker container.
build.sh
#!/usr/bin/env bash
set -e
cd /tmp/workdir
source "$HOME/crossenv/bin/activate"
build-pip install cython
cross-pip install wheel
python setup.py bdist_wheel
4. Start the Docker container and build the module
Run the following command in the same folder where you created the three files.
docker run --rm -v "$PWD:/tmp/workdir" \
tttapa/rpi3-armv8-python-opencv-cross \
bash "/tmp/workdir/build.sh"
When the build is complete, you'll find a file dist/helloworld-0.0.1-cp38-cp38-linux_arm.whl, which you can install using pip install helloworld-0.0.1-cp38-cp38-linux_arm.whl on the Raspberry Py. You'll also find the helloworld.c file generated by Cython.
To be able to run it, you'll probably have to install the cross-compiled libraries and Python itself to the RPi. You can do this by copying everything inside of ~/RPi-staging (inside of the Docker container) to the Pi's root folder /.
Related
Trying to deploy a small python web server to run on a Machine with QNX7.1 OS installed.
For various reasons, I cannot use docker, Pip is also not available on the machine nor can I just pre-install the third party libs I need on the machine (through something like apt-get python-aiohttp etc.). The deployment must be an easily executable file.
I've chosen Pex as the build tool, which will build in a CI Pipe somewhere and be made available over a custom service that behaves like scp.
My issue is that I'm using a couple of third party libraries and am having trouble getting the QNX compatible python 3rd party package installed at pex build time.
These are the libs I require (Not fixated on specific versions):
aiohttp==3.7.1
aiohttp-cors==0.7.0
aiohttp-jinja2==1.5
aiosignal==1.2.0
async-timeout==4.0.2
attrs==21.4.0
charset-normalizer==2.0.12
frozenlist==1.3.0
idna==3.3
Jinja2==3.1.1
MarkupSafe==2.1.1
multidict==6.0.2
RoboConfigApp==20220427
yarl==1.7.2
From the pex documentation; there's an option to specify a platform however I'm clearly not using the correct platform string or not understanding what's happening.
Output of uname -a on the machine is: QNX-7.1.0-x86pc-x86_64 Python is 3.8.6 is system python which I must use (yes I've read the QNX docs relating to it's use.).
I've tried setting the platform as:
PLATFORM=qnx-x86_64-cp-3.8.6-cp38
PLATFORM=QNX-7.1.0-x86pc-x86_64
PLATFORM=unix-x86_64-cp-3.8.6-cp38
PLATFORM=linux-x86_64-cp-3.8.6-cp38
PLATFORM=unix-cp38-cp38-qnx_7_1_0_x86pc
PLATFORM=unix-x86pc-x86_64-cp-3.8.6-cp38
Does someone understand my issue and can point me in the right direction?
For some time I was using opencv (4.5.2 compiled from source) with python 3.8.5 (on Ubuntu 18.04), but for some project, I do need to work with tensorflow, which is not working with my python version. So I installed python3.7 along it (from source), and everything seems to work when starting my pipenv shell, I can work with my python3.7.
But when trying to import cv2, I got the following error:
ImportError: OpenCV loader: missing configuration file:
['config-3.7.py', 'config-3.py']
So I believe I should re-compile OpenCV targeting my specific python3.7, but I also want to make it work with my former python3.8. How can I achieve this, targeting the two versions?
Thanks for you time!
Antoine
Well, in the end I succeeded in having the config files for both python3.7 and 3.8 with opencv built from source doing the following:
I first compiled OpenCv without writing any parameters for pythons. So by default, it detected my Python3.8 and created the config-3.8.py file.
Then, I recompiled OpenCv targeting my Python 3.7 with those arguments:
cmake -DOPENCV_EXTRA_MODULES_PATH=../opencv_contrib-master/modules ../opencv-master -D PYTHON3_EXECUTABLE='/usr/local/bin/python3.7m' \
-D PYTHON_INCLUDE_DIR='/usr/local/include/python3.7m' \
-D PYTHON3_LIBRARY='/home/ubuntu/Documents/Libraries/python3.7/Python-3.7.4/libpython3.7m.so' \
-D PYTHON3_NUMPY_INCLUDE_DIRS='/usr/local/lib/python3.7/site-packages/numpy/core/include' \
-D PYTHON3_PACKAGES_PATH='/usr/local/lib/python3.7/site-packages' ..
Note that the PYTHON3_LIBRARY has to target the shared library(.so and not .a). I first compiled Python3.7 without the flag --enable-shared, so it was not working.
Right before running cmake (from OpenCv), during the config step, you should end up with those printed information, to be sure you will target the right python: (Here it was for python3.8, but following the above steps will show python3.7)
Those PYTHON_FLAG and their usages are not in the latest Opencv documentation like the 4.5.2, but only in the previous ones like the 3.2.0:
https://docs.opencv.org/4.5.2/d7/d9f/tutorial_linux_install.html
https://docs.opencv.org/3.2.0/d7/d9f/tutorial_linux_install.html
Then I was able to import cv2 both in 3.7 and 3.8
to solve that issue on Windows, it's pretty similar. The difference is that you have to make sure that you found the proper folder where pyinstaller is loading info from, because it's tricky once you have other instances other than the cv2 container.
In my machine, for example, I could only solve it when I passed the path as
--path=AppData\Local\pyinstaller\bincache00_py39_64bit\cv2
Add that flag at the end of your pyinstaller command and everything should work just fine.
I am currently using OpenCV 4.5.5 (the most recent version as of today). I was able to get the same output as DiXcipuli using the following:
cmake ../
-DINSTALL_PYTHON_EXAMPLES=ON \
-DBUILD_EXAMPLES=ON \
-DBUILD_opencv_python3=ON
It takes the python interpretter that is currently set as python3 in your machine (say "which python3" on the terminal).
i am trying to install pycairo 1.10.0 for use with my custom-build python 3.1. however,
sudo /flower/bin/easy_install-3.1 pycairo
fails with
XXX#XXXX:/adventures$ sudo /flower/bin/easy_install-3.1 pycairo
install_dir /flower/lib/python3.1/site-packages/
Searching for pycairo
Reading http://pypi.python.org/simple/pycairo/
Reading http://cairographics.org/pycairo
Best match: pycairo 1.10.0
Downloading http://cairographics.org/releases/pycairo-1.10.0.tar.bz2
Processing pycairo-1.10.0.tar.bz2
error: Couldn't find a setup script in /tmp/easy_install-zeG9HB/pycairo-1.10.0.tar.bz2
and indeed, there is no setup.py in the said download; instead, the INSTALL says:
Install Procedure
-----------------
$ ./waf --help # shows available waf options
$ ./waf configure # use --prefix and --libdir if necessary
# --prefix=/usr --libdir=/usr/lib64 for Fedora 64-bit
$ ./waf build
$ ./waf install
Use
$ python3 ./waf ...
if you have python2 and python3 installed, and the default is python 2.
Testing
-------
See test/README
i understand that as telling me that i should
sudo /flower/bin/python3.1 ./waf configure --prefix=/flower/pycairo/
or similar; however, this leads to the following error:
./options()
Setting top to : /tmp/pycairo-1.10.0
Setting out to : /tmp/pycairo-1.10.0/build_directory
./configure()
Checking for 'gcc' (c compiler) : ok
Checking for program python : /usr/bin/python
python executable '/usr/bin/python' different from sys.executable '/flower/bin/python3.1'
Checking for python version : (2, 6, 5, 'final', 0)
The python version is too old, expecting (3, 1, 0)
so i went down into the source code; there is seemingly no way to tell this waf thingie that the targetted python version is simply the one it runs on itself, so i fumbled around and got as far as this:
Checking for 'gcc' (c compiler) : ok
Checking for program python : /usr/bin/python
#############293 /flower/bin/python3.1
Checking for python version : (3, 1, 2, 'final', 0)
Checking for library python3.1 in LIBDIR : yes
Checking for program python3.1-config : not found
Checking for program python-config-3.1 : not found
Checking for header Python.h : Could not find the python development headers
Checking for [] : not found
The configuration failed
i do in fact have a file /flower/bin/python3.1-config so i don't get it. the python code that does all this is rather hard to handle.
any suggestions how to go on? is there a *.deb or similar ready for pycairo + python 3? couldn't find any.
By looking at the python.py file in that subdir I decided to try setting an environment var before executing the waf thru python3:
export PYTHON="python3"
And then the install mysteriously succeeded...
I'm using Ubuntu Karmic, but installing with python2.7 compiled from source, so this is a work around I found in my case. I though it might be helpful, but please use caution.
I found some discussion threads that has similar installation problems here
Although it's not the same python version, there are some flags which you can set for waf.
I got past that Python.h part after using the following command
LDFLAGS="-lm -ldl -lutil" ./waf configure
As I have compiled from source python2.7, I need to reroute the paths with CFLAGS.
The final command looks like this:
CFLAGS="-l/usr/local/include/python2.7 -l/usr/local/bin" LDFLAGS="-lm -ldl -lutil" ./waf configure
I have no idea how the LDFLAGS work, so use with caution.
Alternatively (dangerous method), if you don't want to set the CFLAGS, I did do some symbolic-re-linking in the /local/bin of python to my python2.7, same with python-config to python2.7-config (if you installed from package manager, you might not need to do this)
This is clearly a bug in the pycairo installer, where it tries to figure out where the Python exe is (when what it should do is to simply use the Python exe that it is run with).
I tried to find it, but this code is using its own completely custom build system (it's not onvious why) and the code is laid out in a very weird way, astonishingly with major parts of the code hidden(!) in a directory called .waf3-1.6.4-e3c1e08604b18a10567cfcd2d02eb6e6 and written in some of the most unreadable Python code I've ever seen. As such it would take me hours to understand this code and help you with what probably is a trivial bug.
If you really need this library I suggest you contact the author and ask him nicely if he can fix the bug.
Trying to build an app on the Mac using py2app. Got everything working fine on my machine, but when moving the app to another, it crashes and the console displays this error.
ERROR: pygame.macosx import FAILED
Anybody have a solution to this?
Found the problem and solution after many hours. Turns out other people have experienced similar problems and their articles were quite helpful:
http://b.atcg.us/blog/2010/04/13/py2app-hell-the-first.html
http://www.vijayp.ca/blog/?p=62
In case someone else runs into the issue, this particular problem was caused because the Python framework was not being bundled into the application. You can confirm this by right-clicking your app to view package contents, then proceed to Contents/Frameworks/. If Python.framework is not there, it should be.
Be sure to download Python -
My first issue was reliance on Apple's build in Python package. Don't use this. You need to install your own version of python. Go to http://www.python.org/download/releases/, find a version (I stuck with 2.6), download the gzip (not the mac package), and install with the following if you are running Snow Leopard:
./configure --enable-framework MACOSX_DEPLOYMENT_TARGET=10.6 --with-universal-archs=intel --enable-universalsdk=/
make
sudo make install
Adjust Paths, Install Packages - From here there you need to adjust your paths to ensure you are using your custom-installed version. From here, I reinstalled the following packages - this turned out to be a dependency nightmare so I'm including the version numbers as well:
py2app 0.5.2
macholib 1.3
modulegraph .8.0
If these packages actually worked, you should be able to build and run your app now. Unfortunately, they don't. I'll go into the errors and my hacked solutions in a bit, but there's some settings in the build file that need to be made first.
First the setup.py file should like a little somethin' like this:
setup.py
from setuptools import setup
APP = ['Game.py']
DATA_FILES = ['data']
OPTIONS = {
"argv_emulation": False,
"compressed" : True,
"optimize":2,
"iconfile":'data/game.icns',
}
setup(
app=APP,
data_files=DATA_FILES,
options={'py2app': OPTIONS},
)
then to be extra safe, I use a shell script to call this.
build.sh
## Remove previous builds. Start with clean slate.
rm -rf build dist
## Force python into 32 bit mode.
export VERSIONER_PYTHON_PREFER_32_BIT=yes
## Force build with custom installed python
/Library/Frameworks/Python.framework/Versions/2.6/bin/python setup.py py2app
Running build.sh should compile the app. If it does not compile, I have good news -- it's not your fault. Due to glitches in the libraries, you may run into some (or all) of the following:
Potential Problems
If the build script fails, scan the traceback for some of the following keywords:
pygame not found - basic path problem in py2app. Add...
sys.path.insert(0, os.path.join(os.getcwd(), 'lib', 'python2.6','lib-dynload')) ## Added to fix dynlib bug
after the import statements in boot_app.py in the py2app lib.
pythonNone - This appears to be a bug in the macho package where it cannot determine the version number of your python build. To solve this, I added the following lines to build_app.py in py2app.
## Add these two lines...
if not info["version"]:
info["version"] = "2.6"
## Before this line. (line 941 in method copy_python_framework() at time of writing)
pydir = 'python%s'%(info['version'])
No such file or directory...Python.framework/[lib|include] - py2app is simply looking for directories that exist deeper in the file system tree. Go to the Python.framework directory and symlink up the place...
cd /Library/Frameworks/Python.framework
sudo ln -s Versions/Current/include/ include
sudo ln -s Versions/Current/lib lib
That should do it! - These steps created a compiled app that worked on other intel machines.
Thank you for posting what you found!
I had a similar problem. I tried various combinations of what you suggested, and isolated the single issue for me to be the bug in boot_app.py which you identify above.
Once I added the one-line fix to boot_app.py which you identify above, everything worked, even using the pre-installed Apple build of python (version 2.6.1).
I should note that when I say "everything worked," I really mean building a py2app app for actual distribution, i.e. using the normal command:
python setup.py py2app
The "alias" mode. i.e.
python setup.py py2app -A
which the py2app documentation suggests for use during development, still does not work for me (with the same module not found error). But better the actual distribution build working than nothing at all! Again, thanks.
I'm trying to install PySide v0.3.1 in Mac OS X, for Qt development in python.
As a pre-requisite, I have installed CMake and the Qt SDK.
I have gone through the documentation and come up with the following installation script:
export PYSIDE_BASE_DIR="<my_dir>"
export APIEXTRACTOR_DIR="$PYSIDE_BASE_DIR/apiextractor-0.5.1"
export GENERATORRUNNER_DIR="$PYSIDE_BASE_DIR/generatorrunner-0.4.2"
export SHIBOKEN_DIR="$PYSIDE_BASE_DIR/shiboken-0.3.1"
export PYSIDE_DIR="$PYSIDE_BASE_DIR/pyside-qt4.6+0.3.1"
export PYSIDE_TOOLS_DIR="$PYSIDE_BASE_DIR/pyside-tools-0.1.3"
pushd .
cd $APIEXTRACTOR_DIR
cmake .
cd $GENERATORRUNNER_DIR
cmake -DApiExtractor_DIR=$APIEXTRACTOR_DIR .
cd $SHIBOKEN_DIR
cmake -DApiExtractor_DIR=$APIEXTRACTOR_DIR -DGeneratorRunner_DIR=$GENERATORRUNNER_DIR .
cd $PYSIDE_DIR
cmake -DShiboken_DIR=$SHIBOKEN_DIR/libshiboken -DGENERATOR=$GENERATORRUNNER_DIR .
cd $PYSIDE_TOOLS_DIR
cmake .
popd
Now, I don't know if this installation script is ok, but apparently everything works fine. Each component (apiextractor, generatorrunner, shiboken, pyside-qt and pyside-tools) gets compiled into its own directory.
The problem is that I don't quite understand how PySide gets into the system's python environment. In fact, when I start a python shell, I cannot import PySide:
>>> import PySide
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named PySide
Note: I am aware of the Installing PySide - OSX question, but that question is not relevant anymore, because it is about a specific a dependency on the Boost libraries, but with version 0.3.0 PySide moved from a Boost based source code to a CPython one.
I don't have any MacOS experience but assuming it's similar to any *nix, let's go:
About the script: Isn't it missing some "make, make install" commands? The version you posted just run cmake to configure the build. Also for testing, I set -DCMAKE_INSTALL_PREFIX= for all modules. That way everything is installed in the same place and CMake takes care of finding them for me, as long as I used the same install prefix for each one. The directory layout in your script is quite complicated and mixes build and source directories.
About finding PySide: once everything is properly compiled and installed, the directory where the "PySide" directory was installed must be available in the PYTHONPATH variable. In the example below,
Here's a simple version of a build script(works on Ubuntu):
#!/bin/bash
BUILD_ROOT=/tmp/pyside-build
INSTALL_PREFIX=/tmp/sandbox
function build_module {
cd $BUILD_ROOT
echo Cloning project $1 from url $2
git clone --depth 1 $2 $BUILD_ROOT/$1
BUILD_DIR=$BUILD_ROOT/$1/build
mkdir -p $BUILD_DIR
cd $BUILD_DIR
echo Configuring $1 build.
cmake -DCMAKE_INSTALL_PREFIX=$INSTALL_PREFIX ..
echo Configured $1. Starting build.
make LD_LIBRARY_PATH=$INSTALL_PREFIX/lib
echo Built $1. Installing.
make install
echo Successfully built and installed $1
}
rm -rf $BUILD_ROOT
mkdir -p $BUILD_ROOT
build_module apiextractor git://gitorious.org/pyside/apiextractor.git
build_module generatorrunner git://gitorious.org/pyside/generatorrunner.git
build_module shiboken git://gitorious.org/pyside/shiboken.git
build_module pyside-shiboken git://gitorious.org/pyside/pyside-shiboken.git
Run it and wait a while (Qt is quite big). :)
This script will download all packages into /tmp/pyside-build, build each one in its own "build" directory and install everything into /tmp/sandbox. Then, I just had to set PYTHONPATH to /tmp/sandbox/lib/python2.6/site-packages and PySide worked fine.
In answer to the original post...
What your script would have done is generate the necessary build files to build the pyside bindings, but it wouldn't have done the build itself. To do the build itself you'd need to execute a 'make', then a 'make install' in each of the build directories.
I've gotten most of the way through getting pyside 0.3.1 up and running on the Mac (SnowLeopard 10.6.3), but am hung up on the final compile step. My script is a bit less straightforward than yours, but in essence similar.
I have run into and worked around a number of problems to get to the final compile, and am hoping that I can help some folks along with the solutions/workarounds I've devised. Also, maybe collectively we can figure out how to get through the final step.
I'm going to include the build script that I'm using, and the log of the changes I had to make in the distro to get it running. But first, the step that I'm stuck on... mainly, I don't understand the error message, and the requested file doesn't seem to exist...
I hope this will help move support for the mac along...
I can't post this directly on stack overflow since its too long, so here is the link to the post on the mailing list over at pyside.
http://lists.openbossa.org/pipermail/pyside/2010-June/000812.html
While it's a been couple months since this question originally was asked I ran across the official PySide build script repo at:
http://qt.gitorious.org/pyside/buildscripts
The README covers the build process:
http://qt.gitorious.org/pyside/buildscripts/blobs/master/README
It seems to be fairly automatic with my only caveat so far being that it pulls down libxslt via brew - I can't say authoritatively if libxslt shipped with Snow Leopard or not - but it's definitely in Lion by default.
Seemingly the only manual install step is putting cmake on your system.