I've been having a hard time building python 3.6.3 from source on openSUSE LEAP 42.3.
When I started configuring the build I ran:
./configure --prefix=/opt/python3.6 --with-pydebug --enable-optimizations --enable-shared
and in another rendition
./configure --prefix=/opt/python3.6 --with-pydebug --enable-optimizations --enable-shared --with-system-expat --with-system-ffi
prior to both, CXX was defined with
CXX = "/usr/bin/g++"
Configuration goes well (or it seems to) and then when I start make, after some success, it ALWAYS fails with this:
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3
-Wall -Wstrict-prototypes -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -I. -I./Include -DPy_BUILD_CORE -I./Modules/expat -DHAVE_EXPAT_CONFIG_H -DUSE_PYEXPAT_CAPI -c ./Modules/expat/xmlparse.c -o Modules/xmlparse.o ./Modules/expat/xmlparse.c:92:3: error: #error You do not have support for any sources of high quality entropy
enabled. For end user security, that is probably not what you want.
Your options include: * Linux + glibc >=2.25 (getrandom):
HAVE_GETRANDOM, * Linux + glibc <2.25 (syscall SYS_getrandom):
HAVE_SYSCALL_GETRANDOM, * BSD / macOS >=10.7 (arc4random_buf):
HAVE_ARC4RANDOM_BUF, * BSD / macOS <10.7 (arc4random):
HAVE_ARC4RANDOM, * libbsd (arc4random_buf): HAVE_ARC4RANDOM_BUF +
HAVE_LIBBSD, * libbsd (arc4random): HAVE_ARC4RANDOM + HAVE_LIBBSD, *
Linux / BSD / macOS (/dev/urandom): XML_DEV_URANDOM * Windows
(RtlGenRandom): _WIN32. If insist on not using any of these, bypass
this error by defining XML_POOR_ENTROPY; you have been warned. If you
have reasons to patch this detection code away or need changes to the
build system, please open a bug. Thank you!
I googled and I have yet to see something on this error.
One last note. I tried various permutations of ./configure, removing various feature flags but always keeping the prefix
Can someone suggest WHY this is failing? (and how to fix it please) This is the first time it has happened to me and I suspect very strongly that I had forgotten to install something, but expat and libexpat are all there.
My thanks
OK so this answer is a workaround and for the sake of my need, it will suffice.
I remembered that SUSE has an open build service (which ROCKS \m/ - https://build.opensuse.org/)
From there, I found some enterprising dev who had created a repo for Python 3.6.3 (I will be sending them an email to find out how they did it)
But it was a simple matter of adding the repo [http://download.opensuse.org/repositories/devel:/languages:/python/openSUSE_Leap_42.3/] and then doing a repo specific distro upgrade (zypper dup --repo python3.6.3)
To be extra safe, I create a btrfs snapshot so I can roll back if things went sideways.. they did not and I am a happy camper.
Python 3.6.3 repo: Python 3.6.3 repo on OBS
Related
i need to install Python3 with ssl in a different userdir. I'm working on Debian without root privileges. I'm really trying to find a solution by myself, but after spending serveral hours I'm still not able to get it correctly done.
The Path to install ssl and python is $HOME/.local/
$HOME is always an absolute path like /path/to/install
Logs below
Things I've already tried:
1. Creating installation folders
cd $HOME
mkdir .local
cd .local
mkdir ssl
mkdir python
mkdir src
cd src
2. Downloading latest tar versions of Python and openssl
wget https://www.python.org/ftp/python/3.8.1/Python-3.8.1.tgz
wget https://www.openssl.org/source/openssl-1.1.1d.tar.gz
tar xvfz Python-3.8.1.tgz
tar xvfz openssl-1.1.1d.tar.gz
3. Configure, Make, Install Openssl (in $HOME/.local/ssl)
cd openssl-1.1.1d
./config --prefix=$HOME/.local/ssl
make && make install
4. Edit Modules/Setup, Make, Install Python
cd $HOME/.local/src/Python-3.8.1
cd Modules
vi Setup
Modules/Setup
210 SSL=$HOME/.local/ssl
211 _ssl _ssl.c \
212 -DUSE_SSL -I$(SSL)/include -I$(SSL)/include/openssl \
213 -L$(SSL)/lib -lssl -lcrypto
configure
cd $HOME/.local/src/Python-3.8.1
./configure --prefix=$Home/.local/python
Log-Snippets
Configure Python
checking for openssl/ssl.h in /usr/local/ssl... no
checking for openssl/ssl.h in /usr/lib/ssl... no
checking for openssl/ssl.h in /usr/ssl... no
checking for openssl/ssl.h in /usr/pkg... no
checking for openssl/ssl.h in /usr/local... no
checking for openssl/ssl.h in /usr... no
checking whether compiling and linking against OpenSSL works... no
Alternative Log: Seems it doesn't look in the right places. Already tried to give a hint while configure
./configure --prefix=$Home/.local/python --with-openssl=$HOME/.local/ssl
Log while Configure Changes in:
checking for openssl/ssl.h in $HOME/.local/ssl... yes
checking whether compiling and linking against OpenSSL works... yes
checking for X509_VERIFY_PARAM_set1_host in libssl... yes
checking for --with-ssl-default-suites... python
But then make throws error
Include/internal -I. -I./Include -DPy_BUILD_CORE \
-DGITVERSION="\"`LC_ALL=C `\"" \
-DGITTAG="\"`LC_ALL=C `\"" \
-DGITBRANCH="\"`LC_ALL=C `\"" \
-o Modules/getbuildinfo.o ./Modules/getbuildinfo.c
gcc -pthread -c -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -std=c99 -Wextra -Wno-unused-result -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -I./Include/internal -I. -I./Include -DPy_BUILD_CORE -o Python/frozen.o Python/frozen.c
rm -f libpython3.8.a
ar rcs libpython3.8.a Modules/getbuildinfo.o Parser/acceler.o Parser/grammar1.o Parser/listnode.o Parser/node.o Parser/parser.o Parser/token.o Parser/myreadline.o Parser/parsetok.o Parser/tokenizer.o Objects/abstract.o Objects/accu.o Objects/boolobject.o Objects/bytes_methods.o Objects/bytearrayobject.o Objects/bytesobject.o Objects/call.o Objects/capsule.o Objects/cellobject.o Objects/classobject.o Objects/codeobject.o Objects/complexobject.o Objects/descrobject.o Objects/enumobject.o Objects/exceptions.o Objects/genobject.o Objects/fileobject.o Objects/floatobject.o Objects/frameobject.o Objects/funcobject.o Objects/interpreteridobject.o Objects/iterobject.o Objects/listobject.o Objects/longobject.o Objects/dictobject.o Objects/odictobject.o Objects/memoryobject.o Objects/methodobject.o Objects/moduleobject.o Objects/namespaceobject.o Objects/object.o Objects/obmalloc.o Objects/picklebufobject.o Objects/rangeobject.o Objects/setobject.o Objects/sliceobject.o Objects/structseq.o Objects/tupleobject.o Objects/typeobject.o Objects/unicodeobject.o Objects/unicodectype.o Objects/weakrefobject.o Python/_warnings.o Python/Python-ast.o Python/asdl.o Python/ast.o Python/ast_opt.o Python/ast_unparse.o Python/bltinmodule.o Python/ceval.o Python/codecs.o Python/compile.o Python/context.o Python/dynamic_annotations.o Python/errors.o Python/frozenmain.o Python/future.o Python/getargs.o Python/getcompiler.o Python/getcopyright.o Python/getplatform.o Python/getversion.o Python/graminit.o Python/hamt.o Python/import.o Python/importdl.o Python/initconfig.o Python/marshal.o Python/modsupport.o Python/mysnprintf.o Python/mystrtoul.o Python/pathconfig.o Python/peephole.o Python/preconfig.o Python/pyarena.o Python/pyctype.o Python/pyfpe.o Python/pyhash.o Python/pylifecycle.o Python/pymath.o Python/pystate.o Python/pythonrun.o Python/pytime.o Python/bootstrap_hash.o Python/structmember.o Python/symtable.o Python/sysmodule.o Python/thread.o Python/traceback.o Python/getopt.o Python/pystrcmp.o Python/pystrtod.o Python/pystrhex.o Python/dtoa.o Python/formatter_unicode.o Python/fileutils.o Python/dynload_shlib.o Modules/config.o Modules/getpath.o Modules/main.o Modules/gcmodule.o Modules/posixmodule.o Modules/errnomodule.o Modules/pwdmodule.o Modules/_sre.o Modules/_codecsmodule.o Modules/_weakref.o Modules/_functoolsmodule.o Modules/_operator.o Modules/_collectionsmodule.o Modules/_abc.o Modules/itertoolsmodule.o Modules/atexitmodule.o Modules/signalmodule.o Modules/_stat.o Modules/timemodule.o Modules/_threadmodule.o Modules/_localemodule.o Modules/_iomodule.o Modules/iobase.o Modules/fileio.o Modules/bytesio.o Modules/bufferedio.o Modules/textio.o Modules/stringio.o Modules/faulthandler.o Modules/_tracemalloc.o Modules/hashtable.o Modules/symtablemodule.o
Modules/_ssl.o Modules/xxsubtype.o Python/frozen.o
gcc -pthread -L$HOME/.local/ssl/lib -L$HOME/htdocs/.local/ssl/lib -Xlinker -export-dynamic -o python Programs/python.o libpython3.8.a -lcrypt -lpthread -ldl -lutil -lm -L$HOME/.local/ssl/lib -lssl -lcrypto -lm
./python -E -S -m sysconfig --generate-posix-vars ;\
if test $? -ne 0 ; then \
echo "generate-posix-vars failed" ; \
rm -f ./pybuilddir.txt ; \
exit 1 ; \
fi
./python: error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory
generate-posix-vars failed
Makefile:592: recipe for target 'pybuilddir.txt' failed
make: *** [pybuilddir.txt] Error 1
Guess the Error occurs because of following line
./python: error while loading shared libraries: libssl.so.1.1: cannot open shared object file: No such file or directory
Has somebody a clue where libssl.so.1.1 is supposed to be?
Thanks in advance.
Wip
Solved Issue by adding Path to LD_LIBRARY_PATH
Needed Libs can be found in path/to/ssl/lib
For me adding following Line in .profiles or .bashrc fixed the Issue
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/.local/ssl/lib
I'm trying to run Rscript via Python using subprocess.check_call.
Rscript is very simple it just checks if Rpackage exists, and if not it installs it.
local({r <- getOption("repos")
r["CRAN"] <- "https://cloud.r-project.org/"
options(repos=r)
})
if (!require("glue")) install.packages("glue")
When I run such command in bash
Rscript packages.R
it works fine.
But when I tried to run it using subprocess.check_call
subprocess.check_call(f"Rscript packages.R",
shell=True,
env=self.env)
I got such error:
* installing *source* package 'glue' ...
** package 'glue' successfully unpacked and MD5 sums checked
** libs
gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG -fpic -g -O2 -fdebug-prefix-map=/build/r-base-3.3.3=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c glue.c -o glue.o
gcc: error trying to exec 'cc1': execvp: No such file or directory
/usr/lib/R/etc/Makeconf:132: recipe for target 'glue.o' failed
make: *** [glue.o] Error 1
ERROR: compilation failed for package 'glue'
* removing '/usr/local/lib/R/site-library/glue'
The downloaded source packages are in
'/tmp/Rtmpym9Dy9/downloaded_packages'
You evidently took away (the useful value for) PATH by providing your own environment. If you want to be like the shell, make only the minimum necessary changes to the environment you inherit.
I use Swig.(Mac os 10.13)
My shell script:
swig -c++ -python -o example_wrap.cpp example.i
g++ -c -std=c++17 -fPIC example.cpp
g++ -c -std=c++17 -fPIC example_wrap.cpp -o example_wrap.o \
-I/usr/local/Cellar//python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/include/python3.7m
ld -bundle -macosx_version_min 10.13 -flat_namespace \
-undefined suppress -o _example.so *.o
I spent enough time to seek how to create C++ dynamic library for Python, but I have never used the last line.Most often I create a library from an IDE.
g++ -shared is more familiar, but it doesn't work.
Many such errors appear:
Undefined symbols for architecture x86_64:
"_PyArg_ParseTuple", referenced from:
_wrap_printTree(_object*, _object*) in example_wrap.o
I know about this methods from Python.h.
So, the questions are - how does the last line work(ld -bundle ...)? Are there other methods to create the dynamic library?How can I use g++ -shared?
Here is a small CMakeLists.txt that should work for the example you posted:
cmake_minimum_required(VERSION 3.10) # change this to your needs
project(foo VERSION 0.0 LANGUAGES CXX C)
find_package(SWIG REQUIRED)
include(${SWIG_USE_FILE})
find_package(PythonLibs)
include_directories(${PYTHON_INCLUDE_PATH})
include_directories(${CMAKE_CURRENT_SOURCE_DIR})
set(CMAKE_SWIG_FLAGS "")
add_library(exampleImpl SHARED example.h example.cpp)
target_compile_features(exampleImpl PUBLIC cxx_std_17)
set_source_files_properties(example.i PROPERTIES CPLUSPLUS ON)
swig_add_library(example LANGUAGE python SOURCES example.i)
swig_link_libraries(example ${PYTHON_LIBRARIES} exampleImpl)
To make sure cmake uses the right python library, you can pass an appropriate option upon configure time, see here.
I have anaconda Python first on my path, but a simple Python embedding example shows my Mac system python version instead, even though ProgramFullPath correctly points to anaconda python. Is there a way to correctly find / use anaconda python?
Minimal example:
#include <Python.h>
#include <stdio.h>
int main(void) {
Py_Initialize();
printf("Python version:\n%s\n", Py_GetVersion());
printf("Python Program Full Path:\n%s\n", Py_GetProgramFullPath());
Py_Finalize();
return 0;
}
I compile with,
gcc `python-config --cflags` example.c `python-config --ldflags`
or, expanding the results of the python-config calls,
gcc -I/Users/ryandwyer/anaconda/include/python2.7 \
-I/Users/ryandwyer/anaconda/include/python2.7 \
-fno-strict-aliasing -I/Users/ryandwyer/anaconda/include \
-arch x86_64 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes \
example.c -lpython2.7 -ldl -framework CoreFoundation -u _PyMac_Error
Running the program gives,
Python version:
2.7.5 (default, Mar 9 2014, 22:15:05)
[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.0.68)]
Python Program Full Path:
/Users/ryandwyer/anaconda/bin/python
This seems to be the same problem as Embed python in c++: choose python version. I have also tried setting PYTHONHOME, Py_SetProgramName, Py_SetPythonHome, but cannot get Python_GetVersion() to return the anaconda version.
There was a partial answer in the post you linked.
Option 1: Run your program as follows
LD_LIBRARY_PATH=/path_to_anaconda/lib ./program
Option 2: Run the following command in the terminal, then run your program
export LD_LIBRARY_PATH=/path_to_anaconda/lib
./program
Option 3: Add the following line to the end of your .bashrc file
LD_LIBRARY_PATH=/path_to_anaconda/lib
Why do you have to do this when embedding python, but not when running the interpreter normally? I have no idea, but if some Python/C wizard stumbles on this post I'd love to know why.
I want to cross compile PyZmq for an ARM-Cortex-A8 processor (AM335). But in my case the compiler is being run with flags which are not available for this processor.
I followed this and this post and taking into account that the mentioned patches are obsolete I'm currently following this procedure:
have target toolchain available (tested)
have ZeroMQ compiled with this toolchain (works)
have python3 libs and headers available for target
have Cython for Python3 installed (for host platform)
my setup.cfg looks like this:
[global]
have_sys_un_h = False
[build_ext]
libraries = python3.1
library_dirs = ../path/to/target/python/lib/python3.1
include_dirs = ../path/to/target/python/include/python3.1/
[bdist_egg]
plat-name = linux-armv
then I run:
git clone git://github.com/zeromq/pyzmq.git pyzmq-arm-cortexa8/
cd pyzmq-git-arm-cortexa8/
export PATH=/path/to/compiler/bin:$PATH
export CC="arm-cortexa8-linux-gnueabihf-gcc"
export LDSHARED="arm-cortexa8-linux-gnueabihf-gcc -shared"
python3 ./setup.py configure --zmq=../zeromq-4.0.4-arm-cortexa8/
python3 ./setup.py build
But then building the Cython-generated files fails - the output is:
running build
running build_py
running build_ext
running configure
Warning: Skipping zmq version check
************************************************
skipping 'zmq/backend/cython/_device.c' Cython extension (up-to-date)
building 'zmq.backend.cython._device' extension
arm-cortexa8-linux-gnueabihf-gcc -Wno-unused-result -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I../zeromq-4.0.4-arm-cortexa8/include -Izmq/utils -Izmq/backend/cython -Izmq/devices -I../platform-phyCORE-AM335x/sysroot-target/usr/include/python3.1/ -I/usr/include/python3.3m -c zmq/backend/cython/_device.c -o build/temp.linux-x86_64-3.3/zmq/backend/cython/_device.o
arm-cortexa8-linux-gnueabihf-gcc: error: unrecognized command line option '-fstack-protector-strong'
arm-cortexa8-linux-gnueabihf-gcc: error: unrecognized command line option '-m64'
arm-cortexa8-linux-gnueabihf-gcc: error: unrecognized argument in option '-mtune=generic'
arm-cortexa8-linux-gnueabihf-gcc: note: valid arguments to '-mtune=' are: arm1020e arm1020t arm1022e arm1026ej-s arm10e arm10tdmi arm1136j-s arm1136jf-s arm1156t2-s arm1156t2f-s arm1176jz-s arm1176jzf-s arm2 arm250 arm3 arm6 arm60 arm600 arm610 arm620 arm7 arm70 arm700 arm700i arm710 arm7100 arm710c arm710t arm720 arm720t arm740t arm7500 arm7500fe arm7d arm7di arm7dm arm7dmi arm7m arm7tdmi arm7tdmi-s arm8 arm810 arm9 arm920 arm920t arm922t arm926ej-s arm940t arm946e-s arm966e-s arm968e-s arm9e arm9tdmi cortex-a15 cortex-a5 cortex-a7 cortex-a8 cortex-a9 cortex-m0 cortex-m1 cortex-m3 cortex-m4 cortex-r4 cortex-r4f cortex-r5 ep9312 fa526 fa606te fa626 fa626te fa726te fmp626 generic-armv7-a iwmmxt iwmmxt2 mpcore mpcorenovfp native strongarm strongarm110 strongarm1100 strongarm1110 xscale
error: command 'arm-cortexa8-linux-gnueabihf-gcc' failed with exit status 1
(the same happens when I run python3 setupegg.py build bdist_egg but I don't need the egg anyway)
So for some reason the generated sources are being built with flags which would fit my host system but not the target compiler: -fstack-protector-strong, -m64 and -mtune=generic.
I tried to modify CFLAGS and OPT like mentioned here but despite this doesn't work I doubt that's the most sophisticated approach anyway..
So the question is: what am I doing wrong here? How do I tell the setup process to provide the correct flags to the cross compiler? And why does this procedure seem to work with the Android-NDK which should produce the same errors?
Btw. I'm working on a recent Fedora 20 / x86_64
Alternative solution
Have you ever tried Buildroot? It provides both Python3 and zmq.
git clone git://git.buildroot.net/buildroot
cd buildroot
make beaglebone_defconfig
make menuconfig
In menu select external toolchain, select Python3 and zmq under "Target Packages" and then python-pyzmq under Python3 external packages.
make