Develop Reference

Installation of Python and openssl --- without yum

trying to install OpenSSL for python 3.8.3 in Linux,
make give the following error
I have added all the path required in .bashrc and .bash_profile
I have used /config comand
'''
./configure --prefix=$Home/.local/python --with-openssl=$HOME/.local/ssl
'''
Makefile:1884: warning: overriding commands for target `Modules/_ssl.o'
Makefile:1882: warning: ignoring old commands for target `Modules/_ssl.o'
Makefile:1885: warning: overriding commands for target `Modules/_ssl.cpython-38-x86_64-linux-gnu.so'
Makefile:1883: warning: ignoring old commands for target `Modules/_ssl.cpython-38-x86_64-linux-gnu.so'
gcc -pthread -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -std=c99 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -I./Include/internal -I. -I./Include -DPy_BUILD_CORE_BUILTIN -DUSE_SSL -IOME/.local/openssl1/include -IOME/.local/openssl1/include/openssl -c ./Modules/_ssl.c -o Modules/_ssl.o
./Modules/_ssl.c: In function ‘_ssl_configure_hostname’:
./Modules/_ssl.c:891: error: implicit declaration of function ‘SSL_get0_param’
./Modules/_ssl.c:891: warning: initialization makes pointer from integer without a cast
./Modules/_ssl.c:893: error: implicit declaration of function ‘X509_VERIFY_PARAM_set1_host’
./Modules/_ssl.c:899: error: implicit declaration of function ‘X509_VERIFY_PARAM_set1_ip’
./Modules/_ssl.c: In function ‘_ssl__SSLContext_impl’:
./Modules/_ssl.c:3130: error: ‘X509_CHECK_FLAG_NO_PARTIAL_WILDCARDS’ undeclared (first use in this function)
./Modules/_ssl.c:3130: error: (Each undeclared identifier is reported only once
./Modules/_ssl.c:3130: error: for each function it appears in.)
./Modules/_ssl.c:3240: error: implicit declaration of function ‘SSL_CTX_get0_param’
./Modules/_ssl.c:3240: warning: assignment makes pointer from integer without a cast
./Modules/_ssl.c:3246: error: implicit declaration of function ‘X509_VERIFY_PARAM_set_hostflags’
./Modules/_ssl.c: In function ‘get_verify_flags’:
./Modules/_ssl.c:3555: warning: assignment makes pointer from integer without a cast
./Modules/_ssl.c: In function ‘set_verify_flags’:
./Modules/_ssl.c:3568: warning: assignment makes pointer from integer without a cast
./Modules/_ssl.c: In function ‘set_host_flags’:
./Modules/_ssl.c:3764: warning: assignment makes pointer from integer without a cast

This make output ...
gcc -pthread -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -std=c99 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Werror=implicit-function-declaration -I./Include/internal -I. -I./Include -DPy_BUILD_CORE_BUILTIN -DUSE_SSL -IOME/.local/openssl1/include -IOME/.local/openssl1/include/openssl -c ./Modules/_ssl.c -o Modules/_ssl.o
... suggests that your configure command was not ...
./configure --prefix=$Home/.local/python --with-openssl=$HOME/.local/ssl
In particular, I am looking at these bits of the output:
-IOME/.local/openssl1/include -IOME/.local/openssl1/include/openssl
. That's showing the effect of the text $HOME/.local/openssl1 being expanded by make to OME/.local/openssl1 (because no variable H is defined). That, in turn, indicates that $HOME was not expanded by the shell when you ran configure, so it must have been quoted in the actual command you ran. Something like this, maybe:
(wrong:)
./configure --prefix='$Home/.local/python' --with-openssl='$HOME/.local/openssl1'
Additionally, $Home is not the same as $HOME, so probably what you really want is:
(better:)
./configure --prefix=$HOME/.local/python --with-openssl=$HOME/.local/openssl1
If you do quote then do it with double quotes ("), not single-quotes ('), because the latter suppress shell parameter expansion. If there's anything in your home directory name that requires quoting, however, such as spaces, then you can expect other issues to arise, whether you quote in the configure command or not.

Error: format not a string literal and no format arguments [-Werror=format-security]

I am having this error during installation of 4suite.xml on ubuntu. Trying to install harpia on my ubuntu but couple packages are missing from my system so during to getting those missing dependencies I stuck on this.
username#ubuntu:~/4Suite-XML-1.0.2$ sudo python setup.py install
running install
running build
running config
running build_py
running build_ext
building 'Ft.Xml.Lib.cStreamWriter' extension
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/include/python2.7 -c Ft/Xml/src/StreamWriter.c -o build/temp.linux- x86_64-2.7/Ft/Xml/src/StreamWriter.o
In file included from /usr/include/python2.7/Python.h:94:0,
from Ft/Xml/src/StreamWriter.c:14:
Ft/Xml/src/StreamWriter.c: In function ‘writer_print’:
/usr/include/python2.7/stringobject.h:91:32: error: format not a string literal and no format arguments [-Werror=format-security]
#define PyString_AS_STRING(op) (((PyStringObject *)(op))->ob_sval)
^
Ft/Xml/src/StreamWriter.c:605:15: note: in expansion of macro ‘PyString_AS_STRING’
fprintf(fp, PyString_AS_STRING(repr));
^
Ft/Xml/src/StreamWriter.c: In function ‘entitymap_print’:
/usr/include/python2.7/stringobject.h:91:32: error: format not a string literal and no format arguments [-Werror=format-security]
#define PyString_AS_STRING(op) (((PyStringObject *)(op))->ob_sval)
^
Ft/Xml/src/StreamWriter.c:815:15: note: in expansion of macro ‘PyString_AS_STRING’
fprintf(fp, PyString_AS_STRING(repr));
^
cc1: some warnings being treated as errors
Can you help me to figure that out ?

Trying to use code below might helpful:
fprintf(fp, "%s", PyString_AS_STRING(repr));
Clues are from fprintf, error: format not a string literal and no format arguments [-Werror=format-security] and How to fix this compiler error 'format not a string literal and no format arguments' and similar situation in deploying python codes

A quick and dirty workaround for me was to set CFLAGS to override -Werror=format-security with -Wno-error=format-security (python 2 on CenTOS):
wget https://files.pythonhosted.org/packages/0e/ae/3c5b4fffb12be7c3a80c99475853349e1cf8477f99051921ea06fbf5e3b9/4Suite-XML-1.0.2.tar.gz
tar -xvf 4Suite-XML-1.0.2.tar.gz
export CFLAGS=" -Wno-error=format-security" # no error will be generated, you know what you do
python2 ./setup.py install # 4Suite-XML compilation works without error

How Can I Fix "expected declaration specifiers before ‘initsetsc’" When Installing Guppy (Python 3)?

EDIT: https://stackoverflow.com/a/38860927/1112524 Sorry
When I try to install guppy/heapy for memory profiling,
pip3 install guppy --user
I get
creating build/temp.linux-x86_64-3.4
creating build/temp.linux-x86_64-3.4/src
creating build/temp.linux-x86_64-3.4/src/sets
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.4m -c src/sets/sets.c -o build/temp.linux-x86_64-3.4/src/sets/sets.o
src/sets/sets.c:76:1: warning: return type defaults to ‘int’ [-Wreturn-type]
DL_EXPORT (void)
^
src/sets/sets.c: In function ‘DL_EXPORT’:
src/sets/sets.c:39:18: error: expected declaration specifiers before ‘initsetsc’
#define INITFUNC initsetsc
^
src/sets/sets.c:77:1: note: in expansion of macro ‘INITFUNC’
INITFUNC (void)
^
src/sets/sets.c:108:1: error: expected ‘{’ at end of input
}
^
src/sets/sets.c: At top level:
src/sets/sets.c:45:20: warning: ‘module_methods’ defined but not used [-Wunused-variable]
static PyMethodDef module_methods[] =
^
src/sets/sets.c:67:18: warning: ‘nysets_heapdefs’ defined but not used [-Wunused-variable]
static NyHeapDef nysets_heapdefs[] = {
^
src/sets/sets.c: In function ‘DL_EXPORT’:
src/sets/sets.c:108:1: warning: control reaches end of non-void function [-Wreturn-type]
}
^
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
How can I fix this?
I tried installing things from this question, but it didn't work
error: Setup script exited with error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
Also, just now I happened to try Python 2
pip install guppy --user
And it worked.

Python, ImportError: undefined symbol: g_utf8_skip

There are like tens of similar questions on StackOverflow, but after several hours of lurking I finally gave up.
So I'm trying to write a C extension for Python. Let's call it mylib. Here is the header file:
mylib.h
#ifndef mylib_H
#define mylib_H
#include <Python.h>
< ... >
#include <glib.h>
< ... >
and setup.py:
from distutils.core import setup, Extension
include_list = [
"/usr/include/glib-2.0", "-lglib-2.0",
"/usr/lib/x86_64-linux-gnu/glib-2.0/include"
]
module = Extension('mylib', ['mylib.c'])
setup(name='mylib', version='1.0',
include_dirs=include_list,
ext_modules=[module])
If I run python setup.py install, I get the following (which I take as successful installation):
running install
running build
running build_ext
building 'mylib' extension
creating build
creating build/temp.linux-x86_64-2.7
x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/glib-2.0 -I-lglib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -I/usr/include/python2.7 -c mylib.c -o build/temp.linux-x86_64-2.7/mylib.o
mylib.c: In function ‘c_sound_utf8’:
mylib.c:117:5: warning: ‘g_unicode_canonical_decomposition’ is deprecated (declared at /usr/include/glib-2.0/glib/gunicode.h:627) [-Wdeprecated-declarations]
decomposition = g_unicode_canonical_decomposition(c_composed, &decomposition_len);
^
creating build/lib.linux-x86_64-2.7
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -D_FORTIFY_SOURCE=2 -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/mylib.o -o build/lib.linux-x86_64-2.7/mylib.so
running install_lib
copying build/lib.linux-x86_64-2.7/mylib.so -> /usr/local/lib/python2.7/dist-packages
running install_egg_info
Removing /usr/local/lib/python2.7/dist-packages/mylib-1.0.egg-info
Writing /usr/local/lib/python2.7/dist-packages/mylib-1.0.egg-info
But when I try to use mylib from inside Python, I get the following:
>>> import mylib
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: /usr/local/lib/python2.7/dist-packages/mylib.so: undefined symbol: g_utf8_skip
After rambling around StackOverflow for some time I got an idea that I should either 1. rebuild the needed library or 2. put all the links to needed library after all generated module names.
Rebuilding didn't work (or I did it the wrong way). As for placing links to the needed library after everything else - well, I didn't find out the way to make distutils change the order of links in its compile string. Is there a way?
I also tried providing extra_link_args/extra_compile_args to my extension (without any effect):
module = Extension('mylib', ['mylib.c'],
extra_link_args=["-Xlinker", "-export-dynamic"])
I felt pretty miserable and kept googling on. Then I found out about SWIG. I decided to try it by making another library, (uppercase) MYLIB (I changed filenames and all text occurences of mylib to MYLIB). I wrote a shell script:
#!/bin/bash
GLIB_IMPORT_OPTS="-I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -lglib-2.0"
PY_IMPORT_OPTS="-I/usr/include/python2.7/ -lpython2.7"
swig -Wall -python MYLIB.i
gcc -fPIC -Wall -c MYLIB.c $GLIB_IMPORT_OPTS
gcc -fPIC -Wall -shared MYLIB.o MYLIB_wrap.c -o _MYLIB.so $GLIB_IMPORT_OPTS -L. $PY_IMPORT_OPTS $GLIB_IMPORT_OPTS
When I ran this thing, everything worked fine (I could import the library and do stuff with it). Here, as you can see, links are at the very end of the compile line. So now I'm trying to understand: what did I miss with the distutils way? How can I make it work?

Well, actually I found the solution. A had to add library links to extra_link_args:
extra_link_args=["-I", "/usr/include/glib-2.0", "-l", "glib-2.0", "-I", "/usr/lib/x86_64-linux-gnu/glib-2.0/include"]
which appends them to the end of compile string.

I found adding -fPIC to "extra_compile_args" in the Extension constructor also helped. Like so:
my_module = Extension('modulename',
...
extra_compile_args=["-fPIC"]
sources = ['mycode.c'])

Call python code from c with cython

I'm trying to call some python code from c and I'm trying to follow the cython user guide on how to do that.
I've got a cython script, modulename.pyx that looks like this (taken from the user guide):
cdef public struct Bunny: # public type declaration
int vorpalness
cdef public int spam # public variable declaration
cdef public void grail(Bunny b): # public function declaration
print "Ready the holy hand grenade"
And I've got a c program, main.c, that looks like this (also from the guide):
#include <Python.h>
#include "modulename.h"
int main() {
Py_Initialize();
initmodulename();
struct Bunny b;
grail(b);
Py_Finalize();
}
I've tried to get this to compile in various ways with no luck. I can get the .pyx file compiled to c well enough with cython modulename.pyx, which produces modulename.c and modulename.h. The trouble comes when I want to compile and link those generated files along with my c program.
Here is one example of what I've been trying:
cc -fno-strict-aliasing -fno-common -dynamic -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -pipe -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c main.c modulename.h modulename.c
Which gave me these errors:
modulename.h:25:32: error: expected function body after function declarator
__PYX_EXTERN_C DL_IMPORT(void) grail(struct Bunny);
^
modulename.h:27:31: error: expected function body after function declarator
__PYX_EXTERN_C DL_IMPORT(int) spam;
^
modulename.h:32:1: error: unknown type name 'PyMODINIT_FUNC'
PyMODINIT_FUNC initmodulename(void);
^
3 errors generated.
I guess what I'm looking for is some guidance on getting the example to work or perhaps a pointer towards some example code that calls python code from c with cython.

Alright, so I've figured out the issue. The problem was that I included modulename.h in the argument for the compiler. After I removed that, everyhing ran without any modification to either main.c or modulename.pyx. Also, not all those arguments were necessary for the compiler. The simplest way to compile the above code on my machine (Mac OS 10.9) is:
Compile:
cc -c main.c modulename.c -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7
Link:
cc -L/System/Library/Frameworks/Python.framework/Versions/2.7/lib -L/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/config -lpython2.7 -ldl main.o modulename.o -o main