Develop Reference

Error: format not a string literal and no format arguments [-Werror=format-security]

I am having this error during installation of 4suite.xml on ubuntu. Trying to install harpia on my ubuntu but couple packages are missing from my system so during to getting those missing dependencies I stuck on this.
username#ubuntu:~/4Suite-XML-1.0.2$ sudo python setup.py install
running install
running build
running config
running build_py
running build_ext
building 'Ft.Xml.Lib.cStreamWriter' extension
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/include/python2.7 -c Ft/Xml/src/StreamWriter.c -o build/temp.linux- x86_64-2.7/Ft/Xml/src/StreamWriter.o
In file included from /usr/include/python2.7/Python.h:94:0,
from Ft/Xml/src/StreamWriter.c:14:
Ft/Xml/src/StreamWriter.c: In function ‘writer_print’:
/usr/include/python2.7/stringobject.h:91:32: error: format not a string literal and no format arguments [-Werror=format-security]
#define PyString_AS_STRING(op) (((PyStringObject *)(op))->ob_sval)
^
Ft/Xml/src/StreamWriter.c:605:15: note: in expansion of macro ‘PyString_AS_STRING’
fprintf(fp, PyString_AS_STRING(repr));
^
Ft/Xml/src/StreamWriter.c: In function ‘entitymap_print’:
/usr/include/python2.7/stringobject.h:91:32: error: format not a string literal and no format arguments [-Werror=format-security]
#define PyString_AS_STRING(op) (((PyStringObject *)(op))->ob_sval)
^
Ft/Xml/src/StreamWriter.c:815:15: note: in expansion of macro ‘PyString_AS_STRING’
fprintf(fp, PyString_AS_STRING(repr));
^
cc1: some warnings being treated as errors
Can you help me to figure that out ?

Trying to use code below might helpful:
fprintf(fp, "%s", PyString_AS_STRING(repr));
Clues are from fprintf, error: format not a string literal and no format arguments [-Werror=format-security] and How to fix this compiler error 'format not a string literal and no format arguments' and similar situation in deploying python codes

A quick and dirty workaround for me was to set CFLAGS to override -Werror=format-security with -Wno-error=format-security (python 2 on CenTOS):
wget https://files.pythonhosted.org/packages/0e/ae/3c5b4fffb12be7c3a80c99475853349e1cf8477f99051921ea06fbf5e3b9/4Suite-XML-1.0.2.tar.gz
tar -xvf 4Suite-XML-1.0.2.tar.gz
export CFLAGS=" -Wno-error=format-security" # no error will be generated, you know what you do
python2 ./setup.py install # 4Suite-XML compilation works without error

Issue creating f2py shared object file when linking to libraries and modules

I am able to compile a fortran program normally with these commands:
ifort -O2 -I/work/GSI/include -c create_upperair.f90
ifort -o create_upperair.exe -O2 create_upperair.o -L/work/GSI/lib -lbufr
I am trying to take the same fortran program, but compile it as a python shared object file so that I can call this subroutine from a python script. However the f2py command is giving me issues.
Here is the command I am currently using:
f2py -c -L/work/GSI/lib -lbufr -I/work/GSI/include --f90flags=-O2 --fcompiler=ifort -m create_upperair create_upperair.f90
However that prints out errors at the end of the f2py command with something like:
ld: /work/GSI/lib/libbufr.a(closbf.o): relocation R_X86_64_32S against `nulbfr_' can not be used when making a shared object: recompile with -fPIC
I tried to add --f90flags="-O2 -fPIC" to my f2py command, but it didn't work either. Any ideas of what direction to go with this problem would be much appreciated.
Here is the full error at the end of f2py (note that I changed a few directory names above for simplicity):
ifort:f90: prepbufr_encode_upperair.f90
/glade/apps/opt/modulefiles/ys/cmpwrappers/ifort -shared -shared -nofor_main /tmp/tmpzYKBY5/tmp/tmpzYKBY5/src.linux-x86_64-2.7/prepbufr_encode_upperairmodule.o /tmp/tmpzYKBY5/tmp/tmpzYKBY5/src.linux-x86_64-2.7/fortranobject.o /tmp/tmpzYKBY5/prepbufr_encode_upperair.o - L/glade/p/work/sdegelia/softwares/comGSI_v3.3/lib -L/glade/apps/opt/python/2.7.7/gnu-westmere/4.8.2/lib -lbufr_i4r8 -lpython2.7 -o ./prepbufr_encode_upperair.so
ld: /glade/p/work/sdegelia/softwares/comGSI_v3.3/lib/libbufr_i4r8.a(closbf.o): relocation R_X86_64_32S against `nulbfr_' can not be used when making a shared object; recompile with -fPIC
/glade/p/work/sdegelia/softwares/comGSI_v3.3/lib/libbufr_i4r8.a: could not read symbols: Bad value
ld: /glade/p/work/sdegelia/softwares/comGSI_v3.3/lib/libbufr_i4r8.a(closbf.o): relocation R_X86_64_32S against `nulbfr_' can not be used when making a shared object; recompile with -fPIC
/glade/p/work/sdegelia/softwares/comGSI_v3.3/lib/libbufr_i4r8.a: could not read symbols: Bad value
error: Command "/glade/apps/opt/modulefiles/ys/cmpwrappers/ifort -shared -shared -nofor_main /tmp/tmpzYKBY5/tmp/tmpzYKBY5/src.linux-x86_64-2.7/prepbufr_encode_upperairmodule.o /tmp/tmpzYKBY5/tmp/tmpzYKBY5/src.linux-x86_64-2.7/fortranobject.o /tmp/tmpzYKBY5/prepbufr_encode_upperair.o -L/glade/p/work/sdegelia/softwares/comGSI_v3.3/lib -L/glade/apps/opt/python/2.7.7/gnu-westmere/4.8.2/lib -lbufr_i4r8 -lpython2.7 -o ./prepbufr_encode_upperair.so" failed with exit status 1

How can I wrap a C-Library in SWIG, which has usually to be linked during C-compilation?

Given a C-library, which has to be linked during compilation if I want to use its functions. I want to access these functions in Python using SWIG. I can only find examples and introductions where C-Code (example.c) is wrapped using SWIG, no method how to wrap a dynamic library (example.so).

All you need to do to make the .so (or .a) library case work is to link the library appropriately when you do the compile step of the example build process. You will still have to compile the example_wrap.c that gets generated, this is where you can link against things.
So modified from the SWIG docs that would be:
$ swig -python example.i
$ gcc -O2 -fPIC -c example.c
$ gcc -O2 -fPIC -c example_wrap.c -I/usr/local/include/python2.5
$ gcc -shared example_wrap.o -o _example.so -lmylib.so
In reality you can also skip this at the compile time linker step and use dlopen at runtime instead by injecting some extra code into the Python part of your module that calls dlopen before the shared object from SWIG gets loaded.

Python, ImportError: undefined symbol: g_utf8_skip

There are like tens of similar questions on StackOverflow, but after several hours of lurking I finally gave up.
So I'm trying to write a C extension for Python. Let's call it mylib. Here is the header file:
mylib.h
#ifndef mylib_H
#define mylib_H
#include <Python.h>
< ... >
#include <glib.h>
< ... >
and setup.py:
from distutils.core import setup, Extension
include_list = [
"/usr/include/glib-2.0", "-lglib-2.0",
"/usr/lib/x86_64-linux-gnu/glib-2.0/include"
]
module = Extension('mylib', ['mylib.c'])
setup(name='mylib', version='1.0',
include_dirs=include_list,
ext_modules=[module])
If I run python setup.py install, I get the following (which I take as successful installation):
running install
running build
running build_ext
building 'mylib' extension
creating build
creating build/temp.linux-x86_64-2.7
x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/glib-2.0 -I-lglib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -I/usr/include/python2.7 -c mylib.c -o build/temp.linux-x86_64-2.7/mylib.o
mylib.c: In function ‘c_sound_utf8’:
mylib.c:117:5: warning: ‘g_unicode_canonical_decomposition’ is deprecated (declared at /usr/include/glib-2.0/glib/gunicode.h:627) [-Wdeprecated-declarations]
decomposition = g_unicode_canonical_decomposition(c_composed, &decomposition_len);
^
creating build/lib.linux-x86_64-2.7
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -D_FORTIFY_SOURCE=2 -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/mylib.o -o build/lib.linux-x86_64-2.7/mylib.so
running install_lib
copying build/lib.linux-x86_64-2.7/mylib.so -> /usr/local/lib/python2.7/dist-packages
running install_egg_info
Removing /usr/local/lib/python2.7/dist-packages/mylib-1.0.egg-info
Writing /usr/local/lib/python2.7/dist-packages/mylib-1.0.egg-info
But when I try to use mylib from inside Python, I get the following:
>>> import mylib
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: /usr/local/lib/python2.7/dist-packages/mylib.so: undefined symbol: g_utf8_skip
After rambling around StackOverflow for some time I got an idea that I should either 1. rebuild the needed library or 2. put all the links to needed library after all generated module names.
Rebuilding didn't work (or I did it the wrong way). As for placing links to the needed library after everything else - well, I didn't find out the way to make distutils change the order of links in its compile string. Is there a way?
I also tried providing extra_link_args/extra_compile_args to my extension (without any effect):
module = Extension('mylib', ['mylib.c'],
extra_link_args=["-Xlinker", "-export-dynamic"])
I felt pretty miserable and kept googling on. Then I found out about SWIG. I decided to try it by making another library, (uppercase) MYLIB (I changed filenames and all text occurences of mylib to MYLIB). I wrote a shell script:
#!/bin/bash
GLIB_IMPORT_OPTS="-I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -lglib-2.0"
PY_IMPORT_OPTS="-I/usr/include/python2.7/ -lpython2.7"
swig -Wall -python MYLIB.i
gcc -fPIC -Wall -c MYLIB.c $GLIB_IMPORT_OPTS
gcc -fPIC -Wall -shared MYLIB.o MYLIB_wrap.c -o _MYLIB.so $GLIB_IMPORT_OPTS -L. $PY_IMPORT_OPTS $GLIB_IMPORT_OPTS
When I ran this thing, everything worked fine (I could import the library and do stuff with it). Here, as you can see, links are at the very end of the compile line. So now I'm trying to understand: what did I miss with the distutils way? How can I make it work?

Well, actually I found the solution. A had to add library links to extra_link_args:
extra_link_args=["-I", "/usr/include/glib-2.0", "-l", "glib-2.0", "-I", "/usr/lib/x86_64-linux-gnu/glib-2.0/include"]
which appends them to the end of compile string.

I found adding -fPIC to "extra_compile_args" in the Extension constructor also helped. Like so:
my_module = Extension('modulename',
...
extra_compile_args=["-fPIC"]
sources = ['mycode.c'])

Installing Python 3.3 on Cygwin

I'm having trouble installing Python 3.3 on Cygwin. I've tried installing from source, but make returns:
gcc -Wno-unused-result -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I. -IInclude -I./Include -DPy_BUILD_CORE -c ./Modules/signalmodule.c -o Modules/signalmodule.o
In file included from Include/Python.h:84:0,
from ./Modules/signalmodule.c:6:
./Modules/signalmodule.c: In function `fill_siginfo':
./Modules/signalmodule.c:745:60: error: `siginfo_t' has no member named `si_band'
PyStructSequence_SET_ITEM(result, 6, PyLong_FromLong(si->si_band));
^
Include/tupleobject.h:62:75: note: in definition of macro `PyTuple_SET_ITEM'
#define PyTuple_SET_ITEM(op, i, v) (((PyTupleObject *)(op))->ob_item[i] = v)
^
./Modules/signalmodule.c:745:5: note: in expansion of macro `PyStructSequence_SET_ITEM'
PyStructSequence_SET_ITEM(result, 6, PyLong_FromLong(si->si_band));
^
Makefile:1501: recipe for target `Modules/signalmodule.o' failed
make: *** [Modules/signalmodule.o] Error 1
Makefile:1501: recipe for target 'Modules/signalmodule.o' failed
make: ***[Modules/signalmodule.o] error 1
Any thoughts?

Building Python on Cygwin is not trivial -- I tried. However, the Python community on its bug tracker website is unusually friendly and gentle for a project its size and importance. If you find specific issues, open bugs and follow the discussion. Usually, they will accept tiny patches to fix Cygwin build issues.
This patch will solve your first problem about si->si_band. See related Python issue #21085.
This blog post (in German) is amazing. It will walk you through step-by-step how to build Python3.4 and fix all Cygwin issues.
Good luck. You will need it.