There are like tens of similar questions on StackOverflow, but after several hours of lurking I finally gave up.
So I'm trying to write a C extension for Python. Let's call it mylib. Here is the header file:
mylib.h
#ifndef mylib_H
#define mylib_H
#include <Python.h>
< ... >
#include <glib.h>
< ... >
and setup.py:
from distutils.core import setup, Extension
include_list = [
"/usr/include/glib-2.0", "-lglib-2.0",
"/usr/lib/x86_64-linux-gnu/glib-2.0/include"
]
module = Extension('mylib', ['mylib.c'])
setup(name='mylib', version='1.0',
include_dirs=include_list,
ext_modules=[module])
If I run python setup.py install, I get the following (which I take as successful installation):
running install
running build
running build_ext
building 'mylib' extension
creating build
creating build/temp.linux-x86_64-2.7
x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/glib-2.0 -I-lglib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -I/usr/include/python2.7 -c mylib.c -o build/temp.linux-x86_64-2.7/mylib.o
mylib.c: In function ‘c_sound_utf8’:
mylib.c:117:5: warning: ‘g_unicode_canonical_decomposition’ is deprecated (declared at /usr/include/glib-2.0/glib/gunicode.h:627) [-Wdeprecated-declarations]
decomposition = g_unicode_canonical_decomposition(c_composed, &decomposition_len);
^
creating build/lib.linux-x86_64-2.7
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -D_FORTIFY_SOURCE=2 -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/mylib.o -o build/lib.linux-x86_64-2.7/mylib.so
running install_lib
copying build/lib.linux-x86_64-2.7/mylib.so -> /usr/local/lib/python2.7/dist-packages
running install_egg_info
Removing /usr/local/lib/python2.7/dist-packages/mylib-1.0.egg-info
Writing /usr/local/lib/python2.7/dist-packages/mylib-1.0.egg-info
But when I try to use mylib from inside Python, I get the following:
>>> import mylib
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: /usr/local/lib/python2.7/dist-packages/mylib.so: undefined symbol: g_utf8_skip
After rambling around StackOverflow for some time I got an idea that I should either 1. rebuild the needed library or 2. put all the links to needed library after all generated module names.
Rebuilding didn't work (or I did it the wrong way). As for placing links to the needed library after everything else - well, I didn't find out the way to make distutils change the order of links in its compile string. Is there a way?
I also tried providing extra_link_args/extra_compile_args to my extension (without any effect):
module = Extension('mylib', ['mylib.c'],
extra_link_args=["-Xlinker", "-export-dynamic"])
I felt pretty miserable and kept googling on. Then I found out about SWIG. I decided to try it by making another library, (uppercase) MYLIB (I changed filenames and all text occurences of mylib to MYLIB). I wrote a shell script:
#!/bin/bash
GLIB_IMPORT_OPTS="-I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -lglib-2.0"
PY_IMPORT_OPTS="-I/usr/include/python2.7/ -lpython2.7"
swig -Wall -python MYLIB.i
gcc -fPIC -Wall -c MYLIB.c $GLIB_IMPORT_OPTS
gcc -fPIC -Wall -shared MYLIB.o MYLIB_wrap.c -o _MYLIB.so $GLIB_IMPORT_OPTS -L. $PY_IMPORT_OPTS $GLIB_IMPORT_OPTS
When I ran this thing, everything worked fine (I could import the library and do stuff with it). Here, as you can see, links are at the very end of the compile line. So now I'm trying to understand: what did I miss with the distutils way? How can I make it work?
Well, actually I found the solution. A had to add library links to extra_link_args:
extra_link_args=["-I", "/usr/include/glib-2.0", "-l", "glib-2.0", "-I", "/usr/lib/x86_64-linux-gnu/glib-2.0/include"]
which appends them to the end of compile string.
I found adding -fPIC to "extra_compile_args" in the Extension constructor also helped. Like so:
my_module = Extension('modulename',
...
extra_compile_args=["-fPIC"]
sources = ['mycode.c'])
Related
QUESTION
When creating a Python extension in C++ that uses Armadillo, I get the errors:
A) In Mac OS Mojave 10.14.4, Python 3.7.5:
Traceback (most recent call last):
File "./py_program.py", line 5, in <module>
import cmodule
ImportError: dlopen(/Users/angel/.pyenv/versions/3.7.5/lib/python3.7/site-packages/cmodule.cpython-37m-darwin.so, 2): Symbol not found: __ZTWN4arma23arma_rng_cxx11_instanceE
Referenced from: /Users/angel/.pyenv/versions/3.7.5/lib/python3.7/site-packages/cmodule.cpython-37m-darwin.so
Expected in: flat namespace in /Users/angel/.pyenv/versions/3.7.5/lib/python3.7/site-packages/cmodule.cpython-37m-darwin.so
B) In Ubuntu 20, Python 3.8.2:
Traceback (most recent call last):
File "./py_program.py", line 5, in <module>
import cmodule
ImportError: /usr/local/lib/python3.8/dist-packages/cmodule.cpython-38-x86_64-linux-gnu.so: undefined symbol: _ZN4arma23arma_rng_cxx11_instanceE
Both of them due to the use of arma::arma_rng::randn<double>(), see below.
How can I fix it?
DETAILS
I want py_program.py to import the C++ module (extension) defined in cmodule.cpp.
Following the documentation https://docs.python.org/3/extending/extending.html , I have the files py_program.py, setup.py and cmodule.cpp.
Their contents are:
py_program.py
#!/usr/bin/env python3
"""Import and use cmodule."""
import cmodule
cmodule.printvol(3.)
setup.py
"""To install the module defined in cmodule.cpp."""
from distutils.core import setup, Extension
setup(name='cmodule', version='1.0', \
ext_modules=[
Extension(
'cmodule', ['cmodule.cpp'],
extra_compile_args=['-std=c++11'],
language='c++')],
)
cmodule.cpp
/* Module to be used in Python.
All Python stuff follows Sec. 1.8 of
https://docs.python.org/3/extending/extending.html */
#define PY_SSIZE_T_CLEAN
#include <Python.h>
#include <armadillo>
double f()
// Fails at using Armadillo.
// The module works if I delete this function.
{
double rn_y = arma::arma_rng::randn<double>();
return rn_y;
}
arma::cx_double g()
// Succeeds at using Armadillo.
{
arma::cx_double value(0., 1.);
return value;
}
static PyObject *
cmodule_printvol(PyObject *self, PyObject *args)
// A method of the module.
{
double voltage;
if (!PyArg_ParseTuple(args, "d", &voltage))
return NULL;
printf("voltage is %f.\n", voltage);
Py_RETURN_NONE;
}
static PyMethodDef cmodule_methods[] = {
// Declare the modules methods.
{"printvol", cmodule_printvol, METH_VARARGS, "Print voltage."},
{NULL, NULL, 0, NULL} /* sentinel */
};
static struct PyModuleDef cmodule = {
// Create the module.
PyModuleDef_HEAD_INIT,
"diff",
NULL,
-1,
cmodule_methods
};
PyMODINIT_FUNC
PyInit_cmodule(void)
// Initialize the module.
{
return PyModule_Create(&cmodule);
}
I run them as follows:
python setup.py install
python py_program.py
In Ubuntu, the output of python3 setup.py install is:
running install
running build
running build_ext
building 'cmodule' extension
creating build
creating build/temp.linux-x86_64-3.8
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/usr/include/python3.8 -c cmodule.cpp -o build/temp.linux-x86_64-3.8/cmodule.o -std=c++11
creating build/lib.linux-x86_64-3.8
x86_64-linux-gnu-g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fwrapv -O2 -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 build/temp.linux-x86_64-3.8/cmodule.o -o build/lib.linux-x86_64-3.8/cmodule.cpython-38-x86_64-linux-gnu.so
running install_lib
copying build/lib.linux-x86_64-3.8/cmodule.cpython-38-x86_64-linux-gnu.so -> /usr/local/lib/python3.8/dist-packages
running install_egg_info
Removing /usr/local/lib/python3.8/dist-packages/cmodule-1.0.egg-info
Writing /usr/local/lib/python3.8/dist-packages/cmodule-1.0.egg-info
What causes the problem is
double rn_y = arma::arma_rng::randn<double>();
Actually, if I delete the function f(), I get no error.
Notice that Armadillo is loaded successfully, since g() uses it no problem.
What is happening?
In setup.py, the argument libraries=['armadillo'] to Extension() fixes the problem:
"""To install the module defined in cmodule.cpp."""
from distutils.core import setup, Extension
setup(name='cmodule', version='1.0', \
ext_modules=[
Extension(
'cmodule', ['cmodule.cpp'],
extra_compile_args=['-std=c++11'],
libraries=['armadillo'], // this solves the problem
language='c++')],
)
Mysteriously, without it, arma:: can be used correctly. But not 'submodules' like arma::arma_rng.
This solution is general: the same problem happens with other libraries. Actually, I reproduced the same (and made it work) with the GNU Scientific Library (libraries=['gsl']).
I have written a wrapper in Cython for an integration function from NAG (https://www.nag.co.uk/content/nag-library-c) c library.
It compiles using the python setup.py build --inplace, where setup file is:
from setuptools import Extension, setup
from Cython.Build import cythonize
import re
def loadMacros(headerFile):
""" Given a .h file, return dict of touples with macros """
regex = re.compile("#define +(\w+) *(\w*)")
with open(headerFile) as f:
macros = dict(map(lambda x: re.match(regex, x).groups(),
[l for l in f if re.match(regex, l)]))
# Remove recursive entries - Note this is not foolproof..
# while not set(macros.keys()).isdisjoint(macros.values()):
# for k, v in macros.items():
# if v in macros:
# macros[k] = macros[v]
return macros
nagHome = "/Users/hfmw1m17/NAG/nlmi627dbl" # "/opt/NAG/cll6a23dhl"
macros = loadMacros(nagHome + "/lp64/include/nag.h")
macros = list(macros.items())
e = Extension("nag_integrate",
define_macros=macros,
sources=["nag_integrate.pyx"],
include_dirs=[nagHome + "/lp64/include",nagHome + "/lp64/lib"],
library_dirs=[nagHome + "/lp64/lib"],libraries=["nag_nag"],extra_objects=[
nagHome+"/lp64/lib/libnag_nag.dylib"],runtime_library_dirs=[nagHome+"/lp64/lib/"],extra_link_args=['-Wl,-rpath']
)
setup(ext_modules=cythonize(e,annotate=True,language_level=3))enter code here
with output:
/Users/hfmw1m17/anaconda3/envs/TowingTankAcoustics/bin/python3.7 setup.py build_ext --inplace
Compiling nag_integrate.pyx because it changed.
[1/1] Cythonizing nag_integrate.pyx
running build_ext
building 'nag_integrate' extension
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/hfmw1m17/anaconda3/envs/TowingTankAcoustics/include -arch x86_64 -I/Users/hfmw1m17/anaconda3/envs/TowingTankAcoustics/include -arch x86_64 -DNAG_H= -DNAG_MICROSOFT_THREAD_SAFE= -DNAG_THREAD_SAFE= -DNULLFN=0 -DNULLDFN=0 -DNAGERR_DEFAULT= -DNAGUSER_DEFAULT= -DNAGCOMM_NULL= -DNAGMESG_DEFAULT= -DE01_DEFAULT= -DE04_DEFAULT= -DG13_DEFAULT= -DH02_DEFAULT= -DINIT_FAIL= -DSET_FAIL= -DINIT_MESG= -DINIT_STREAM= -DRDUMMY= -DIDUMMY= -DINIT2DUMMY= -DVprintf= -DVfprintf= -DVsprintf= -DVscanf= -DVfscanf= -DVstrcpy= -DABS= -DFABS= -DSIGN= -DMAX= -DMIN= -DDROUND= -DROUND= -DSQZABS= -DCONJ= -DVCONJ= -DZMULT= -D_nag_expand= -Dnag_stringize= -I/Users/hfmw1m17/NAG/nlmi627dbl/lp64/include -I/Users/hfmw1m17/NAG/nlmi627dbl/lp64/lib -I/Users/hfmw1m17/anaconda3/envs/TowingTankAcoustics/include/python3.7m -c nag_integrate.c -o build/temp.macosx-10.9-x86_64-3.7/nag_integrate.o
gcc -bundle -undefined dynamic_lookup -L/Users/hfmw1m17/anaconda3/envs/TowingTankAcoustics/lib -arch x86_64 -L/Users/hfmw1m17/anaconda3/envs/TowingTankAcoustics/lib -arch x86_64 -arch x86_64 build/temp.macosx-10.9-x86_64-3.7/nag_integrate.o /Users/hfmw1m17/NAG/nlmi627dbl/lp64/lib/libnag_nag.dylib -L/Users/hfmw1m17/NAG/nlmi627dbl/lp64/lib -L/Users/hfmw1m17/NAG/nlmi627dbl/lp64/lib/ -lnag_nag -o build/lib.macosx-10.9-x86_64-3.7/nag_integrate.cpython-37m-darwin.so -Wl,-rpath
copying build/lib.macosx-10.9-x86_64-3.7/nag_integrate.cpython-37m-darwin.so ->
Process finished with exit code 0
However when i import a function from the .so object created i get the following error:
ImportError: dlopen(/Users/hfmw1m17/WaterTankISM/WaterTankISM/nag_integration/nag_integrate.cpython-37m-darwin.so, 2): Library not loaded: libnag_nag.dylib
Referenced from: /Users/hfmw1m17/WaterTankISM/WaterTankISM/nag_integration/nag_integrate.cpython-37m-darwin.so
Reason: image not found
libnag_nag.dylib is a dynamic library produced by NAG.
Using otool -L on the shared object of my wrapper results in:
libnag_nag.dylib (compatibility version 0.0.0, current version 27.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1)
I think its an issue with the linker not being able to find the dynamic library when compiling. Any suggestions on how to solve this problem?
many thanks
Solved using:
install_name_tool -add_rpath path_to_dylib_directory
after running python setup.py build --inplace.
Would like a way of doing this in the setup.py file if anyone can figure that out.
many thanks
Is it possible to disable the creation of shared objects with distutils.core.Extension? I want to stop the compiler before linking (i.e. g++ -c ...).
I am swigging a native file, which creates an object file and a python file. I have other code to compile that I'll later link with this object file, so I don't want this to proceed after the .o compilation.
$ python setup.py build
running build
....
building 'foo' extension
swigging src/foobar.i to src/foobar.cpp
swig -python -c++ -o src/foobar.cpp src/foobar.i
I want to stop here, but it continues.
creating build/temp.linux-x86_64-2.7
creating build/temp.linux-x86_64-2.7/src
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -Isrc -I/usr/include/python2.7 -c src/foobar.cpp -o build/temp.linux-x86_64-2.7/src/foobar.o
g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro build/temp.linux-x86_64-2.7/src/foobar.o -o build/lib.linux-x86_64-2.7/foobar.so
Do I need to use the CCompiler class directly? Or is there a way to wrangle the Extension class?
23 ext_modules=[
24 # Swig
25 Extension(
26 name='foobar',
27 sources=['src/foobar.i'],
28 include_dirs=['src'],
29 swig_opts=['-c++'],
30 ),
31 ]
It is not possible to stop the linking step without modifying the underlying ccompiler object. One could theoretically override the link_shared_object function of the underlying ccompiler to do nothing (See the build_ext source).
However, to answer the original intent behind this question, the C/C++ files can be passed to the Extension with the Swig interface file without needing to compile them independently and link later. It is not necessary to separate the swig file generation and the library compilation.
You could do something like this:
from distutils.command import build_ext
def cmd_ex(command_subclass):
orig_ext = command_subclass.build_extension
def build_ext(self, ext):
sources = self.swig_sources(list(ext.sources), ext)
command_subclass.build_extension = build_ext
return command_subclass
#cmd_ex
class build_ext_ex(build_ext):
pass
setup(
name = ...,
cmdclass = {'build_ext': build_ext_ex},
ext_modules = ...
)
to override the default behavior of distutils command.
Setuptools – run custom code in setup.py
I am having this error during installation of 4suite.xml on ubuntu. Trying to install harpia on my ubuntu but couple packages are missing from my system so during to getting those missing dependencies I stuck on this.
username#ubuntu:~/4Suite-XML-1.0.2$ sudo python setup.py install
running install
running build
running config
running build_py
running build_ext
building 'Ft.Xml.Lib.cStreamWriter' extension
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/include/python2.7 -c Ft/Xml/src/StreamWriter.c -o build/temp.linux- x86_64-2.7/Ft/Xml/src/StreamWriter.o
In file included from /usr/include/python2.7/Python.h:94:0,
from Ft/Xml/src/StreamWriter.c:14:
Ft/Xml/src/StreamWriter.c: In function ‘writer_print’:
/usr/include/python2.7/stringobject.h:91:32: error: format not a string literal and no format arguments [-Werror=format-security]
#define PyString_AS_STRING(op) (((PyStringObject *)(op))->ob_sval)
^
Ft/Xml/src/StreamWriter.c:605:15: note: in expansion of macro ‘PyString_AS_STRING’
fprintf(fp, PyString_AS_STRING(repr));
^
Ft/Xml/src/StreamWriter.c: In function ‘entitymap_print’:
/usr/include/python2.7/stringobject.h:91:32: error: format not a string literal and no format arguments [-Werror=format-security]
#define PyString_AS_STRING(op) (((PyStringObject *)(op))->ob_sval)
^
Ft/Xml/src/StreamWriter.c:815:15: note: in expansion of macro ‘PyString_AS_STRING’
fprintf(fp, PyString_AS_STRING(repr));
^
cc1: some warnings being treated as errors
Can you help me to figure that out ?
Trying to use code below might helpful:
fprintf(fp, "%s", PyString_AS_STRING(repr));
Clues are from fprintf, error: format not a string literal and no format arguments [-Werror=format-security] and How to fix this compiler error 'format not a string literal and no format arguments' and similar situation in deploying python codes
A quick and dirty workaround for me was to set CFLAGS to override -Werror=format-security with -Wno-error=format-security (python 2 on CenTOS):
wget https://files.pythonhosted.org/packages/0e/ae/3c5b4fffb12be7c3a80c99475853349e1cf8477f99051921ea06fbf5e3b9/4Suite-XML-1.0.2.tar.gz
tar -xvf 4Suite-XML-1.0.2.tar.gz
export CFLAGS=" -Wno-error=format-security" # no error will be generated, you know what you do
python2 ./setup.py install # 4Suite-XML compilation works without error
I have tried, but am failing, to get a minimum working example. As I do not need to expose much of my fortran code to python, I don't need f2py to wrap large parts of it. Also, due to allocatable arrays being passed and derived types being used, I specifically want f2py to only wrap the interface module I created (in the following example 'main.f90'). But I am having problems to get the other modules which I compile separately to link to my main module.
The Code:
Note all source files are in a single directory.
I have created a fortran module I want to compile (libtest.f90):
module testmod
implicit none
contains
subroutine testsub(arr)
real, allocatable, intent(in) :: arr(:,:)
print *, 'testsub executed'
end subroutine testsub
end module testmod
and a fortran module that I want to wrap with f2py (main.f90):
module mainmod
use testmod
implicit none
contains
subroutine mainsub
real, allocatable :: arr(:,:)
call testsub(arr)
end subroutine main sub
end module mainmod
I use the following compile commands:
gfortran -c -fPIC libtest.f90
which generates 'libtest.o' and 'testmod.mod', and
f2py -c --fcompiler=gfortran -L. -I. -llibtest -m Main main.f90
Which gives me 'ld: library not found for -llibtest'.
I don't understand why this occurs, since it seems to work for others (F2PY doesn't find a module).
If I take out the -llibtest, I of course get (in my python script):
Traceback (most recent call last):
File "./script.py", line 7, in <module>
import Main
ImportError: dlopen(/Users/gmueller/Workspace/Minimum_PySpin/Main.so, 2): Symbol not found: ___testmod_MOD_testsub
Referenced from: /Users/gmueller/Workspace/Minimum_PySpin/Main.so
Expected in: dynamic lookup
Edit: Note I am on OSX(10.9.5), in case that makes some difference (but it shouldn't, since I do not need to pass any -shared (linux) or -dynamiclib (osx) to gfortran, right?).
Here the full output from f2py:
f2py -c --fcompiler=gfortran -L. -I. -llibtest -m Main main.f90
Unknown vendor: "gfortran"
running build
running config_cc
unifing config_cc, config, build_clib, build_ext, build commands --compiler options
running config_fc
unifing config_fc, config, build_clib, build_ext, build commands --fcompiler options
running build_src
build_src
building extension "Main" sources
f2py options: []
f2py:> /var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T/tmpJSf2e8/src.macosx-10.4-x86_64-2.7/Mainmodule.c
creating /var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T/tmpJSf2e8/src.macosx-10.4-x86_64-2.7
Reading fortran codes...
Reading file 'main.f90' (format:free)
Post-processing...
Block: Main
Block: mainmod
In: :Main:main.f90:mainmod
get_useparameters: no module testmod info used by mainmod
Block: mainsub
In: :Main:main.f90:mainmod:mainsub
get_useparameters: no module testmod info used by mainsub
Post-processing (stage 2)...
Block: Main
Block: unknown_interface
Block: mainmod
Block: mainsub
Building modules...
Building module "Main"...
Constructing F90 module support for "mainmod"...
Constructing wrapper function "mainmod.mainsub"...
mainsub()
Wrote C/API module "Main" to file "/var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T/tmpJSf2e8/src.macosx-10.4-x86_64-2.7/Mainmodule.c"
Fortran 90 wrappers are saved to "/var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T/tmpJSf2e8/src.macosx-10.4-x86_64-2.7/Main-f2pywrappers2.f90"
adding '/var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T/tmpJSf2e8/src.macosx-10.4-x86_64-2.7/fortranobject.c' to sources.
adding '/var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T/tmpJSf2e8/src.macosx-10.4-x86_64-2.7' to include_dirs.
copying /usr/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/f2py/src/fortranobject.c -> /var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T/tmpJSf2e8/src.macosx-10.4-x86_64-2.7
copying /usr/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/f2py/src/fortranobject.h -> /var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T/tmpJSf2e8/src.macosx-10.4-x86_64-2.7
adding '/var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T/tmpJSf2e8/src.macosx-10.4-x86_64-2.7/Main-f2pywrappers2.f90' to sources.
build_src: building npy-pkg config files
running build_ext
customize UnixCCompiler
customize UnixCCompiler using build_ext
customize Gnu95FCompiler
Found executable /usr/local/bin/gfortran
customize Gnu95FCompiler using build_ext
building 'Main' extension
compiling C sources
C compiler: gcc -fno-strict-aliasing -fno-common -dynamic -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes
creating /var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T/tmpJSf2e8/var
creating /var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T/tmpJSf2e8/var/folders
creating /var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T/tmpJSf2e8/var/folders/yg
creating /var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T/tmpJSf2e8/var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq
creating /var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T/tmpJSf2e8/var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T
creating /var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T/tmpJSf2e8/var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T/tmpJSf2e8
creating /var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T/tmpJSf2e8/var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T/tmpJSf2e8/src.macosx-10.4-x86_64-2.7
compile options: '-I. -I/var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T/tmpJSf2e8/src.macosx-10.4-x86_64-2.7 -I/usr/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/include -I/usr/local/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c'
gcc: /var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T/tmpJSf2e8/src.macosx-10.4-x86_64-2.7/fortranobject.c
In file included from /var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T/tmpJSf2e8/src.macosx-10.4-x86_64-2.7/fortranobject.c:2:
In file included from /var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T/tmpJSf2e8/src.macosx-10.4-x86_64-2.7/fortranobject.h:13:
In file included from /usr/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/include/numpy/arrayobject.h:4:
In file included from /usr/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/include/numpy/ndarrayobject.h:17:
In file included from /usr/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/include/numpy/ndarraytypes.h:1804:
/usr/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:15:2: warning: "Using deprecated NumPy API, disable it by " "#defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-W#warnings]
#warning "Using deprecated NumPy API, disable it by " \
^
1 warning generated.
gcc: /var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T/tmpJSf2e8/src.macosx-10.4-x86_64-2.7/Mainmodule.c
In file included from /var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T/tmpJSf2e8/src.macosx-10.4-x86_64-2.7/Mainmodule.c:17:
In file included from /var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T/tmpJSf2e8/src.macosx-10.4-x86_64-2.7/fortranobject.h:13:
In file included from /usr/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/include/numpy/arrayobject.h:4:
In file included from /usr/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/include/numpy/ndarrayobject.h:17:
In file included from /usr/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/include/numpy/ndarraytypes.h:1804:
/usr/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:15:2: warning: "Using deprecated NumPy API, disable it by " "#defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-W#warnings]
#warning "Using deprecated NumPy API, disable it by " \
^
1 warning generated.
compiling Fortran 90 module sources
Fortran f77 compiler: /usr/local/bin/gfortran -Wall -g -ffixed-form -fno-second-underscore -fPIC -O3 -funroll-loops
Fortran f90 compiler: /usr/local/bin/gfortran -Wall -g -fno-second-underscore -fPIC -O3 -funroll-loops
Fortran fix compiler: /usr/local/bin/gfortran -Wall -g -ffixed-form -fno-second-underscore -Wall -g -fno-second-underscore -fPIC -O3 -funroll-loops
compile options: '-I. -I/var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T/tmpJSf2e8/src.macosx-10.4-x86_64-2.7 -I/usr/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/include -I/usr/local/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c'
extra options: '-J/var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T/tmpJSf2e8/ -I/var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T/tmpJSf2e8/'
gfortran:f90: main.f90
compiling Fortran sources
Fortran f77 compiler: /usr/local/bin/gfortran -Wall -g -ffixed-form -fno-second-underscore -fPIC -O3 -funroll-loops
Fortran f90 compiler: /usr/local/bin/gfortran -Wall -g -fno-second-underscore -fPIC -O3 -funroll-loops
Fortran fix compiler: /usr/local/bin/gfortran -Wall -g -ffixed-form -fno-second-underscore -Wall -g -fno-second-underscore -fPIC -O3 -funroll-loops
compile options: '-I. -I/var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T/tmpJSf2e8/src.macosx-10.4-x86_64-2.7 -I/usr/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/core/include -I/usr/local/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c'
extra options: '-J/var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T/tmpJSf2e8/ -I/var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T/tmpJSf2e8/'
gfortran:f90: /var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T/tmpJSf2e8/src.macosx-10.4-x86_64-2.7/Main-f2pywrappers2.f90
/usr/local/bin/gfortran -Wall -g -Wall -g -undefined dynamic_lookup -bundle /var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T/tmpJSf2e8/var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T/tmpJSf2e8/src.macosx-10.4-x86_64-2.7/Mainmodule.o /var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T/tmpJSf2e8/var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T/tmpJSf2e8/src.macosx-10.4-x86_64-2.7/fortranobject.o /var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T/tmpJSf2e8/main.o /var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T/tmpJSf2e8/var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T/tmpJSf2e8/src.macosx-10.4-x86_64-2.7/Main-f2pywrappers2.o -L. -L/usr/local/lib/gcc/x86_64-apple-darwin/4.8.2 -llibtest -lgfortran -o ./Main.so
ld: library not found for -llibtest
collect2: error: ld returned 1 exit status
ld: library not found for -llibtest
collect2: error: ld returned 1 exit status
error: Command "/usr/local/bin/gfortran -Wall -g -Wall -g -undefined dynamic_lookup -bundle /var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T/tmpJSf2e8/var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T/tmpJSf2e8/src.macosx-10.4-x86_64-2.7/Mainmodule.o /var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T/tmpJSf2e8/var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T/tmpJSf2e8/src.macosx-10.4-x86_64-2.7/fortranobject.o /var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T/tmpJSf2e8/main.o /var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T/tmpJSf2e8/var/folders/yg/65v1lbd153v1jlt1kb91vcbm0000gq/T/tmpJSf2e8/src.macosx-10.4-x86_64-2.7/Main-f2pywrappers2.o -L. -L/usr/local/lib/gcc/x86_64-apple-darwin/4.8.2 -llibtest -lgfortran -o ./Main.so" failed with exit status 1
Your command:
gfortran -c -fPIC libtest.f90
produces an object file with position independent code. This is a pre-requisite of a shared library, not a shared library.
If you want to use the object as is, you can modify your f2py invocation:
f2py -c --fcompiler=gfortran -I. libtest.o -m Main main.f90
This will link the object file and produce the file Main.cpython-33.so (the python version number may differ for you) and you can then import main in your python code.
If you would rather actually produce a shared object, you need to compile to a shared library. One way to do this is:
gfortran -shared -O2 -o libtest.so -fPIC libtest.f90
This produces libtest.so and now your original f2py command will work with one small change:
f2py -c --fcompiler=gfortran -L. -I. -ltest -m Main main.f90
The small change I am referring to is changing -llibtest to -ltest, as the -l option will add lib to the front of the library and .so to the end, e.g. -ltest will look for libtest.so. This produces Main.cpython-33.so with a dynamic link dependency to libtest.so, so you will need to distribute both shared libraries in order to use the python module.