Develop Reference

Compiling a wrapper for a c library using Cython - Linker cant find .dylib of external c library on OSX

I have written a wrapper in Cython for an integration function from NAG (https://www.nag.co.uk/content/nag-library-c) c library.
It compiles using the python setup.py build --inplace, where setup file is:
from setuptools import Extension, setup
from Cython.Build import cythonize
import re
def loadMacros(headerFile):
""" Given a .h file, return dict of touples with macros """
regex = re.compile("#define +(\w+) *(\w*)")
with open(headerFile) as f:
macros = dict(map(lambda x: re.match(regex, x).groups(),
[l for l in f if re.match(regex, l)]))
# Remove recursive entries - Note this is not foolproof..
# while not set(macros.keys()).isdisjoint(macros.values()):
# for k, v in macros.items():
# if v in macros:
# macros[k] = macros[v]
return macros
nagHome = "/Users/hfmw1m17/NAG/nlmi627dbl" # "/opt/NAG/cll6a23dhl"
macros = loadMacros(nagHome + "/lp64/include/nag.h")
macros = list(macros.items())
e = Extension("nag_integrate",
define_macros=macros,
sources=["nag_integrate.pyx"],
include_dirs=[nagHome + "/lp64/include",nagHome + "/lp64/lib"],
library_dirs=[nagHome + "/lp64/lib"],libraries=["nag_nag"],extra_objects=[
nagHome+"/lp64/lib/libnag_nag.dylib"],runtime_library_dirs=[nagHome+"/lp64/lib/"],extra_link_args=['-Wl,-rpath']
)
setup(ext_modules=cythonize(e,annotate=True,language_level=3))enter code here
with output:
/Users/hfmw1m17/anaconda3/envs/TowingTankAcoustics/bin/python3.7 setup.py build_ext --inplace
Compiling nag_integrate.pyx because it changed.
[1/1] Cythonizing nag_integrate.pyx
running build_ext
building 'nag_integrate' extension
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I/Users/hfmw1m17/anaconda3/envs/TowingTankAcoustics/include -arch x86_64 -I/Users/hfmw1m17/anaconda3/envs/TowingTankAcoustics/include -arch x86_64 -DNAG_H= -DNAG_MICROSOFT_THREAD_SAFE= -DNAG_THREAD_SAFE= -DNULLFN=0 -DNULLDFN=0 -DNAGERR_DEFAULT= -DNAGUSER_DEFAULT= -DNAGCOMM_NULL= -DNAGMESG_DEFAULT= -DE01_DEFAULT= -DE04_DEFAULT= -DG13_DEFAULT= -DH02_DEFAULT= -DINIT_FAIL= -DSET_FAIL= -DINIT_MESG= -DINIT_STREAM= -DRDUMMY= -DIDUMMY= -DINIT2DUMMY= -DVprintf= -DVfprintf= -DVsprintf= -DVscanf= -DVfscanf= -DVstrcpy= -DABS= -DFABS= -DSIGN= -DMAX= -DMIN= -DDROUND= -DROUND= -DSQZABS= -DCONJ= -DVCONJ= -DZMULT= -D_nag_expand= -Dnag_stringize= -I/Users/hfmw1m17/NAG/nlmi627dbl/lp64/include -I/Users/hfmw1m17/NAG/nlmi627dbl/lp64/lib -I/Users/hfmw1m17/anaconda3/envs/TowingTankAcoustics/include/python3.7m -c nag_integrate.c -o build/temp.macosx-10.9-x86_64-3.7/nag_integrate.o
gcc -bundle -undefined dynamic_lookup -L/Users/hfmw1m17/anaconda3/envs/TowingTankAcoustics/lib -arch x86_64 -L/Users/hfmw1m17/anaconda3/envs/TowingTankAcoustics/lib -arch x86_64 -arch x86_64 build/temp.macosx-10.9-x86_64-3.7/nag_integrate.o /Users/hfmw1m17/NAG/nlmi627dbl/lp64/lib/libnag_nag.dylib -L/Users/hfmw1m17/NAG/nlmi627dbl/lp64/lib -L/Users/hfmw1m17/NAG/nlmi627dbl/lp64/lib/ -lnag_nag -o build/lib.macosx-10.9-x86_64-3.7/nag_integrate.cpython-37m-darwin.so -Wl,-rpath
copying build/lib.macosx-10.9-x86_64-3.7/nag_integrate.cpython-37m-darwin.so ->
Process finished with exit code 0
However when i import a function from the .so object created i get the following error:
ImportError: dlopen(/Users/hfmw1m17/WaterTankISM/WaterTankISM/nag_integration/nag_integrate.cpython-37m-darwin.so, 2): Library not loaded: libnag_nag.dylib
Referenced from: /Users/hfmw1m17/WaterTankISM/WaterTankISM/nag_integration/nag_integrate.cpython-37m-darwin.so
Reason: image not found
libnag_nag.dylib is a dynamic library produced by NAG.
Using otool -L on the shared object of my wrapper results in:
libnag_nag.dylib (compatibility version 0.0.0, current version 27.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1281.100.1)
I think its an issue with the linker not being able to find the dynamic library when compiling. Any suggestions on how to solve this problem?
many thanks

Solved using:
install_name_tool -add_rpath path_to_dylib_directory
after running python setup.py build --inplace.
Would like a way of doing this in the setup.py file if anyone can figure that out.
many thanks

OSX Caffe Compilation Fails With Expected Expression Error

I have been trying to install Caffe on my mac running OSX 10.13.6
I have followed several guides, including the installation guide on the caffe site. I have erased everything and restarted several times. I get the same problem no matter what. When I go to compile everything I am getting this
Scanning dependencies of target caffeproto
[ 1%] Building CXX object src/caffe/CMakeFiles/caffeproto.dir/__/__/include/caffe/proto/caffe.pb.cc.o
In file included from /Users/Name/Documents/Programming/PythonEnv/caffe/build/include/caffe/proto/caffe.pb.cc:4:
In file included from /Users/Name/Documents/Programming/PythonEnv/caffe/build/include/caffe/proto/caffe.pb.h:9:
/usr/local/include/google/protobuf/stubs/common.h:209:17: error: expected
expression
OnShutdownRun([](const void* p) { delete static_cast<const T*>(p); }, p);
^
In file included from /Users/Name/Documents/Programming/PythonEnv/caffe/build/include/caffe/proto/caffe.pb.cc:4:
In file included from /Users/Name/Documents/Programming/PythonEnv/caffe/build/include/caffe/proto/caffe.pb.h:25:
In file included from /usr/local/include/google/protobuf/generated_message_table_driven.h:34:
In file included from /usr/local/include/google/protobuf/map.h:49:
In file included from /usr/local/include/google/protobuf/map_type_handler.h:35:
In file included from /usr/local/include/google/protobuf/wire_format_lite_inl.h:43:
/usr/local/include/google/protobuf/message_lite.h:117:3: error: unknown type
name 'constexpr'
constexpr const T& get() const { return reinterpret_cast<const T&>(union_); }
I have also tried using CMake and run into the same problem. I'm not sure where to go from here. I am not incredibly knowledgable about building processes so I apologize if this is vague and will be happy to provide whatever other information might help fix this.
Thank you!

If compiling with make, change Makefile as follows:
- CXXFLAGS += -pthread -fPIC $(COMMON_FLAGS) $(WARNINGS)
+ CXXFLAGS += -pthread -fPIC $(COMMON_FLAGS) $(WARNINGS) -std=c++11

This is the same problem as reported here: https://trac.macports.org/ticket/57093#comment:1
The compiler needs to be using C++11. Try making this change in CMakeLists.txt:
if(UNIX OR APPLE)
- set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fPIC -Wall")
+ set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fPIC -Wall -std=c++11")
endif()

You just need to replace the newest version of protobuf with protobuf v3.5.1.
wget https://github.com/protocolbuffers/protobuf/archive/v3.5.1.zip
You need to download to source code, and compile it yourself.

install gcc4.0 on mac os x 10.8

I am trying to read and learn the pytho2.5.6 source code, and try to recompile the code everytime i make some change on it.
But it doesn't seem to work when I comile the source code via the following commands:
./configure --prefix=/path/to/somewhere/that/don't/messup/myenv/
make
After make, the console just show errors like this:
cc1: error: unrecognized command line option "-Wno-long-double"
I search the google , and it seems like the gcc(4.2.1) that i am using is not compatiable with the python2.5.6.
some info of my sys:
gcc version 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2336.11.00)
Xcode 4.6.2(installed via the xcode command cline tool )
python2.5.6 source code(download from python.org)
I have try some solutions like these:
1.download diff python code version , like 2.5.4, and do the steps again.
2.after configure,modify the Makefile,and remove the --Wno-long-double,but new errors showed like below:
➜ Python-2.5.6 make
gcc -c -fno-strict-aliasing -no-cpp-precomp -mno-fused-madd -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I. -IInclude -I./Include -DPy_BUILD_CORE -o Python/mactoolboxglue.o Python/mactoolboxglue.c
In file included from Include/Python.h:57,
from Python/mactoolboxglue.c:26:
Include/pyport.h:547: warning: ‘struct winsize’ declared inside parameter list
Include/pyport.h:547: warning: its scope is only this definition or declaration, which is probably not what you want
Include/pyport.h:548: warning: ‘struct winsize’ declared inside parameter list
In file included from /System/Library/Frameworks/CoreFoundation.framework/Headers/CFBase.h:67,
from /System/Library/Frameworks/CoreFoundation.framework/Headers/CoreFoundation.h:38,
from /System/Library/Frameworks/CoreServices.framework/Frameworks/CarbonCore.framework/Headers/CarbonCore.h:18,
from /System/Library/Frameworks/CoreServices.framework/Frameworks/AE.framework/Headers/AE.h:20,
from /System/Library/Frameworks/CoreServices.framework/Headers/CoreServices.h:18,
from /System/Library/Frameworks/Carbon.framework/Headers/Carbon.h:20,
from Include/pymactoolbox.h:10,
from Python/mactoolboxglue.c:27:
/usr/include/AvailabilityMacros.h:109:14: warning: #warning Building for Intel with Mac OS X Deployment Target < 10.4 is invalid.
In file included from Python/mactoolboxglue.c:27:
.......
Python/mactoolboxglue.c: In function ‘MediaObj_Convert’:
Python/mactoolboxglue.c:431: error: ‘cobj’ undeclared (first use in this function)
Python/mactoolboxglue.c:431: error: too many arguments to function ‘PyMacGluePtr_MediaObj_Convert’
make: *** [Python/mactoolboxglue.o] Error 1
So, i wonder if there are some way to intall gcc4.0 manually?
PS:the reason why i use python2.5.6 is because i read a book called which is written in chinese.Or maybe the best solution is to read the 2.7 source code =.=....hmmm.....

Reinstalling Python 2.5.6 On Lion

I am having the identical problem as mentioned in this thread when I try to install Python 2.5.6 on Lion: Python 2.5.6 build error on Mac Lion
Specifically, the error I receive is when attempting a make is:
/usr/bin/gcc-4.2 -c -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I. -IInclude -I./Include -DPy_BUILD_CORE -o Modules/python.o ./Modules/python.c
cc1: error: unrecognized command line option "-Wno-long-double"
make: *** [Modules/python.o] Error 1
In my case, it is as a result of me deleting the python 2.5 folder thinking I would never need it while I was troubleshooting another problem.
Now, when I try to launch Google App Engine, I get the error: "Sorry, pieces of GoogleAppEngineLauncher.app appear missing or corrupted, or I can't run python2.5 properly" and then it shuts down, so unfortunately I can't just ignore the problem as is recommended in the previous thread.
EDIT:
It looks like I messed things up pretty royally by deleting the folder. The solution (though I haven't implemented it yet) is described here: Mac Python "Installation step failed: run postflight script"
EDIT x2:
Solved it! Though I have no idea what kind of damage I've done. I ran the .DMG installer, and even though it failed it placed the folder for Python 2.5.4. in /Library/Frameworks/Python.framework/Versions. I then copied that along with 2.7 into the /System/Library/Frameworks/Python.framework/Versions folder. After that, launched Google App Engine and everything worked! (For now.)

I think your best bet is to install the Universal Binary provided for OS X, see http://www.python.org/ftp/python/2.5.4/python-2.5.4-macosx.dmg .
This is well documented on http://docs.python.org/2/using/mac.html.

How to actually build 64-bit Python on OS X 10.6.2

Why? I want to do this because installation of SciPy recommends it, and I thought it would be a good learning experience. This question has been asked before (e.g. here). The preferred answer seems to be to use MacPorts, but as I say, I'd like to understand how it's done.
Anyway, I grab the source (Python-2.6.4.tgz) and unzip. I read the instructions on how to build a 64-bit "framework" build. As I understand it, I should run
./configure --enable-framework --enable-universalsdk=/ --with-univeral-archs=intel
configure runs for a while...and finishes. When I do make, it's obviously got a problem:
$ make
gcc -c -arch ppc -arch i386 -isysroot / -fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I. -IInclude -I./Include -DPy_BUILD_CORE -o Modules/python.o ./Modules/python.c
In file included from //usr/include/architecture/i386/math.h:626,
from //usr/include/math.h:28,
from Include/pyport.h:235,
from Include/Python.h:58,
from ./Modules/python.c:3:
//usr/include/AvailabilityMacros.h:108:14: warning: #warning Building for Intel with Mac OS X Deployment Target < 10.4 is invalid.
gcc is being called with the wrong arguments. Do I have the wrong arguments to configure, or should I set compiler flags in the environment, or what?
Edit: I don't see any errors in the output from configure...and I see this line:
checking for OSX 10.5 SDK or later... yes
it ends with
creating Modules/Setup
creating Modules/Setup.local
creating Makefile
Edit2: I thought I copied from the readme...
I did! There's a typo in the readme spec! My age-related dyslexia is acting up again. ;)

Your ./configure option is not correct. --enable-universalsdk should be set to the correct SDK, not /!
That's why gcc got confused, see the option -isysroot.
So, check what SDKs you have in /Developer/SDKs, and set the correct one.
Moreover, your gcc is called only with -arch ppc -arch i386, which do not include -arch x86_64 which is the intel 64 bit flag.

In order to choose an answer as correct, I'm paraphrasing comments above:
As noticed by Virgil Dupras, there was a typo in this flag:
--with-universal-archs=intel
It originates from the file Mac/readme, but I should've caught it before posting.
Also I recommend that you read Ned Deily's very helpful comments. Check out those guys and vote 'em up.