Develop Reference

PyCaffe build fails, lboost_python not found

I am on MacOS 10.13.3 (17D47) with Python 2.7.14. I am in the process of building caffe w/ python. The project is CPU only. I am able to build caffe through make run test however, after attempting make pycaffe I get an error. This is the result:
touch python/caffe/proto/__init__.py
CXX/LD -o python/caffe/_caffe.so python/caffe/_caffe.cpp
PROTOC (python) src/caffe/proto/caffe.proto
In file included from python/caffe/_caffe.cpp:17:
In file included from ./include/caffe/caffe.hpp:12:
./include/caffe/net.hpp:41:5: warning: unused typedef 'INVALID_REQUESTED_LOG_SEVERITY' [-Wunused-local-typedef]
LOG_EVERY_N(WARNING, 1000) << "DEPRECATED: ForwardPrefilled() "
^
/usr/local/include/glog/logging.h:943:30: note: expanded from macro 'LOG_EVERY_N'
INVALID_REQUESTED_LOG_SEVERITY); \
^
1 warning generated.
ld: library not found for -lboost_python
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [python/caffe/_caffe.so] Error 1
I have a Makefile.config that calls out boost-python from the brew install set as:
# Whatever else you find you need goes here.
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/local/Cellar/boost-python/1.67.0/lib
My two initial questions are:
1)Am I not allowed to link to the boost library using the system python? 2)What is the correct way to tell make to grab boost-python?

I was actually having the same problem and couldn't find any answers for days. However, I found this question: Build caffe with Python ( cannot find -lboost_python3 ) .
That question is for python3, but I'm using python2.7 . Basically, I went into the Makefile (not Makefile.config), searched for boost_python, and changed it to boost_python27 . I did the following steps:
vim Makefile
use vim command :?boost_python (there should only be 1 occurrence)
changed PYTHON_LIBRARIES ?= boost_python python2.7
to PYTHON_LIBRARIES ?= boost_python27 python2.7
ran the command sudo make pycaffe
I hope this helps you!

Goto Makefile (and not the .config file) and change the name of the ld library to python27.

Cannot find -lgcc

I am trying to follow an example for Cython builds from here http://blog.perrygeo.net/2008/04/19/a-quick-cython-introduction/, except my c1.pyx is simply helloworld.pyx with "print "Hello World"":
# this will create a c1.c file - the C source code to build a python extension
cython c1.pyx
# Compile the object file
gcc -c -fPIC -I/usr/include/python2.5/ c1.c
# Link it into a shared library
gcc -shared c1.o -o c1.so
The reason I am doing it in the command line is because my setup.py is giving me an error so I am doing it manually to see what is wrong.
The last step (gcc -shared c1.o -o c1.so) is where I hit the following error:
c:/users/MyName/anaconda/mingw/bin/../lib/gcc/x86_64-w64-mingw32/4.7.0/../../../..
/x86_64-w64-mingw32/bin/ld.exe: cannot find -lgcc
c:/users/MyName/anaconda/mingw/bin/../lib/gcc/x86_64-w64-mingw32/4.7.0/../../../..
/x86_64-w64-mingw32/bin/ld.exe: cannot find -lgcc
collect2.exe: error: ld returned 1 exit status
error: command 'C:\\Users\\MyName\\Anaconda\\Scripts\\gcc.bat' failed with exit st
atus 1
Any suggestions?
EDIT1: Thank you #mat for the edit. Also, I was noticing I didn't have Visual C++ 2008 installed on my computer. Not sure if Anaconda requires this for GCC and if this is the solution.

Linking Libraries using f2py

I have a fortran program that uses some library files. I am trying to link them along with the module file being created.
The library file I am trying to link is called ulib.a and is located in the directory /home/replace/lib/
The command I am using is:
f2py -L/home/replace/lib/ -lulib.a -c main.f -m progs
I am getting the following error:
/usr/bin/ld: cannot find -lulib.a
collect2: ld returned 1 exit status
/usr/bin/ld: cannot find -lulib.a
collect2: ld returned 1 exit status
I would appreciate any help!

Try leaving off the .a - I am reasonably sure that the linker already knows that libraries are .a so in your example it will be looking for ulib.a.a and failing.

I had to remove the extension from the library name and also provide the full path. For some reason providing the path using the -L argument did not work.
f2py -l/home/replace/lib/ulib -c main.f -m progs

The library should have the full name libxxx.a where xxx is the given name. Then do
f2py -L. -lxxx -c main.f90 -m progs
Note that only xxx comes after -l. If you create the library yourself remember to include -fPIC. For example, it could look like this:
gfortran -c -fPIC source1.f90 source2.f90
ar crs libxxx.a obj1.o obj2.o
f2py -L. -lxxx -c main.f90 -m progs
Found guidance in this example: https://modelingguru.nasa.gov/docs/DOC-2343

cannot find -lgsl and -lgslcblas

I am trying to compile a software called MRtrix in windows. It uses a python build script, (I have uploaded it here: http://pastebin.com/XnufSz53)
when I run this script I get the error pasted bellow.
I guess that the GSL libraries are not being found and I need to add a -L pointing to the library but because the compilation is being done by the python script I do not really now how to do this. Currently GnuWin is installed at C:\GnuWin32\
Thank you in advance,
Jesse
============================================================
ERROR: [LD] lib\mrtrix-0_2_11.dll
g++ -shared lib\image\format\base.o lib\math\vector.o lib\file\mmap.o lib\args.o
lib\image\format\dicom.o lib\file\dicom\select_cmdline.o lib\file\key_value.o l
ib\image\axis.o lib\image\format\mrtrix.o lib\image\interp.o lib\data_type.o lib
\point.o lib\image\mapper.o lib\file\dicom\element.o lib\file\dicom\mapper.o lib
\file\dicom\patient.o lib\file\dicom\quick_scan.o lib\image\object.o lib\mrtrix.
o lib\image\format\xds.o lib\file\dicom\dict.o lib\app.o lib\math\matrix.o lib\f
ile\config.o lib\image\format\mri.o lib\file\dicom\image.o lib\math\linalg.o lib
\file\dicom\study.o lib\file\dicom\series.o lib\image\name_parser.o lib\image\fo
rmat\list.o lib\file\dicom\tree.o lib\image\fft.o lib\image\header.o lib\image\f
ormat\analyse.o lib\image\format\nifti1.o -LC:/gtkmm64/lib -lglibmm-2.4 -lgobjec
t-2.0 -lsigc-2.0 -lgthread-2.0 -lglib-2.0 -lintl -lgsl -lgslcblas -lz -o lib\mrt
rix-0_2_11.dll
failed with output:
c:/mingw/bin/../lib/gcc/mingw32/4.7.2/../../../../mingw32/bin/ld.exe: cannot fin
d -lgsl
c:/mingw/bin/../lib/gcc/mingw32/4.7.2/../../../../mingw32/bin/ld.exe: cannot fin
d -lgslcblas
collect2.exe: error: ld returned 1 exit status
============================================================

Using libspotify .dll/.lib files in MinGW32 compiling pySpotify

Using MinGW32 on a Windows PC I am trying to compile pySpotify. The first error was that libspotify/api.h was missing. I fixed this by copying the appropriate folder from libspotify into C:\MinGW\include. However now dllwrap is now failing with ld linking. The binaries Spotify distribute are libspotify.dll and libspotify.lib. No matter where I put them (pySpotify folder/subfolders, temp build folder/subfolders and MinGW folder/subfolders) or what I name them (.a, .o & .so) it still shows the same error messages.
The pertinent error is:
C:\MinGW\bin\dllwrap.exe -mdll -static --output-lib build\temp.win32-2.7\Release\src\lib_spotify.a --def build\temp.win32-2.7\Release\src\_spotify.def -s build\temp.win32-2.7\Release\src\module.o build\temp.win32-2.7\Release\src\session.o build\temp.win32-2.7\Release\src\link.o build\temp.win32-2.7\Release\src\track.obuild\temp.win32-2.7\Release\src\album.o build\temp.win32-2.7\Release\src\albumbrowser.o build\temp.win32-2.7\Release\src\artist.o build\temp.win32-2.7\Release\src\artistbrowser.o build\temp.win32-2.7\Release\src\search.o build\temp.win32-2.7\Release\src\playlist.o build\temp.win32-2.7\Release\src\playlistcontainer.o build\temp.win32-2.7\Release\src\playlistfolder.o build\temp.win32-2.7\Release\src\image.o build\temp.win32-2.7\Release\src\user.o build\temp.win32-2.7\Release\src\pyspotify.o build\temp.win32-2.7\Release\src\toplistbrowser.o -LC:\Python26\libs -LC:\Python26\PCbuild -lspotify -lpython26 -lmsvcr90 -o build\lib.win32-2.7\spotify\_spotify.pyd
c:/mingw/bin/../lib/gcc/mingw32/4.7.2/../../../../mingw32/bin/ld.exe: cannot find -lspotify
collect2.exe: error: ld returned 1 exit status
dllwrap: gcc exited with status 1
error: command 'dllwrap' failed with exit status 1
bok says on github that:
You need to add the API headers in the include path (add something like -I~\libspotify\include to your compiler options) and the shared library in the library path (add -L~\libspotify\lib to your linker options). This will allow the compiler to find the necessary include files, and your linker to find the necessary binary objects.
However the distutils Extension class seems to be deprecated and hard to find documentation for (I believe this is where custom compiler options need to go). Appreciate that ~ may need changed to %USERPROFILE% or similar. Similarly %PYTHONPATH%\Lib\distutils\distutils.cfg has little documentation beyond the [build] compiler=mingw32 stanza. This makes editing the compile/link commands and their options impossible to change.
How do you compile pySpotify on Windows?
EDIT:
By using Python 2.6 and copying libspotify.dll/libspotify.lib to C:\Python26\PCbuild and renaming them to spotify.dll/libspotify.lib I now receive another error message from ld:
C:\MinGW\bin\dllwrap.exe -mdll -static --output-lib build\temp.win32-2.6\Release\src\lib_spotify.a --def build\temp.win32-2.6\Release\src\_spotify.def -s build\temp.win32-2.6\Release\src\module.o build\temp.win32-2.6\Release\src\session.o build\temp.win32-2.6\Release\src\link.o build\temp.win32-2.6\Release\src\track.obuild\temp.win32-2.6\Release\src\album.o build\temp.win32-2.6\Release\src\albumbrowser.o build\temp.win32-2.6\Release\src\artist.o build\temp.win32-2.6\Release\src\artistbrowser.o build\temp.win32-2.6\Release\src\search.o build\temp.win32-2.6\Release\src\playlist.o build\temp.win32-2.6\Release\src\playlistcontainer.o build\temp.win32-2.6\Release\src\playlistfolder.o build\temp.win32-2.6\Release\src\image.o build\temp.win32-2.6\Release\src\user.o build\temp.win32-2.6\Release\src\pyspotify.o build\temp.win32-2.6\Release\src\toplistbrowser.o -LC:\Python26\libs -LC:\Python26\PCbuild -lspotify -lpython26 -lmsvcr90 -o build\lib.win32-2.6\spotify\_spotify.pyd
_spotify.exp: file not recognized: File format not recognized
collect2.exe: error: ld returned 1 exit status
dllwrap: gcc exited with status 1
error: command 'dllwrap' failed with exit status 1

Not having access to a mingw installation at the moment, I can suggest a few things.
First, ld is known to be picky about order of arguments. Strangely, when I googled for "ld argument order", I got a bunch of pages suggesting that the order doesn't matter, but I have been burned several times by this. I've had the most success with the following argument order:
Switches to ld (ie, -Wall)
Library search paths (ie, -LPATH)
Object files
Libraries (ie, -lspotify)
I noticed in your linker output some references to amd64. I'm not sure how you compiled the other object files that you have, but since libspotify is 32-bit on Windows, I'm guessing that mixing 32/64 bit here is not going to work out too well.
The last thing I can think of is that perhaps the dll extension is confusing to ld, have you tried changing the filename to libspotify.so? I know this is kind of a shot in the dark, but otherwise I'm not sure how to help you further.

I guess you are trying to link a 64bit version of pyspotify against a 32bit version of libspotify. Try to rebuild pyspotify adding the -m32 to CFLAGS and CXXFLAGS

I've added some instructions on how I got it to compile on the actual ticket for the issue. Let me Know if it helped you!
https://github.com/mopidy/pyspotify/issues/63