Develop Reference

Python/C++ Extension, Undefined symbol error when linking a library

I made a project that has glfw as a library, my directory looks like this:
main_dir
|--include
|--|--glfw_binder.h
|--src
|--|--glfw_binder.cpp
|--lib
|--|--glfw
|--|--|--src
|--|--|--|--libglfw.so
|--|--|--|--libglfw.so.3
|--|--|--|--libglfw.so.3.3
|--|--|--|--...
|--|--|--include
|--|--|-- ...
|--main.cpp
|--setup.py
|--test.py
I've installed glfw from its website and simply extracted it to lib/ directory. Then I ran cmake -G "Unix Makefiles" followed by make to build it.
When I try to run it as pure C++ with this cmake file it runs just fine, it displays the glfw window:
cmake_minimum_required(VERSION 3.0)
project(Test)
add_subdirectory(lib/glfw)
find_package(PythonLibs REQUIRED)
include_directories(${PYTHON_INCLUDE_DIRS})
include_directories(${CMAKE_CURRENT_SOURCE_DIR}/include)
add_executable(Test main.cpp src/glfw_binder.cpp)
target_link_libraries(Test PRIVATE
${PYTHON_LIBRARIES}
glfw
)
And here is what my setup.py looks like:
from setuptools import setup, Extension, find_packages
module1 = Extension('test',
sources = ['main.cpp', 'src/glfw_binder.cpp'],
include_dirs=["include", "lib/glfw/include"],
library_dirs=["lib/glfw"],
libraries=["lib/glfw"],
extra_link_args=["lib/glfw"]
)
setup (name = 'python_test',
version = '1.0',
description = 'This is a demo package',
packages=find_packages("glfw"),
ext_modules = [module1]
)
Then I run python3 setup.py build followed by sudo python3 setup.py install. It creates the so files just fine. Then I run this test code:
import test
test.create_glfw_window()
When I run this script via python3 test.py I got this error:
Traceback (most recent call last):
File "/home/turgut/Desktop/TestDir/Python-Binder-Test/test.py", line 3, in <module>
import test
ImportError: /usr/local/lib/python3.10/dist-packages/python_test-1.0-py3.10-linux-x86_64.egg/test.cpython-310-x86_64-linux-gnu.so: undefined symbol: glfwDestroyWindow
It's not specific to glfwDestroyWindow it says this for all glfw functions. How am I supposed to link this?
Update: I've added these lines:
packages=['glfw'],
package_dir={'glfw': 'lib/glfw'},
#package_data={'glfw': ['lib/glfw/src/*.so']},
package_data={'glfw': ['lib/glfw/src/libglfw.so', "lib/glfw/src/libglfw.so.3", "lib/glfw/src/libglfw.so.3.3"]},
Now it gives me the following error:
ImportError: libglfw.so.3: cannot open shared object file: No such file or directory
Even though I've specified all the .so files together. I don't think this is the way to go but I thought it's worth mentioning.
Better update:
I've removed the above update and added these two lines to my extension:
include_dirs=["include", "lib/glfw/include"],
#extra_link_args=[ "-lm", "-lGL", "-lGLU", "-lglfw"],
library_dirs=["lib/glfw"],
libraries=['glfw']
and now I'm getting this error which I think is closer to what I'm looking for:
/usr/bin/ld: cannot find -lglfw: No such file or directory
collect2: error: ld returned 1 exit status
error: command '/usr/bin/x86_64-linux-gnu-g++' failed with exit code 1

For anyone facing a similar issue, you need to specify runtime_library_dirs so the setup.py file looks like this:
from setuptools import setup, Extension, find_packages
module1 = Extension('nerveblox',
sources = ['main.cpp', 'src/nerveblox_VM.cpp'],
include_dirs=["include", "lib/glfw/include"],
#extra_link_args=[ "-lm", "-lGL", "-lGLU", "-lglfw"],
library_dirs=["lib/glfw/src"],
libraries=['glfw', 'GL'],
runtime_library_dirs=['lib/glfw/src']
)
setup (
name = 'Nerveblox_python_test',
version = '1.0',
description = 'This is a demo package',
ext_modules = [module1],
install_requires=[
'importlib-metadata; python_version == "3.8"',
],
)

Python missing DLL from installed module

I got a project which consists of Python and C++. I don't understand all of it but in order to compile and run everything I run an included BAT file. I have already installed the dependencies needed. Now when I run the script, I get this:
MainProcess - [INFO] os_utils: Disabling idle sleep not supported on this OS version.
world - [ERROR] launchables.world: Process Capture crashed with trace:
Traceback (most recent call last):
File "C:\work\pupil\pupil_src\launchables\world.py", line 118, in world
from plugin_manager import Plugin_Manager
File "C:\work\pupil\pupil_src\shared_modules\plugin_manager.py", line 15, in <module>
from video_capture import Base_Manager, Base_Source
File "C:\work\pupil\pupil_src\shared_modules\video_capture\__init__.py", line 36, in <module>
from .file_backend import FileCaptureError, FileSeekError
File "C:\work\pupil\pupil_src\shared_modules\video_capture\file_backend.py", line 13, in <module>
import av
File "C:\Users\XXX\AppData\Local\Programs\Python\Python36\lib\site-packages\av\__init__.py", line 9, in <module>
from av._core import time_base, pyav_version as __version__
ImportError: DLL load failed: The specified module could not be found.
It couldn't find pyav? But if I run:
C:\Users\XXX\Downloads>pip install av-0.3.1-cp36-cp36m-win_amd64.whl
Requirement already satisfied: av==0.3.1 from file:///C:/Users/XXX/Downloads/av-0.3.1-cp36-cp36m-win_amd64.whl in c:\users\anton\appdata\local\programs\python\python36\lib\site-packages
I already have it installed. What am doing wrong here?
If I open ...site-packages\av__init__.py" I can see this:
from av._core import time_base, pyav_version as __version__
is it something here?

I suspect that the module has successfully installed, but is dynamically linked against FFMPEG. You can obtain built distributions of FFMPEG from their website (https://www.ffmpeg.org/download.html). They provide both statically linked and dynamically linked builds, though it is going to be the dynamically linked one that is providing the dll's you need. Looking at my own copy of PyAV, it seems that the current release version (3.4.2) is the one that it is linked against.
When you download it, it will have a name like ffmpeg-date-build-win64-shared. In the bin directory, you will find all the relevant DLLs. You can either add this directory to the PATH, or more easily, copy the DLLs to your python location.

sphinx python with boost-python library not found

I'm trying to use sphinx to generate documentation for a package that I wrote. Everything was working great until I added some code that used another package I wrote that wraps some boost-python modules.
Now when I run make html I get this (edited for clarity):
$ make html
sphinx-build -b html -d build/doctrees source build/html
Running Sphinx v1.3.1
loading pickled environment... done
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 0 source files that are out of date
updating environment: 0 added, 1 changed, 0 removed
reading sources... [100%] my_project
/path/to/my_project/my_project/docs/source/my_project.rst:19: WARNING: autodoc: failed to import module u'my_project.my_module'; the following exception was raised:
Traceback (most recent call last):
File "/Users/harding/anaconda/lib/python2.7/site-packages/Sphinx-1.3.1-py2.7.egg/sphinx/ext/autodoc.py", line 385, in import_object
__import__(self.modname)
File "/path/to/my_project/__init__.py", line 12, in <module>
from . import module_that_uses_boost_python_classes
File "/path/to/module_that_uses_boost_python_classes.py", line 10, in <module>
import package_that_wraps_boost_python_classes
File "/path/to/package_that_wraps_boost_python_classes/__init__.py", line 21, in <module>
from native_boost_python_module import *
ImportError: dlopen(/path/to/native_boost_python_module.so, 2): Library not loaded: cpp_library.dylib
Referenced from: /path/to/native_boost_python_module.so
Reason: image not found
The problem is that when sphinx trys to import my module, it fails to load the linked c++ library from my boost-python related package. That package is installed via pip and it loads fine in all other situations.
I manually specified the full path to all the c++ libraries with install_name_tool -change and then sphinx works fine.
My question is this: how can I get sphinx to import my boost-python package without manually editing the c++ libraries manually with install_name_tool?
I have DYLD_LIBRARY_PATH set to include the directory that contains all the c++ libraries.
I'm on MacOS Sierra

ImportError: DLL load failed: The operating system cannot run %1

I want to build M2Crypto for Windows x64 to be used in python 2.7 in Windows x64.
I downloaded the library package named: "M2CryptoWin64-master" from: https://github.com/martinpaljak/M2Crypto
Then, I followed these steps from enter link description here:
In setup.py:
replace self.openssl = 'C:\pkg' by self.openssl =
'C:\grr-build\openssl' replace name = 'M2Crypto.__m2crypto' , by name
= '__m2crypto' Build:
set PATH=%PATH%;C:\grr-build\swigwin-2.0.9 C:\Python27\python.exe
setup.py build C:\Python27\python.exe setup.py install To get M2Crypto
to work for PyInstaller:
Rename C:\Python27\lib\site-packages\M2Crypto-0.21.1-py2.7-win32.egg
to C:\Python27\lib\site-packages\M2Crypto-0.21.1-py2.7-win32.egg.zip
extract M2Crypto-0.21.1-py2.7-win32.egg.zip into
C:\Python27\lib\site-packages\ without a
M2Crypto-0.21.1-py2.7-win32.egg sub directory remove
M2Crypto-0.21.1-py2.7-win32.egg.zip
When I test the library by going to Python.exe and type:
>>> import M2Crypto
I get this message:
Traceback (most recent call last): File "", line 1, in
File
"C:\Python27\lib\site-packages\m2crypto-0.22.3-py2.7-win-amd64.egg\M2Cryp
to__init__.py", line 22, in
import __m2crypto ImportError: DLL load failed: The operating system cannot run %1.
Can you please help me figure out how to solve the issue?
EDIT:
I also did the following to get M2Crypto to work for PyInstaller:
Rename C:\Python27\lib\site-packages\M2Crypto-0.21.1-py2.7-win32.egg to
C:\Python27\lib\site-packages\M2Crypto-0.21.1-py2.7-win32.egg.zip
extract M2Crypto-0.21.1-py2.7-win32.egg.zip into
C:\Python27\lib\site-packages\ without a
M2Crypto-0.21.1-py2.7-win32.egg sub directory remove
M2Crypto-0.21.1-py2.7-win32.egg.zip

AttributeError: Extension instance has no attribute '__version__'

Here is the script..
from distutils.core import setup, Extension
nmap = Extension('nmap',sources = ['nmap/nmap.py',
'nmap/__init__.py', 'nmap/example.py'])
from nmap import *
setup (
name = 'python-nmap',
version = nmap.__version__,
author = 'Alexandre Norman',
author_email = 'norman#xael.org',
license ='gpl-3.0.txt',
keywords="nmap, portscanner, network, sysadmin",)
... and i got this error:
Traceback (most recent call last):
File "C:\Python27\nmap.py", line 6, in <module>
from nmap import *
File "C:\Python27\nmap.py", line 17, in <module>
version = nmap.__version__,
AttributeError: Extension instance has no attribute '__version__'

There are a number of problems here.
Your nmap package isn't an extension, it's a pure-Python package; don't create an Extension object for it. Python extension are written in C or C++.
You're trying to access nmap.__version__, presumably because you defined that variable in nmap/__init__.py, but nmap here is that Extension object you created; it's trying to access the variable from the wrong thing.
Even if you remove the Extension object, you still wouldn't be able to access nmap.__version__, because you imported your package incorrectly; you meant to use import nmap.
You never actually pass your package to setup, so distutils won't know about it. There are a few examples of how to do that in the documentation.
The distutils documentation is pretty big, but it's a good idea to read through all of it at least once.