So I am trying to make a C/C++ dll for a project but any C++ library I include in my header, ends up causing loading problems when I try to load it using ctypes in python. I'm guessing maybe ctypes doesn't have c++ libs paths included? I made a simple demonstration to my problem.
init2.h: Generic Header file for dll and execs using the dll
#ifndef INIT2_H
#define INIT2_H
#ifdef __cplusplus
extern "C" {
#endif
#include <stdio.h>
void func11();
#ifdef __cplusplus
}
#endif
#endif
init2.cc: dll code (loads fine)
// This works fine
#include "init2.h"
void func11() {
printf("Func11\n");
}
init.c: exe code that loads dll
#include "init2.h"
int main() {
func11();
return 0;
}
Makefile
CFLAGS = -Wall -std=c11 -g
CCFLAGS = -Wall -std=c++11 -g
all: libmain.dll main
main.o: init2.cc
g++ $(CCFLAGS) init2.cc -fpic -c -o main.o
libmain.dll: main.o
g++ $(CCFLAGS) -shared main.o -o libmain.dll
main: init.c
gcc $(CFLAGS) -L. -lmain init.c -o main
clean:
del *.o *.exe *.dll *.txt
script.py
from ctypes import *
lib = CDLL('D:\Projects\Learning-C++\libmain.dll')
lib.func11()
The above compiles, links and loads fine in the generated main.exe and script.py. The problem occurs when I add a c++ lib in my dll.
init2.cc: With iostream (loads in main.exe only)
#include "init2.h"
#include <iostream>
void func11() {
std::cout << "Func11\n";
}
Compiling that into the dll, while everything stays the same loads fine into the main.exe (init.c code) but gives me the following loading error in script.py.
Traceback (most recent call last):
File "D:\Projects\Learning-C++\script.py", line 3, in <module>
lib = CDLL('D:\Projects\Learning-C++\libmain.dll')
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.3312.0_x64__qbz5n2kfra8p0\lib\ctypes\__init__.py", line 374, in __init__
self._handle = _dlopen(self._name, mode)
FileNotFoundError: Could not find module 'D:\Projects\Learning-C++\libmain.dll' (or one of its dependencies). Try using the full path with constructor syntax.
How can I fix this? My bad if these seems obvious, but I'm fairly new to C++ and haven't got the integration down yet. Thanks in advance!
The problem was as I expected due to cdll not able to find the C++ library. The file name for my case was libstdc++-6.dll. Which is located in the bin folder in my compiler's directory. To make cdll search for the dependencies in that folder I did the following:
import os
# add dependency directory
os.add_dll_directory('C:\\Program Files\\CodeBlocks\\MinGW\\bin')
# load the dll
lib = CDLL('D:\\Projects\\Learning-C++\\libmain.dll')
lib.func11()
The dll may have many dependencies located in other directories e.g. You defined libDLL.dll and bin/libDependency.dll. You will need to add the /path/to/dir/bin directory using the above method if you're trying to load libDLL.dll in python. You can find all the dependencies of a dll using dependency walker or
dumpbin /dependents libDLL.dll
in the VS dev powershell.
Related
I currently have a source files written in C++ and wrapped into a Python module (uses Boost and Parser libs).
I ported the C++ folder under go/src and .so files along with main.go
Program Structure
src
/main
main.go
network.so
/network
file1.cpp (this has a function **object DBdata::getTable())
file1.hpp (#define FILE1_H_)
main.go
package main
// #cgo pkg-config: python3
// #cgo CFLAGS : -I./ -I/usr/include/python3.6
// #cgo LDFLAGS: -L/usr/lib/python3.6/config-3.6m-x86_64-linux-gnu -L/usr/lib -lpython3.6 -lpthread -ldl -lutil -lm
// #include <Python.h>
import "C"
import "fmt"
func main() {
cmd := exec.Command("python", "-c", "import network; network.getTable()")
cmd.Dir = "/home/username/go/src/network" //directory where is my python code
out,err := cmd.CombinedOutput()
}
After building this main.go, I get the error as
/usr/include/boost/config/no_tr1/memory.hpp: fatal error: memory: No
such file or directory
> # include
> ^~~~~~~~
> compilation terminated.
How to import .so as Python modules in Go?
Can Swig be used in this place?
What is the better approach to expose Python module to Go?
Issue fixed. The .so placed in /bin and the Go was build and could access the functions under network
I am trying to learn how to embed a Python interpreter into my C/C++ programs.
I like this concept because it may enable me to extend C programs at run time as well as enable users to write custom scripts/plugins.
Detailed instruction on embedding Python in C is provided at: https://docs.python.org/3/extending/embedding.html
I'm using example code provided in the Python documentation to figure out the mechanics involved:
embedded.c
#include <Python.h>
int
main(int argc, char *argv[])
{
Py_SetProgramName(argv[0]); /* optional but recommended */
Py_Initialize();
PyRun_SimpleString("from time import time,ctime\n"
"print 'Today is',ctime(time())\n");
Py_Finalize();
return 0;
}
The problem is I can't figure out how to compile it on Windows using gcc.
C:\Users\user\embedded_python> gcc embedded.c
embedded.c:2:10: fatal error: Python.h: No such file or directory
#include <Python.h>
^~~~~~~~~~
compilation terminated.
I think the problem is I need to link to the Python.h file, but I can't seem to get the compiler flags right:
C:\Users\user\Desktop\embedded_python>gcc -IC:\Users\user\AppData\Local\Programs\Python\Python38 -lPython38 embedded.c
embedded.c:2:10: fatal error: Python.h: No such file or directory
#include <Python.h>
^~~~~~~~~~
compilation terminated.
How can I get this program to compile on Windows (preferably with gcc/g++ to avoid Visual Studio bloat)?
In Linux, but I suppose if you activate your Bash from windows and can perform a work around. First you have to install python3-dev
Then use this command line where your file is located, to compile (test is your filename)
gcc -Wall test.c -o test $(pkg-config --cflags --libs python3)
I am trying to compile some C++ Code with OpenCv and Pybind with this header:
https://github.com/patrikhuber/eos/blob/v0.12.2/python/pybind11_opencv.hpp
This has worked for me before, so I don't think the header file is the Problem.
I can compile the code without problems, but when i try to import the created file to Python I get the following error:
ImportError: /usr/lib/libgtk-3.so.0: undefined symbol: g_mount_operation_set_is_tcrypt_hidden_volume
Here is the C++ Code:
#include <opencv2/imgproc.hpp>
#include <opencv2/highgui.hpp>
#include <pybind11/pybind11.h>
#include "pybind11_opencv.hpp"
using namespace std;
namespace py = pybind11;
cv::Mat func(cv::Mat Image1,cv::Mat Image2)
{
return Image1;
}
PYBIND11_MODULE(pybind_module, m)
{
m.doc() = "Text";
m.def("func", &func, "Function",
py::arg("Image1"),
py::arg("Image2"));
}
I am guessing it's a problem with my setup (arch linux) since I got something similar working before and not even this minimal example is working.
I was able to solve the problem myself using the following compiler settings.
c++ -msse4 -O3 -Wall -shared -std=c++11 -fPIC -lopencv_imgcodecs `python3 -m pybind11 --includes` main.cpp -o executable`python3-config --extension-suffix` /usr/local/include/ -L/usr/local/lib/ -march=native
I guess there was a mistake in the previous compilation.
I just recently discovered Boost.Python and I am trying to figure out how it works. I tried to go through the tutorial on the official website. However, I got
link.jam: No such file or directory
when running bjam as in the example (which appears to be just a warning),
and
Traceback (most recent call last):
File "hello.py", line 7, in <module>
import hello_ext
ImportError: libboost_python.so.1.55.0: cannot open shared object file: No such file or directory
when running python hello.py.
I also tried to compile a module as described in another tutorial with similar results. I am running Ubuntu14.04 with boost1.55 compiled myself.
I tried to compile the following:
#include <boost/python.hpp>
char const* greet()
{
return "hello, world";
}
BOOST_PYTHON_MODULE(hello_ext)
{
using namespace boost::python;
def("greet", greet);
}
with the following command from command line:
g++ -o hello_ext.so hello.cpp -I /usr/include/python2.7/ -I /home/berardo/boost_1_55_0/ -L /usr/lib/python2.7/ -L /home/berardo/boost/lib/ -lboost_python -lpython2.7 -Wl, -fPIC -expose-dynamic
which still gives me a:
/usr/bin/ld: impossibile trovare : File o directory non esistente
collect2: error: ld returned 1 exit status.
Finally, I was able to make it work. First, I fixed the linker issues, as suggested by Dan. It finally compiled but I still got:
ImportError: libboost_python.so.1.55.0: cannot open shared object file: No such file or directory
The problem was that the python module was not able to load correctly so I needed to add another linker option. Here, I report the final Makefile:
# location of the Python header file
PYTHON_VERSION = 2.7
PYTHON_INCLUDE = /usr/include/python$(PYTHON_VERSION)
# location of the Boost Python include files and library
BOOST_INC = ${HOME}/boost/include
BOOST_LIB = ${HOME}/boost/lib
# compile mesh classes
TARGET = hello_ext
$(TARGET).so: $(TARGET).o
g++ -shared -Wl,-rpath,$(BOOST_LIB) -Wl,--export-dynamic $(TARGET).o -L$(BOOST_LIB) -lboost_python -L/usr/lib/python$(PYTHON_VERSION)/config -lpython$(PYTHON_VERSION) -o $(TARGET).so
$(TARGET).o: $(TARGET).C
g++ -I$(PYTHON_INCLUDE) -I$(BOOST_INC) -fPIC -c $(TARGET).C
Notice the -Wl,-rpath, option, which apparently makes the new created shared library available to the python script.
#Dan: Thanks for the valuable hints.
My C code:
#include<stdio.h>
#include "Python.h"
int main()
{
printf("Hello World");
return 0;
}
I have python-dev installed for python2.7. Moreover, Python.h is available in /usr/include/python2.7.
gcc myfile.c # Python.h: No such file or directory
I even tried :
gcc -L/usr/include/python2.7/ myfile.c # Python.h: No such file or directory
I tried building a python c module ujson with pip that uses Python.h, it was able to compile.
What am I missing / doing wrong ?
It should be -I, not -L:
gcc -I/usr/include/python2.7 myfile.c
Use
#include <Python.h>
instead of
#include "Python.h"
to include the header file.
The Python.h file should be the first file which is included.
#see Extending Python with C or C++ (Section 1.1 Note)
Since Python may define some pre-processor definitions which affect
the standard headers on some systems, you must include Python.h before
any standard headers are included.