I'm trying to design an external DLL for my python program. Now I could use the C++ and Visual Studio 2010 to produce a file with a postfix of ".pyd". If the .pyd is not attached with other .dll files produced by C++, this python library could work well.
However, now I need to design a .pyd file with such a struction:
A.pyd -> 1.dll
-> 2.dll
in which the files 1,2 are C libraries. The functions in these libraries are called when producing A.pyd.
Although this .pyd file could be produced without errors by VS 2010, it could not work in python 3.6. The error report is as follow:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: DLL load failed, could not find the target program.
Even when the referred 1.dll and 2.dll are stored in the same folder that contains A.pyd, this error still exists. I wonder how could I enable the python-C++ library to call the functions in these dynamic link libraries based on C.
OK! Now I have found the correct way to perform this operation!
Wrap the DLL
At first, if the dll (I call it dll-A) that you need to call has not exported any function (you could use dumpbin in VS command to check the exported functions), you need to wrap the original dll-A. Use the implicit call to include dll-A. If there is a function declared in the original .h/.lib, like this:
int func(int a, int b);
Then you need to create a new dll project and wrap the above function by this:
.h
extern "C" _declspec(dllexport) int w_func(int a, int b);
.cpp
int w_func(int a, int b){
return func(a, b);
}
Certainly, if the dumpbin shows that dll-A have avaliable exported functions, you could skip this step.
After exporting this dll (I call it dll-B), you will get 'dll-B' and its depending files (including dll-A and dll-A's depending files).
Write the .pyd file
Use the explicit call to refer dll-B. When using this method, you should not include any lib/.h files, because dll-B itself could provide you with enough interfaces. You could load the dll by this method:
.h
typedef int (*func_ptr)(int a, int b);
.cpp (part of a function named Testdll when you write the .pyd project)
func_ptr FUNC_API = NULL;
HINSTANCE h = LoadLibraryA("libdll/dllB.dll");
//Certainly! you could set the folder of stored dlls!
if (h){
FUNC_API = (func_ptr)GetProcAddress(h, "w_func");
//You could load more functions here.
}
else{ // If the dll could not be found
FreeLibrary(h);
return 0x100;
}
int errorflag=0;
if (FUNC_API==NULL){ //Check whether the function is valid.
cout << "Could not find: func" << endl;
errorflag = errorflag | 0x001;
}
//You could check more functions here.
if (errorflag!=0){ // if any function could not be found.
FreeLibrary(h);
return 0x100 | errorflag;
}
//process functions.
int a,b,c;
c = FUNC_API(a,b);
//Free the lib
if (h)
FreeLibrary(h);
After build your own .pyd, you could get your python database (I call it pyd-C).
Link the .pyd with Python
In python project, you could test this file by this method:
.py
import pydC
if __name__ == '__main__':
X = pydC.cclass()
X.Testdll();
Then you can find that the function is performed well.
Noted that your .pyd should be in the same folder with where .py is. Because you have set dll-B in libdll/dllB.dll, dll-B should be put in the folder named libdll. However, because dll-B calls dll-A and other depending dlls implicitly, dll-A and other files should be in your workspace folder, i.e. the same folder with where .py is.
In short, you need to enter the workspace folder, and the folder formation is as follow:
./
Test.py
pydC.pyd
dllA.dll
dllA_depended1.dll
...
libdll/
dllB.dll
Related
Let's say I have a simple class in hello.h
#ifndef LIB_HELLO_GREET_H_
#define LIB_HELLO_GREET_H_
class A{
public:
int a = 0;
int b = 0;
int add(){
return a+b;
}
};
#endif
with bazel build file:
load("#rules_cc//cc:defs.bzl", "cc_binary", "cc_library")
cc_library(
name = "hello",
hdrs = ["hello.h"],
)
cc_binary(
name = "hello.so",
deps = [
":hello",
],
linkshared=True,
linkstatic=False
)
After I run bazel build hello.so, there is a shared object file generated in bazel-bin/main and bazel-bin/main/hello.so.runfiles/__main__/main/hello.so. Using those files, I want call class A with a python script. Ideally, I'd want to use cppyy or something similar.
I've tried with simple python scripts
import cppyy
cppyy.load_reflection_info('hello')
print(dir(cppyy.gbl))
or
import cppyy
cppyy.load_library('hello')
print(dir(cppyy.gbl))
with both .so files, but cppyy can't seem to detect class A - it is never inside cppyy.gbl
I'm wondering the best way to solve this problem, any help appreciated!
With cppyy, you also need to give it the header file, that is:
cppyy.include("hello.h")
and optionally use cppyy.add_include_path() to add the directory where hello.h resides, so that it can be found.
An alternative is to use so-called "dictionaries." These package (customizable) directory locations, names of necessary headers files and (through linking) the shared library with the implementation into a single, new, shared library (named "dictionary" because of history). There can be a so-called "rootmap" file on the side that allows a class loader to find A automatically on first use. See the example in the documentation.
This is the structure of my program:
I'm trying to bind my program in C++ with a GUI in python. I'm using pybind11 and I have a python_binding.cpp file for the bind and some ".h" and ".cpp" with the methods in other directories. I include the ".h" files but somehow the python_binding.cpp it's not able to recognize them.
The file config.cpp only has one void method, "cargar_configuracion()" and this is how it looks like in the binding:
#include "Ejemplo/config.h"
PYBIND11_MODULE(Example, m) {
m.doc() = "Binding"; // optional module docstring
m.def("cargar_configuracion", &cargar_configuracion);
The result of this is the following error:
undefined reference to `cargar_configuracion()'
What am I doing wrong? Should I have my .cpp and .h with the binding.cpp in the same directory?
Thanks in advance!
Your pybind11 looks fine, this is a linker error. It looks like config.cpp is in another project within your solution, and is being built within a separate executable. You have two options here, either copy config.cpp into the same directory or reconfigure Ejemplo to be a static library and specify it as a dependency in the properties of the python wrapper project.
Change your code from:
bViewResult = QtWidgets.QPushButton('View Results', self)
bViewResult.clicked.connect(self.openCSV)
to:
bViewResult = QtWidgets.QPushButton('View Results', self)
bViewResult.clicked.connect(cargar_configuracion())
I have a shared library (DLL) in C++ with some C-style API which I use from Python. There is a function which takes a C callback as an argument. As far as I understood, Python extension module (a separate DLL) is required to do that. This module shall pass a native "proxy" callback to API, then call Python object from that callback. I try do use it as follows (it fails):
from ctypes import CDLL
from test_extension import set_python_callback
def callback():
pass
if __name__ == '__main__':
library = CDLL('test_dll.shared-library')
set_python_callback(library, callback)
library.trigger_callback()
I'm using Windows 8 x64 environment with Python 2.7.6 (32 bit) and MinGW 32-bit compiler (mingw32). Python extension is built by distutils. API library can be changed if necessary, however, Python-specific code has to be kept in a separate library.
Here is my code, reduced. All error-checking was removed, however, when performed it showed no errors either from Python API or from Windows API.
API library:
/* typedef void (*callback_t)(); */
static callback_t s_callback = nullptr;
void DLL_PUBLIC set_callback(callback_t callback) {
s_callback = callback;
}
void DLL_PUBLIC trigger_callback() {
s_callback(); // <--------------------------(1)
}
Python extension module function set_python_callback(library, callback) takes a ctypes.CDLL object and Python callable. It then extracts native DLL handle from the former:
PyObject* handle_object = PyObject_GetAttrString(library, "_handle");
const HMODULE handle = static_cast<const HMODULE>(
PyLong_AsVoidPtr(handle_object));
const native_set_callback_t native_api_routine =
reinterpret_cast<const native_set_callback_t>(
GetProcAddress(handle, "set_callback"));
native_api_routine(common_callback);
C function common_callback() is implemented as follows:
extern "C" void DLL_PUBLIC common_callback() {
// <--------------------------------------- (2)
PyObject *arguments = Py_BuildValue("()");
PyObject *callback_result = PyEval_CallObject(s_callback, arguments);
Py_DECREF(arguments);
Py_XDECREF(callback_result);
}
The error message says:
Invoking callback from DLL... Traceback (most recent call last):
File "script.py", line 14, in <module>
library.trigger_callback()
WindowsError: exception: access violation reading 0x00000028
Using debug print, I traced the error down to (1). Any code at (2) point doesn't execute. The strange thing is, changing the way common_callback() call Python code (e. g. passing Py_None instead of empty tuple) changes the address in the error message.
Maybe this is somehow related to the calling convention, but I have no idea where it's wrong exactly or how to fix it.
Solved the issue by registering the thread for GIL as described in documentation.
Please note that original API DLL, Python extension DLL, and the script were executed in single-threaded mode and no additional thread had been created, either Python-managed, or not. However, there had been a synchronization issue for sure, because using Py_AddPendingCall() also worked. (Using is is discouraged; I found it analyzing signals module sources, it lead to the solution above.)
This statement of mine was incorrect:
Using debug print, I traced the error down to (1). Any code at (2) point doesn't execute.
I was mislead by the fact that some of Python API functions still worked (e. g. Py_BuildValue or Py_Initialize), but most didn't (e. g. PySys_WriteStdout, PyObject_Call, etc).
I have a .cpp file loading python file and calling a function. I have a logger class in the .cpp and I want to use it from .py file.
Example:
# python-file.py
def FunctionCalledFromC_API():
log("some string")
log("some error", error)
log("some debug info", debug)
# etc...
And the .cpp
// cpp-file.cpp
// Load python file, do stuff...
PyObject *args = PyTuple_New(0);
PyObject_CallObject(pFunctionCalledFromC_API, args);
Py_DECREF(args);
I want that the log("some string") function of the .py file calls my logger.log(...) function from the C++ application.
As Markku K. suggested, I made all my app a DLL and I'll try soon.
That's what I'll do (for people with the same question):
Make my app a single DLL
Make an executable to run it (obvious)
Make a Python file containing the functions (example: logger.log()). That functions should call the DLL's equivalent functions.
Do "trial&error" until it works ;)
EDIT:
After a lot of "trial&error", I've found a way to do it!
Using boost::python to expose my API and the standard Python API to load the .py file I made this work!
But I'll have to start the application with a python file importing the shared object and calling the main function :(
Anyway, Thanks Markku K. for your help!
What is the easiest way to use a DLL file from within Python?
Specifically, how can this be done without writing any additional wrapper C++ code to expose the functionality to Python?
Native Python functionality is strongly preferred over using a third-party library.
For ease of use, ctypes is the way to go.
The following example of ctypes is from actual code I've written (in Python 2.5). This has been, by far, the easiest way I've found for doing what you ask.
import ctypes
# Load DLL into memory.
hllDll = ctypes.WinDLL ("c:\\PComm\\ehlapi32.dll")
# Set up prototype and parameters for the desired function call.
# HLLAPI
hllApiProto = ctypes.WINFUNCTYPE (
ctypes.c_int, # Return type.
ctypes.c_void_p, # Parameters 1 ...
ctypes.c_void_p,
ctypes.c_void_p,
ctypes.c_void_p) # ... thru 4.
hllApiParams = (1, "p1", 0), (1, "p2", 0), (1, "p3",0), (1, "p4",0),
# Actually map the call ("HLLAPI(...)") to a Python name.
hllApi = hllApiProto (("HLLAPI", hllDll), hllApiParams)
# This is how you can actually call the DLL function.
# Set up the variables and call the Python name with them.
p1 = ctypes.c_int (1)
p2 = ctypes.c_char_p (sessionVar)
p3 = ctypes.c_int (1)
p4 = ctypes.c_int (0)
hllApi (ctypes.byref (p1), p2, ctypes.byref (p3), ctypes.byref (p4))
The ctypes stuff has all the C-type data types (int, char, short, void*, and so on) and can pass by value or reference. It can also return specific data types although my example doesn't do that (the HLL API returns values by modifying a variable passed by reference).
In terms of the specific example shown above, IBM's EHLLAPI is a fairly consistent interface.
All calls pass four void pointers (EHLLAPI sends the return code back through the fourth parameter, a pointer to an int so, while I specify int as the return type, I can safely ignore it) as per IBM's documentation here. In other words, the C variant of the function would be:
int hllApi (void *p1, void *p2, void *p3, void *p4)
This makes for a single, simple ctypes function able to do anything the EHLLAPI library provides, but it's likely that other libraries will need a separate ctypes function set up per library function.
The return value from WINFUNCTYPE is a function prototype but you still have to set up more parameter information (over and above the types). Each tuple in hllApiParams has a parameter "direction" (1 = input, 2 = output and so on), a parameter name and a default value - see the ctypes doco for details
Once you have the prototype and parameter information, you can create a Python "callable" hllApi with which to call the function. You simply create the needed variable (p1 through p4 in my case) and call the function with them.
This page has a very simple example of calling functions from a DLL file.
Paraphrasing the details here for completeness:
It's very easy to call a DLL function in Python. I have a self-made DLL file with two functions: add and sub which take two arguments.
add(a, b) returns addition of two numbers
sub(a, b) returns substraction of two numbers
The name of the DLL file will be "demo.dll"
Program:
from ctypes import*
# give location of dll
mydll = cdll.LoadLibrary("C:\\demo.dll")
result1= mydll.add(10,1)
result2= mydll.sub(10,1)
print("Addition value:"+result1)
print("Substraction:"+result2)
Output:
Addition value:11
Substraction:9
Building a DLL and linking it under Python using ctypes
I present a fully worked example on how building a shared library and using it under Python by means of ctypes. I consider the Windows case and deal with DLLs. Two steps are needed:
Build the DLL using Visual Studio's compiler either from the command line or from the IDE;
Link the DLL under Python using ctypes.
The shared library
The shared library I consider is the following and is contained in the testDLL.cpp file. The only function testDLL just receives an int and prints it.
#include <stdio.h>
extern "C" {
__declspec(dllexport)
void testDLL(const int i) {
printf("%d\n", i);
}
} // extern "C"
Building the DLL from the command line
To build a DLL with Visual Studio from the command line run
"C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\Tools\vsdevcmd"
to set the include path and then run
cl.exe /D_USRDLL /D_WINDLL testDLL.cpp /MT /link /DLL /OUT:testDLL.dll
to build the DLL.
Building the DLL from the IDE
Alternatively, the DLL can be build using Visual Studio as follows:
File -> New -> Project;
Installed -> Templates -> Visual C++ -> Windows -> Win32 -> Win32Project;
Next;
Application type -> DLL;
Additional options -> Empty project (select);
Additional options -> Precompiled header (unselect);
Project -> Properties -> Configuration Manager -> Active solution platform: x64;
Project -> Properties -> Configuration Manager -> Active solution configuration: Release.
Linking the DLL under Python
Under Python, do the following
import os
import sys
from ctypes import *
lib = cdll.LoadLibrary('testDLL.dll')
lib.testDLL(3)
ctypes can be used to access dlls, here's a tutorial:
http://docs.python.org/library/ctypes.html#module-ctypes
Maybe with Dispatch:
from win32com.client import Dispatch
zk = Dispatch("zkemkeeper.ZKEM")
Where zkemkeeper is a registered DLL file on the system...
After that, you can access functions just by calling them:
zk.Connect_Net(IP_address, port)
ctypes will be the easiest thing to use but (mis)using it makes Python subject to crashing. If you are trying to do something quickly, and you are careful, it's great.
I would encourage you to check out Boost Python. Yes, it requires that you write some C++ code and have a C++ compiler, but you don't actually need to learn C++ to use it, and you can get a free (as in beer) C++ compiler from Microsoft.
If the DLL is of type COM library, then you can use pythonnet.
pip install pythonnet
Then in your python code, try the following
import clr
clr.AddReference('path_to_your_dll')
# import the namespace and class
from Namespace import Class
# create an object of the class
obj = Class()
# access functions return type using object
value = obj.Function(<arguments>)
then instantiate an object as per the class in the DLL, and access the methods within it.