Develop Reference

How to call function/classes in c++ .so files, generated by Bazel, in Python?

Let's say I have a simple class in hello.h
#ifndef LIB_HELLO_GREET_H_
#define LIB_HELLO_GREET_H_
class A{
public:
int a = 0;
int b = 0;
int add(){
return a+b;
}
};
#endif
with bazel build file:
load("#rules_cc//cc:defs.bzl", "cc_binary", "cc_library")
cc_library(
name = "hello",
hdrs = ["hello.h"],
)
cc_binary(
name = "hello.so",
deps = [
":hello",
],
linkshared=True,
linkstatic=False
)
After I run bazel build hello.so, there is a shared object file generated in bazel-bin/main and bazel-bin/main/hello.so.runfiles/__main__/main/hello.so. Using those files, I want call class A with a python script. Ideally, I'd want to use cppyy or something similar.
I've tried with simple python scripts
import cppyy
cppyy.load_reflection_info('hello')
print(dir(cppyy.gbl))
or
import cppyy
cppyy.load_library('hello')
print(dir(cppyy.gbl))
with both .so files, but cppyy can't seem to detect class A - it is never inside cppyy.gbl
I'm wondering the best way to solve this problem, any help appreciated!

With cppyy, you also need to give it the header file, that is:
cppyy.include("hello.h")
and optionally use cppyy.add_include_path() to add the directory where hello.h resides, so that it can be found.
An alternative is to use so-called "dictionaries." These package (customizable) directory locations, names of necessary headers files and (through linking) the shared library with the implementation into a single, new, shared library (named "dictionary" because of history). There can be a so-called "rootmap" file on the side that allows a class loader to find A automatically on first use. See the example in the documentation.

How to enable the pyd to call functions in external C++ dll?

I'm trying to design an external DLL for my python program. Now I could use the C++ and Visual Studio 2010 to produce a file with a postfix of ".pyd". If the .pyd is not attached with other .dll files produced by C++, this python library could work well.
However, now I need to design a .pyd file with such a struction:
A.pyd -> 1.dll
-> 2.dll
in which the files 1,2 are C libraries. The functions in these libraries are called when producing A.pyd.
Although this .pyd file could be produced without errors by VS 2010, it could not work in python 3.6. The error report is as follow:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: DLL load failed, could not find the target program.
Even when the referred 1.dll and 2.dll are stored in the same folder that contains A.pyd, this error still exists. I wonder how could I enable the python-C++ library to call the functions in these dynamic link libraries based on C.

OK! Now I have found the correct way to perform this operation!
Wrap the DLL
At first, if the dll (I call it dll-A) that you need to call has not exported any function (you could use dumpbin in VS command to check the exported functions), you need to wrap the original dll-A. Use the implicit call to include dll-A. If there is a function declared in the original .h/.lib, like this:
int func(int a, int b);
Then you need to create a new dll project and wrap the above function by this:
.h
extern "C" _declspec(dllexport) int w_func(int a, int b);
.cpp
int w_func(int a, int b){
return func(a, b);
}
Certainly, if the dumpbin shows that dll-A have avaliable exported functions, you could skip this step.
After exporting this dll (I call it dll-B), you will get 'dll-B' and its depending files (including dll-A and dll-A's depending files).
Write the .pyd file
Use the explicit call to refer dll-B. When using this method, you should not include any lib/.h files, because dll-B itself could provide you with enough interfaces. You could load the dll by this method:
.h
typedef int (*func_ptr)(int a, int b);
.cpp (part of a function named Testdll when you write the .pyd project)
func_ptr FUNC_API = NULL;
HINSTANCE h = LoadLibraryA("libdll/dllB.dll");
//Certainly! you could set the folder of stored dlls!
if (h){
FUNC_API = (func_ptr)GetProcAddress(h, "w_func");
//You could load more functions here.
}
else{ // If the dll could not be found
FreeLibrary(h);
return 0x100;
}
int errorflag=0;
if (FUNC_API==NULL){ //Check whether the function is valid.
cout << "Could not find: func" << endl;
errorflag = errorflag | 0x001;
}
//You could check more functions here.
if (errorflag!=0){ // if any function could not be found.
FreeLibrary(h);
return 0x100 | errorflag;
}
//process functions.
int a,b,c;
c = FUNC_API(a,b);
//Free the lib
if (h)
FreeLibrary(h);
After build your own .pyd, you could get your python database (I call it pyd-C).
Link the .pyd with Python
In python project, you could test this file by this method:
.py
import pydC
if __name__ == '__main__':
X = pydC.cclass()
X.Testdll();
Then you can find that the function is performed well.
Noted that your .pyd should be in the same folder with where .py is. Because you have set dll-B in libdll/dllB.dll, dll-B should be put in the folder named libdll. However, because dll-B calls dll-A and other depending dlls implicitly, dll-A and other files should be in your workspace folder, i.e. the same folder with where .py is.
In short, you need to enter the workspace folder, and the folder formation is as follow:
./
Test.py
pydC.pyd
dllA.dll
dllA_depended1.dll
...
libdll/
dllB.dll

Exchanging info between running C++ code and python script

I have a python script that is called inside a C++ program. The python script creates a directory based on the current time, puts files in there, and then execution returns to C++. I want to save a parameters file into the directory created in the python program.
I figure my options are:
Pass in the text to save in the parameters file to the Python program and have it create and save the file
Return the location of the directory to C++ from python so it knows where to save the file
Use C++ to locate the most recently created directory after execution of the python script and put file there
I'm not sure how to do any of this. My python script is not embedded. I use
std::string test = "python analyzeData2.py";
system(test.c_str());
to call a python script.
Any ideas how to do this?

I'd go with option B -- return the location of the directory to c++ from python so it knows where to save the file.
If you plan on using system(), something like this:
char* dirname[64];
FILE* fin;
system("python analyzeData2.py > created.log");
fin = fopen("created.log", "r");
fgets(dirname, sizeof(dirname), fin);
fclose(fin);
/* dirname has contents of created.log */
...

Python C API - Call C function from embedded python (callback)

I have a .cpp file loading python file and calling a function. I have a logger class in the .cpp and I want to use it from .py file.
Example:
# python-file.py
def FunctionCalledFromC_API():
log("some string")
log("some error", error)
log("some debug info", debug)
# etc...
And the .cpp
// cpp-file.cpp
// Load python file, do stuff...
PyObject *args = PyTuple_New(0);
PyObject_CallObject(pFunctionCalledFromC_API, args);
Py_DECREF(args);
I want that the log("some string") function of the .py file calls my logger.log(...) function from the C++ application.

As Markku K. suggested, I made all my app a DLL and I'll try soon.
That's what I'll do (for people with the same question):
Make my app a single DLL
Make an executable to run it (obvious)
Make a Python file containing the functions (example: logger.log()). That functions should call the DLL's equivalent functions.
Do "trial&error" until it works ;)
EDIT:
After a lot of "trial&error", I've found a way to do it!
Using boost::python to expose my API and the standard Python API to load the .py file I made this work!
But I'll have to start the application with a python file importing the shared object and calling the main function :(
Anyway, Thanks Markku K. for your help!

How do I set up the python/c library correctly?

I have been trying to get the python/c library to like my mingW compiler. The python online doncumentation; http://docs.python.org/c-api/intro.html#include-files only mentions that I need to import the python.h file. I grabbed it from the installation directory (as is required on the windows platform), and tested it by compiling the script:
#include "Python.h". This compiled fine. Next, I tried out the snippet of code shown a bit lower on the python/c API page:
PyObject *t;
t = PyTuple_New(3);
PyTuple_SetItem(t, 0, PyInt_FromLong(1L));
PyTuple_SetItem(t, 1, PyInt_FromLong(2L));
PyTuple_SetItem(t, 2, PyString_FromString("three"));
For some reason, the compiler would compile the code if I'd remove the last 4 lines (so that only the pyObject variable definition would be left), yet calling the actual constructor of the tuple returned errors.
I am probably missing something completely obvious here, given I am very new to C, but does anyone know what it is?

I've done some crafty Googling, and if you are getting errors at the linker stage (the error messages might have hex strings or references to ld), you may need to make sure the Python library that ships with the Windows version has been converted to a format that GCC (MinGW) can read; see here, among other sites. Also ensure that GCC can find and is using the library file if needs be, using -L/my/directory and -lpython26 (substituting appropriately for your path and Python version).
If the errors are at the compilation stage (if line numbers are given in the messages), make sure that you don't need to add any other directories to the include search path. Python might (I've not used its C API) include other header files in Python.h which are stored in some other directory. If this is the case, use the -I/my/directory/ flag to GCC to tell it to search there as well.
Exact (copied-and-pasted) error messages would help, though.
Warning: The text below does not answer the question!
Did you put the code inside a function? Try putting it in main, like so:
int main(int argc, char *argv[]) {
PyObject *t;
t = PyTuple_New(3);
PyTuple_SetItem(t, 0, PyInt_FromLong(1L));
PyTuple_SetItem(t, 1, PyInt_FromLong(2L));
PyTuple_SetItem(t, 2, PyString_FromString("three"));
return 0;
}
This code will be run on execution of the program. You can then use whatever other methods are provided to examine the contents of the tuple. If it isn't to be run separately as an executable program, then stick it in a differently-named method; I assume you have another way to invoke the function.
The PyObject *t; definition is valid outside the function as a global variable definition, as well as inside a function, declaring it as a local variable. The other four lines are function calls, which must be inside another function.
The above code on its own does not a program make. Are you trying to write a C extension to Python? If so, look at some more complete documentation here.

I have made some progress since I asked my question, and I thought I would just share it in case someone else is having similar problems.
These were the errors I got:
In function `main':
undefined reference to `_imp__PyTuple_New'
undefined reference to `_imp__PyInt_FromLong'
undefined reference to `_imp__PyTuple_SetItem'
undefined reference to `_imp__PyInt_FromLong'
undefined reference to `_imp__PyTuple_SetItem'
undefined reference to `_imp__PyString_FromString'
undefined reference to `_imp__PyTuple_SetItem'
The errors I got were the result of missing libraries from the mingW compiler. So only including the header file in the source ode is not enough, there is also a special file required (.lib, .o, .a, ..) that needs to be included for compilation. It is possible to use the -l[path] flag on the mingW command line, but I found that codeBlocks ( http://www.codeblocks.org/ ) is the most convenient to use here. After creating a project, and going to Project>Build options.., you can specify the location of the library file under the linker settings tab. When you are done, build the project, and it will hopefully work.
I hope anyone struggling with similar problems have help from this :)