I'm writing a custom operation for Tensorflow that is supposed to load a video. For this, I need to include OpenCV.
For now, the operation simply tries to open a VideoCapture and returns an empty tensor.
Here's the C++ code:
#include "opencv2/opencv.hpp"
#include "opencv2/core/core.hpp"
#include "opencv2/highgui/highgui.hpp"
#include "tensorflow/core/framework/op.h"
#include "tensorflow/core/framework/shape_inference.h"
#include "tensorflow/core/framework/op_kernel.h"
#include <iostream>
using namespace tensorflow;
using namespace cv;
using namespace std;
using shape_inference::ShapeHandle;
using shape_inference::DimensionHandle;
REGISTER_OP("LoadVideo")
.Input("filename: string")
.Output("frame: float32")
.SetShapeFn([](::tensorflow::shape_inference::InferenceContext* c) {
TensorShape outputTensorShape({224, 224, 3});
ShapeHandle outputShapeHandle;
c->MakeShapeFromTensorShape(outputTensorShape, &outputShapeHandle);
c->set_output(0, outputShapeHandle);
return Status::OK();
});
class LoadVideoOp : public OpKernel {
public:
explicit LoadVideoOp(OpKernelConstruction* context) : OpKernel(context) {}
void Compute(OpKernelContext* context) override {
// Grab the input tensor
const Tensor& input_tensor = context->input(0);
auto input = input_tensor.flat<string>();
string filename = input(0);
VideoCapture cap = VideoCapture("data/0eRkpTGq5pA.mp4");
Tensor* output_tensor = NULL;
OP_REQUIRES_OK(context, context->allocate_output(0, {224, 224, 3}, &output_tensor));
}
};
REGISTER_KERNEL_BUILDER(Name("LoadVideo").Device(DEVICE_CPU), LoadVideoOp);
Then, I use the following command to compile the code:
g++ -std=c++11 -shared -fPIC \
-I /home/master/anaconda3/envs/tf/lib/python3.6/site-packages/tensorflow/include \
-I ~/anaconda3/envs/tf/include/opencv2/ -I ~/anaconda3/envs/tf/include/opencv/ -O2 \
-L ~/anaconda3/envs/tf/lib \
load_video.cc -o load_video.so \
-lopencv_core -lopencv_videoio -lopencv_highgui \
-lopencv_imgproc -lopencv_video -lopencv_objdetect
When I load the compiled code into a Python script (using tf.load_op_library) and try to run the op I get the following error:
tensorflow.python.framework.errors_impl.NotFoundError: lib/ops/load_video.so: undefined symbol: _ZN2cv12VideoCaptureC1ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE
It looks like the compiled C++ code cannot access the appropriate OpenCV object. I don't know much about C++ compilation and linking, so the problem probably is that I'm compiling the custom op in a wrong way.
Could you please help me compile the op in such way, that it can be successfully loaded and run by tensorflow?
EDIT 1:
This is the Python script I use to load the custom op:
import tensorflow as tf
load_video_module = tf.load_op_library('lib/ops/load_video.so')
with tf.Session():
x = load_video_module.load_video("data/0eRkpTGq5pA.mp4").eval()
print(x)
The error happens on the line 2 (i.e. when trying to load the compiled C++ code).
Solution:
I managed to successfully compile and run the custom tensorflow op after rebuilding OpenCV. The compilation command is:
g++ -std=c++11 -ggdb -shared -I`python -c 'import tensorflow as tf; print(tf.sysconfig.get_include())'` `pkg-config --cflags opencv` -o load_video.so load_video.cc `pkg-config --libs opencv` -fPIC
You can check if your library requires some missing library using ldd.
Just check ldd load_video.so.
However, probably you're not linking some shared library that some OpenCV method you're using requires.
To be sure to link and include every library needed you can use pkg-config.
Remove your manually -I and -l flags that point to OpenCV libs and just add pkg-config --libs --cflags opencv that do the complete work (of including and linking libraries) for you
If you are running on Linux, cd into the directory that contains your video file and then perform, without the parentheses:
sudo chmod 777 (the name of your video file)
This should give your program access to the video file.
I don't know much about C++, but TensorFlow often throws this error when it's denied permission, so give it a shot and good luck!
Related
I'm have a C++ program that is pretty far developed to do it's own job and now we'd like to add an addition functionality to it and we thought that making said functionality in Python and then calling that python with required inputs from the C++ when needed would be the best way to go as it keeps them separated and allows us to use this python script from elsewhere too.
As a first step I decided to try to make a test program to see how this would work and seems like it was a good idea because I can't get it to work.
How do I run separate python from c++?
I have tried following this guide and while it seems good it doesn't give any information on what compiler options should I run this with?
I have two files, cpp.cpp and python.py
This is my cpp.cpp file:
#include <stdio.h>
#include <stdlib.h>
#include <iostream>
#include <ncurses.h>
#include <Python.h>
using namespace std;
int main() {
std::cout << "C++ program started!\n";
char filename[] = "python.py";
FILE* fp;
Py_Initialize();
fp = _Py_fopen(filename, "r");
PyRun_SimpleFile(fp, filename);
Py_Finalize();
std::cout << "C++ program is ending!\n";
return 0;
}
and my python file is just two printf line:
#print('External Python program running...')
#print('Hello World from Python program')
I then try to compile this, give it all the includes it seems to want and then execute the output file:
g++ -I . -I /home/ahomm/python3.6/Include -I /home/ahomm/python3.6/release cpp.cpp && ./a.out
This is the output I get:
/tmp/cccQsh1p.o: In function `main':
cpp.cpp:(.text+0x3f): undefined reference to `Py_Initialize'
cpp.cpp:(.text+0x52): undefined reference to `_Py_fopen'
cpp.cpp:(.text+0x70): undefined reference to `PyRun_SimpleFileExFlags'
cpp.cpp:(.text+0x75): undefined reference to `Py_Finalize'
collect2: error: ld returned 1 exit status
What am I missing? is just something just a little or completely wrong?
cpp and py files and located in the same directory.
And how do I then read the output of python in C++? Haven't even got to that yet...
You have to link your code with libpython3.x.a/python3.x.lib (x - version of python you use). Which file to link: *.a or *.lib depends of your OS. The files are available with python distribution.
Here is a code with cmake that works for me:
cmake_minimum_required(VERSION 2.8.9)
project (embpy)
add_executable(embpy embpy.cpp)
target_include_directories(embpy PRIVATE /path-to-python/Python38/include/python3.8)
target_link_libraries(embpy /path-to-python/Python38/lib/libpython3.8.a)
the embpy.cpp is the same as yours
Figured it out myself then, the problem was incomplete compiler arguments.
This is what I got it to works with:
g++ -fPIC $(python3.6-config --cflags) cpp.cpp $(python3.6-config --ldflags)
the key missing parts were $(python3.6-config --cflags) before and $(python3.6-config --ldflags) after the file that was to be compiled. The first one gives g++ the compile options and the latter gives the flags for linking.
Found the solution from python docs, part 1.6.
I am trying to learn how to embed a Python interpreter into my C/C++ programs.
I like this concept because it may enable me to extend C programs at run time as well as enable users to write custom scripts/plugins.
Detailed instruction on embedding Python in C is provided at: https://docs.python.org/3/extending/embedding.html
I'm using example code provided in the Python documentation to figure out the mechanics involved:
embedded.c
#include <Python.h>
int
main(int argc, char *argv[])
{
Py_SetProgramName(argv[0]); /* optional but recommended */
Py_Initialize();
PyRun_SimpleString("from time import time,ctime\n"
"print 'Today is',ctime(time())\n");
Py_Finalize();
return 0;
}
The problem is I can't figure out how to compile it on Windows using gcc.
C:\Users\user\embedded_python> gcc embedded.c
embedded.c:2:10: fatal error: Python.h: No such file or directory
#include <Python.h>
^~~~~~~~~~
compilation terminated.
I think the problem is I need to link to the Python.h file, but I can't seem to get the compiler flags right:
C:\Users\user\Desktop\embedded_python>gcc -IC:\Users\user\AppData\Local\Programs\Python\Python38 -lPython38 embedded.c
embedded.c:2:10: fatal error: Python.h: No such file or directory
#include <Python.h>
^~~~~~~~~~
compilation terminated.
How can I get this program to compile on Windows (preferably with gcc/g++ to avoid Visual Studio bloat)?
In Linux, but I suppose if you activate your Bash from windows and can perform a work around. First you have to install python3-dev
Then use this command line where your file is located, to compile (test is your filename)
gcc -Wall test.c -o test $(pkg-config --cflags --libs python3)
I am trying to compile some C++ Code with OpenCv and Pybind with this header:
https://github.com/patrikhuber/eos/blob/v0.12.2/python/pybind11_opencv.hpp
This has worked for me before, so I don't think the header file is the Problem.
I can compile the code without problems, but when i try to import the created file to Python I get the following error:
ImportError: /usr/lib/libgtk-3.so.0: undefined symbol: g_mount_operation_set_is_tcrypt_hidden_volume
Here is the C++ Code:
#include <opencv2/imgproc.hpp>
#include <opencv2/highgui.hpp>
#include <pybind11/pybind11.h>
#include "pybind11_opencv.hpp"
using namespace std;
namespace py = pybind11;
cv::Mat func(cv::Mat Image1,cv::Mat Image2)
{
return Image1;
}
PYBIND11_MODULE(pybind_module, m)
{
m.doc() = "Text";
m.def("func", &func, "Function",
py::arg("Image1"),
py::arg("Image2"));
}
I am guessing it's a problem with my setup (arch linux) since I got something similar working before and not even this minimal example is working.
I was able to solve the problem myself using the following compiler settings.
c++ -msse4 -O3 -Wall -shared -std=c++11 -fPIC -lopencv_imgcodecs `python3 -m pybind11 --includes` main.cpp -o executable`python3-config --extension-suffix` /usr/local/include/ -L/usr/local/lib/ -march=native
I guess there was a mistake in the previous compilation.
I am working on writing a wrapper around C++ functions to use in Python. Thus, I am trying to use Boost.Python as an experiment first. Below are the function I want to wrap:
hello_exp.cpp:
char const* greet()
{
return "hello, world!";
}
#include <boost/python.hpp>
BOOST_PYTHON_MODULE(hello_ext)
{
using namespace boost::python;
def("greet", greet);
}
my Makefile:
COMPILER = g++
CPPFLAGS = -g -Wall -std=c++11 -stdlib=libc++
# Python and BoostPython links.
BOOSTHEADERS = -I/usr/local/Cellar/boost/1.64.0_1/include/boost/
BOOSTLIBRARIES = -L/usr/local/Cellar/boost-python/1.64.0/lib/
BOOSTLIB = -lboost_python
PYTHONHEADERS = -I/usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/include/python3.6m
PYTHONLIBRARIES = -L/usr/local/Cellar/python3/3.6.1/Frameworks/Python.framework/Versions/3.6/lib
PYTHONLIB = -lpython3.6
# Collect links.
LIBRARIES = $(BOOSTLIBRARIES) $(PYTHONLIBRARIES) $(PYTHONLIB) $(BOOSTLIB)
HEADERS = $(BOOSTHEADERS) $(PYTHONHEADERS)
# Build target.
TARGET = hello_ext
# BEGIN MAKE
all: $(TARGET)
$(TARGET): $(TARGET).cpp
$(COMPILER) -shared $(TARGET).cpp $(LIBRARIES) $(HEADERS) -o $(TARGET).so
clean:
$(RM) $(TARGET)
However, after some experiments, I am perpetually stuck at this error...:
Undefined symbols for architecture x86_64:
"boost::python::detail::init_module(PyModuleDef&, void (*)())", referenced from:
_PyInit_hello_ext in hello_ext-476eb2.o
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [hello_ext.so] Error 1
So I have a installed python3 through HomeBrew, as well as boost and boost-python. Note that I actually installed the boost-python library without python2 support and only with python3 support.
Thank you in advance!
After some fixing and digging around, it turns out the problem is that the boost-python library I installed is still in python2. so instead, make sure you do
brew rm boost-python
brew install boost-python --with-python3 --without-python
to get the right version. Then the make file is just change
BOOSTLIB = -lboost_python
to
BOOSTLIB = -lboost_python3
And then just hit make :)
I was looking at here to see how to expose c++ to Python. I have built Python deep learning code which uses boost-python to connect c++ and python and it is running ok, so my system has things for boost-python alread setup.
Here is my hello.cpp code (where I used WorldC and WorldP to clearly show the C++ and Python class name usage in the declaration. I don't know why the original web page is using the same class name World to cause confusion to beginners.)
#include <boost/python.hpp>
using namespace boost::python;
struct WorldC
{
void set(std::string msg) { this->msg = msg; }
std::string greet() { return msg; }
std::string msg;
};
BOOST_PYTHON_MODULE(hello)
{
class_<WorldC>("WorldP")
.def("greet", &WorldC::greet)
.def("set", &WorldC::set)
;
}
and this is how I made hello.so
g++ -shared -c -o hello.so -fPIC hello.cpp -lboostpython -lpython2.7 -I/usr/local/include/python2.7
When I run import hello in python, it gives me this error.
>>> import hello
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: ./hello.so: only ET_DYN and ET_EXEC can be loaded
Can anybody tell me what's wrong?
(I'm using anaconda2 under my home directory for python environment, but since my deep learning code builds ok with boost-python, there should be no problem including boost/python.hpp in my system directory)
I've long forgotten about this problem and got to revisit this issue today.
I found two problems. The first problem was that I gave -c option which made the compiler only compile the source and not link. The second problem was that the library name was wrong(I search /usr/lib64 and there was libboost_python.so, so it should be -lboost_python instead of -lboostpython). So the correct way to build it is :
g++ -shared -o hello.so -fPIC hello.cpp -lboost_python -lpython2.7 -I/usr/local/include/python2.7
I found it runs ok in python. :)