I've been following this tutorial for creating LMDBs in Python. The code in the tutorial depends on Caffe, namely the caffe.proto.caffe_pb2.Datum() object for serializing the data.
However, those not using Caffe as a deep learning framework might find it tedious to install Caffe just for using the protobuf. What other ways exist for creating LMDBs?
LMDB is a general purpose DB that can be used for many applications in many ways. However, if you want to use it with caffe you are restricted to the interface written in caffe's "Data" layer. This interface is expecting a "Datum" elements in the LMDB. Other stored elements in the LMDB will simply results with a read error by caffe failing to interpret the stored elements.
Therefore, if you are going to use LMDB as an input for a caffe "Data" layer, you must store elements as "Datum" in the LMDB.
However, if you are going to use LMDB for other purposes, you may store any type of objects you wish, as long as you know how to read them correctly.
Related
I am trying to train a custom object detector using tflite model maker (https://www.tensorflow.org/lite/tutorials/model_maker_object_detection). I want to deploy trained tflite model to coral edgeTPU. I want to use tensorflow tfrecord (multiple) as input for training a model like object detection API. I tried with
tflite_model_maker.object_detector.DataLoader(
tfrecord_file_patten, size, label_map, annotations_json_file=None
) but I am not able to work around it. I have following questions.
Is it possible to tfrecord for training like mentioned above?
Is it also possible to pass multiple CSV files for training?
For multiple CSV files, you could probably just append one file to the other. Then you'd just have to pass one csv file.
As for passing a tfrecord instead, this should be possible. I'm also attempting to do this, so if I get it working I'll update my post. Looking at the source, it seems from_cache is the function internally used. Following that structure, should be able to create a DataLoader object similarly:
train_data = DataLoader(tfrecord_file_patten, meta_data['size'],
meta_data['label_map'], ann_json_file)
In this case, tfrecord_file_patten should be a tfrecord of your training data. You can construct the validation and test data the same way. This will work provided you're constructing your TFRecords correctly. There appears to be some inconsistency to how it's done in different places, so make sure you follow the same structure in creating the TFRecords as found in the ModelMaker source. This worked for me. One specific thing to watch out for is to use an integer for the 'image/source_id' feature in your TFExamples. If you use a string it'll throw an error.
I am trying to check if my .onnx model is correct, and need to run inference to verify the output for the same.
I know we can run validation on .mlmodel using coremltools in Python - basically load the model and input and get the prediction. I am trying to do a similar thing for the .onnx model.
I found the MXNet framework but I can't seem to understand how to import the model - I just have the .onnx file and MXNet requires some extra input besides the onnx model.
Is there any other simple way to do this in Python? I am guessing this is a common problem but can't seem to find any relevant libraries/frameworks to do this as easily as coremltools for .mlmodel.
I do not wish to convert .onnx to another type of model (like say PyTorch) as I want to check the .onnx model as is, not worrying if the conversion was correct. Just need a way to load the model and input, run inference and print the output.
This is my first time encountering these formats, so any help or insight would be appreciated.
Thanks!
I figured out a way to do this using Caffe2 - just posting in case someone in the future tries to do the same thing.
The main code snippet is:
import onnx
import caffe2.python.onnx.backend
from caffe2.python import core, workspace
import numpy as np
# make input Numpy array of correct dimensions and type as required by the model
modelFile = onnx.load('model.onnx')
output = caffe2.python.onnx.backend.run_model(modelFile, inputArray.astype(np.float32))
Also it is important to note that the input to run_model can only be a numpy array or a string. The output will be an object of the Backend.Outputs type. I was able to extract the output numpy array from it.
I was able to execute inference on the CPU, and hence did not need the Caffe2 installation with GPU (requiring CUDA and CDNN).
I've read caffe2 tutorials and tried pre-trained models. I knew caffe2 will leverge GPU to run the model/net. But the input data seems always be given from CPU(ie. Host) memory. For example, in Loading Pre-Trained Models, after model is loaded, we can predict an image by
result = p.run([img])
However, image "img" should be read in CPU scope. What I look for is a framework that can pipline the images (which is decoded from a video and still resides in GPU memory) directly to the prediction model, instead of copying it from GPU to CPU scope, and then transfering to GPU again to predict result. Is Caffe or Caffe2 provides such functions or interfaces for python or C++? Or should I need to patch Caffe to do so? Thanks at all.
Here is my solution:
I'd found in tensor.h, function ShareExternalPointer() can exactly do what I want.
Feed gpu data this way,
pInputTensor->ShareExternalPointer(pGpuInput, InputSize);
then run the predict net through
pPredictNet->Run();
where pInputTensor is the entrance tensor for the predict net pPredictNet
I don't think you can do it in caffe with python interface.
But I think that it can be accomplished using the c++: In c++ you have access to the Blob's mutable_gpu_data(). You may write code that run on device and "fill" the input Blob's mutable_gpu_data() directly from gpu. Once you made this update, caffe should be able to continue its net->forward() from there.
UPDATE
On Sep 19th, 2017 PR #5904 was merged into master. This PR exposes GPU pointers of blobs via the python interface.
You may access blob._gpu_data_ptr and blob._gpu_diff_ptr directly from python at your own risk.
As you've noted, using a Python layer forces data in and out of the GPU, and this can cause a huge hit to performance. This is true not just for Caffe, but for other frameworks too. To elaborate on Shai's answer, you could look at this step-by-step tutorial on adding C++ layers to Caffe. The example given should touch on most issues dealing with layer implementation. Disclosure: I am the author.
I am trying to convert a pretrained InceptionV3 model (.ckpt) from the Open Images Dataset to a .pb file for use in the Tensorflow for Mobile Poets example. I have searched the site as well as the GitHub Repository and have not found any conclusive answers.
(OpenImages Inception Model: https://github.com/openimages/dataset)
Thank you for your responses.
Below I've included some draft documentation I'm working on that might be helpful. One other thing to look out for is that if you're using Slim, you'll need to run export_inference_graph.py to get a .pb GraphDef file initially.
In most situations, training a model with TensorFlow will give you a folder containing a GraphDef file (usually ending with the .pb or .pbtxt extension) and a set of checkpoint files. What you need for mobile or embedded deployment is a single GraphDef file that’s been ‘frozen’, or had its variables converted into inline constants so everything’s in one file.
To handle the conversion, you’ll need the freeze_graph.py script, that’s held in tensorflow/pythons/tools/freeze_graph.py. You’ll run it like this:
bazel build tensorflow/tools:freeze_graph
bazel-bin/tensorflow/tools/freeze_graph \
--input_graph=/tmp/model/my_graph.pb \ --input_checkpoint=/tmp/model/model.ckpt-1000 \ --output_graph=/tmp/frozen_graph.pb \
--input_node_names=input_node \
--output_node_names=output_node \
The input_graph argument should point to the GraphDef file that holds your model architecture. It’s possible that your GraphDef has been stored in a text format on disk, in which case it’s likely to end in ‘.pbtxt’ instead of ‘.pb’, and you should add an extra --input_binary=false flag to the command.
The input_checkpoint should be the most recent saved checkpoint. As mentioned in the checkpoint section, you need to give the common prefix to the set of checkpoints here, rather than a full filename.
output_graph defines where the resulting frozen GraphDef will be saved. Because it’s likely to contain a lot of weight values that take up a large amount of space in text format, it’s always saved as a binary protobuf.
output_node_names is a list of the names of the nodes that you want to extract the results of your graph from. This is needed because the freezing process needs to understand which parts of the graph are actually needed, and which are artifacts of the training process, like summarization ops. Only ops that contribute to calculating the given output nodes will be kept. If you know how your graph is going to be used, these should just be the names of the nodes you pass into Session::Run() as your fetch targets. If you don’t have this information handy, you can get some suggestions on likely outputs by running the summarize_graph tool.
Because the output format for TensorFlow has changed over time, there are a variety of other less commonly used flags available too, like input_saver, but hopefully you shouldn’t need these on graphs trained with modern versions of the framework.
I want to save a model that I have trained.
Since it is using shared variables (like Weights, bias and so on) and since it should be readable on machines without Theano installed, I wanted to use the theano.misc.pkl_utils.dump() function.
However, it seems as if that is only installed in bleeding edge installations (the current github file looks different than my local one).
Is that really the case? And why is the description in the docs then?
I am using theano 0.7.0 and I'm seriously confused about this.
If that feature is not yet available (I can't install bleeding edge right now), what are other ways? I'm sure that I am not the only one trying to save a trained model the easiest way possible ;-)
Thank you a lot,
Roman
If you train your model using Theano, the parameters of the model will be eventually shared variables (probably a list of shared variables if the network consists of several layers). It is possible to pickle the list of shared variables and unpickle it later. However you might have problems to unpickle such variables in another machine, e.g. with no Theano installation or if you train in a GPU-capable machine that generates CudaNdarrays and then you want to load back the model in a non-GPU-capable machine. What I recommend you is the following: convert every shared variable of the list of parameters into a numpy ndarray:
params_numpy = [numpy.asarray(p.get_value()) for p in params]
where params is a list of shared variables. Then you can safely pickle/unpickle params_numpy.