After using YOLOv5 to train model weights as .pt file,
how can I convert the weights file (model.pt) to hdf5 file (model.h5)?
Running python train.py --batch 16 --epochs 3 --data mydata.yaml --weights yolov5s.pt, the result is given by best.pt file at subfolder of YOLOv5, how can I convert it to h5 file?
Do this installation and get started here
This could be a lengthy procedure ... as you are aware that .pt only contains weights and not model architecture hence your model class should also be present in your conversion code
Edit: New links are added
Related
I'm a beginner with PyTorch and I'm trying to download the MNIST dataset using PyTorch with the following code:
torchvision.datasets.MNIST('data/', download=True)
This succeeds by downloading the raw files. But there is no 'processed' folder containing the training.pt and test.pt files.
How do I generate them?
I tried to add options like train=True or transform=torchvision.transforms.ToTensor to the code. But there files are not generated.
I am creating a custom object detection sample for android, I used the ssd_mobilenet_v1_coco pretrained model for transfer learning and got a decent accuracy. I also successfully managed to export the model.ckpt-XXXX to a .pb tflite graph using this line of code in the terminal (ran from the object_detection folder after cloning Tensorflow Object Detection API from github):
python export_tflite_ssd_graph.py --pipeline_config_path=training/ssd_mobilenet_v1_coco.config --trained_checkpoint_prefix=training/model.ckpt-40500--output_directory=tflite --add_postprocessing_op=true
The above created a folder tflite and it contained 2 files :
tflite_graph.pb
tflite_graph.pbtxt
However, when I want to convert the tflite_graph.pb to detect.tflite I get the following error and the program ends abruptly:
"TOCO failed. See console for info.\n%s\n%s\n" % (stdout, stderr))
tensorflow.lite.python.convert.ConverterError: TOCO failed.
.
.
.
Check failed: input_array_dims[i] == input_array_proto.shape().dims(i) (300 vs. 128)
Fatal Python error: Aborted
.
.
.
This is the command I used to convert the .pb to .tflite:
tflite_convert --graph_def_file=tflite/tflite_graph.pb --output_file=tflite/detect.tflite --input_shapes=1,128,128,3 --input_arrays=normalized_input_image_tensor --output_arrays=TFLite_Detection_PostProcess,TFLite_Detection_PostProcess:1,TFLite_Detection_PostProcess:2,TFLite_Detection_PostProcess:3 --allow_custom_ops
The images I used had a size of 128x128 hence why I assumed that would be the input_shapes. I do have Toco installed as well.
Any help or advice will be highly appreciated.
So upon doing additional research, I found out that it was because the config file of the model was looking at images of sizes 300 x 300. So I changed the images dimension in the config file to 128 and it worked.
As we know, when using tensorflow to save checkpoint, we have 3 files, for e.g.:
model.ckpt.data-00000-of-00001
model.ckpt.index
model.ckpt.meta
I check on the faster rcnn and found that they have an evaluation.py script which helps evaluate the pre-trained model, but the script only accept .ckpt file (as they provided some pre-trained models above).
I have run some finetuning from their pre-trained model
And then I wonder if there's a way to convert all the .data-00000-of-00001, .index and .meta into one single .ckpt file to run the evaluate.py script on the checkpoint?
(I also notice that the pre-trained models they provided in the repo do have only 1 .ckpt file, how can they do that when the save-checkpoint function generates 3 files?)
These
{
model.ckpt.data-00000-of-00001
model.ckpt.index
model.ckpt.meta
}
are the more recent checkpoint format
while
{model.ckpt}
is a previous checkpoint format
It will be in the same concept as to convert a Nintendo Switch to NES ... Or a 3 pieces CD bundle to a single ROM cartridge...
You don't need to convert, You can save the variables in the network using
saver = tf.train.Saver()
saver.save(sess, 'path of save/fileName.ckpt')
To restore the network for reuse later or in another script, use:
saver = tf.train.Saver()
saver.restore(sess, tf.train.latest_checkpoint('path of save/')
sess.run(....)
Important points:
sess must be same between first and later runs (coherent structure).
saver.restore needs the path of the folder of the saved files, not
an individual file path.
I need to train from scratch a CNN over a COCO dataset with a specific configuration: https://github.com/tensorflow/models/blob/master/research/object_detection/samples/configs/embedded_ssd_mobilenet_v1_coco.config
Thus, I installed TF Object Detection API and I downloaded the COCO dataset. However the dataset is in .h5 extension.
Is it possible to run the training with this kind of file or do I need to convert it in images in someway? If that is possible, what would the command be?
PS: I was not able to find a pre-trained model with that config, this is why I need to train a cnn from scratch.
My suggestion would be to convert the .hdf5 file to a .tfrecord file, you can find examples of how to do this here.
I am trying an example of car evaluation classification from
http://archive.ics.uci.edu/ml/datasets/Car+Evaluation
I have successfully trained model and printing predictions successfully
using following code
I am following this page for converting .pb model to .tflite
I have successfully build frozen graph
bazel build tensorflow/python/tools:freeze_graph
Now I am facing problems in running following command
bazel-bin/tensorflow/python/tools/freeze_graph\
--input_graph=/CarEvaluation/mobilenet_v1_224.pb \
--input_checkpoint=/CarEvaluation/checkpoints/mobilenet-10202.ckpt \
--input_binary=true --output_graph=/CarEvaluation/frozen_mobilenet_v1_224.pb \
--output_node_names=CarEvaluation/Predictions/Reshape_1
Problem is that in model directory I have .pbtxt file instead of .pb
and also I couldn't find .ckpt file in model directory, I have a simple checkpoint file and several .ckpt meta and index files with some number as suffix.
I have tried running above command with .pbtxt file and I am getting this exception
input_graph_def.ParseFromString(f.read())
google.protobuf.message.DecodeError: Error parsing message
Use the .pbtxt and the highest numbered .ckpt
i.e. something like:
bazel-bin/tensorflow/python/tools/freeze_graph\
--input_graph=/CarEvaluation/mobilenet_v1_224.pbtxt \
--input_checkpoint=/CarEvaluation/checkpoints/mobilenet-10202.ckpt-2000 \
--input_binary=true --output_graph=/CarEvaluation/frozen_mobilenet_v1_224.pb \
--output_node_names=CarEvaluation/Predictions/Reshape_1
As far as I have understood from the freeze_graph code, when you want to use it with a pbtxt file, you need to omit the --input_binary=true option, as the input file is no longer a binary one.