Develop Reference

Loading TFlite model for Inference (Python)

I'm using Tensorflow Lite to train an image classifier. I now have a bunch of *.tflite models stored, and I'm trying to write some code that allows me to pick a tflite model file, pick a dataset, and test that model on that dataset (inference).
When I train a model using:
model = image_classifier.create(trainData, validation_data=valData, shuffle=True, use_augmentation=False)
I am able to easily test this model on a test dataset right away because the model is actually stored in the variable 'model', by using:
model.evaluate_tflite('model.tflite', test_data)
or
loss, accuracy = model.evaluate(test_data)
However, if I simply want to load an already existing *.tflite model, without having trained it in the same run, I can't figure out a simple way to do that.
Following these instructions, it seems to be a lot of steps for what I'm trying to do. In other Machine Learning libraries (like PyTorch), you are able to define the model and then quickly load the saved weights and then get to testing, like:
model = models.densenet201(progress=True, pretrained=pretrained)
model.load_state_dict(torch.load("models/model.pt"))
Is there a simple way for me to initialise the model into the 'model' variable, load the saved weights from a *.tflite file, and then run inference?
Thank you for your help

A simple example of image classification:
import tensorflow as tf
import numpy as np
import cv2
class TFLiteModel:
def __init__(self, model_path: str):
self.interpreter = tf.lite.Interpreter(model_path)
self.interpreter.allocate_tensors()
self.input_details = self.interpreter.get_input_details()
self.output_details = self.interpreter.get_output_details()
def predict(self, *data_args):
assert len(data_args) == len(self.input_details)
for data, details in zip(data_args, self.input_details):
self.interpreter.set_tensor(details["index"], data)
self.interpreter.invoke()
return self.interpreter.get_tensor(self.output_details[0]["index"])
model = TFLiteModel("mobilenet_v2_1.0_224_1_default_1.tflite")
image = cv2.imread("hand_blower.png")
image = cv2.resize(image, (224, 224))
image = image.astype(np.float32)[np.newaxis]
image = (image - 127.5) / 127.5
label = model.predict(image)[0].argmax()
print(label)
Please refer to the official documentation for detailed information:
https://www.tensorflow.org/api_docs/python/tf/lite/Interpreter
The model was loaded from:
https://tfhub.dev/tensorflow/lite-model/mobilenet_v2_1.0_224/1/default/1

Problem with loading language_model_learner fastai

I have problem with fastai library. My code below:
import fastai
from fastai.text import *
import os
import pandas as pd
import fastai
from fastai import *
lab = df.columns[0]
data_lm = TextLMDataBunch.from_csv(r'/AWD', 'data.csv', label_cols = lab, text_cols = ['text'])
data_clas = TextClasDataBunch.from_csv(r'/AWD', 'data.csv', vocab = data_lm.train_ds.vocab, bs = 256,label_cols = lab, text_cols=['text'])
data_lm.save('data_lm_export.pkl')
data_clas.save('data_clas.pkl')
learn = language_model_learner(data_lm,AWD_LSTM,drop_mult = 0.3)
learn.lr_find()
learn.recorder.plot(skip_end=10)
learn.fit_one_cycle(10,1e-2,moms=(0.8,0.7))
learn.save('fit_head')
learn.load('fit_head')
My data is quite big, so each epoch in fit_one_cycle lasts about 6h. My resources enables me only to train model in SLURM JOB 70h, so my whole script will be cancelled. I wanted to divide my script into pieces and the first longest part has to learn and save fit_head. Everything was ok, and after that I wanted to load my model to train it again, but i got this error:
**RuntimeError: Error(s) in loading state_dict for SequentialRNN:
size mismatch for 0.encoder.weight: copying a param with shape torch.Size([54376, 400]) from checkpoint, the shape in current model is torch.Size([54720, 400]).
**
I have checked similar problems on github/stack posts and I tried those solutions like this below, but i cannot find anything usefull.
data_clas.vocab.stoi = data_lm.vocab.stoi
data_clas.vocab.itos = data_lm.vocab.itos
Is there any possibility to load trained model without having this issue ?

When you do learner.save() only the model weights are saved on your disk and not the model state dict which contains the model architecture information.
To train the model in a different session you must first define the model itself. Remember to use the same code to define your new model. Since your data is quite heavy as you mentioned you can use a very small subset (~16 records) of your data to create this new model and then do learn.load(model_path) and you should be able to resume training.
you can modify the training data with learn.data.train_dl = new_dl

Understanding what fc1000_softmax output from tensorflow CNN classification means

I am using python 3.6.10 and tensorflow 1.5 on a cpu. I have trained a cnn and saved it as a .onnx file. I am now trying to make a binary classification of my images using the code below:
import onnx
import warnings
from onnx_tf.backend import prepare
import numpy as np
from numpy import array
from IPython.display import display
from PIL import Image
warnings.filterwarnings('ignore')
onnx_model = onnx.load("trainednet.onnx") # load onnx model
tf_rep = prepare(onnx_model) # Import the ONNX model to Tensorflow
img = Image.open('Im025.jpg').resize((224, 224))
img = array(img).reshape(1,3, 224,224)
classification = tf_rep.run(img)
print(classification)
The print(classification) gives me an output like this:
Outputs(fc1000_softmax=array([[9.9967182e-01, 3.2823894e-04]], dtype=float32))
What does this output mean and how can I use it to understand what tensorflow classified my image as?

Well, what you have is the array of the output of the softmax layer of your model. This output can be interpreted as a probability assigned to each class ( in your case 2). You can also see it as "how confident the model is for each class".
So to have the final classification you need to take the max value of this array and map it to your label map { 0 -> class_one, 1-> class_two}.

Yes exactly, so after you model is not perfect (ie the model doesn't have 100% of accuracy). You should test on more images of class two and you will see the second number will be higher .
You should run it on a test set or refer to the metrics of the trained model.

Launch Tensorboard with keras graph (for visualize accuracy, loss and predict result)

I don't understand how to use tensor board to visualize the training step of my keras network.
I already launch tensor board with the command line : tensorboard --logdir=/run1
But he raise this error :
No dashboards are active for the current data set. Probable causes:You
haven’t written any data to your event files. TensorBoard can’t find
your event files.
import tensorflow as tf
from tensorflow import keras
import numpy as np
# Create the array of data
train_data = [[1.0,2.0,3.0],[4.0,5.0,6.0]]
train_data_np = np.asarray(train_data)
train_label = [[1,2,3],[4,5,6]]
train_label_np = np.asarray(train_data)
### Build the model
model = keras.Sequential([
keras.layers.Dense(3,input_shape =(3,2)),
keras.layers.Dense(3,activation=tf.nn.sigmoid)
])
model.compile(optimizer='sgd',loss='sparse_categorical_crossentropy',metrics=['accuracy'])
#Train the model
tensorboard = TensorBoard(log_dir="run1")
model.fit(train_data_np,train_label_np,epochs=10,callbacks=tensorboard)
#test the model
restest = model.evaluate(test_data_np,test_label_np)

Adding formal answer here; looks like there is a typo in the tensorboard logdir parameter. You need to remove the slash at the beginning of the directory
tensorboard --logdir=run1

Caffe execution

I am starting to use caffe for deep learning. I have the .caffemodel file with my trained weights and a particular neural network. I am using python interface.
I've seen that I can load my network and my weights by doing this:
solver=caffe.get_solver('prototxtfile.prototxt')
solver.net.copy_from('weights.caffemodel')
But I do not want to fine-tuned my application. I just want to use those weights. I want to execute the network and for each image from the Imagenet data set I want to obtain the result of the classification (not the accuracy of an entire batch). How can I do that?
Thank you very much.

Try to understand the attached lines of python code and adjust them to your needs. It's not my code but I wrote a similar piece to test my models.
The source is:
https://www.cc.gatech.edu/~zk15/deep_learning/classify_test.py
If you don't want to fine-tune a pre-trained model, it's obvious that you don't need a solver. The solver is what optimizes the model. If you want to predict the class probability for an image, you actually just have to do a forward pass. Keep in mind that your deploy.prototxt must have a proper last layer which uses either a softmax or sigmoid function (depending on the architecture). You can't use the loss function from the train_val.prototxt for this.
import numpy as np
import matplotlib.pyplot as plt
# Make sure that caffe is on the python path:
caffe_root = '../' # this file is expected to be in {caffe_root}/examples
import sys
sys.path.insert(0, caffe_root + 'python')
import caffe
# Set the right path to your model definition file, pretrained model weights,
# and the image you would like to classify.
MODEL_FILE = '../models/bvlc_reference_caffenet/deploy.prototxt'
PRETRAINED = '../models/bvlc_reference_caffenet/bvlc_reference_caffenet.caffemodel'
IMAGE_FILE = 'images/cat.jpg'
caffe.set_mode_cpu()
net = caffe.Classifier(MODEL_FILE, PRETRAINED,
mean=np.load(caffe_root + 'python/caffe/imagenet/ilsvrc_2012_mean.npy').mean(1).mean(1),
channel_swap=(2,1,0),
raw_scale=255,
image_dims=(256, 256))
input_image = caffe.io.load_image(IMAGE_FILE)
plt.imshow(input_image)
prediction = net.predict([input_image]) # predict takes any number of images, and formats them for the Caffe net automatically
print 'prediction shape:', prediction[0].shape
plt.plot(prediction[0])
print 'predicted class:', prediction[0].argmax()
plt.show()

This is the code I use when I need to forward an image through my network:
import caffe
caffe.set_mode_cpu() #If you are using CPU
#caffe.set_mode_gpu() #or if you are using GPU
model_def = "path/to/deploy.prototxt" #architecture
model_weights = "path/to/weights.caffemodel" #weights
net = caffe.Net(model_def, # defines the structure of the model
model_weights,
caffe.TEST) # use test mode (e.g., don't perform dropout)
#Let's forward a single image (let's say inputImg)
#'data' is the name of my input blob
net.blobs["data"].data[0] = inputImg
out = net.forward()
# to get the final softmax probability
# in my case, 'prob' is the name of our last blob
# a softmax layer that will output the score/probability for our problem
outputScore = net.blobs["prob"].data[0] #[0] here because we forwarded a single image
In this example, the inputImg dimensions must match the dimensions of the images used during training, as well as all preprocessing done.