Develop Reference

Loading TFlite model for Inference (Python)

I'm using Tensorflow Lite to train an image classifier. I now have a bunch of *.tflite models stored, and I'm trying to write some code that allows me to pick a tflite model file, pick a dataset, and test that model on that dataset (inference).
When I train a model using:
model = image_classifier.create(trainData, validation_data=valData, shuffle=True, use_augmentation=False)
I am able to easily test this model on a test dataset right away because the model is actually stored in the variable 'model', by using:
model.evaluate_tflite('model.tflite', test_data)
or
loss, accuracy = model.evaluate(test_data)
However, if I simply want to load an already existing *.tflite model, without having trained it in the same run, I can't figure out a simple way to do that.
Following these instructions, it seems to be a lot of steps for what I'm trying to do. In other Machine Learning libraries (like PyTorch), you are able to define the model and then quickly load the saved weights and then get to testing, like:
model = models.densenet201(progress=True, pretrained=pretrained)
model.load_state_dict(torch.load("models/model.pt"))
Is there a simple way for me to initialise the model into the 'model' variable, load the saved weights from a *.tflite file, and then run inference?
Thank you for your help

A simple example of image classification:
import tensorflow as tf
import numpy as np
import cv2
class TFLiteModel:
def __init__(self, model_path: str):
self.interpreter = tf.lite.Interpreter(model_path)
self.interpreter.allocate_tensors()
self.input_details = self.interpreter.get_input_details()
self.output_details = self.interpreter.get_output_details()
def predict(self, *data_args):
assert len(data_args) == len(self.input_details)
for data, details in zip(data_args, self.input_details):
self.interpreter.set_tensor(details["index"], data)
self.interpreter.invoke()
return self.interpreter.get_tensor(self.output_details[0]["index"])
model = TFLiteModel("mobilenet_v2_1.0_224_1_default_1.tflite")
image = cv2.imread("hand_blower.png")
image = cv2.resize(image, (224, 224))
image = image.astype(np.float32)[np.newaxis]
image = (image - 127.5) / 127.5
label = model.predict(image)[0].argmax()
print(label)
Please refer to the official documentation for detailed information:
https://www.tensorflow.org/api_docs/python/tf/lite/Interpreter
The model was loaded from:
https://tfhub.dev/tensorflow/lite-model/mobilenet_v2_1.0_224/1/default/1

How to Load Fastai model and predict it on single image

I have trained a fastai model using Kaggle notebook, it has saved the model but how to load the model is the problem, i have tried different methods like the method given below. Even it does load the model it doesn't have any predict function only thing I can see is model.eval().
The second problem is when the model was trained on google collab it didn't even get the single image, I did try to convert the image to NumPy way and another way but both didn't work out.
I am attaching the kaggle link of model training, the saved model and the test images in last after this code
#Code for Loading model
from fastai import *
from fastai.vision import *
import torch
loc = torch.load('/content/gdrive/MyDrive/Data Exports/35k data/stage-1.pth')
body = create_body(models.resnet18, True, None)
data_classes = 4
nf = callbacks.hooks.num_features_model(body) * 2
head = create_head(nf, data_classes, None, ps=0.5, bn_final=False)
model = nn.Sequential(body, head)
Kaggle Model
Test Images From Kaggle Dataset
Saved Model

How to load pytorch models:
loc = torch.load('/content/gdrive/MyDrive/Data Exports/35k data/stage-1.pth')
model = ... # build your model
model.load_state_dict(loc)
model.eval()
Now you should be able to simply use the forward pass to generate your predictions:
input = ... # your input image
pred = model(input) # your class predictions
Don't forget to convert your inputs to torch tensors first, you might want to use a DataLoader for ease of use.

How to deploy my .pkl machine learnt model made with fastai on my other machine with python on the desktop

I have followed the fastai documentation and the videos on how to create a ML model that can detect the different home care products like soaps and deodorants and so on.
I have now come to where I have the model that supposedly works with an error rate of 0.03...
to my understanding its about a 97% accurate model, however I have no idea on how to predict on other images on another machine. I have exported it using the "learn.export('Home_Care_Model.pkl')" as said in the documentation, with no luck.
Now in the documentation is states that I would need to define the model again with a classes and training set and so on again but now I'm on another computer so i don't have those files on it and I can't go through running it on the web as it is supposed to be run as a python script on any desktop (end goal).
What I'm going towards is where I have one file with unsorted images that then when i run the model on will separate the images into two different folders according to the prediction.
I have been searching for an answer to this and to be honest i'm not sure if i am just not understanding it well enough or something as i have come up empty with every attempt i make to get this model working.
Here is my training code:
from fastai import *
from fastai.vision import *
%matplotlib inline
%reload_ext autoreload
%autoreload 2
import os
os.environ['KMP_DUPLICATE_LIB_OK']='True'
path = Path('my files path...')
print(path)
for folder in ('soap','deo'): # I have more but it will waist space.
print (folder)
verify_images(path/folder, max_size=500)
np.random.seed(42)
data = ImageDataBunch.from_folder(path, train='.', valid_pct=0.3, ds_tfms=get_transforms(),
size=224, num_workers=4).normalize(imagenet_stats)
data.classes
from fastai.metrics import error_rate
learn= create_cnn(data, models.resnet34, metrics=error_rate)
learn
defaults.device = torch.device('cuda')
learn.fit_one_cycle(5)
learn.unfreeze()
learn.lr_find()
learn.recorder.plot()
learn.fit_one_cycle(4, max_lr=slice(3e-6,3e-5))
learn.save('day2Test_02')
from fastai.widgets import *
ds, idxs = DatasetFormatter().from_toplosses(learn)
ImageCleaner(ds, idxs, path)
df = pd.read_csv(path/'cleaned.csv', header='infer')
df.head()
df[(df['name'].apply(lambda x: len(x)<5))]
np.random.seed(42)
db =(ImageList.from_df(df,path).random_split_by_pct(0.2).label_from_df().transform(get_transforms(), size= 224).databunch(bs=8)).normalize(imagenet_stats)
data.classes, data.c, len(data.train_ds), len(data.valid_ds)
db.classes, db.c,len(db.train_ds), len(data.valid_ds)
learn.data = db
learn.freeze()
learn.fit_one_cycle(4)
learn.save('day2Test_02_01')
learn.unfreeze()
learn.lr_find()
learn.recorder.plot()
learn.fit_one_cycle(4, max_lr=slice(3e-5,3e-4))
learn.save('dat2Test_test2')
interp = ClassificationInterpretation.from_learner(learn)
interp.plot_confusion_matrix()
learn.export('day2Test_test2.pkl')

The clear_session() method of keras.backend does not clean up the fitting data

I am working on a comparison of the fitting accuracy results for the different types of data quality. A "good data" is the data without any NA in the feature values. A "bad data" is the data with NA in the feature values. A "bad data" should be fixed by some value correction. As a value correction, it might be replacing NA with zero or mean value.
In my code, I am trying to perform multiple fitting procedures.
Review the simplified code:
from keras import backend as K
...
xTrainGood = ... # the good version of the xTrain data
xTrainBad = ... # the bad version of the xTrain data
...
model = Sequential()
model.add(...)
...
historyGood = model.fit(..., xTrainGood, ...) # fitting the model with
# the original data without
# NA, zeroes, or the feature mean values
Review the fitting accuracy plot, based on historyGood data:
After that, the code resets a stored the model and re-train the model with the "bad" data:
K.clear_session()
historyBad = model.fit(..., xTrainBad, ...)
Review the fitting process results, based on historyBad data:
As one can notice, the initial accuracy > 0.7, which means the model "remembers" previous fitting.
For the comparison, this is the standalone fitting results of "bad" data:
How to reset the model to the "initial" state?

K.clear_session() isn't enough to reset states and ensure reproducibility. You'll also need to:
Set (& reset) random seeds
Reset TensorFlow default graph
Delete previous model
Code accomplishing each below.
reset_seeds()
model = make_model() # example function to instantiate model
model.fit(x_good, y_good)
del model
K.clear_session()
tf.compat.v1.reset_default_graph()
reset_seeds()
model = make_model()
model.fit(x_bad, y_bad)
Note that if other variables reference the model, you should del them also - e.g. model = make_model(); model2 = model --> del model, model2 - else they may persist. Lastly, tf random seeds aren't as easily reset as random's or numpy's, and require the graph to be cleared beforehand.
Function/modules used:
import tensorflow as tf
import numpy as np
import random
import keras.backend as K
def reset_seeds():
np.random.seed(1)
random.seed(2)
if tf.__version__[0] == '2':
tf.random.set_seed(3)
else:
tf.set_random_seed(3)
print("RANDOM SEEDS RESET")

I had a similar issue when did training many models in a loop in a single file. I tried many things on Keras/TF (version 2.5), including the answers in this thread. Nothing helped apart from one thing - running one file from another file using subprocess module, which ensures the kernel restart every single time.
In the simplest way, you can keep a training code in a single file, and access it to run your initial model or rerun the consequent model from a different file. To run one file from another simply do it in the second file:
run_no = [0,1,2,3]
for i in range(len()):
subprocess.run(["ipython", "your_main_file.ipynb", str(i)]) # for jupyter
#subprocess.run(["python3", "your_main_file.py", str(i)]) # for python

You are using K.clear_session() in the wrong way, to get a model with randomly initialized weights, you should delete the old model (using the del keyword), and then proceed to create a new model, and train it.
The you can use K.clear_session() after each fitting procedure.

Instantiating a new model object with the same name is not enough?
model = make_model()

use a saved trained model to predict on new dataset

I am using theano, sklearn and numpy in Python. I found this code for saving my trained network and predict on my new dataset in this link https://github.com/lzhbrian/RBM-DBN-theano-DL4J/blob/master/src/theano/code/logistic_sgd.py. the part of the code I am using is this :
"""
An example of how to load a trained model and use it
to predict labels.
"""
def predict():
# load the saved model
classifier = pickle.load(open('best_model.pkl'))
# compile a predictor function
predict_model = theano.function(
inputs=[classifier.input],
outputs=classifier.y_pred)
# We can test it on some examples from test test
dataset='mnist.pkl.gz'
datasets = load_data(dataset)
test_set_x, test_set_y = datasets[2]
test_set_x = test_set_x.get_value()
predicted_values = predict_model(test_set_x[:10])
print("Predicted values for the first 10 examples in test set:")
print(predicted_values)
if __name__ == '__main__':
sgd_optimization_mnist()
The code for the neural network model I want to save and load and predict with is https://github.com/aseveryn/deep-qa. I could save and load the model with cPickle but I continuously get errors in # compile a predictor function part:
predict_model = theano.function(inputs=[classifier.input],outputs=classifier.y_pred)
Actually I am not certain what I need to put in the inputs according to my code. Which one is right?
inputs=[main.predict_prob_batch.batch_iterator], outputs=test_nnet.layers[-1].
y_pred)
inputs=[predict_prob_batch.batch_iterator],
outputs=test_nnet.layers[-1].y_pred)
inputs=[MiniBatchIteratorConstantBatchSize.dataset],
outputs=test_nnet.layers[-1].y_pred)
inputs=[
sgd_trainer.MiniBatchIteratorConstantBatchSize.dataset],
outputs=test_nnet.layers[-1].y_pred)
or none of them???
Each of them I tried I got the errors:
ImportError: No module named MiniBatchIteratorConstantBatchSize
or
NameError: global name 'predict_prob_batch' is not defined
I would really appreciate if you could help me.
I also used these commands for running the code but still the errors.
python -c 'from run_nnet import predict; from sgd_trainer import MiniBatchIteratorConstantBatchSize; from MiniBatchIteratorConstantBatchSize import dataset; print predict()'
python -c 'from run_nnet import predict; from sgd_trainer import *; from MiniBatchIteratorConstantBatchSize import dataset; print predict()'
Thank you and let me know please if you know a better way to predict for new dataset on the loaded trained model.