I am working on creating a custom model for image classification. The model will run on a microcontroller. So the goal is to make the model as small as possible while still being some what accurate. The model has three classes to identify two types of flowers and an "other" class for anything that is not one of the flowers. I have a directory flower_photos which contains the subdirectories roses, daisy, and other. Each subdirectory has hundreds of .jpg files. I believe this is the correct preparation structure required.
For optimization, I believe it will help by reducing the image size and convert them to grayscale. Will this help and does this need to be done prior to running it through model maker? I haven't been able to find a way to do it directly in model maker. I am also using for_int8() method for QuantizationConfig. It is my understanding that this is the best post training quatization for microcontrollers. Is this correct?
Below is my code so far. Is this correct for optimization for use on a microcontroller? Thanks!
import tensorflow as tf
assert tf.__version__.startswith('2')
from tflite_model_maker import image_classifier
from tflite_model_maker.config import QuantizationConfig
from tflite_model_maker.image_classifier import DataLoader
data = DataLoader.from_folder('C:\\Users\\username\\Python\\flower_photos')
train_data, test_data = data.split(0.9)
model = image_classifier.create(train_data)
loss, accuracy = model.evaluate(test_data)
config = QuantizationConfig.for_int8(representative_data=test_data)
model.export(export_dir='C:\\Users\\username\\Python\\model_int8', quantization_config=config)
Related
I am dynamically creating tensorflow models (all types classification, timeseries, etc) and I want to standardise the process of saving and calling them. So far I have been able to save the models to model.json using
model_json = model.to_json()
and the model weights using
model.save_weights('model_weights.h5')
and then load them in from the json and h5 files. Will this method work with all tensorflow models (i.e. can ui standardise this part of the pipeline)
Thanks in advance
Tl;DR: How could I access the pytorch pre-trained model for Swin-Transformer so that I could extract features from it to train it on segmentation task using DeepLabv3+ head on a custom data set with image sizes of 512
I am testing SwinTransformer backbone with Deeplabv3+ as head for semantic segmentation.
I already have the code for Head and it get the features from backbone and then process the features in different way. It is working fine for ResNet and Xception. The thing is that I want to work with pre-trained SWIN. Now there are many ways but each of those has it's own set of problems.
1. Using timm library for pre-trainned models`
! pip install timm
import timm
import torch
all_swins = timm.list_models('*swin*')
print(all_swins)
model = timm.create_model('swin_large_patch4_window12_384_in22k', in_chans = 3, pretrained = True,)
print(model.default_cfg)
dummy_image = torch.randn(1,3,512,512) # create an image of size 512
model.forward_features(dummy_image) # extract features : Will result in error
new_model = timm.create_model('swin_large_patch4_window12_384_in22k', in_chans = 3, pretrained = True, img_size = 512) # This won't work I'll explain below
The problem is that None of the above code work. Reason being that the model accepts here an image of size 384 but my images are of 512. So you could change the argument img_size for other CNNs but as the author has clarified in this question
It should work with the vit, vit_deit, vit_deit_distilled. Has not been implemented for pit, swin, and tnt yet.
2. Using MMcv / MMSeg library:
Please open this colab notebook. I have commented and documented the part
Problem: The pre-trained weights are for only for a specific method which produced SOTA results i.e ADE dataset using UperNet backbone. I can not use it with DeepLabv3+ on a custom dataset.
3. Segmentations Models Pytorch Library which uses timm encoders
Problem: Again, as it uses timm, so the image resolutions can't be changed.
4. PaddleSeg Library
It has Swin transformer but Deeplabv3+ works only with Resnet50 and 101
Last Resort: In the end, I pulled up the official code from microsoft where I found couple of useful things:
configuration yml file
Code which they use for model building def build_model()
Code inside def main() which parses the arguments and builds the whole model
I don't really know how to build the model changing the image size configurations. If anyone has any idea, please help.
I am trying to use transfer learning on resnet 152 to classify a data set of images that are custom labeled 0/1 as containing the object of interest or not. I have referenced multiple tutorials and haven't been able to figure it out. I will put some links below that I have previously referenced, but first code I am attempting to use.
I started with trying to use this.
PyTorch transfer learning with pre-trained ImageNet model
# Load the pretrained model
model = models.resnet152(pretrained=True)
classifier_name, old_classifier = model._modules.popitem()
for param in model.parameters():
param.requires_grad = False
classifier_input_size = old_classifier.in_features
classifier = nn.Sequential(OrderedDict([
('fc1', nn.Linear(classifier_input_size, hidden_layer_size)),
('activation', nn.SELU()),
('dropout', nn.Dropout(p=0.5)),
('fc2', nn.Linear(hidden_layer_size, output_layer_size)),
('output', nn.LogSoftmax(dim=1))
]))
But I get NameError, "OrderedDict" is not defined. I would like to understand what I am doing wrong on my classifier step here. After that I'm still struggling to understand how I could use this new model and classifier on my own dataset of images (how the images should be fed, specified as 0/1,specified as training/test, etc). Any help or tutorials on that you can point me to would be greatly appreciated.
you need to import OrderedDict using below command
from collections import OrderedDict
then it will work.
for more information visit Link
I have the Ensemble model that combines both tensorflow and scikit-learn. And I would like to save this Ensemble model as a box to feed data in and generate the output. My code is as below
def model_base_LSTM(***):
***
model = model_base_LSTM(***)
ensem_model = BaggingRegressor(base_estimator=model, n_estimators=15)
ensem_model.fit(x_train, y_train)
bag_mod_pred = ensem_model.predict(x_test_bag)
from joblib import dump, load
dump(ensem_model, 'LSTM_Ensemble.joblib')
TypeError: can't pickle _thread._local objects
So, how to solve this problem??
You can save your TensorFlow (and even PyTorch) models with Scikit-Learn, but only if you use Neuraxle and its saving mechanics.
Neuraxle is an extension of Scikit-Learn to make it more compatible with all deep learning libraries.
The trick is performed by using Neuraxle-TensorFlow or Neuraxle-PyTorch.
Why so?
Using one of Neuraxle-TensorFlow or Neuraxle-PyTorch will provide you with a saver to allow your thing to be serialized correctly. You want it to be serialized correctly to be able to ensure compatibility between scikit-learn and your Deep Learning framework when it comes time to save or parallelize things and so forth. You can read how Neuraxle solves this with savers here.
Code Examples
Here is a full project example from A to Z where TensorFlow is used with Neuraxle as if it was used with Scikit-Learn.
Here is another practical example where TensorFlow is used within a scikit-learn-like pipeline
I'm trying to understand how to read local images, use them as TensorFlow Dataset and train Keras model with TF Dataset. I'm following TF Keras MNIST TPU tutorial. The only difference that I want to read my set of images and train on them.
Let's say I have list of images (file names) and corresponding list of labels.
files = [...] # list of file names
labels = [...] # list of labels (integers)
images = tf.constant(files) # or tf.convert_to_tensor(files)
labels = tf.constant(labels) # or tf.convert_to_tensor(labels)
dataset = tf.data.Dataset.from_tensor_slices((images, labels))
dataset = dataset.shuffle(len(files))
dataset = dataset.repeat()
dataset = dataset.map(parse_function).batch(batch_size)
The parse_function is a simple function which reads the input file name and yields the image data and corresponding label, e.g.
def parse_function(filename, label):
image_string = tf.read_file(filename)
image_decoded = tf.image.decode_image(image_string)
image = tf.cast(image_decoded, tf.float32)
return image, label
At this point I have a dataset which is a tf.data.Dataset type (more precisely tf.data.BatchDataset) and I pass it along to keras model trained_model from tutorial, e.g.
history = trained_model.fit(dataset, ...)
But at this point code breaks with the following error:
AttributeError: 'BatchDataset' object has no attribute 'ndim'
The error comes from keras which performs the check on given input like that
from keras import backend as K
K.is_tensor(dataset) # which returns false
Keras tries to determine type of the input and since it is not a tensor it assumes it is numpy array and tries to get its dimension. That's why the error occurs.
My questions here are the following:
am I reading TF dataset correctly? I looked up plenty of examples on internet and it seems I'm reading it as people suggest
why my dataset is not a tensor? may be I need to perform additional conversion, but it is not the case of TF tutorial
why in TF tutorial everything works with tf datasets, I really don't see any difference from they way how they read MNIST data (which is in different data-format, but eventually they get images) and what I'm doing here.
Any suggestion would be greatly appreciated.
Please note, even TF tutorial is about TPUs it is structured such that it works on both TPUs and CPU/GPUs.
Turns out the problem was in using Keras model. The example in TF tutorial relies on Keras model build using tf.keras module (all layers, model, etc. came from tf.keras). While the model I was using (DenseNet) relies on pure keras module, i.e. all layers came from keras module and not from tf.keras. This cause the tf.data.Dataset to be checked for ndim in fit method of keras model. Once I adjusted my DenseNet to use tf.keras layers everything become working again.