I have transformed MNIST images saved as .pt files in a folder in Google drive. I'm writing my Pytorch code in Colab.
I would like to use these files, and create a Dataset that stores these images as Tensors. How can I do this?
Transforming images during training took too long. Hence, transformed them and saved them all as .pt files. I just want to load them back as a dataset and use them in my model.
The approach you are following to save images is indeed a good idea. In such a case, you can simply write your own Dataset class to load the images.
from torch.utils.data import Dataset, DataLoader
from torch.utils.data.sampler import RandomSampler
class ReaderDataset(Dataset):
def __init__(self, filename):
# load the images from file
def __len__(self):
# return total dataset size
def __getitem__(self, index):
# write your code to return each batch element
Then you can create Dataloader as follows.
train_dataset = ReaderDataset(filepath)
train_sampler = RandomSampler(train_dataset)
train_loader = DataLoader(
train_dataset,
batch_size=args.batch_size,
sampler=train_sampler,
num_workers=args.data_workers,
collate_fn=batchify,
pin_memory=args.cuda,
drop_last=args.parallel
)
# args is a dictionary containing parameters
# batchify is a custom function that prepares each mini-batch
Related
I've trained a Tensorflow Lite (TFLite) model saved as a *.tflite file.
I'm writing code that lets me pick a tflite file, and a folder containing images, and then runs inference on this images using that model.
Here is what I have written:
def testModel(self, testData):
#Test any model on any dataset
model = "**path to model file**"
#Loading TFLite model and allocating tensors.
interpreter = tf.lite.Interpreter(model_path=model)
interpreter.allocate_tensors()
# Get input and output tensors.
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
rawImg = "**path to test images folder**"
imgNameList = glob.glob(os.path.join(os.getcwd(), rawImg) + os.sep + '*') # gets list of image names in dir
#creates dataset and dataloader from images
testDataset = SalObjDataset(img_name_list = imgNameList,lbl_name_list = [], transform=transforms.Compose([RescaleT(224),ToTensorLab(flag=0)]))
testDataloader = DataLoader(testDataset,batch_size=1,shuffle=False)
#loops through dataloader (goes through each image file)
for _, data in enumerate(testDataloader):
inputImg = data['image']
if torch.cuda.is_available():
inputImg = Variable(inputImg.cuda())
else:
inputImg = Variable(inputImg)
#rearranges dimensions in image file to match the expected input dimensions
#also changes the type to uint8 as expected
inputImg = tf.transpose(inputImg.cpu(), perm = [0,2,3,1])
inputImg = tf.cast(inputImg, tf.uint8)
interpreter.set_tensor(input_details[0]['index'], inputImg)
interpreter.invoke()
output_data = interpreter.get_tensor_details()
print(output_data)
if __name__ == '__main__':
#initialise object with the modelID of the model you want to test
#pass the testing data folder name to testModel()
#this is the folder where the model is
modelID = "model_1"
tester = ModelTrainer(modelID)
#this is the folder where the testing images are
tester.testModel("model_1/model_1/plant")
The way our it's setup, the images for each label are stored in their own subdirectory, so all images of a 'plant' would be in folder/plant/image-1.jpg.
I'm not sure if I'm using 'interpreter.set_tensor' correctly, I've gone through the documentation quite intensively and I'm still a bit confused.
I'm also not sure how to make sense of the output, I would like to somehow get a loss/accuracy value, how do I go about doing this?
My output is currently just [[255]] for each image.
Thanks!
I'm assuming you have trained a object detection Have you added necessary metadata needed for interpreter to get the Outputs according to image given below Outputs
if not make sure to use metadata writer API
use this notebook for writing metadata in which pass the labels and model which
does not have any metadata in it
https://colab.research.google.com/github/tensorflow/tensorflow/blob/master/tensorflow/lite/g3doc/models/convert/metadata_writer_tutorial.ipynb
TensorFlow Lite Metadata Writer API provides an easy-to-use API to create Model Metadata for popular ML tasks supported by the TFLite Task Library. This notebook shows examples on how the metadata should be populated for the following tasks below:
Image classifiers
Object detectors
Image segmenters
Natural language classifiers
Audio classifiers
https://www.tensorflow.org/lite/models/convert/metadata_writer_tutorial
I am working on an image recognition project using TensorFlow and Keras, that I would like to implement to my Android project. And I am new to Tensorflow...
I would like to find the closest match between an image to a folder with +2000 images. Images are similar in background and size, like so:
For now I have this following Python code that works okay.
from tensorflow.keras.preprocessing import image
from tensorflow.keras.applications.vgg16 import VGG16, preprocess_input
from tensorflow.keras.models import Model
import numpy as np
from PIL import Image
base_model = VGG16(weights='imagenet')
model = Model(inputs=base_model.input, outputs=base_model.get_layer('fc1').output)
def extract(img):
img = img.resize((224, 224)) # Resize the image
img = img.convert('RGB') # Convert the image color space
x = image.img_to_array(img) # Reformat the image
x = np.expand_dims(x, axis=0)
x = preprocess_input(x)
feature = model.predict(x)[0] # Extract Features
return feature / np.linalg.norm(feature)
# Iterate through images and extract Features
images = ["img1.png","img2.png","img3.png","img4.png","img5.png"...+2000 more]
all_features = np.zeros(shape=(len(images),4096))
for i in range(len(images)):
feature = extract(img=Image.open(images[i]))
all_features[i] = np.array(feature)
# Match image
query = extract(img=Image.open("image_to_match.png")) # Extract its features
dists = np.linalg.norm(all_features - query, axis=1) # Calculate the similarity (distance) between images
ids = np.argsort(dists)[:5] # Extract 5 images that have lowest distance
Now I am a bit lost to where to go from here. To my understanding I need to create a .h5 file with all extracted image features and a .tflite file containing the model.
UPDATE after answer
I can convert the model with:
# Convert the model.
base_model = VGG16(weights='imagenet')
model = Model(inputs=base_model.input, outputs=base_model.get_layer('fc1').output)
converter = tf.lite.TFLiteConverter.from_keras_model(model)
tflite_model = converter.convert()
# Save the model.
with open('model.tflite', 'wb') as f:
f.write(tflite_model)
But how can I get the extracted features to my Android project? Also, the file size of the model is +400 mb so Android doesnt allow to import it.
Hope you can help me, thanks.
From Tensorflows own site:
# Convert the model.
converter = tf.lite.TFLiteConverter.from_keras_model(model)
tflite_model = converter.convert()
# Save the model.
with open('model.tflite', 'wb') as f:
f.write(tflite_model)
Please find the below diagram for better understanding of the conversion process. The TensorFlow Lite converter takes a tensorflow/keras model and generates a tensoflow lite (.tflite) model. Even though there is a command line way of converting the model (https://www.tensorflow.org/lite/convert/index#cmdline) you are recommended to use the Python API to do the same, because it allows to add metadata(if required) and apply optimizations to the model. The following steps lets you convert your keras model to tflite.
# Convert the model.
converter = tf.lite.TFLiteConverter.from_keras_model(model)
tflite_model = converter.convert()
# Save the model.
with open('model.tflite', 'wb') as f:
f.write(tflite_model)
Many a times when you are dealing with bigger models, your model size will be much bigger than the allowed size and might not perform as good as you want. So you have apply optimizations to make the model work. This is done using tf.lite.Optimize. It allows you to optimize your model for speed, storage etc. before tensorflow allowed a lot of manual control where you were able to specify what you need to optimize upon using tf.lite.Optimize.OPTIMIZE_FOR_LATENCY or tf.lite.Optimize.OPTIMIZE_FOR_SIZE nowadays default comes with both these optimizations. Now the conversion code becomes like this.
# Convert the model.
converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.optimizations = [tf.lite.Optimize.DEFAULT] #optimization
tflite_quant_model = converter.convert()
# Save the model.
with open('model.tflite', 'wb') as f:
f.write(tflite_quant_model)
What this does is a dynamic range quantization. Check the size of your model after this step.
If you want to further quantize the model, for example convert all float32 to float16 which will reduce the model size to approx. half the size as original, you can do it specifying a target spec. then your code will look like this. (understand that this will affect the model accuracy)
# Convert the model.
converter = tf.lite.TFLiteConverter.from_keras_model(model)
converter.optimizations = [tf.lite.Optimize.DEFAULT] #optimization
converter.target_spec.supported_types = [tf.float16] #target spec
tflite_fp16_model = converter.convert()
tflite_model_fp16_file = tflite_models_dir/"model_quant_f16.tflite"
tflite_model_fp16_file.write_bytes(tflite_fp16_model)
There are other types of post training quantizations also, which you can find in this page.
https://www.tensorflow.org/lite/performance/post_training_quantization
All this is post training quantizations, you may quantize the model before that also. refer to this link for the same, you can also find a lot of tutorials via search https://www.tensorflow.org/api_docs/python/tf/quantization/
After this you will have to run these tflite models to test them using python.
the steps are as below.
Load the model onto interpreters.
Test the model with sample images(s)
Evaluate the model
you can find a detailed example on this page
https://www.tensorflow.org/lite/performance/post_training_float16_quant
there are many other types of quantization as well based on the precision required and the type of edge device you are going to use. Please refer to the below link for details on how to apply them to your model.
https://www.tensorflow.org/lite/performance/model_optimization.
After quantization check your model size, this should reduce the model size to the required value if not repeat the operation will lower precisions.
You have 2 options:
Post-training quantization
import tensorflow as tf
converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
tflite_quant_model = converter.convert()
Selective builds
TensorFlow Lite enables you to reduce model binary sizes by using selective builds (Mobilenet). It says:
Selective builds skip unused operations in your model set and produce
a compact library with just the runtime and the op kernels required
for the model to run on your mobile device.
I am trying to train a segmentation algorithm with FastAi. I have training and validation data in separate folders, so was planning on using GrandparentSplitter() but for some reason the validation set is empty.
My files are organised as below:
Path ---> train ---> images
---> masks
---> valid ---> images
---> masks
And this is how I set up my datablock and dataloader:
codes = np.array(['background', 'prostate'])
def label_func(x): return path/'train/masks'/f'{x.stem}_mask.png'
db = DataBlock(blocks=(ImageBlock(), MaskBlock(codes)),
splitter=GrandparentSplitter(train_name='train', valid_name='valid'),
get_items=get_image_files,
get_y=label_func)
dls = db.dataloaders(path/'train/images', bs=1)
dls.show_batch()
I am assuming there is something wrong with how I organised the files.
i could not make it work with the grandparent
just implement my own get_files
I am working on a project with Tensorflow federated. I have managed to use the libraries provided by TensorFlow Federated Learning simulations in order to load, train, and test some datasets.
For example, i load the emnist dataset
emnist_train, emnist_test = tff.simulation.datasets.emnist.load_data()
and it got the data sets returned by load_data() as instances of tff.simulation.ClientData. This is an interface that allows me to iterate over client ids and allow me to select subsets of the data for simulations.
len(emnist_train.client_ids)
3383
emnist_train.element_type_structure
OrderedDict([('pixels', TensorSpec(shape=(28, 28), dtype=tf.float32, name=None)), ('label', TensorSpec(shape=(), dtype=tf.int32, name=None))])
example_dataset = emnist_train.create_tf_dataset_for_client(
emnist_train.client_ids[0])
I am trying to load the fashion_mnist dataset with Keras to perform some federated operations:
fashion_train,fashion_test=tf.keras.datasets.fashion_mnist.load_data()
but I get this error
AttributeError: 'tuple' object has no attribute 'element_spec'
because Keras returns a Tuple of Numpy arrays instead of a tff.simulation.ClientData like before:
def tff_model_fn() -> tff.learning.Model:
return tff.learning.from_keras_model(
keras_model=factory.retrieve_model(True),
input_spec=fashion_test.element_spec,
loss=loss_builder(),
metrics=metrics_builder())
iterative_process = tff.learning.build_federated_averaging_process(
tff_model_fn, Parameters.server_adam_optimizer_fn, Parameters.client_adam_optimizer_fn)
server_state = iterative_process.initialize()
To sum up,
Is any way to create tuple elements of tff.simulation.ClientData from Keras Tuple Numpy arrays?
Another solution that comes to my mind is to use the
tff.simulation.HDF5ClientData and load
manually the appropriate files in aHDF5format (train.h5, test.h5) in order to get the tff.simulation.ClientData, but my problem is that i cant find the url for fashion_mnist HDF5 file format i mean something like that for both train and test:
fileprefix = 'fed_emnist_digitsonly'
sha256 = '55333deb8546765427c385710ca5e7301e16f4ed8b60c1dc5ae224b42bd5b14b'
filename = fileprefix + '.tar.bz2'
path = tf.keras.utils.get_file(
filename,
origin='https://storage.googleapis.com/tff-datasets-public/' + filename,
file_hash=sha256,
hash_algorithm='sha256',
extract=True,
archive_format='tar',
cache_dir=cache_dir)
dir_path = os.path.dirname(path)
train_client_data = hdf5_client_data.HDF5ClientData(
os.path.join(dir_path, fileprefix + '_train.h5'))
test_client_data = hdf5_client_data.HDF5ClientData(
os.path.join(dir_path, fileprefix + '_test.h5'))
return train_client_data, test_client_data
My final goal is to make the fashion_mnist dataset work with the TensorFlow federated learning.
You're on the right track. To recap: the datasets returned by tff.simulation.dataset APIs are tff.simulation.ClientData objects. The object returned by tf.keras.datasets.fashion_mnist.load_data is a tuple of numpy arrays.
So what is needed is to implement a tff.simulation.ClientData to wrap the dataset returned by tf.keras.datasets.fashion_mnist.load_data. Some previous questions about implementing ClientData objects:
Federated learning : convert my own image dataset into tff simulation Clientdata
How define tff.simulation.ClientData.from_clients_and_fn Function?
Is there a reasonable way to create tff clients datat sets?
This does require answering an important question: how should the Fashion MNIST data be split into individual users? The dataset doesn't include features that that could be used for partitioning. Researchers have come up with a few ways to synthetically partition the data, e.g. randomly sampling some labels for each participant, but this will have a great effect on model training and is useful to invest some thought here.
I have a standard directory structure of train, validation, test, and each contain class subdirectories.
...
|train
|class A
|1
|1_1.raw
|1_2.raw
...
|2
...
|class B
...
|test
...
I want to use the flow_from_directory API, but all I can find is an ImageDataGenerator, and the files I have are raw numpy arrays (generated with arr.tofile(...)).
Is there an easy way to use ImageDataGenerator with a custom file loader?
I'm aware of flow_from_dataframe, but that doesn't seem to accomplish what I want either; it's for reading images with more custom organization. I want a simple way to load raw binary files instead of having to re-encode 100,000s of files into jpgs with some precision loss along the way (and wasted time, etc.).
Tensorflow is an entire ecosystem with IO capabilities and ImageDataGenerator is one of the least flexible approaches. Read here on How to Load Numpy Data in Tensorflow.
import tensorflow as tf
import numpy as np
DATA_URL = 'https://storage.googleapis.com/tensorflow/tf-keras-datasets/mnist.npz'
path = tf.keras.utils.get_file('mnist.npz', DATA_URL)
with np.load(path) as data:
train_examples = data['x_train']
train_labels = data['y_train']
test_examples = data['x_test']
test_labels = data['y_test']
train_dataset = tf.data.Dataset.from_tensor_slices((train_examples, train_labels))
test_dataset = tf.data.Dataset.from_tensor_slices((test_examples, test_labels))