I'm learning ML model training following this tutorial from Tensorflow. I have uploaded my own dataset from my computer to a folder named "sample_arrow" in Google Colab and specified the path to it:
image_path = 'sample_arrow'
The folder contains images, the size is not 0. But I get an error when executing this line of code:
data = DataLoader.from_folder(image_path) train_data, test_data = data.split(0.9)
ValueError: Image size is zero
What is wrong here? Maybe the folder path is not specified correctly? I'm completely new to the topic, unfamiliar with Pyhon (have Java skills) and would appreciate a detailed answer.
At last, I've found the solution.
Import os and the correct path definition were missing:
import os
root_path = "/content/"
image_path = os.path.join(os.path.dirname(root_path), 'sample_arrow')
Use the following code in google colab:
import tensorflow as tf
data_path = tf.keras.utils.get_file(
'flower_photos',
'https://storage.googleapis.com/download.tensorflow.org/example_images/flower_photos.tgz',
untar=True)
from tflite_model_maker import image_classifier
from tflite_model_maker.image_classifier import DataLoader
# Load input data specific to an on-device ML app.
data = DataLoader.from_folder(data_path)
train_data, test_data = data.split(0.9)
# Customize the TensorFlow model.
model = image_classifier.create(train_data)
# Evaluate the model.
loss, accuracy = model.evaluate(test_data)
# Export to Tensorflow Lite model and label file in `export_dir`.
model.export(export_dir='/tmp/')
So you need to download the data! took me a while to find the solution!!
Related
I'm trying to use Tensorflow Lite Image Searcher with mobilenet_v3 to query a database for a similar image and I'm getting surprisingly bad results.
Thus, I suspect there is a mistake in one of my steps.
Here is my code:
from tflite_model_maker import searcher
from tflite_support.task import vision
# Load pretrained model:
model_name = "lite-model_imagenet_mobilenet_v3_large_100_224_feature_vector_5_metadata_1.tflite"
data_loader = searcher.ImageDataLoader.create(model_name)
# Load db images, calc feature vectors
path_db = 'dir_with_db_jpg_images'
data_loader.load_from_folder(path_db)
# Set model:
scann_options = searcher.ScaNNOptions(
distance_measure="dot_product",
tree=searcher.Tree(num_leaves=10, num_leaves_to_search=2),
score_ah=searcher.ScoreAH(2, anisotropic_quantization_threshold=0.2))
model = searcher.Searcher.create_from_data(data_loader, scann_options)
# Export as tflite model:
model.export(
export_filename="searcher.tflite",
userinfo="",
export_format=searcher.ExportFormat.TFLITE)
# Load model:
image_searcher = vision.ImageSearcher.create_from_file("searcher.tflite")
# Predict NN for query image:
image = vision.TensorImage.create_from_file('path_query_img.jpg')
result = image_searcher.search(image)
result.nearest_neighbors
Do the bad results stem from missing preprocessing steps for the input images (DB or query)?
What happens to the images once load_from_folder(path) is called before the feature vector is created? And is this different from image_searcher.search(vision.TensorImage.create_from_file('path_query_img.jpg')) which might explain the bad results.
I couldn't figure out a way to first load the DB images and then feed them to the model for it to extract the feature vector and append it to the searcher model. Do such methods exist? That would allow me to experiment more with preprocessing steps (e.g. resizing the images to the 224x244 expected image size of the network - which I tried).
Maybe there is another problem with my code?
Thank you!
i have train detection model (CRAFT) for my custom dataset and in exp/custom_dataset folder i got weights ( train on google colab using GPU).when i am trying to use those train weights in my system (CPU) with EasyOCR i got errors like this
RuntimeError: Error(s) in loading state_dict for CRAFT:
Missing key(s) in state_dict: "basenet.slice1.0.weight", "basenet.slice1.0.bias", "basenet.slice1.1.weight" Etc
Unexpected key(s) in state_dict: "iter", "craft", "optimizer", "scaler".
for this i followed this github Link : https://github.com/JaidedAI/EasyOCR/blob/master/trainer/craft/README.md.
i have used the following code
import easyocr
import os
import pandas as pd
fol_path = "/home/satyam/PycharmProjects/PivotProject/NP_Images"
## reader = easyocr.Reader(['en'], model_storage_directory="/home/satyam/PycharmProjects/Custom_EasyOcr/model")
reader = easyocr.Reader(['en'])
i have make changes in config.py comment out their default weights & put mine
so please tell me how can i train EasyOCR CRAFT Detection model for custom dataset(for number plate) and use it in easyocr
I want to change this code to put my own dataset(from my drive it's an Arabic data with 2 classes) tried to read it normally but i get an error in 'train' .. what i want is to use the dataset that i have not the ones from huggingface.
!pip install openprompt
!git clone https://github.com/thunlp/OpenPrompt.git
!cd OpenPrompt
# load dataset
from datasets import load_dataset
# raw_dataset = load_dataset('super_glue', 'cb', cache_dir="../datasets/.cache/huggingface_datasets")
# raw_dataset['train'][0]
from datasets import load_from_disk
raw_dataset = load_from_disk("/home/hushengding/huggingface_datasets/saved_to_disk/super_glue.cb")
# Note that if you are running this scripts inside a GPU cluster, there are chances are you are not able to connect to huggingface website directly.
# In this case, we recommend you to run `raw_dataset = load_dataset(...)` on some machine that have internet connections.
# Then use `raw_dataset.save_to_disk(path)` method to save to local path.
# Thirdly upload the saved content into the machiine in cluster.
# Then use `load_from_disk` method to load the dataset.
from openprompt.data_utils import InputExample
dataset = {}
for split in ['train', 'validation', 'test']:
dataset[split] = []
for data in raw_dataset[split]:
input_example = InputExample(text_a = data['premise'], text_b = data['hypothesis'], label=int(data['label']), guid=data['idx'])
dataset[split].append(input_example)
print(dataset['train'][0])
The data should be in here:
from datasets import load_dataset
raw_dataset = load_dataset('csv', data_files='/content/drive/MyDrive/TEST2.csv')
I am running into this error , i can't unpickle a file on my jupyter notebook:
import os
import pickle
import joblib
import pandas as pd
from sklearn.preprocessing import MinMaxScaler
filename = open("loan_model3.pkl", "rb")
mdl = pickle.load(filename)
mdl.close()
and it always shows the below error message , even tho i'vce upgraded all my libraries
Error Message:
FileNotFoundError: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for ram://89506590-ec42-44a9-b67c-3ee4cc8e884e/variables/variables You may be trying to load on a different device from the computational device. Consider setting the experimental_io_deviceoption intf.saved_model.LoadOptions to the io_device such as '/job:localhost'.
I tried to upgrade my libraries but still didn't work.
I got the same error too when I was trying to store my Sequential model in .pkl file, since Sequential model is a TensorFlow Keras model so we have to store it in .h5 file and Keras saves models in this format as it can easily store the weights and model configuration in a single file.
Code:
from keras.models import load_model
model.save('model.h5')
model_final = load_model('model.h5')
Idk if you are still here but I found the solution. basically you should not save the tensorflow model into a pickle file but instead into h5 file
## save model
save_path = './model.h5'
model.save(save_path)
## load tensorflow model
model = keras.models.load_model(save_path)
This worked for me. Hope this helps you too.
this worked for me:
import tensorflow as tf
path = './model.h5'
model.save(path )
loaded_model= tf.keras.models.load_model(path )
I have faced the same issue, but by saving the model as .h5 file worked for me. Now i'm able to load .h5 model.
I am working on a Image Claasification TF Lite model to detect mask or no mask from human faces using this link. I followed the link and trained an image multi class classification in vertex AI and downloaded the TF lite model. The labels of the model are "mask" and "no_mask". In order to test the model, I wrote the following code:
interpret= tf.lite.Interpreter(model_path="<FILE_PATH>")
input= interpret.get_input_details()
output= interpret.get_output_details()
interpret.allocate_tensors()
pprint(input)
pprint(output)
data= cv2.imread("file.jpeg")
new_image= cv2.resize(data,(224,224))
interpret.resize_tensor_input(input[0]["index"],[1,224,224,3])
interpret.allocate_tensors()
interpret.set_tensor(input[0]["index"],[new_image])
interpret.invoke()
result= interpret.get_tensor(output[0]['index'])
print (" Prediction is - {}".format(result))
Using this code for one of my image is giving me the result as :
[[30 246]]
Now I want to print the label in the result as well. For example:
mask: 30
no_mask: 46
Is there any way I can implement this?
Please help as I am new to TF Lite
I solved it myself. The .tflite model downloaded from Vertex AI contains the label file called 'dict.txt' that contains all the labels. Check the GCP doc here. To get this label file we first need to unzip the .tflite file which will give us the dict.txt. For more information, check out the tflite documentation and how to read associate file from the models.
After that I executed the following code taking reference from the github link label.py:
import argparse
import time
import numpy as np
from PIL import Image
import tensorflow as tf
interpret= tf.lite.Interpreter(model_path="<FILE_PATH>")
input= interpret.get_input_details()
output= interpret.get_output_details()
interpret.allocate_tensors()
pprint(input)
pprint(output)
data= cv2.imread("file.jpeg")
new_image= cv2.resize(data,(224,224))
interpret.resize_tensor_input(input[0]["index"],[1,224,224,3])
interpret.allocate_tensors()
interpret.set_tensor(input[0]["index"],[new_image])
interpret.invoke()
floating_model= input[0]['dtype'] == np.float32
op_data= interpret.get_tensor(output[0]['index'])
result= np.squeeze(op_data)
top_k=result.agrsort()[-5:][::1]
labels=load_labels("dict.txt")
for i in top_k:
if floating_model:
print('{:08.6f}: {}'.format(float(result[i]), labels[i]))
else:
print('{:08.6f}: {}'.format(float(result[i] / 255.0), labels[i]))