Related
I'm using canva/konva.js to allow users to draw digits frontend and sending them to backend to do some image detection. I'm using EMNIST because I'm allowing all letters but my detection was very very very bad...
I originally thought it might be the data I was getting, but MNIST turns out to not work with it either.
Here's my detection code in python:
from extra_keras_datasets import emnist
(input_train, target_train), (input_test, target_test) = emnist.load_data(type='balanced')
input_train = tf.keras.utils.normalize(input_train, axis = 1)
input_test = tf.keras.utils.normalize(input_test, axis = 1)
def train():
model = tf.keras.models.Sequential()
model.add(tf.keras.layers.Flatten(input_shape=(28, 28)))
model.add(tf.keras.layers.Dense(256, activation='relu'))
model.add(tf.keras.layers.Dense(256, activation='relu'))
model.add(tf.keras.layers.Dense(62, activation='softmax'))
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
model.fit(input_train, target_train, epochs = 20)
Here's my canva code (using konva.js) :
<Stage
width={window.innerWidth/4}
height={window.innerHeight/3}
onMouseDown={handleMouseDown}
onMousemove={handleMouseMove}
onMouseup={handleMouseUp}
onTouchStart={handleTouchStart}
onTouchMove={handleTouchMove}
onTouchEnd={handleTouchEnd}
id="stage"
ref={stageRef}
className="stage" >
<Layer>
<Rect
x={0}
y={0}
width={window.innerWidth}
height={window.innerHeight}
fill="white"
shadowBlur={0} />
{lines.map((line, i) => (
<Line
key={i}
points={line.points}
stroke="#000000"
strokeWidth={4}
tension={0.5}
lineCap="round"
lineJoin="round"
globalCompositeOperation={
line.tool === 'eraser' ? 'destination-out' : 'source-over'
}
/>
))}
</Layer>
</Stage>
Here's how I handle my images:
I did this because what I get is a black text on white background image.
I remove the whitespace and rotate it and invert the colour (because that's how it seems to look like).
I've tried making my strokes thicker in the canvas but it doesn't seem to work.
I'm just wondering how I can process my image/set up canvas so that it generates an image that EMNIST can detect?
img = opencv.remove_whitespace(path)
img = cv2.imread(path)[:, :, 0]
img = cv2.resize(img, (28, 28))
img = cv2.bitwise_not(img)
# img = cv2.rotate(img, cv2.ROTATE_90_COUNTERCLOCKWISE)
# img = cv2.flip(img, 0)
cv2.imshow("img", img)
cv2.waitKey(0)
img = np.array([img])
model = tf.keras.models.load_model('model.model')
prediction = model.predict(img)
print("The predicted value is : ", prediction)
classes = np.argmax(prediction,axis=1)
print("Predicted class: ", classes)
I have an image classification problem for which I'd like to combine different models
I've trained.
for example I have two models:
mobilenet_v2_100_96 = tf.keras.models.load_model("saved_model_mobilenet_v2_100_96")
mobilenet_v2_100_224 = tf.keras.models.load_model("saved_model_mobilenet_v2_100_224")
After which I lock the layers of the models and combine them into an array called "models" and create a new model to combine these like so:
ensemble_visible = [model.input for model in models]
ensemble_outputs = [model.output for model in models]
merge = tf.keras.layers.concatenate(ensemble_outputs)
merge = tf.keras.layers.Dense(200, activation='relu')(merge)
output = tf.keras.layers.Dense(200, activation='sigmoid')(merge)
model = tf.keras.models.Model(inputs=ensemble_visible, outputs=ensemble_outputs)
model.compile(optimizer=tf.keras.optimizers.SGD(learning_rate=0.005, momentum=0.9),
loss=tf.keras.losses.CategoricalCrossentropy(from_logits=True, label_smoothing=0.1),
metrics=['accuracy'])
As you can see I need two differently sized inputs in order to make it work.
I obtain my two datasets through the tf.keras.preprocessing.image_dataset_from_directory function.
Now I need a way to combine these two datasets. I made sure to set the seed value of both calls the same, so the images should be in the same order.
I've tried splitting the datasets via this tf_unzip (https://stackoverflow.com/a/68661507/20617581) function. And combining them like this:
model_inputs = tf.data.Dataset.zip(({"inputmobilenet_v2_100_96":train_x_96, "inputmobilenet_v2_100_224":train_x_224}, train_y_96))
When I run the model.fit function however the model is not able to learn above 1% which is way less than the original models. If I run the same model using only one input everything works as expected.
My complete code is following:
import tensorflow as tf
from utils.model import Model
from tqdm import tqdm
import numpy as np
import pandas as pd
import tensorflow_datasets as tfds
def tfdata_unzip(
tfdata: tf.data.Dataset,
*,
recursive: bool=False,
eager_numpy: bool=False,
num_parallel_calls: int=tf.data.AUTOTUNE,
):
"""
Unzip a zipped tf.data pipeline.
Args:
tfdata: the :py:class:`tf.data.Dataset`
to unzip.
recursive: Set to ``True`` to recursively unzip
multiple layers of zipped pipelines.
Defaults to ``False``.
eager_numpy: Set this to ``True`` to return
Python lists of primitive types or
:py:class:`numpy.array` objects. Defaults
to ``False``.
num_parallel_calls: The level of parallelism to
each time we ``map()`` over a
:py:class:`tf.data.Dataset`.
Returns:
Returns a Python list of either
:py:class:`tf.data.Dataset` or NumPy
arrays.
"""
if isinstance(tfdata.element_spec, tf.TensorSpec):
if eager_numpy:
return list(tfdata.as_numpy_iterator())
return tfdata
def tfdata_map(i: int) -> list:
return tfdata.map(
lambda *cols: cols[i],
deterministic=True,
num_parallel_calls=num_parallel_calls,
)
if isinstance(tfdata.element_spec, tuple):
num_columns = len(tfdata.element_spec)
if recursive:
return [
tfdata_unzip(
tfdata_map(i),
recursive=recursive,
eager_numpy=eager_numpy,
num_parallel_calls=num_parallel_calls,
)
for i in range(num_columns)
]
else:
return [
tfdata_map(i)
for i in range(num_columns)
]
raise ValueError(
"Unknown tf.data.Dataset element_spec: " +
str(tfdata.element_spec)
)
print("GPU is", "available" if tf.config.list_physical_devices('GPU') else "NOT AVAILABLE")
models = []
# mobilenet_v2_050_160 = tf.keras.models.load_model("saved_model_mobilenet_v2_050_160")
mobilenet_v2_100_96 = tf.keras.models.load_model("saved_model_mobilenet_v2_100_96")
mobilenet_v2_100_224 = tf.keras.models.load_model("saved_model_mobilenet_v2_100_224")
# models.append(mobilenet_v2_050_160)
models.append(mobilenet_v2_100_96)
models.append(mobilenet_v2_100_224)
for i, model in enumerate(models):
for layer in model.layers:
layer.trainable = False
layer._name = layer.name + str(i)
model.input._name = "input_" + str(i)
model.input.type_spec._name = "input_" + str(i)
# model.summary()
ensemble_visible = [model.input for model in models]
ensemble_outputs = [model.output for model in models]
merge = tf.keras.layers.concatenate(ensemble_outputs)
merge = tf.keras.layers.Dense(200, activation='relu')(merge)
output = tf.keras.layers.Dense(200, activation='sigmoid')(merge)
model = tf.keras.models.Model(inputs=ensemble_visible, outputs=output)
model.compile(optimizer=tf.keras.optimizers.SGD(learning_rate=0.005, momentum=0.9),
loss=tf.keras.losses.CategoricalCrossentropy(from_logits=True, label_smoothing=0.1),
metrics=['accuracy'])
def build_dataset(image_size):
return tf.keras.preprocessing.image_dataset_from_directory(
"img",
validation_split=.20,
subset="both",
label_mode="categorical",
# Seed needs to provided when using validation_split and shuffle = True.
# A fixed seed is used so that the validation set is stable across runs.
seed=123,
image_size=image_size,
batch_size=16
)
batch_size = 16
def gen_datasets(image_size):
train_ds, val_ds = build_dataset(image_size)
class_names = tuple(train_ds.class_names)
train_size = train_ds.cardinality().numpy()
train_ds = train_ds.unbatch().batch(batch_size)
train_ds = train_ds.repeat()
normalization_layer = tf.keras.layers.Rescaling(1. / 255)
preprocessing_model = tf.keras.Sequential([normalization_layer])
do_data_augmentation = False # #param {type:"boolean"}
if do_data_augmentation:
preprocessing_model.add(tf.keras.layers.RandomRotation(40))
preprocessing_model.add(tf.keras.layers.RandomTranslation(0, 0.2))
preprocessing_model.add(tf.keras.layers.RandomTranslation(0.2, 0))
preprocessing_model.add(tf.keras.layers.RandomZoom(0.2, 0.2))
preprocessing_model.add(tf.keras.layers.RandomFlip(mode="horizontal"))
train_ds = train_ds.map(lambda images, labels: (preprocessing_model(images), labels))
valid_size = val_ds.cardinality().numpy()
val_ds = val_ds.unbatch().batch(batch_size) #self.batch_size)
val_ds = val_ds.map(lambda images, labels:
(normalization_layer(images), labels))
return train_ds, val_ds, train_size, valid_size, class_names
train_ds_96, val_ds_96, train_size, valid_size, class_names = gen_datasets([96,96])
train_ds_224, val_ds_224, train_size, valid_size, class_names = gen_datasets([224,224])
print("aquired data")
train_x_96, train_y_96 = tfdata_unzip(train_ds_96)
train_x_224, train_y_224 = tfdata_unzip(train_ds_224)
val_x_96, val_y_96 = tfdata_unzip(val_ds_96)
val_x_224, val_y_224 = tfdata_unzip(val_ds_224)
model.summary()
model_inputs = tf.data.Dataset.zip(({"inputmobilenet_v2_100_96":train_x_96, "inputmobilenet_v2_100_224":train_x_224}, train_y_96))
model_vals = tf.data.Dataset.zip(({"inputmobilenet_v2_100_96":val_x_96, "inputmobilenet_v2_100_224":val_x_224}, val_y_96))
steps_per_epoch = train_size // batch_size
validation_steps = valid_size // batch_size
hist = model.fit(
model_inputs,
epochs=1,
steps_per_epoch=steps_per_epoch,
validation_data=model_vals,
validation_steps=validation_steps).history
model.save("joined_model")
What i've tried so far
I've tried:
splitting the dataset with:
def fit_generator(dataset,len):
df = tfds.as_numpy(dataset)
X_ret = np.array([])
Y_ret = np.array([])
for a,b in tqdm(df, total=len):
np.append(X_ret,a)
np.append(Y_ret,b)
return X_ret, Y_ret
which was to slow to handle the amount of data.
I've tried using a generator, but it was not accepted by the model.fit function like this:
def fit_generator(dataset,len):
df = tfds.as_numpy(dataset)
X_ret = np.array(e[0] for e in df)
Y_ret = np.array(e[1] for e in df)
return X_ret, Y_ret
Lastly i've tried the answer provided in the problem description tf_unzip (https://stackoverflow.com/a/68661507/20617581)
But the model does not learn in any significant way.
I am following this tutorial to build a simple deep learning app for traffic sign recognition. link
I made an own model, and I also tried it with the model in this repository: link
When I run the app from xcode on my iPhone, I can see the picture of the camera, but the text always says "Label", no matter what is on the screen.
The only thing I modified from the tutorial is that I hardcoded the classes before converting to mlmodel:
# import necessary packages
from keras.models import load_model
import coremltools
import argparse
import pickle
# construct the argument parser and parse the arguments
# load the class labels
print("[INFO] loading class labels from label binarizer")
# lb = pickle.loads(open(args["labelbin"], "rb").read())
# class_labels = lb.classes_.tolist()
class_labels = list(range(1, 43))
print("[INFO] class labels: {}".format(class_labels))
# load the trained convolutional neural network
print("[INFO] loading model...")
model = load_model('my_model.h5')
# convert the model to coreml format
print("[INFO] converting model")
coreml_model = coremltools.converters.keras.convert(model,
input_names="image",
image_input_names="image",
image_scale=1/255.0,
class_labels=class_labels,
is_bgr=True)
# save the model to disk
output = "mymodel.mlmodel"
print("[INFO] saving model as {}".format(output))
coreml_model.save(output)
So instead of using laber binarizer, I told the converter that there are 43 classes in my model.
Here is my AppDelegate.swift:
//
// AppDelegate.swift
// trafficsign
//
// Created by administrator on 2020. 11. 11..
// Copyright © 2020. administrator. All rights reserved.
//
import UIKit
#UIApplicationMain
class AppDelegate: UIResponder, UIApplicationDelegate {
var window: UIWindow?
func application(_ application: UIApplication, didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?) -> Bool {
// Override point for customization after application launch.
// Override point for customization after application launch.
window = UIWindow()
window?.makeKeyAndVisible()
let vc = ViewController()
window?.rootViewController = vc
return true
}
}
My SceneDelegate.swift:
//
// SceneDelegate.swift
// trafficsign
//
// Created by administrator on 2020. 11. 11..
// Copyright © 2020. administrator. All rights reserved.
//
import UIKit
class SceneDelegate: UIResponder, UIWindowSceneDelegate {
var window: UIWindow?
func scene(_ scene: UIScene, willConnectTo session: UISceneSession, options connectionOptions: UIScene.ConnectionOptions) {
// Use this method to optionally configure and attach the UIWindow `window` to the provided UIWindowScene `scene`.
// If using a storyboard, the `window` property will automatically be initialized and attached to the scene.
// This delegate does not imply the connecting scene or session are new (see `application:configurationForConnectingSceneSession` instead).
guard let windowScene = (scene as? UIWindowScene) else { return }
window = UIWindow(windowScene: windowScene)
window?.rootViewController = ViewController()
window?.makeKeyAndVisible()
}
func sceneDidDisconnect(_ scene: UIScene) {
// Called as the scene is being released by the system.
// This occurs shortly after the scene enters the background, or when its session is discarded.
// Release any resources associated with this scene that can be re-created the next time the scene connects.
// The scene may re-connect later, as its session was not neccessarily discarded (see `application:didDiscardSceneSessions` instead).
}
func sceneDidBecomeActive(_ scene: UIScene) {
// Called when the scene has moved from an inactive state to an active state.
// Use this method to restart any tasks that were paused (or not yet started) when the scene was inactive.
}
func sceneWillResignActive(_ scene: UIScene) {
// Called when the scene will move from an active state to an inactive state.
// This may occur due to temporary interruptions (ex. an incoming phone call).
}
func sceneWillEnterForeground(_ scene: UIScene) {
// Called as the scene transitions from the background to the foreground.
// Use this method to undo the changes made on entering the background.
}
func sceneDidEnterBackground(_ scene: UIScene) {
// Called as the scene transitions from the foreground to the background.
// Use this method to save data, release shared resources, and store enough scene-specific state information
// to restore the scene back to its current state.
}
}
And most importantly my SceneDelegate.swift:
//
// ViewController.swift
// trafficsign
//
// Created by administrator on 2020. 11. 11..
// Copyright © 2020. administrator. All rights reserved.
//
import UIKit
import AVFoundation
import Vision
class ViewController: UIViewController, AVCaptureVideoDataOutputSampleBufferDelegate {
let label: UILabel = {
let label = UILabel()
label.textColor = .white
label.translatesAutoresizingMaskIntoConstraints = false
label.text = "Label"
label.font = label.font.withSize(30)
return label
}()
override func viewDidLoad() {
super.viewDidLoad()
setupCaptureSession()
view.addSubview(label)
setupLabel()
}
override func didReceiveMemoryWarning() {
// call the parent function
super.didReceiveMemoryWarning()
// Dispose of any resources that can be recreated.
}
func setupCaptureSession() {
// create a new capture session
let captureSession = AVCaptureSession()
// find the available cameras
let availableDevices = AVCaptureDevice.DiscoverySession(deviceTypes: [.builtInWideAngleCamera], mediaType: AVMediaType.video, position: .back).devices
do {
// select a camera
if let captureDevice = availableDevices.first {
captureSession.addInput(try AVCaptureDeviceInput(device: captureDevice))
}
} catch {
// print an error if the camera is not available
print(error.localizedDescription)
}
// setup the video output to the screen and add output to our capture session
let captureOutput = AVCaptureVideoDataOutput()
captureSession.addOutput(captureOutput)
let previewLayer = AVCaptureVideoPreviewLayer(session: captureSession)
previewLayer.frame = view.frame
view.layer.addSublayer(previewLayer)
// buffer the video and start the capture session
captureOutput.setSampleBufferDelegate(self, queue: DispatchQueue(label: "videoQueue"))
captureSession.startRunning()
// // creates a new capture session
// let captureSession = AVCaptureSession()
//
// // search for available capture devices
// let availableDevices = AVCaptureDevice.DiscoverySession(deviceTypes: [.builtInWideAngleCamera], mediaType: AVMediaType.video, position: .back).devices
//
// // get capture device, add device input to capture session
// do {
// if let captureDevice = availableDevices.first {
// captureSession.addInput(try AVCaptureDeviceInput(device: captureDevice))
// }
// } catch {
// print(error.localizedDescription)
// }
//
// // setup output, add output to capture session
// let captureOutput = AVCaptureVideoDataOutput()
// captureSession.addOutput(captureOutput)
//
// captureOutput.setSampleBufferDelegate(self, queue: DispatchQueue(label: "videoQueue"))
//
// let previewLayer = AVCaptureVideoPreviewLayer(session: captureSession)
// previewLayer.frame = view.frame
// previewLayer.videoGravity = .resizeAspectFill
// view.layer.addSublayer(previewLayer)
//
// captureSession.startRunning()
}
// called everytime a frame is captured
func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {
// load our CoreML Pokedex model
guard let model = try? VNCoreMLModel(for: model_squeezeNet_TSR().model) else { return }
// run an inference with CoreML
let request = VNCoreMLRequest(model: model) { (finishedRequest, error) in
// grab the inference results
guard let results = finishedRequest.results as? [VNClassificationObservation] else { return }
// grab the highest confidence result
guard let Observation = results.first else { return }
// create the label text components
let predclass = "\(Observation.identifier)"
let predconfidence = String(format: "%.02f%", Observation.confidence * 100)
// set the label text
DispatchQueue.main.async(execute: {
self.label.text = "\(predclass) \(predconfidence)"
})
}
// create a Core Video pixel buffer which is an image buffer that holds pixels in main memory
// Applications generating frames, compressing or decompressing video, or using Core Image
// can all make use of Core Video pixel buffers
guard let pixelBuffer: CVPixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) else { return }
// execute the request
try? VNImageRequestHandler(cvPixelBuffer: pixelBuffer, options: [:]).perform([request])
// guard let model = try? VNCoreMLModel(for: model_squeezeNet_TSR().model) else { return }
// let request = VNCoreMLRequest(model: model) { (finishedRequest, error) in
// guard let results = finishedRequest.results as? [VNClassificationObservation] else { return }
// guard let Observation = results.first else { return }
//
// DispatchQueue.main.async(execute: {
// self.label.text = "\(Observation.identifier)"
// print(Observation.confidence)
// })
// }
// guard let pixelBuffer: CVPixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) else { return }
// // executes request
// try? VNImageRequestHandler(cvPixelBuffer: pixelBuffer, options: [:]).perform([request])
}
func setupLabel() {
label.centerXAnchor.constraint(equalTo: view.centerXAnchor).isActive = true
label.bottomAnchor.constraint(equalTo: view.bottomAnchor, constant: -50).isActive = true
}
}
I don't know if this fixes it, but in your conversion script try the following:
class_labels = list(range(1, 43))
class_labels = [str(x) for x in class_labels] # add this line
Currently your class labels are integers. It's possible this confuses Core ML or Vision at some point.
So I added a label binarizer to my model generator script. I used the original coremlconverter script, which gets the labels from the label binarizer. When I run it, it also lists the 42 labels, so it gets it.
But when I run the app, it still shows inconsistent things. Not the "Label" text, but random values, no matter what is on the camera.
here is a video: link
Here is my model creating python script (Those comments that are in hungarian are not important, those are obvious for those who know python):
#!/usr/bin/env python
# coding: utf-8
# In[1]:
pip install tensorflow==1.15.0 keras==2.2.4 scikit-learn==0.19.2 matplotlib pandas pillow opencv-python
# In[9]:
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import cv2
import pickle
#import tensorflow as tf
from PIL import Image
import os
from sklearn.model_selection import train_test_split
from keras.utils import to_categorical
from keras.models import Sequential, load_model
from keras.layers import Conv2D, MaxPool2D, Dense, Flatten, Dropout
from sklearn.preprocessing import LabelBinarizer
data = []
labels = []
classes = 43
cur_path = os.getcwd()
#Képek és címkék betöltése
for i in range(classes):
path = os.path.join(cur_path,'train',str(i))
images = os.listdir(path)
for a in images:
try:
image = Image.open(path + '\\'+ a)
image = image.resize((30,30))
image = np.array(image)
#sim = Image.fromarray(image)
data.append(image)
labels.append(i)
except:
print("Error loading image")
#Listák numpy tömbökbe konvertálása
data = np.array(data)
labels = np.array(labels)
#these two lines are commented when the script is used without label binarizer
lb = LabelBinarizer()
labels = lb.fit_transform(labels)
# In[10]:
print(data.shape, labels.shape)
#Az adathalmaz szétválasztása teszt és edző adathalmazokra
X_train, X_test, y_train, y_test = train_test_split(data, labels, test_size=0.2, random_state=42)
print(X_train.shape, X_test.shape, y_train.shape, y_test.shape)
#Címkék konvertálása one-hot kódolásúra
#the next two lines are uncommented, when the script is used without label binarizer
#y_train = to_categorical(y_train, 43)
#y_test = to_categorical(y_test, 43)
print(X_train.shape, X_test.shape, y_train.shape, y_test.shape)
# In[11]:
#A modell kialakítása
model = Sequential()
model.add(Conv2D(filters=32, kernel_size=(5,5), activation='relu', input_shape=X_train.shape[1:]))
model.add(Conv2D(filters=32, kernel_size=(5,5), activation='relu'))
model.add(MaxPool2D(pool_size=(2, 2)))
model.add(Dropout(rate=0.25))
model.add(Conv2D(filters=64, kernel_size=(3, 3), activation='relu'))
model.add(Conv2D(filters=64, kernel_size=(3, 3), activation='relu'))
model.add(MaxPool2D(pool_size=(2, 2)))
model.add(Dropout(rate=0.25))
model.add(Flatten())
model.add(Dense(256, activation='relu'))
model.add(Dropout(rate=0.5))
model.add(Dense(43, activation='softmax'))
#A modell összeállítása
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
# In[12]:
epochs = 15
history = model.fit(X_train, y_train, batch_size=32, epochs=epochs, validation_data=(X_test, y_test))
model.save("my_model.h5")
# In[5]:
# save the label binarizer to disk
import pickle
print("[INFO] serializing label binarizer...")
f = open("labelbin", "wb")
f.write(pickle.dumps(lb))
f.close()
# In[13]:
#Pontosság ábrázolása grafikonon
plt.figure(0)
plt.plot(history.history['acc'], label='training accuracy')
plt.plot(history.history['val_acc'], label='val accuracy')
plt.title('Accuracy')
plt.xlabel('epochs')
plt.ylabel('accuracy')
plt.legend()
plt.show()
plt.figure(1)
plt.plot(history.history['loss'], label='training loss')
plt.plot(history.history['val_loss'], label='val loss')
plt.title('Loss')
plt.xlabel('epochs')
plt.ylabel('loss')
plt.legend()
plt.show()
# In[8]:
#Pontosság tesztelése a tényleges tesztadathalmazon
from sklearn.metrics import accuracy_score
print(os.path.abspath(path))
y_test = pd.read_csv('Test.csv')
labels = y_test["ClassId"].values
imgs = y_test["Path"].values
data=[]
for img in imgs:
image = Image.open(img)
image = image.resize((30,30))
data.append(np.array(image))
X_test=np.array(data)
pred = model.predict_classes(X_test)
#Pontosság teszt adatokkal
from sklearn.metrics import accuracy_score
print(accuracy_score(labels, pred))
# In[13]:
import os
import time
from sklearn.metrics import accuracy_score
cur_path = os.getcwd()
path = os.path.join(cur_path)
print(os.path.abspath(path))
y_test = pd.read_csv('Test.csv')
labels = y_test["ClassId"].values
imgs = y_test["Path"].values
data=[]
icount=0
timesum=0
timecur=0
#Átlagos futásidő kiszámítása
for img in imgs:
start_time = time.time()
image = Image.open(img)
image = image.resize((30,30))
image = np.expand_dims(image, axis=0)
image = np.array(image)
pred = model.predict_classes([image])[0]
timecur = (time.time() - start_time)
print("--- %s seconds ---" % (timecur))
timesum = timesum + timecur
icount = icount+1
for img in imgs:
image = Image.open(img)
image = image.resize((30,30))
data.append(np.array(image))
X_test=np.array(data)
print("Átlag: ", (timesum/icount))
pred = model.predict_classes(X_test)
from sklearn.metrics import accuracy_score
print(accuracy_score(labels, pred))
# In[7]:
import tkinter as tk
import time
from tkinter import filedialog
from tkinter import *
from PIL import ImageTk, Image
import numpy
#Az edzett modell betöltése
from keras.models import load_model
model = load_model('my_model.h5')
#Az osztályok nevének listája a kiiratáshoz
classes = { 1:'Speed limit (20km/h)',
2:'Speed limit (30km/h)',
3:'Speed limit (50km/h)',
4:'Speed limit (60km/h)',
5:'Speed limit (70km/h)',
6:'Speed limit (80km/h)',
7:'End of speed limit (80km/h)',
8:'Speed limit (100km/h)',
9:'Speed limit (120km/h)',
10:'No passing',
11:'No passing veh over 3.5 tons',
12:'Right-of-way at intersection',
13:'Priority road',
14:'Yield',
15:'Stop',
16:'No vehicles',
17:'Veh > 3.5 tons prohibited',
18:'No entry',
19:'General caution',
20:'Dangerous curve left',
21:'Dangerous curve right',
22:'Double curve',
23:'Bumpy road',
24:'Slippery road',
25:'Road narrows on the right',
26:'Road work',
27:'Traffic signals',
28:'Pedestrians',
29:'Children crossing',
30:'Bicycles crossing',
31:'Beware of ice/snow',
32:'Wild animals crossing',
33:'End speed + passing limits',
34:'Turn right ahead',
35:'Turn left ahead',
36:'Ahead only',
37:'Go straight or right',
38:'Go straight or left',
39:'Keep right',
40:'Keep left',
41:'Roundabout mandatory',
42:'End of no passing',
43:'End no passing veh > 3.5 tons' }
#GUI betöltése
top=tk.Tk()
top.geometry('800x600')
top.title('Traffic sign classification')
top.configure(background='#CDCDCD')
label=Label(top,background='#CDCDCD', font=('arial',15,'bold'))
sign_image = Label(top)
def classify(file_path):
start_time = time.time()
global label_packed
image = Image.open(file_path)
image = image.resize((30,30))
image = numpy.expand_dims(image, axis=0)
image = numpy.array(image)
pred = model.predict_classes([image])[0]
sign = classes[pred+1]
print("--- %s seconds ---" % (time.time() - start_time))
print(sign)
label.configure(foreground='#011638', text=sign)
def show_classify_button(file_path):
classify_b=Button(top,text="Classify Image",command=lambda: classify(file_path),padx=10,pady=5)
classify_b.configure(background='#364156', foreground='white',font=('arial',10,'bold'))
classify_b.place(relx=0.79,rely=0.46)
def upload_image():
try:
file_path=filedialog.askopenfilename()
uploaded=Image.open(file_path)
uploaded.thumbnail(((top.winfo_width()/2.25),(top.winfo_height()/2.25)))
im=ImageTk.PhotoImage(uploaded)
sign_image.configure(image=im)
sign_image.image=im
label.configure(text='')
show_classify_button(file_path)
except:
pass
upload=Button(top,text="Upload an image",command=upload_image,padx=10,pady=5)
upload.configure(background='#364156', foreground='white',font=('arial',10,'bold'))
upload.pack(side=BOTTOM,pady=50)
sign_image.pack(side=BOTTOM,expand=True)
label.pack(side=BOTTOM,expand=True)
heading = Label(top, text="Know Your Traffic Sign",pady=20, font=('arial',20,'bold'))
heading.configure(background='#CDCDCD',foreground='#364156')
heading.pack()
top.mainloop()
# In[ ]:
We are using tensorflow library for face recognition. Our code works fine for single images. But when we run it as an API, the prediction time increases for every subsequent request. This happens because it searches for previously predicted images as well which should ideally not happen. Please find below the code I am using.
def train:
with tf.Session(config=tf.ConfigProto(log_device_placement=False)) as sess:
test_set = _get_test_data(input_directory)
images, labels = _load_images_and_labels(test_set, image_size=160, batch_size=batch_size,
num_threads=4, num_epochs=1)
_load_model(model_filepath=model_path)
init_op = tf.group(tf.global_variables_initializer(), tf.local_variables_initializer())
sess.run(init_op)
images_placeholder = tf.get_default_graph().get_tensor_by_name("input:0")
embedding_layer = tf.get_default_graph().get_tensor_by_name("embeddings:0")
phase_train_placeholder = tf.get_default_graph().get_tensor_by_name("phase_train:0")
coord = tf.train.Coordinator()
threads = tf.train.start_queue_runners(sess=sess,coord=coord)
emb_array, label_array = _create_embeddings(embedding_layer, images, labels, images_placeholder,
phase_train_placeholder, sess)
classifier_filename = classifier_output_path
class_name, prob = _evaluate_classifier(emb_array, label_array, classifier_filename)
coord.request_stop()
coord.join(threads)
def _create_embeddings(embedding_layer, images, labels, images_placeholder, phase_train_placeholder, sess):
emb_array = None
label_array = None
try:
i = 0
while True:
print("batch images")
batch_images, batch_labels = sess.run([images, labels])
print('Processing iteration {} batch of size: {}'.format(i, len(batch_labels)))
emb = sess.run(embedding_layer,
feed_dict={images_placeholder: batch_images, phase_train_placeholder: False})
emb_array = np.concatenate([emb_array, emb]) if emb_array is not None else emb
label_array = np.concatenate([label_array, batch_labels]) if label_array is not None else batch_labels
i += 1
except tf.errors.OutOfRangeError:
pass
return emb_array, label_array
It searches for previously predicted images at
`batch_images, batch_labels = sess.run([images, labels])`
in the create embedding function. I think this is the problem of some unclosed threads because of which sess.run runs for all queued threads. Can anyone help me with this
During debugging I found that previously predicted images information was in default graph which is picked during execution of following lines
images_placeholder = tf.get_default_graph().get_tensor_by_name("input:0")
embedding_layer = tf.get_default_graph().get_tensor_by_name("embeddings:0")
so by resetting graph before start of session will solve the problem of scanning previously predicted images as
tf.reset_default_graph()
Hey I am trying to set an input point for the model that i have writen in tensorflow
this is the code for the classification
n_dim = training_features.shape[1]
x = tf.placeholder(tf.float32, [None,n_dim])
classifier = (...)
init_op = tf.initialize_all_variables()
with tf.Session() as sess:
sess.run(init_op)
classifier.fit(training_features, training_labels, steps=100)
accuracy_score = classifier.evaluate(testing_features, testing_labels, steps=100)["accuracy"]
print('Accuracy', accuracy_score)
pred_a = np.asarray([x])
prediction = format(list(classifier.predict(pred_a)))
prediction_result = np.array(prediction)
output = tf.convert_to_tensor(prediction_result,dtype=None,name="output", preferred_dtype=None)
and here is my building code
export_path_base = sys.argv[-1]
export_path = os.path.join(
compat.as_bytes(export_path_base),
compat.as_bytes(str(FLAGS.model_version)))
print('Exporting trained model to', export_path)
builder = saved_model_builder.SavedModelBuilder(export_path)
classification_inputs = utils.build_tensor_info(y)
classification_outputs_classes = utils.build_tensor_info(output)
print('classification_signature...')
classification_signature = signature_def_utils.build_signature_def(
inputs={signature_constants.CLASSIFY_INPUTS: classification_inputs},
outputs={
signature_constants.CLASSIFY_OUTPUT_CLASSES:
classification_outputs_classes
},
method_name=signature_constants.CLASSIFY_METHOD_NAME)
tensor_info_x = utils.build_tensor_info(x)
print('prediction_signature...')
prediction_signature = signature_def_utils.build_signature_def(
inputs={'input': tensor_info_x},
outputs={
'classes' : classification_outputs_classes
},
method_name=signature_constants.PREDICT_METHOD_NAME)
print('Exporting...')
legacy_init_op = tf.group(tf.tables_initializer(), name='legacy_init_op')
builder.add_meta_graph_and_variables(
sess, [tag_constants.SERVING],
signature_def_map={
'predict_sound':
prediction_signature,
signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY:
classification_signature,
},
legacy_init_op=legacy_init_op)
builder.save()
print('Saved...')
I have tried manually passing dummy data before the build and that works but i am trying to have the client stub pass data into the model dynamically.
When i try run that code to build i get this error
InvalidArgumentError (see above for traceback): Shape in
shape_and_slice spec [1,280] does not match the shape stored in
checkpoint: [193,280] [[Node: save/RestoreV2_1 =
RestoreV2[dtypes=[DT_FLOAT],
_device="/job:localhost/replica:0/task:0/cpu:0"](_recv_save/Const_0, save/RestoreV2_1/tensor_names, save/RestoreV2_1/shape_and_slices)]]
May main goal is to have x as an input and output return the results, the output works but cant get the input to work.
Edit: If you just put the np.array as input without going through an input function, it will work but you also give up the chance to check the input.
Tensorflow won't check your input even if it's in the wrong shape or type or somehow corrupted but to throw such an error in the middle of session. Since you can run it with your test data successfully, the problem should be your actual data. Thus, it's recommended to write an input function to check your data before put it into classifier. Note that input function should return tf.Tensor with the shape of [x,1] (x is the number of your features) instead of an np.array.
Please refer to https://www.tensorflow.org/get_started/input_fn to see how to write your own input function and pass it to the classifier.
An example of input function:
def input_fn_predict(): # returns x, None
#do your check here or you can just print it out
feature_tensor = tf.constant(pred_a,shape=[1,pred_a.size])
return feature_tensor,None