I want to use a pretrained Inception-V3 model from Keras, paired with an input pipeline from Tensorflow (i.e. giving the network's input input via a tensor).
This is my code:
import tensorflow as tf
from keras.preprocessing.image import load_img, img_to_array
from keras.applications.inception_v3 import InceptionV3, decode_predictions, preprocess_input
import numpy as np
img_sample_filename = 'my_image.jpg'
img = img_to_array(load_img(img_sample_filename, target_size=(299,299)))
img = preprocess_input(img)
img_tensor = tf.constant(img[None,:])
# WITH KERAS:
model = InceptionV3()
pred = model.predict(img[None,:])
pred = decode_predictions(np.asarray(pred)) #<------ correct prediction!
print(pred)
# WITH TF:
model = InceptionV3(input_tensor=img_tensor)
init = tf.global_variables_initializer()
with tf.Session() as sess:
from keras import backend as K
K.set_session(sess)
sess.run(init)
pred = sess.run([model.output], feed_dict={K.learning_phase(): 0})
pred = decode_predictions(np.asarray(pred)[0])
print(pred) #<------ wrong prediction!
where my_image.jpg is any image I want to classify.
If I use keras' predict function to compute the prediction, the result is correct. If, however, I make a tensor out of the image array and feed that tensor to the model via input_tensor=... and then compute the prediction via sess.run([model.output], ...) the results are very wrong.
What is the reason of the different behaviour? Can't I use the Keras network in this way?
Finally, digging through the InceptionV3 code, I found the issue: sess.run(init) overwrites the weigts loaded in InceptionV3's constructor.
The -dirty- fix I found to this problem is reloading the weights after the sess.run(init).
from keras.applications.inception_v3 import get_file, WEIGHTS_PATH
with tf.Session() as sess:
from keras import backend as K
K.set_session(sess)
sess.run(init)
weights_path = get_file(
'inception_v3_weights_tf_dim_ordering_tf_kernels.h5',
WEIGHTS_PATH,
cache_subdir='models',
md5_hash='9a0d58056eeedaa3f26cb7ebd46da564')
model.load_weights(weights_path)
pred = sess.run([model.output], feed_dict={K.learning_phase(): 0})
Note: The parameters for get_file() are taken directly from InceptionV3's constructor and, in my example, are specific to restoring the full network's weights with image_data_format='channels_last'.
I asked in this Github issue if there's a better workaround for this. I'll update this answer if I should get more information.
Related
from keras.applications.resnet50 import ResNet50
from keras.preprocessing import image
from keras.applications.resnet50 import preprocess_input, decode_predictions
import numpy as np
model = ResNet50(weights='imagenet')
In this code, there is the "wrapper" (that's what it's referred to) ResNet50. What are the other types of weights I can use for this? I tried looking around but I don't even understand the source code; there is nothing conclusive there either
You can find it on the keras doc
Or github code
There is only two options, either None if you just want the architecture without the weights, or imagenet to load imagenet weights.
Edit : how to use our own weights :
# Take a DenseNET201
backbone = tf.keras.applications.DenseNet201(input_shape=input_shape, weights=None, include_top=False)
# Change the model a little bit, because why not
input_image = tf.keras.layers.Input(shape=input_shape)
x = backcone(input_image)
x = tf.keras.layers.Conv2D(classes, (3, 3), padding='same', name='final_conv')(input)
x = tf.keras.layers.Activation(activation, name=activation)(x)
model = tf.keras.Model(input, x)
#... some additional code
# training part
optimizer = tf.keras.optimizers.Adam(lr=FLAGS.learning_rate)
model.compile(loss=loss,
optimizer=optimizer,
metrics=['accuracy', f1_m, recall_m, precision_m])
callbacks = [tf.keras.callbacks.ModelCheckpoint(filepath=ckpt_name)]
model.fit(train_generator, validation_data=validation_generator, validation_freq=1, epochs=10, callbacks=callbacks)
# using the callback there will weights saved in cktp_name each epoch
# Inference part, just need to reinstance the model (lines after #Change part comment)
model.load_weights(ckpt_name)
results = model.predict(test_generator, verbose=1)
You don't need to change the model obviously, you could have used x = backbone(x) and then model = tf.keras.Model(input, x)
I would like to know how to get tf.gradients from a model built using the Keras API.
import Tensorflow as tf
from tensorflow import keras
from sklearn.datasets.samples_generator import make_blobs
# Create the model
inputs = keras.Input(shape=(2,))
x = keras.layers.Dense(12, activation='relu')(inputs)
x = keras.layers.Dense(8, activation='relu')(x)
predictions = keras.layers.Dense(3, activation='softmax')(x)
model = keras.models.Model(inputs=inputs, outputs=predictions)
model.compile(optimizer=tf.train.AdamOptimizer(0.001),
loss='categorical_crossentropy',
metrics=['accuracy'])
# Generate random data
X, y = make_blobs(n_samples=1000, centers=3, n_features=2)
labels = keras.utils.to_categorical(y, num_classes=3)
# Compute the gradients wrt inputs
y_true = tf.convert_to_tensor(labels)
y_pred = tf.convert_to_tensor(np.round(model.predict(X)))
sess = tf.Session()
sess.run(tf.global_variables_initializer())
grads = tf.gradients(model.loss_functions[0](y_true, y_pred),
model.inputs[0])
sess.run(grads, input_dict={model.inputs[0]: X, model.outputs: y})
First attempt above: my grads are None. With my second try below:
sess.run(grads, input_dict={model.inputs: X, model.outputs: y })
I get the following error:
TypeError: unhashable type: 'list'
I think you shouldn't create a new session directly with Tensorflow when using Keras. Instead, it is better to use the session implicitly created by Keras:
import keras.backend as K
sess = K.get_session()
However, I think in this case you don't need to retrieve the session at all. You can easily use backend functions, like K.gradients() and K.function(), to achieve your aim.
I want to view the CPU/memory usage in TensorBoard with Keras.
For this purpose, I need to execute the method of add_run_metadata.
But I cannot found the way to pass the add_run_metadata method in Keras's TensorBoard callback.
Is this good way to implement CPU/memory usage for Keras.
Reference
See following "Runtime Statistics" in TensorFlow
https://www.tensorflow.org/programmers_guide/graph_viz
add_run_metadata is defined in following location (in TensorFlow)
https://github.com/tensorflow/tensorflow/blob/v1.5.0/tensorflow/python/summary/writer/writer.py#L248
TensorBoard callback in Keras is defined here
https://github.com/keras-team/keras/blob/2.1.3/keras/callbacks.py#L587
EDIT: I encountered the same problem. I'm editing to share how I attempted to approch this.
I changed the keras source for: callbacks.py, and replaced this line in on_epoch_end() with -
run_options = tf.RunOptions(trace_level=tf.RunOptions.FULL_TRACE)
run_metadata = tf.RunMetadata()
result = self.sess.run([self.merged], feed_dict=feed_dict, options=run_options, run_metadata=run_metadata)
self.writer.add_run_metadata(run_metadata, 'epoch%d_step%d' % (epoch, i))
However I end up with the following error:
...\tensorflow\stream_executor\dso_loader.cc:141] Couldn't open CUDA library cupti64_90.dll
...\tensorflow/stream_executor/lib/statusor.h:212] Non-OK-status: status_ status: Failed precondition: could not dlopen DSO: cupti64_90.dll; dlerror: cupti64_90.dll not found
Which is puzzling to me as it seems to be related to the proper installation of cuda and not related in any obvious (to me) way to the change I made.
I'm using keras version 2.1.6 and tensorflow version 1.6.0
The solution is to run the Keras model in a TF session, and is based on the blog post: keras-as-a-simplified-interface-to-tensorflow#calling-keras-layers-on-tensorflow-tensors. Bellow is a detailed full and minimal working example.
First of all, dummy generation of data:
data.py
import numpy as np
def load_data(n = 1000):
x = np.random.rand(n, 100)
y = np.sum(x, axis=1, keepdims=True)
return x, y
The core idea is to run the model in TF session, so the main code is pure TF, and only the model itself is defined with Keras. For this to work (following the above mentioned tutorial):
The model needs to be built on top of a tf.placeholder, instead of the keras.layers.Input.
Remain as a tensor, and not compiled into a keras.models.Model.
from keras.layers import Dense
model.py
def load_network(input_tensor):
x = Dense(100, activation='relu')(input_tensor)
x = Dense(100, activation='relu')(x)
x = Dense(1, activation='sigmoid')(x)
return x
And the TF session that runs the keras model (a clean, but full, version of the TensorBoard tutorial):
run_runtime_stats.py
import tensorflow as tf
sess = tf.Session()
from keras import backend as K
from keras.objectives import mean_squared_error
K.set_session(sess)
from model import load_network
from data import load_data
# load your keras model as a tf.Tensor
input = tf.placeholder(tf.float32, shape=(None, 100)) # is passed as input to our keras layers
labels = tf.placeholder(tf.float32, shape=(None, 1))
net = load_network(input) # type(net) == tf.Tensor
loss = tf.reduce_mean(mean_squared_error(labels, net))
opt = tf.train.GradientDescentOptimizer(0.1).minimize(loss)
writer = tf.summary.FileWriter(r'./logs', sess.graph)
sess.run(tf.global_variables_initializer())
with sess.as_default():
x, y = load_data(64)
run_options = tf.RunOptions(trace_level=tf.RunOptions.FULL_TRACE)
run_metadata = tf.RunMetadata()
sess.run([opt],
feed_dict={input: x, labels: y},
options=run_options,
run_metadata=run_metadata)
writer.add_run_metadata(run_metadata, 'runtime-statistics')
writer.close()
I have saved a trained model based on recurrent neural networks. When I run the following function 'lstm_vector_predict()', it returns a different value every time even though it is loading the same model. Does tensor flow use some random number generation when predicting values?
import get_list_of_values_to_input
import tensorflow as tf
import tensorflow.contrib.learn as tflearn
import tensorflow.contrib.layers as tflayers
from tensorflow.contrib.learn.python.learn import learn_runner
import tensorflow.contrib.metrics as metrics
import tensorflow.contrib.rnn as rnn
import numpy as np
from backend.common.numpy_array_to_numpy_array_of_arrays import get_numpy_arrays_from_numpy_matrix
def lstm_vector_predict(model_name='sample_model_vector.meta', number_of_tickers=2, batch_size=20,number_of_points=100, start_time=1489462200):
tf.reset_default_graph()
inputs = number_of_tickers
hidden = 100
output = number_of_tickers
current_time = start_time
X = tf.placeholder(tf.float32, [None, batch_size, inputs])
# This is low level tensor flow stuff used for preparing output of data generation
basic_cell = tf.contrib.rnn.BasicRNNCell(num_units=hidden, activation=tf.nn.relu)
rnn_output, states = tf.nn.dynamic_rnn(basic_cell, X, dtype=tf.float32)
stacked_rnn_output = tf.reshape(rnn_output, [-1, hidden])
stacked_outputs = tf.layers.dense(stacked_rnn_output, output)
outputs = tf.reshape(stacked_outputs, [-1, batch_size, output])
# We get the saver ready
saver = tf.train.import_meta_graph(model_name)
init = tf.global_variables_initializer()
# Later, launch the model, use the saver to restore variables from disk, and
# do some work with the model.
return_values = []
with tf.Session() as sess:
# Restore variables from disk.
saver.restore(sess, tf.train.latest_checkpoint('./'))
print("Model restored.")
# Check the values of the variables
sess.run(init)
for i in range(number_of_points):
last_values = get_list_of_values_to_input()
print("Generating point", i)
#x_generators = last_values[-batch_size:]
x_generators = last_values[-batch_size:].reshape(-1, batch_size, number_of_tickers)
y_forecast = sess.run(outputs, feed_dict={X: x_generators})
return_values.append(y_forecast[-1][-1])
current_time += 300
return return_values
You will see different results because of the stochastic nature of the LSTM model, and because it is hard to fix the random seed for LSTM models to get 100% reproducible results.
I would like to include my custom pre-processing logic in my exported Keras model for use in Tensorflow Serving.
My pre-processing performs string tokenization and uses an external dictionary to convert each token to an index for input to the Embedding layer:
from keras.preprocessing import sequence
token_to_idx_dict = ... #read from file
# Custom Pythonic pre-processing steps on input_data
tokens = [tokenize(s) for s in input_data]
token_idxs = [[token_to_idx_dict[t] for t in ts] for ts in tokens]
tokens_padded = sequence.pad_sequences(token_idxs, maxlen=maxlen)
Model architecture and training:
model = Sequential()
model.add(Embedding(max_features, 128, input_length=maxlen))
model.add(LSTM(128, activation='sigmoid'))
model.add(Dense(n_classes, activation='softmax'))
model.compile(loss='sparse_categorical_crossentropy', optimizer='adam')
model.fit(x_train, y_train)
Since the model will be used in Tensorflow Serving, I want to incorporate all pre-processing logic into the model itself (encoded in the exported model file).
Q: How can I do so using the Keras library only?
I found this guide explains how to combine Keras and Tensorflow. But I'm still unsure how to export everything as one model.
I know Tensorflow has built-in string splitting, file I/O, and dictionary lookup operations.
Pre-processing logic using Tensorflow operations:
# Get input text
input_string_tensor = tf.placeholder(tf.string, shape={1})
# Split input text by whitespace
splitted_string = tf.string_split(input_string_tensor, " ")
# Read index lookup dictionary
token_to_idx_dict = tf.contrib.lookup.HashTable(tf.contrib.lookup.TextFileInitializer("vocab.txt", tf.string, 0, tf.int64, 1, delimiter=","), -1)
# Convert tokens to indexes
token_idxs = token_to_idx_dict.lookup(splitted_string)
# Pad zeros to fixed length
token_idxs_padded = tf.pad(token_idxs, ...)
Q: How can I use these Tensorflow pre-defined pre-processing operations and my Keras layers together to both train and then export the model as a "black box" for use in Tensorflow Serving?
I figured it out, so I'm going to answer my own question here.
Here's the gist:
First, (in separate code file) I trained the model using Keras only with my own pre-processing functions, exported the Keras model weights file and my token-to-index dictionary.
Then, I copied just the Keras model architecture, set the input as the pre-processed tensor output, loaded the weights file from the previously trained Keras model, and sandwiched it between the Tensorflow pre-processing operations and the Tensorflow exporter.
Final product:
import tensorflow as tf
from keras import backend as K
from keras.models import Sequential, Embedding, LSTM, Dense
from tensorflow.contrib.session_bundle import exporter
from tensorflow.contrib.lookup import HashTable, TextFileInitializer
# Initialize Keras with Tensorflow session
sess = tf.Session()
K.set_session(sess)
# Token to index lookup dictionary
token_to_idx_path = '...'
token_to_idx_dict = HashTable(TextFileInitializer(token_to_idx_path, tf.string, 0, tf.int64, 1, delimiter='\t'), 0)
maxlen = ...
# Pre-processing sub-graph using Tensorflow operations
input = tf.placeholder(tf.string, name='input')
sparse_tokenized_input = tf.string_split(input)
tokenized_input = tf.sparse_tensor_to_dense(sparse_tokenized_input, default_value='')
token_idxs = token_to_idx_dict.lookup(tokenized_input)
token_idxs_padded = tf.pad(token_idxs, [[0,0],[0,maxlen]])
token_idxs_embedding = tf.slice(token_idxs_padded, [0,0], [-1,maxlen])
# Initialize Keras model
model = Sequential()
e = Embedding(max_features, 128, input_length=maxlen)
e.set_input(token_idxs_embedding)
model.add(e)
model.add(LSTM(128, activation='sigmoid'))
model.add(Dense(num_classes, activation='softmax'))
# Load weights from previously trained Keras model
weights_path = '...'
model.load_weights(weights_path)
K.set_learning_phase(0)
# Export model in Tensorflow format
# (Official tutorial: https://github.com/tensorflow/serving/blob/master/tensorflow_serving/g3doc/serving_basic.md)
saver = tf.train.Saver(sharded=True)
model_exporter = exporter.Exporter(saver)
signature = exporter.classification_signature(input_tensor=model.input, scores_tensor=model.output)
model_exporter.init(sess.graph.as_graph_def(), default_graph_signature=signature)
model_dir = '...'
model_version = 1
model_exporter.export(model_dir, tf.constant(model_version), sess)
# Input example
with sess.as_default():
token_to_idx_dict.init.run()
sess.run(model.output, feed_dict={input: ["this is a raw input example"]})
The accepted answer is super helpful, however it uses an outdated Keras API as #Qululu mentioned, and an outdated TF Serving API (Exporter), and it does not show how to export the model so that its input is the original tf placeholder (versus Keras model.input, which is post preprocessing). Following is a version that works well as of TF v1.4 and Keras 2.1.2:
sess = tf.Session()
K.set_session(sess)
K._LEARNING_PHASE = tf.constant(0)
K.set_learning_phase(0)
max_features = 5000
max_lens = 500
dict_table = tf.contrib.lookup.HashTable(tf.contrib.lookup.TextFileInitializer("vocab.txt",tf.string, 0, tf.int64, TextFileIndex.LINE_NUMBER, vocab_size=max_features, delimiter=" "), 0)
x_input = tf.placeholder(tf.string, name='x_input', shape=(None,))
sparse_tokenized_input = tf.string_split(x_input)
tokenized_input = tf.sparse_tensor_to_dense(sparse_tokenized_input, default_value='')
token_idxs = dict_table.lookup(tokenized_input)
token_idxs_padded = tf.pad(token_idxs, [[0,0],[0, max_lens]])
token_idxs_embedding = tf.slice(token_idxs_padded, [0,0], [-1, max_lens])
model = Sequential()
model.add(InputLayer(input_tensor=token_idxs_embedding, input_shape=(None, max_lens)))
...REST OF MODEL...
model.load_weights("model.h5")
x_info = tf.saved_model.utils.build_tensor_info(x_input)
y_info = tf.saved_model.utils.build_tensor_info(model.output)
prediction_signature = tf.saved_model.signature_def_utils.build_signature_def(inputs={"text": x_info}, outputs={"prediction":y_info}, method_name=tf.saved_model.signature_constants.PREDICT_METHOD_NAME)
builder = saved_model_builder.SavedModelBuilder("/path/to/model")
legacy_init_op = tf.group(tf.tables_initializer(), name='legacy_init_op')
init_op = tf.group(tf.global_variables_initializer(), tf.local_variables_initializer())
sess.run(init_op)
# Add the meta_graph and the variables to the builder
builder.add_meta_graph_and_variables(
sess, [tag_constants.SERVING],
signature_def_map={
signature_constants.DEFAULT_SERVING_SIGNATURE_DEF_KEY:
prediction_signature,
},
legacy_init_op=legacy_init_op)
builder.save()
UPDATE Doing pre-processing for inference with Tensorflow is a CPU op, and is not carried out efficiently if the model is deployed on a GPU server. The GPU stalls really bad, and the throughput is very low. Therefore, we ditched this for efficient pre-processing in the client process, instead.