Develop Reference

Tensorflow For Poets - Upgrade to Tensorflow 2

I used the "Tensorflow For Poets" tutorial a few years ago to create an image classifier. It's amazing and I've been using it regularly ever since.
Today I have attempted to migrate my image classifier to a new Docker environment but it's running the new version Tensorflow 2 and so my script breaks.
Can anyone help to upgrade this famous tutorial script to Tensorflow 2?
directory = '/imageFolder'
# Tensorflow labels
label_lines = [line.rstrip() for line in tf.gfile.GFile('/tf_files/retrained_labels.txt')]
# Unpersists graph from file
with tf.gfile.FastGFile('/tf_files/retrained_graph.pb', 'rb') as f:
graph_def = tf.GraphDef()
graph_def.ParseFromString(f.read())
_ = tf.import_graph_def(graph_def, name='')
with tf.Session() as sess:
# Feed the image_data as input to the graph and get first prediction
softmax_tensor = sess.graph.get_tensor_by_name('final_result:0')
# Count the folders
def fcount(path, map = {}):
count = 0
for f in os.listdir(path):
child = os.path.join(path, f)
if os.path.isdir(child):
child_count = fcount(child, map)
count += child_count + 1 # unless include self
map[path] = count
return count
map = {}
totalDirectories = fcount(directory, map)
# Walk the directory
for dirpath, dirnames, filenames in os.walk(directory):
splicedDirpath = dirpath[len(directory):]
print "Processing ", splicedDirpath
counter = 0
for name in filenames:
if name.lower().endswith(('.jpg', '.jpeg', '.tiff')):
print name
image_data = tf.gfile.FastGFile(os.path.join(dirpath, name), 'rb').read()
predictions = sess.run(softmax_tensor, \
{'DecodeJpeg/contents:0': image_data})
# Sort to show labels of first prediction in order of confidence
top_k = predictions[0].argsort()[-len(predictions[0]):][::-1]
firstElt = top_k[0];
for node_id in top_k:
human_string = label_lines[node_id]
score = predictions[0][node_id]

There is already a new updated version of the TensorFlow for Poets in this notebook.
If you need to migrate your code from TensorFlow 1.x to TensorFlow 2.x.
You can use the compat library of TensorFlow 2.x by basically importing tf.compat.v1 and executing tf.compat.v1.disable_v2_behavior().
You can also use the upgrade script provided by TensorFlow to assist in migrating your code, this will also show you parts of the code that needs manual changes.
The guide in using the automatic upgrade script is in this link.
A more in-depth discussion on the guide of Migrating your code from TensorFlow 1.x to 2.x is in this link.

Tensor Flow using old graph instead of new one

I have retrained two different classification models models using retrain.py.
For predicting labels for two images I have created getLabel method from Label_image.py as follows:
def getLabel(localFile, graphKey, labelKey):
image_data_str = tf.gfile.FastGFile(localFile, 'rb').read()
# Loads label file, strips off carriage return
label_lines = [line.rstrip() for line
in tf.gfile.GFile(labelKey)]
# Unpersists graph from file
with tf.gfile.FastGFile(graphKey, 'rb') as f:
graph_def = tf.GraphDef()
graph_def.ParseFromString(f.read())
_ = tf.import_graph_def(graph_def, name='')
sess = tf.Session()
with sess:
# Feed the image_data as input to the graph and get first prediction
softmax_tensor = sess.graph.get_tensor_by_name('final_result:0')
predictions = sess.run(softmax_tensor, {'DecodeJpeg/contents:0': image_data_str})
# Sort to show labels of first prediction in order of confidence
top_k = predictions[0].argsort()[-len(predictions[0]):][::-1]
series = []
count = 1
for node_id in top_k:
human_string = label_lines[node_id]
if count==1:
label = human_string
count+=1
score = predictions[0][node_id]
print('%s (score = %.5f)' % (human_string, score))
series.append({"name": human_string, "data": [score * 100]})
sess.close()
return label, series
And I am calling them as
label,series = predict.getLabel(localFile, 'graph1.pb', 'labels1.txt')
label,series = predict.getLabel(localFile, 'graph2.pb', 'labels2.txt')
But for the second function call it is using the old graph i.e. graph1.pb & it is giving below error since model 1 has more categories than model 2.
human_string = label_lines[node_id]
IndexError: list index out of range
I am not able to understand why is this happening. Can someone tell how to load second graph??

It looks like what is happening is that you are calling the same session for both calls to predict.getFinalLabel. What you should do is define two separate sessions, and initialize each separately (e.g. have predict1.getFinalLabel and predict2.getFinalLabel). If you post more of your code, I can provide more detail and code.

Tensorflow Label_Image for PNG, GIF etc

I was playing around with Tensorflow for image classification. I used the image_retraining/retrain.py to retrain the inception library with new categories and used it to classify images using label_image.py from https://github.com/llSourcell/tensorflow_image_classifier/blob/master/src/label_image.py as below:
import tensorflow as tf
import sys
# change this as you see fit
image_path = sys.argv[1]
# Read in the image_data
image_data = tf.gfile.FastGFile(image_path, 'rb').read()
# Loads label file, strips off carriage return
label_lines = [line.rstrip() for line
in tf.gfile.GFile("/root/tf_files/output_labels.txt")]
# Unpersists graph from file
with tf.gfile.FastGFile("/root/tf_files/output_graph.pb", 'rb') as f:
graph_def = tf.GraphDef()
graph_def.ParseFromString(f.read())
_ = tf.import_graph_def(graph_def, name='')
with tf.Session() as sess:
# Feed the image_data as input to the graph and get first prediction
softmax_tensor = sess.graph.get_tensor_by_name('final_result:0')
#predictions = sess.run(softmax_tensor,{'DecodeJpeg/contents:0': image_data})
predictions = sess.run(softmax_tensor,{'DecodePng/contents:0': image_data})
# Sort to show labels of first prediction in order of confidence
top_k = predictions[0].argsort()[-len(predictions[0]):][::-1]
for node_id in top_k:
human_string = label_lines[node_id]
score = predictions[0][node_id]
print('%s (score = %.5f)' % (human_string, score))
I noticed two issues. When I retrain with new categories, it only trains JPG images. I am a noob in machine learning so not sure whether this is a limitation or is it possible to train other extension images like PNG, GIF?
Another one is when classifying the images the input is again only for JPG. I tried to change DecodeJpeg to DecodePng in label_image.py above but couldn't work. Another way I tried was to convert other formats into JPG before passing them in for classification like:
im = Image.open('/root/Desktop/200_s.gif').convert('RGB')
im.save('/root/Desktop/test.jpg', "JPEG")
image_path1 = '/root/Desktop/test.jpg'
Is there any other way to do this? Does Tensorflow have functions to handle other image formats other than JPG?
I tried the following by feeding in parsed image as compared to JPEG as suggested by #mrry
import tensorflow as tf
import sys
import numpy as np
from PIL import Image
# change this as you see fit
image_path = sys.argv[1]
# Read in the image_data
image_data = tf.gfile.FastGFile(image_path, 'rb').read()
image = Image.open(image_path)
image_array = np.array(image)[:,:,0:3] # Select RGB channels only.
# Loads label file, strips off carriage return
label_lines = [line.rstrip() for line
in tf.gfile.GFile("/root/tf_files/output_labels.txt")]
# Unpersists graph from file
with tf.gfile.FastGFile("/root/tf_files/output_graph.pb", 'rb') as f:
graph_def = tf.GraphDef()
graph_def.ParseFromString(f.read())
_ = tf.import_graph_def(graph_def, name='')
with tf.Session() as sess:
# Feed the image_data as input to the graph and get first prediction
softmax_tensor = sess.graph.get_tensor_by_name('final_result:0')
predictions = sess.run(softmax_tensor,{'DecodeJpeg:0': image_array})
# Sort to show labels of first prediction in order of confidence
top_k = predictions[0].argsort()[-len(predictions[0]):][::-1]
for node_id in top_k:
human_string = label_lines[node_id]
score = predictions[0][node_id]
print('%s (score = %.5f)' % (human_string, score))
It works for JPEG images but when I use PNG or GIF it throws
Traceback (most recent call last):
File "label_image.py", line 17, in <module>
image_array = np.array(image)[:,:,0:3] # Select RGB channels only.
IndexError: too many indices for array

The model can only train on (and evaluate) JPEG images, because the GraphDef that you've saved in /root/tf_files/output_graph.pb only contains a tf.image.decode_jpeg() op, and uses the output of that op for making predictions. There are at least a couple of options for using other image formats:
Feed in parsed images rather than JPEG data. In the current program, you feed in a JPEG-encoded image as a string value for the tensor "DecodeJpeg/contents:0". Instead, you can feed in a 3-D array of decoded image data for the tensor "DecodeJpeg:0" (which represents the output of the tf.image.decode_jpeg() op), and you can use NumPy, PIL, or some other Python library to create this array.
Remap the image input in tf.import_graph_def(). The tf.import_graph_def() function enables you to connect two different graphs together by remapping individual tensor values. For example, you could do something like the following to add a new image-processing op to the existing graph:
image_string_input = tf.placeholder(tf.string)
image_decoded = tf.image.decode_png(image_string_input)
# Unpersists graph from file
with tf.gfile.FastGFile("/root/tf_files/output_graph.pb", 'rb') as f:
graph_def = tf.GraphDef()
graph_def.ParseFromString(f.read())
softmax_tensor, = tf.import_graph_def(
graph_def,
input_map={"DecodeJpeg:0": image_decoded},
return_operations=["final_result:0"])
with tf.Session() as sess:
# Feed the image_data as input to the graph and get first prediction
predictions = sess.run(softmax_tensor, {image_string_input: image_data})
# ...

You should have a look at the tf.image package. It's got good functions to decode / encode JPEGs, GIFs and PNGs.

Following #mrry's suggestion to feed in parsed image, converted the image data into array and convert into RGB as stated below in the code. Now I am able to feed in JPG,PNG and GIF.
import tensorflow as tf
import sys
import numpy as np
from PIL import Image
# change this as you see fit
image_path = sys.argv[1]
# Read in the image_data
image_data = tf.gfile.FastGFile(image_path, 'rb').read()
image = Image.open(image_path)
image_array = image.convert('RGB')
# Loads label file, strips off carriage return
label_lines = [line.rstrip() for line
in tf.gfile.GFile("/root/tf_files/output_labels.txt")]
# Unpersists graph from file
with tf.gfile.FastGFile("/root/tf_files/output_graph.pb", 'rb') as f:
graph_def = tf.GraphDef()
graph_def.ParseFromString(f.read())
_ = tf.import_graph_def(graph_def, name='')
with tf.Session() as sess:
# Feed the image_data as input to the graph and get first prediction
softmax_tensor = sess.graph.get_tensor_by_name('final_result:0')
predictions = sess.run(softmax_tensor,{'DecodeJpeg:0': image_array})
# Sort to show labels of first prediction in order of confidence
top_k = predictions[0].argsort()[-len(predictions[0]):][::-1]
for node_id in top_k:
human_string = label_lines[node_id]
score = predictions[0][node_id]
print('%s (score = %.5f)' % (human_string, score))

Tensorflow Inception_Resnet_V2 Classify Image

I trained an inception_resnet_v2 model for the flowers images following the README at https://github.com/tensorflow/models/tree/master/slim
I got my graph.pbtxt file out of this after training with which I converted to a graph.pb file with the following code:
import tensorflow as tf
from google.protobuf import text_format
def convert_pbtxt_to_graphdef(filename):
"""Returns a `tf.GraphDef` proto representing the data in the given pbtxt file.
Args:
filename: The name of a file containing a GraphDef pbtxt (text-formatted
`tf.GraphDef` protocol buffer data).
Returns:
A `tf.GraphDef` protocol buffer.
"""
with tf.gfile.FastGFile(filename, 'r') as f:
graph_def = tf.GraphDef()
file_content = f.read()
# Merges the human-readable string in `file_content` into `graph_def`.
text_format.Merge(file_content, graph_def)
return graph_def
with tf.gfile.FastGFile('/foo/bar/workspace/results/graph.pb', 'wb') as f:
f.write(convert_pbtxt_to_graphdef('/foo/bar/workspace/results/graph.pbtxt'))
After getting this file I tried feeding the trained model a random image using tensorflow's classify_image.py found here: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/models/image/imagenet/classify_image.py
using my .pb, .pbtxt, and my labels file, however, I get the following error:
Traceback (most recent call last):
File "classify_image.py", line 212, in <module>
tf.app.run()
File "/usr/local/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 30, in run
sys.exit(main(sys.argv[:1] + flags_passthrough))
File "classify_image.py", line 208, in main
run_inference_on_image(image)
File "classify_image.py", line 170, in run_inference_on_image
softmax_tensor = sess.graph.get_tensor_by_name('softmax:0')
File "/usr/local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2615, in get_tensor_by_name
return self.as_graph_element(name, allow_tensor=True, allow_operation=False)
File "/usr/local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2466, in as_graph_element
return self._as_graph_element_locked(obj, allow_tensor, allow_operation)
File "/usr/local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2508, in _as_graph_element_locked
"graph." % (repr(name), repr(op_name)))
KeyError: "The name 'softmax:0' refers to a Tensor which does not exist. The operation, 'softmax', does not exist in the graph."

The problem with slim, in fact tensorflow/models, is that the framework and so the produced models don't really fit the prediction use case:
TF-slim is a new lightweight high-level API of TensorFlow
(tensorflow.contrib.slim) for defining, training and evaluating
complex models.
(Source https://github.com/tensorflow/models/tree/master/slim)
The main problem at prediction time is, that it only works well with the checkpoint files created by the Saver class. When using checkpoint files the assign_from_checkpoint_fn() method can be used to initialize all the variables with the trained parameters contained in the checkpoint. On the other hand, in the situation when having the GraphDef file *.pb only, you kinda lost. There is a nice trick though.
The key idea is to inject a tf.placeholder variable for the input image(s) into the computation graph after you saved your trained model as a checkpoint. The following script (convert_checkpoint_to_pb.py) reads a checkpoint, inserts a placeholder, converts the graph variables to constants and dumps it to a *.pb file.
import tensorflow as tf
from tensorflow.contrib import slim
from nets import inception
from tensorflow.python.framework.graph_util import convert_variables_to_constants
from tensorflow.python.tools.optimize_for_inference_lib import optimize_for_inference
from preprocessing import inception_preprocessing
checkpoints_dir = '/path/to/your/checkpoint_dir/'
OUTPUT_PB_FILENAME = 'minimal_graph.proto'
NUM_CLASSES = 2
# We need default size of image for a particular network.
# The network was trained on images of that size -- so we
# resize input image later in the code.
image_size = inception.inception_resnet_v2.default_image_size
with tf.Graph().as_default():
# Inject placeholder into the graph
input_image_t = tf.placeholder(tf.string, name='input_image')
image = tf.image.decode_jpeg(input_image_t, channels=3)
# Resize the input image, preserving the aspect ratio
# and make a central crop of the resulted image.
# The crop will be of the size of the default image size of
# the network.
# I use the "preprocess_for_eval()" method instead of "inception_preprocessing()"
# because the latter crops all images to the center by 85% at
# prediction time (training=False).
processed_image = inception_preprocessing.preprocess_for_eval(image,
image_size,
image_size, central_fraction=None)
# Networks accept images in batches.
# The first dimension usually represents the batch size.
# In our case the batch size is one.
processed_images = tf.expand_dims(processed_image, 0)
# Load the inception network structure
with slim.arg_scope(inception.inception_resnet_v2_arg_scope()):
logits, _ = inception.inception_resnet_v2(processed_images,
num_classes=NUM_CLASSES,
is_training=False)
# Apply softmax function to the logits (output of the last layer of the network)
probabilities = tf.nn.softmax(logits)
model_path = tf.train.latest_checkpoint(checkpoints_dir)
# Get the function that initializes the network structure (its variables) with
# the trained values contained in the checkpoint
init_fn = slim.assign_from_checkpoint_fn(
model_path,
slim.get_model_variables())
with tf.Session() as sess:
# Now call the initialization function within the session
init_fn(sess)
# Convert variables to constants and make sure the placeholder input_image is included
# in the graph as well as the other neccesary tensors.
constant_graph = convert_variables_to_constants(sess, sess.graph_def, ["input_image", "DecodeJpeg",
"InceptionResnetV2/Logits/Predictions"])
# Define the input and output layer properly
optimized_constant_graph = optimize_for_inference(constant_graph, ["eval_image"],
["InceptionResnetV2/Logits/Predictions"],
tf.string.as_datatype_enum)
# Write the production ready graph to file.
tf.train.write_graph(optimized_constant_graph, '.', OUTPUT_PB_FILENAME, as_text=False)
(The models/slim code must be in your python path to execute this code)
To predict new images with the converted model (now present as a *.pb file) use the code from file minimal_predict.py:
import tensorflow as tf
import urllib2
def create_graph(model_file):
"""Creates a graph from saved GraphDef file and returns a saver."""
# Creates graph from saved graph_def.pb.
with tf.gfile.FastGFile(model_file, 'rb') as f:
graph_def = tf.GraphDef()
graph_def.ParseFromString(f.read())
_ = tf.import_graph_def(graph_def, name='')
model_file = "/your/path/to/minimal_graph.proto"
url = ("http://pictureparadise.net/funny-babies/funny-babies02/funny-babies-053.jpg")
# Open specified url and load image as a string
image_string = urllib2.urlopen(url).read()
with tf.Graph().as_default():
with tf.Session() as new_sess:
create_graph(model_file)
softmax = new_sess.graph.get_tensor_by_name("InceptionResnetV2/Logits/Predictions:0")
# Loading the injected placeholder
input_placeholder = new_sess.graph.get_tensor_by_name("input_image:0")
probabilities = new_sess.run(softmax, {input_placeholder: image_string})
print probabilities
To use these scripts simply run python
python convert_checkpoint_to_pb.py
python minimal_predicty.py
while having tensorflow and tensorflow/models/slim in your PYTHONPATH.

In the convert_checkpoint_to_pb.pyprovided by #Maximilian,["eval_image"] in
optimized_constant_graph = optimize_for_inference(constant_graph, ["eval_image"],["InceptionResnetV2/Logits/Predictions"],tf.string.as_datatype_enum)
should be replaced by ["input_image"] else you will get an error stating "The following input nodes were not found: {'eval_image'}\n".

Memory not freed when running TensorFlow Inception v3?

I am trying to classify a set of photos (<10000) using Tensorflow Inception v3.
I'm using the TensorFlow Docker installation CPU Binary image plus source code on a Macbook Air with 4gb DDR3.
I've written a python script to spawn a subprocess to classify each image:
"""
classifier_spawner.py
Runs custom_classify_image.py in subprocess
"""
if __name__ == "__main__":
import sqlite3
import subprocess as sub
from ast import literal_eval
conn = sqlite3.connect("database_name.db")
c = conn.cursor()
c.execute("SELECT * FROM images")
images = c.fetchall()
for image in images:
p = sub.Popen(["python", "custom_classify_image.py", "--image_file=image_dir/" + str(image[0]) + ".jpg"], stdout=sub.PIPE, stderr=sub.PIPE)
output, errors = p.communicate()
categories = literal_eval(output)
for cat in categories:
c.execute("""INSERT OR IGNORE INTO classification (image_id, class_id, probability) VALUES(?,?,?)""", (str(image[0]), str(cat[0]), float(cat[1])))
conn.commit()
print("Classification complete, exiting.")
conn.close()
I have amended run_inference_on_image(image) in classify_image.py as provided with TensorFlow to print a python list of tuples of the top 5 classifications, as shown.
"""My custom run_inference_on_image(image) """
def run_inference_on_image(image):
"""Runs inference on an image.
Args:
image: Image file name.
Returns:
Nothing
"""
if not tf.gfile.Exists(image):
tf.logging.fatal('File does not exist %s', image)
image_data = tf.gfile.FastGFile(image, 'rb').read()
# Creates graph from saved GraphDef.
"""Creates a graph from saved GraphDef file and returns a saver."""
# Creates graph from saved graph_def.pb.
with tf.gfile.FastGFile(os.path.join(
FLAGS.model_dir, 'classify_image_graph_def.pb'), 'rb') as f:
graph_def = tf.GraphDef()
graph_def.ParseFromString(f.read())
_ = tf.import_graph_def(graph_def, name='')
with tf.Session() as sess:
# Some useful tensors:
# 'softmax:0': A tensor containing the normalized prediction across
# 1000 labels.
# 'pool_3:0': A tensor containing the next-to-last layer containing 2048
# float description of the image.
# 'DecodeJpeg/contents:0': A tensor containing a string providing JPEG
# encoding of the image.
# Runs the softmax tensor by feeding the image_data as input to the graph.
softmax_tensor = sess.graph.get_tensor_by_name('softmax:0')
predictions = sess.run(softmax_tensor,
{'DecodeJpeg/contents:0': image_data})
predictions = np.squeeze(predictions)
# Creates node ID --> English string lookup.
node_lookup = NodeLookup()
top_k = predictions.argsort()[-FLAGS.num_top_predictions:][::-1]
top_k_tup_list = []
for node_id in top_k:
human_string = node_lookup.id_to_string(node_id)
score = predictions[node_id]
top_k_tup_list.append((human_string, score))
sess.close()
print(top_k_tup_list)
My issue is that when I run classifier_spawner.py, after the subprocess has completed, the memory that it used is not freed and after ~10 classifications my disk is full (after writing ~15gb to disk). I then have to delete my virtual machine.
I don't understand why this is happening, shouldn't the virtual memory the subprocess uses be freed after (1) the TensorFlow session is ended and (2) the process has exited?
Thanks in advance, if you need any clarification please let me know.