I've trained a DCGAN model and would now like to load it into a library that visualizes the drivers of neuron activation through image space optimization.
The following code works, but forces me to work with (1, width, height, channels) images when doing subsequent image analysis, which is a pain (the library assumptions about the shape of network input).
# creating TensorFlow session and loading the model
graph = tf.Graph()
sess = tf.InteractiveSession(graph=graph)
new_saver = tf.train.import_meta_graph(model_fn)
new_saver.restore(sess, './')
I'd like to change the input_map, After reading the source, I expected this code to work:
graph = tf.Graph()
sess = tf.InteractiveSession(graph=graph)
t_input = tf.placeholder(np.float32, name='images') # define the input tensor
t_preprocessed = tf.expand_dims(t_input, 0)
new_saver = tf.train.import_meta_graph(model_fn, input_map={'images': t_input})
new_saver.restore(sess, './')
But got an error:
ValueError: tf.import_graph_def() requires a non-empty name if input_map is used.
When the stack gets down to tf.import_graph_def() the name field is set to import_scope, so I tried the following:
graph = tf.Graph()
sess = tf.InteractiveSession(graph=graph)
t_input = tf.placeholder(np.float32, name='images') # define the input tensor
t_preprocessed = tf.expand_dims(t_input, 0)
new_saver = tf.train.import_meta_graph(model_fn, input_map={'images': t_input}, import_scope='import')
new_saver.restore(sess, './')
Which netted me the following KeyError:
KeyError: "The name 'gradients/discriminator/minibatch/map/while/TensorArrayWrite/TensorArrayWriteV3_grad/TensorArrayReadV3/RefEnter:0' refers to a Tensor which does not exist. The operation, 'gradients/discriminator/minibatch/map/while/TensorArrayWrite/TensorArrayWriteV3_grad/TensorArrayReadV3/RefEnter', does not exist in the graph."
If I set 'import_scope', I get the same error whether or not I set 'input_map'.
I'm not sure where to go from here.
In the newer version of tensorflow>=1.2.0, the following step works fine.
t_input = tf.placeholder(np.float32, shape=[None, width, height, channels], name='new_input') # define the input tensor
# here you need to give the name of the original model input placeholder name
# For example if the model has input as; input_original= tf.placeholder(tf.float32, shape=(1, width, height, channels, name='original_placeholder_name'))
new_saver = tf.train.import_meta_graph(/path/to/checkpoint_file.meta, input_map={'original_placeholder_name:0': t_input})
new_saver.restore(sess, '/path/to/checkpointfile')
So, the main issue is that you're not using the syntax right. Check the documentation for tf.import_graph_def for the use of input_map (link).
Let's breakdown this line:
new_saver = tf.train.import_meta_graph(model_fn, input_map={'images': t_input}, import_scope='import')
You didn't outline what model_fn is, but it needs to be a path to the file.
For the next part, in input_map, you're saying: replace the input in the original graph (DCGAN) whose name is images with my variable (in the current graph) called t_input. Problematically, t_input and images are referencing the same object in different ways as per this line:
t_input = tf.placeholder(np.float32, name='images')
In other words, images in input_map should actually be whatever the variable name is that you're trying to replace in the DCGAN graph. You'll have to import the graph in its base form (i.e., without the input_map line) and figure out what the name of the variable you want to link to is. It'll be in the list returned by tf.get_collection('variables') after you have imported the graph. Look for the dimensions (1, width, height, channels), but with the values in place of the variable names. If it's a placeholder, it'll look something like scope/Placeholder:0 where scope is replaced with whatever the variable's scope is.
Word of caution:
Tensorflow is very finicky about what it expects graphs to look like. So, if in the original graph specification the width, height, and channels are explicitly specified, then Tensorflow will complain (throw an error) when you try to connect a placeholder with a different set of dimensions. And, this makes sense. If the system was trained with some set of dimensions, then it only knows how to generate images with those dimensions.
In theory, you can still stick all kinds of weird stuff on the front of that network. But, you will need to scale it down so it meets those dimensions first (and the Tensorflow documentation says it's better to do that with the CPU outside of the graph; i.e., before inputing it with feed_dict).
Hope that helps!
Related
I am using Chainer's pertained model vgg (here named net). Every time I run the following code, I get a different result:
img = Image.open("/Users/macintosh/Desktop/Code/Ger.jpg")
img = Variable(vgg.prepare(img))
img = img.reshape((1,) + img.shape)
print(net(img,layers=['prob'])['prob'])
I have checked vgg.prepare() several times but its output is the same, and there is no random initialization here (net is a pre-trained vgg network). So why is this happening?
As you can see VGG implementation, it has dropout function. I think this causes the randomness.
When you want to forward the computation in evaluation mode (instead of training mode), you can set chainer config 'train' to False as follows:
with chainer.no_backprop_mode(), chainer.using_config('train', False):
result = net(img,layers=['prob'])['prob']
when train flag is False, dropout is not executed (and some other function behaviors also change, e.g., BatchNormalization uses trained statistics).
I have a frozen graph in tensorflow that is set up to take (batchsize, 224,224, 3) as input. Thus not taking an input function. I want to change it to take 50000 real images from a data folder. How do I feed those images into the metagraph without going over the memory limit?
I know that I could feed it as an input function but that would mean changing my frozen graph. Any suggestions of avoiding that?
Is the only way to convert the input to take in a function. I would like not to modify the frozen graphs.
This is the code I use to load my graph:
dummy_input = np.random.random_sample((batch_size,224,224,3))
tf.reset_default_graph()
g = tf.Graph()
outlist=[]
with g.as_default():
#processing the data
inc=tf.constant(dummy_input, dtype=tf.float32)
dataset=tf.data.Dataset.from_tensors(inc)
dataset=dataset.repeat()
iterator=dataset.make_one_shot_iterator()
next_element=iterator.get_next()
# loading the graph
out = tf.import_graph_def(
graph_def=gdef,
input_map={input_layer:next_element}, # input layer name of the model is required fro this
return_elements=[output_layer] # here is output layer name.
)
out = out[0].outputs[0]
outlist.append(out)
I have a meta and checkpoint files from which I load the weights of a pre-trained model. This works fine.
To test this model with a new image I need to do some pre-processing for the image (converting from grayscale to RGB, etc) which is basically done using opencv library. Doing this I do get my desired output.
But now what I want to do is add this pre-processing code to tensorflow itself so that when I save this model and re-use it I can only pass the image path as an argument and I don't need to do any pre-processing before passing it to tensorflow. I want tensorflow to handle all this.
I have tried the following already
The following I have used to implement the preprocessing the images within the tensorflow itself and save the new meta and checkpoint files
graph = tf.Graph()
with graph.as_default():
def dataprocess(x):
#convert from gray to rgb,etc
return y
path = ["images/test.jpg"]
filenames = tf.constant(path)
dataset = tf.contrib.data.Dataset.from_tensor_slices((filenames))
dataset = dataset.map(
lambda path : tf.py_func(
dataprocess[path], [tf.float32]))
iterator = dataset.make_initializable_iterator()
next_element = iterator.get_next()
next_element = tf.reshape(next_element,[-1,3,224,224]) #reshape as tensorflow shows unknown
The below code I use to restore my previous model
with tf.Session(graph=graph) as sess:
sess.run(iterator.initializer)
element1 = sess.run(next_element)
saver = tf.train.import_meta_graph('./.meta')
saver.restore(sess,'./')
saver1 = tf.train.Saver()
input= graph.get_tensor_by_name('input_1:0')
output= graph.get_tensor_by_name('predictions/Sigmoid:0')
print(sess.run(output,{input:element1}))
saver1.save(sess,'/tmp/test1/')
This all works fine
Next, I use the newly created meta and checkpoint files to test for an image(path)
path = ["images/test.jpg"]
with tf.Session() as sess:
saver = tf.train.import_meta_graph('./.meta')
saver.restore(sess,'./')
graph = tf.get_default_graph()
input = graph.get_tensor_by_name('Const:0')
output= graph.get_tensor_by_name('predictions/Sigmoid:0')
print(sess.run(output,{input:path}))
Using I get the following error
InvalidArgumentError: You must feed a value for placeholder tensor 'input_1' with dtype float and shape [?,3,224,224]
[[Node: input_1 = Placeholder[dtype=DT_FLOAT, shape=[?,3,224,224], _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]
input_1 corresponds to the above input of the graph
So I am assuming I am not passing the path to the correct place
I am new to tensorflow and there is pretty less documentation regarding this
Thank you
But now what I want to do is add this pre-processing code to
tensorflow itself so that when I save this model and re-use it I can
only pass the image path as an argument and I don't need to do any
pre-processing before passing it to tensorflow. I want tensorflow to
handle all this.
Unfortunately this is not possible if you use the tf.py_func operation. When you save the graph the python code inside the tf.py_func will not be saved as it is not part of the graph. (See py_func limitations here.) The only way to make the pre-processing part of the graph would be to rewrite it with tensorflow code without using the tf.py_func.
I wrote a script to do the classification of a single input image using a model I trained with MxNet. To classify the incoming image I feedforward them in through network.
In short here is what I am doing:
symbol, arg_params, aux_params = mx.model.load_checkpoint('model-prefix', 42)
model = mx.mod.Module(symbol=symbol, context=mx.cpu())
model.bind(data_shapes=[('data', (1, 3, 224, 244))], for_training=False)
model.set_params(arg_params, aux_params)
# ... loading the image & resizing ...
# img is the image to classify as numpy array of shape (3, 244, 244)
Batch = namedtuple('Batch', ['data'])
self._model.forward(Batch(data=[mx.nd.array(img)]))
probabilities = self._model.get_outputs()[0].asnumpy()
print(str(probabilities))
This works fine, except that I am getting the following warning
UserWarning: Data provided by label_shapes don't match names specified by label_names ([] vs. ['softmax_label'])
What should I change to avoid getting this warning? It is not clear to me what the label_shapes and label_names parameters are meant for, and what I am expect to fill them with.
Note: I found some thread about them, but none enabled me to solve the problem. Similarly the MxNet documentation doesn't provide much details on what those parameters are and on how they are supposed to be filled.
Set label_names=None and allow_missing=True. That should get rid of the warning.
model = mx.mod.Module(symbol=symbol, context=mx.cpu(), label_names=None)
...
model.set_params(arg_params, aux_params, allow_missing=True)
If you are curious why the warning is printed in the first place,
Every module has associated label. When this model was trained, softmax_label was used as the label (most likely because the output layer was a softmax layer named 'softmax'). When the model was loaded from file, the module that was created had softmax_label as the module's label.
>>>print(model.label_names)
['softmax_label']
model.bind is then called without providing label_shapes.
model.bind(data_shapes=[('data', (1, 3, 224, 244))], for_training=False)
MXNet sees that the module has a label in it which was not provided during bind and complains about it - which is the warning message you see.
I think if bind is called with for_training=False, MXNet shouldn't complain about the missing label. I've created this issue: https://github.com/dmlc/mxnet/issues/6958
However, for this particular case where we load a model from disk, we can load it with None as the label so that MXNet doesn't later complain when bind doesn't provide label - which is what the suggested fix does.
I think I'm losing my mind at this point.
I'm using Lasagne for a small convolutional neural network. It trains perfectly, I can compute the error on training and validation set as well, but I cannot save the trained model on the disk. Better, I can save it and load it, but I cannot use it to predict for new data.
This is what I do after training
model = {'network': network, 'params': get_all_params(network), 'params_values': get_all_param_values(network)}
pickle.dump(model, open('models/model_1.pkl', 'wb'), protocol=pickle.HIGHEST_PROTOCOL)
And this is what I do to load the model
with open('models/model.pkl', 'rb') as pickle_file:
model = pickle.load(pickle_file)
network = model['network']
values = model['params_values']
set_all_param_values(network, values)
T_input = T.tensor4('input', dtype='float32')
T_target = T.ivector('target')
predictions = get_output(network, deterministic=True)
loss = (cross_entropy(predictions, T_target)).mean()
acc = T.mean(T.eq(T.argmax(predictions, axis=1), T_target), dtype=config.floatX)
test_fn = function([T_input, T_target], [loss, acc])
I cannot even pass the real numpy input, that I get this error
theano.compile.function_module.UnusedInputError: theano.function was asked to create a
function computing outputs given certain inputs, but the provided input variable at index 0
is not part of the computational graph needed to compute the outputs: input.
To make this error into a warning, you can pass the parameter
on_unused_input='warn' to theano.function. To disable it completely, use
on_unused_input='ignore'.
I tried to set the parameter on_unused_input='warn' then, and this is the result
theano.gof.fg.MissingInputError: An input of the graph, used to compute (..)
was not provided and not given a value.Use the Theano flag
exception_verbosity='high',for more information on this error.
The problem is that your T_input is not tied to the input layer and hence theano can't compile it
T_input = lasagne.layers.get_all_layers(network)[0].input_var