I am trying to see what my data looks like after going through a convolutional layer in my Keras model. I am using the Theano backend. The code I have has been cobbled together from the Keras Github:
def get_layer0_outputs(model, test_data):
output = model.layers[0].output
inputs = [K.learning_phase()] + model.inputs
func = K.function(inputs, [output])
return func([0] + [test_data])
What I'm trying to do here is compile a function for the first layer (a Conv2D layer) in my network. The test_data argument is an np.ndarray. My model is loaded correctly, and I have already trained it with decent accuracy.
However, when I call this function, I get a cryptic stacktrace:
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/theano/compile/function_module.py", line 884, in __call__
self.fn() if output_subset is None else\
File "/usr/local/lib/python3.5/dist-packages/theano/gof/op.py", line 872, in rval
r = p(n, [x[0] for x in i], o)
File "/usr/local/lib/python3.5/dist-packages/theano/tensor/nnet/abstract_conv.py", line 1626, in perform
conv_out = self.conv(img, kern, mode="valid", dilation=self.filter_dilation)
File "/usr/local/lib/python3.5/dist-packages/theano/tensor/nnet/abstract_conv.py", line 1531, in conv
dilated_kern[n, im0, ...],
IndexError: index 1 is out of bounds for axis 1 with size 1
What does this mean? Am I calling my function incorrectly?
Your function works for me using the following model:
a = Input(shape=(224,224,3))
b = Conv2D(8, 3, strides=(2,2))(a)
model = Model(inputs=a, outputs=b)
model.compile(optimizer='sgd', loss='mse')
def get_layer0_outputs(model, test_data):
output = model.layers[0].output
inputs = [K.learning_phase()] + model.inputs
func = K.function(inputs, [output])
return func([0] + [test_data])
print get_layer0_outputs(model, np.zeros((1, 224, 224, 3)))[0].shape
Note that layer 0 is an Input layer not a Conv2D, but the code also works for layer 1. I'm using the tensorflow backend so I don't know if the difference is your model or the theano backend.
Related
This is a toy model I am trying to implement with tensorflow. The input is a set (10) of real number pairs. And the underlying function I want to approximate is . The implemented model should look like this:
I also need to mention that "Hidden Layer" is the same layer (same parameters) for all X_i.
What I implemented so far:
import tensorflow as tf
import numpy as np
def tf_model():
# Define the inputs
inputs = tf.keras.Input(shape=[10, 2])
# Define common hidden layer
hidden_layer = tf.keras.layers.Dense(64, activation="relu")(inputs)
# Propagate and average
outputs = tf.keras.layers.Dense(1, activation="sigmoid")(hidden_layer)
outputs = tf.keras.layers.Average()(outputs)
return tf.keras.Model(inputs=inputs, outputs=output)
X = np.random.rand(1000,10,2) * 100
y = 1 / (1 + X[...,0]**2 + X[...,1]**4)
y = np.average(y, axis=1)
model = tf_model()
model.fit(X, y)
What I get from running this:
Traceback (most recent call last):
File "model_test.py", line 21, in <module>
model = tf_model()
File "model_test.py", line 13, in tf_model
outputs = tf.keras.layers.Average()(outputs)
File "/home/redbull/.local/lib/python3.8/site-packages/keras/utils/traceback_utils.py", line 67, in error_handler
raise e.with_traceback(filtered_tb) from None
File "/home/redbull/.local/lib/python3.8/site-packages/keras/layers/merge.py", line 88, in build
raise ValueError(
ValueError: A merge layer should be called on a list of inputs. Received: input_shape=(None, 10, 1) (not a list of shapes)
I think the issue is that tf.keras.layers.Average() only works with a list of inputs, but not a tf layer/ tensor.
Since tf.keras.layers.Average() does not seem to be suitable in this scenario, how can I implement the wished functionality?
You can use tf.reduce_mean as below.
outputs = tf.reduce_mean(outputs, axis=1)
I am building a prediction model for sequence data using conv1d layer provided by Keras. This is how I did
input_layer = Input(shape=(500,))
layer = Conv1D(128,5,activation="relu")(input_layer)
layer = MaxPooling1D(pool_size=2)(layer)
layer = Flatten()(layer)
layer = Dense(128, activation='relu')(layer)
output_layer = Dense(10, activation='softmax')(layer)
classifier = Model(input_layer, output_layer)
classifier.summary()
classifier.compile(optimizer=optimizers.Adam(), loss='sparse_categorical_crossentropy', metrics=['accuracy'])
return classifier
However, am facing the following error:
Traceback (most recent call last):
File "train.py", line 71, in <module>
classifier = create_cnn_model()
File "train.py", line 60, in create_cnn_model
layer = Conv1D(128,5, activation="relu")(input_layer)
File "C:\Python368\lib\site-packages\keras\backend\tensorflow_backend.py", line 75, in symbolic_fn
_wrapper
return func(*args, **kwargs)
File "C:\Python368\lib\site-packages\keras\engine\base_layer.py", line 446, in __call__
self.assert_input_compatibility(inputs)
File "C:\Python368\lib\site-packages\keras\engine\base_layer.py", line 342, in assert_input_compat
ibility
str(K.ndim(x)))
ValueError: Input 0 is incompatible with layer conv1d_1: expected ndim=3, found ndim=2
I think the input_shape in the first layer is not setup right. How to set it up?
Right, conv layers need 3 dimensional input.
I am assuming you have a univariate time series with 500 samples.
You need to write a function to split the time series into steps.
For example:
x y
[t-n,...,t-2,t-1] t
So you are basically using the last n values to predict the next value in your series.
Then your input shape will be [len(x), n, 1]
I want to use a custom loss function, by extracting features from the penultimate layer of VGG16 network. So, in the model.compile() function, I pass a function vgg_loss() which returns the required loss. It is shown below:
model_loss = VGG16(include_top=False, input_shape=input_size)
model.compile(optimizer = Adam(lr = lrate, decay = 1e-6), loss = vgg_loss(model_loss))
The vgg_loss() function is defined as follows:
import keras.backend as K
from keras.backend import reshape
from keras.applications.vgg16 import preprocess_input
def vgg_loss(model):
def loss(gt, pred):
print(" Inside loss function ")
pred = reshape(pred, (1,) + (128,128,3))
gt = reshape(gt, (1,) + (128,128,3))
gt_in = preprocess_input(gt)
pred_in = preprocess_input(pred)
pred_out = model.predict(pred_in)
gt_out = model.predict(gt_in)
return K.mean(K.mean((pred_out - gt_out)**2, axis = (1,2)), axis = 1)
return loss
When run model.compile, I get the following error related to the usage of symbolic tensors:
Traceback (most recent call last):
File "C:\Users\Dell\.spyder-py3\UNet_keras\train_SO.py", line 32, in <module>
model = unet(input_size = (height,width,nc))
File "C:\Users\Dell\.spyder-py3\UNet_keras\model_SO.py", line 68, in unet
loss = vgg_loss(model_loss), metrics = ['mse'])
File "C:\Users\Dell\Anaconda3\envs\viji_aip\lib\site-packages\keras\engine\training.py", line 229, in
compile
self.total_loss = self._prepare_total_loss(masks)
File "C:\Users\Dell\Anaconda3\envs\viji_aip\lib\site-packages\keras\engine\training.py", line 692, in
_prepare_total_loss
y_true, y_pred, sample_weight=sample_weight)
File "C:\Users\Dell\Anaconda3\envs\viji_aip\lib\site-packages\keras\losses.py", line 71, in __call__
losses = self.call(y_true, y_pred)
File "C:\Users\Dell\Anaconda3\envs\viji_aip\lib\site-packages\keras\losses.py", line 132, in call
return self.fn(y_true, y_pred, **self._fn_kwargs)
File "C:\Users\Dell\.spyder-py3\UNet_keras\data_SO.py", line 28, in loss
pred_out = model.predict(pred)
File "C:\Users\Dell\Anaconda3\envs\viji_aip\lib\site-packages\keras\engine\training.py", line 1464,
in predict
callbacks=callbacks)
File "C:\Users\Dell\Anaconda3\envs\viji_aip\lib\site-packages\keras\engine\training_arrays.py", line
250, in predict_loop
steps_name='steps')
File "C:\Users\Dell\Anaconda3\envs\viji_aip\lib\site-packages\keras\engine\training_utils.py", line
571, in check_num_samples
'you should specify the `' + steps_name + '` argument '
ValueError: If your data is in the form of symbolic tensors, you should specify the `steps` argument
(instead of the `batch_size` argument, because symbolic tensors are expected to produce batches of
input data).
What am I doing wrong here?
You have to use tensorflow-2 instead of keras.
You can't add anything in the loss function and expect it to work, it must be differentiable. You may not need to add preprocessing_input of vgg16 inside the loss, your output from the model shouldn't be perfect images (with range 0-255), they should be already normalized tensors (due to some activations like sigmoid). (Make sure, the VGG16 you're using was also trained with same normalization scheme (0-1) range, as the output from the network will probably pass through an activation function like sigmoid, your output will most likely be in the range(0-1))
Here's a simple model with the loss function of yours:
import tensorflow.keras.backend as K
from tensorflow.keras.applications import *
from tensorflow.keras.layers import *
from tensorflow.keras.models import *
def vgg_loss(model):
def loss(y_true, y_pred):
return K.mean(K.square(model(y_pred) - model(y_true)), axis=-1)
return loss
input_size = (224,224,3)
model_loss = VGG16(include_top=False, input_shape=input_size)
# simple model
ip = Input(input_size)
base_model = Dense(3)(ip)
model = Model(ip, base_model)
model.summary()
model.compile(optimizer = 'adam', loss = vgg_loss(model_loss))
Model: "model"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_3 (InputLayer) [(None, 224, 224, 3)] 0
_________________________________________________________________
dense (Dense) (None, 224, 224, 3) 12
=================================================================
Total params: 12
Trainable params: 12
Non-trainable params: 0
I understand that there are several options for kernel_constraint in Conv2D in Keras: max_norm, non_neg or unit_norm..
But what I needed is to set the anchor (center) position in the filter kernels to be zero.
For example, if we have a filter kernel with its size of (width, height) = (5, 5), and that we have 3 channels in the input. I need to constrain the anchor (center) point of this kernel for every channel to be 0, like w(2,2,:)=0, assuming we put the channel dimension the 3rd dimension. If there are multiple filters, the anchor position of each filter should be with zero. How could I implement this?
I assume a custom kernel constraint is needed. This link gives suggest how to create one class inheriting from Constraint: https://github.com/keras-team/keras/issues/8196. This shows how in-built constraints are implemented:
https://github.com/keras-team/keras/blob/master/keras/constraints.py
But still, I do not know how the dimensions of w are manipulated, and how to set the desired position to be zero. Any help is appreciated. Thanks.
Update:
Daniel Möller's answer was tried. The error message is as follows:
raise ValueError('An operation has None for gradient. '
ValueError: An operation has None for gradient. Please make sure that all of your ops have a gradient defined (i.e. are differentiable). Common ops without gradient: K.argmax, K.round, K.eval.
Since Daniel can run that without problem on his side, to inspect what goes wrong in my program, I post my simplified code here. My data have 8 channels, but it shouldn't matter how many you have.
from keras.layers import Input, Conv2D
from keras.models import Model, optimizers
import numpy as np
import tensorflow as tf
from keras import backend as K
from keras.callbacks import ModelCheckpoint
class ZeroCenterConv2D(Conv2D):
def __init__(self, filters, kernel_size, **kwargs):
super(ZeroCenterConv2D, self).__init__(filters, kernel_size, **kwargs)
def call(self, inputs):
assert self.kernel_size[0] % 2 == 1, "Error: the kernel size is an even number"
assert self.kernel_size[1] % 2 == 1, "Error: the kernel size is an even number"
centerX = (self.kernel_size[0] - 1) // 2
centerY = (self.kernel_size[1] - 1) // 2
kernel_mask = np.ones(self.kernel_size + (1, 1))
kernel_mask[centerX, centerY] = 0
kernel_mask = K.constant(kernel_mask)
customKernel = self.kernel * kernel_mask
outputs = K.conv2d(
inputs,
customKernel,
strides=self.strides,
padding=self.padding,
data_format=self.data_format,
dilation_rate=self.dilation_rate)
if self.activation is not None:
return self.activation(outputs)
return outputs
size1 = 256
size2 = 256
input_img = Input(shape=(size1, size2, 8))
conv1 = ZeroCenterConv2D(8, (5, 5), padding='same', activation='relu')(input_img)
autoencoder = Model(input_img, conv1)
adam = optimizers.Adam(lr=0.001, beta_1=0.9, beta_2=0.999, epsilon=1e-8)
autoencoder.compile(optimizer=adam, loss='mean_squared_error')
import scipy.io
A = scipy.io.loadmat('data_train')
x_train = A['data']
x_train = np.reshape(x_train, (1, 256, 256, 8))
from keras.callbacks import TensorBoard
autoencoder.fit(x_train, x_train,
epochs=5,
batch_size=1,
shuffle=False,
validation_data=(x_train, x_train),
callbacks=[TensorBoard(log_dir='/tmp/autoencoder')])
decoded_imgs = autoencoder.predict(x_train)
When conv1 = ZeroCenterConv2D... was replaced by the conventional conv1 = Conv2D..., everything works.
The full error message:
Connected to pydev debugger (build 181.4668.75)
/home/allen/kerasProject/keras/venv/py2.7/local/lib/python2.7/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
from ._conv import register_converters as _register_converters
Using TensorFlow backend.
Traceback (most recent call last):
File "/snap/pycharm-community/60/helpers/pydev/pydevd.py", line 1664, in <module>
main()
File "/snap/pycharm-community/60/helpers/pydev/pydevd.py", line 1658, in main
globals = debugger.run(setup['file'], None, None, is_module)
File "/snap/pycharm-community/60/helpers/pydev/pydevd.py", line 1068, in run
pydev_imports.execfile(file, globals, locals) # execute the script
File "/home/allen/autotion/temptest", line 62, in <module>
callbacks=[TensorBoard(log_dir='/tmp/autoencoder')])
File "/home/allen/kerasProject/keras/venv/py2.7/local/lib/python2.7/site-packages/keras/engine/training.py", line 1682, in fit
self._make_train_function()
File "/home/allen/kerasProject/keras/venv/py2.7/local/lib/python2.7/site-packages/keras/engine/training.py", line 992, in _make_train_function
loss=self.total_loss)
File "/home/allen/kerasProject/keras/venv/py2.7/local/lib/python2.7/site-packages/keras/legacy/interfaces.py", line 91, in wrapper
return func(*args, **kwargs)
File "/home/allen/kerasProject/keras/venv/py2.7/local/lib/python2.7/site-packages/keras/optimizers.py", line 445, in get_updates
grads = self.get_gradients(loss, params)
File "/home/allen/kerasProject/keras/venv/py2.7/local/lib/python2.7/site-packages/keras/optimizers.py", line 80, in get_gradients
raise ValueError('An operation has `None` for gradient. '
ValueError: An operation has `None` for gradient. Please make sure that all of your ops have a gradient defined (i.e. are differentiable). Common ops without gradient: K.argmax, K.round, K.eval.
Process finished with exit code 1
A further update: adding "bias" part in the code in Daniel's answer (already done), problems solved!
You need a custom Conv2D layer for that, where you change its call method to apply the zero at the center.
class ZeroCenterConv2D(Conv2D):
def __init__(self, filters, kernel_size, **kwargs):
super(ZeroCenterConv2D, self).__init__(filters, kernel_size, **kwargs)
def call(self, inputs):
assert self.kernel_size[0] % 2 == 1, "Error: the kernel size is an even number"
assert self.kernel_size[1] % 2 == 1, "Error: the kernel size is an even number"
centerX = (self.kernel_size[0] - 1) // 2
centerY = (self.kernel_size[1] - 1) // 2
kernel_mask = np.ones(self.kernel_size + (1, 1))
kernel_mask[centerX, centerY] = 0
kernel_mask = K.variable(kernel_mask)
customKernel = self.kernel * kernel_mask
outputs = K.conv2d(
inputs,
customKernel,
strides=self.strides,
padding=self.padding,
data_format=self.data_format,
dilation_rate=self.dilation_rate)
if self.use_bias:
outputs = K.bias_add(
outputs,
self.bias,
data_format=self.data_format)
if self.activation is not None:
return self.activation(outputs)
return outputs
This will not replace the actual weights, though, but the center ones will never be used.
When you use layer.get_weights() of model.get_weights(), you will see the center weights as they were initialized (not as zeros).
I have a model network implemented in keras with theano as the backend which currently initialises with random weights for the filters:
# Combine and reshape for convolution
seq = concat(embeddings)
cshape = (config.window_size, sum(f.output_dim for f in features))
seq = Reshape((1,)+cshape)(seq)
# Convolutions
conv_outputs = []
for filter_size, filter_num in zip(config.filter_sizes, config.filter_nums):
conv = Convolution2D(filter_num, filter_size, cshape[1], activation='relu')(seq)
cout = Flatten()(conv)
conv_outputs.append(cout)
seq = concat(conv_outputs)
But now I want to be able to load in previously generated weights which were stored as numpy arrays.
This is the altered code which attempts to read in the arrays:
# Combine and reshape for convolution
seq = concat(embeddings)
cshape = (config.window_size, sum(f.output_dim for f in features))
seq = Reshape((1,)+cshape)(seq)
# Convolutions
conv_outputs = []
for filter_size, filter_num in zip(config.filter_sizes, config.filter_nums):
filters = np.load('path/to/filters/size-%d.npy' % filter_size)
conv = Convolution2D(filter_num, filter_size, cshape[1], weights=filters)(seq)
cout = Flatten()(conv)
conv_outputs.append(cout)
seq = concat(conv_outputs)
When I try to run the augmented script, I run into the following error:
Traceback (most recent call last):
File "conv.py", line 78, in <module>
weights=filters)(seq)
File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 458, in __call__
self.build(input_shapes[0])
File "/usr/local/lib/python2.7/dist-packages/keras/layers/convolutional.py", line 324, in build
self.set_weights(self.initial_weights)
File "/usr/local/lib/python2.7/dist-packages/keras/engine/topology.py", line 846, in set_weights
' weights. Provided weights: ' + str(weights))
Exception: You called `set_weights(weights)` on layer "convolution2d_1" with a weight list of length 3, but the layer was expecting 2 weights.
I've tried searching online but I can't figure out why keras is throwing this error up.