I'm working with the keras.datasets.fashion_mnist dataset, which contains 28 x 28 grayscale images. I've built a pretty simple convolutional neural network that accepts a placeholder of images defined as:
X = tf.placeholder(tf.float32, [None, 28, 28, INPUT_CHANNELS], name='X_placeholder')
I'm starting out with a <type 'numpy.ndarray'> of shape (100, 28, 28). 100 here represents the batch size that I've chosen to train with.
Obviously, the dimensionality doesn't line up here. The graph I've built should work with RGB images as well, hence the INPUT_CHANNEL dimension. As expected, when I try to train, I get the following error:
ValueError: Cannot feed value of shape (100, 28, 28) for Tensor u'X_placeholder:0', which has shape '(?, 28, 28, 1)'
Being relatively new to TF and numpy, I'm failing to see how to add in that extra dimension. Having pieced together my code from various sources, I can't say that I chose the placeholder input shape [None, 28, 28, INPUT_CHANNELS], but I want to stick with it instead of trying to work around it.
Question
How can I reshape my training data to match the expected placeholder dimensionality?
In numpy:
You can use np.newaxis,np.expand_dims and reshape() to add dimension.
import numpy as np
train_data = np.random.normal(size=(100,28,28))
print(train_data.shape)
new_a = train_data[...,np.newaxis]
print(new_a.shape)
new_a = np.expand_dims(train_data,axis=-1)
print(new_a.shape)
new_a = train_data.reshape(100,28,28,1)
print(new_a.shape)
(100, 28, 28)
(100, 28, 28, 1)
(100, 28, 28, 1)
(100, 28, 28, 1)
In tensorflow:
You can use tf.newaxis,tf.expand_dims and tf.reshape to add dimension.
import tensorflow as tf
train_data = tf.placeholder(shape=(None,28,28),dtype=tf.float64)
print(train_data.shape)
new_a = train_data[...,tf.newaxis]
print(new_a.shape)
new_a = tf.reshape(train_data,shape=(-1,28,28,1))
print(new_a.shape)
new_a = tf.expand_dims(train_data,axis=-1)
print(new_a.shape)
(?, 28, 28)
(?, 28, 28, 1)
(?, 28, 28, 1)
(?, 28, 28, 1)
Related
I found the method of resizing the MNIST training dataset from (60000, 28, 28) to (60000, 14, 14).
This is the code and results:
import tensorflow as tf
import numpy as np
(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()
x_train, x_test = x_train[..., np.newaxis], x_test[..., np.newaxis]
x_train_small = tf.image.resize(x_train, (14,14)).numpy()
x_test_small = tf.image.resize(x_test, (14,14)).numpy()
print(x_train.shape)
print(x_test.shape)
print(x_train_small.shape)
print(x_test_small.shape)
>>>(60000, 28, 28, 1)
>>>(10000, 28, 28, 1)
>>>(60000, 14, 14, 1)
>>>(10000, 14, 14, 1)
I'm confused about why it has to add a new axis to change the shape that I want.
I would like to know whether there is another method to do the resize work without adding a new axis.
This is all described in the docs:
The first argument of resize is images: "4-D Tensor of shape [batch, height, width, channels] or 3-D Tensor of shape [height, width, channels]."
The second is size: "A 1-D int32 Tensor of 2 elements: new_height, new_width. The new size for the images."
Conclusion: You need the fourth dimension because those are the channels which tf.image.resize expects no matter what. The size along that dimension is 1 because the MNIST image are grayscale.
Of course you could use a some other library to resize, but personally I would avoid unnecessary dependencies, just for the sake of cleanliness.
I am wondering how should we concatenate multiple tensors with different shapes into one tensor in keras. I tried tf.keras.layers.concatenate for that as follow:
import tensorflow as tf
from tf.keras.layers import concatenate
print(tensor_1.shape)
print(tensor_2.shape)
new_tensor = concatenate([tensor_1, tensor_2],axis=1)
new_tensor
but I got the following value error:
shape of tensor_1 (?, 30, 30, 128)
shape of tensor_2 (?, 26, 26, 128)
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-24-96d471a8e99e> in <module>()
----> 1 concatenate([tensor_1, tensor_2], axis=1)
4 frames
/usr/local/lib/python3.6/dist-packages/tensorflow/python/keras/layers/merge.py in build(self, input_shape)
517 shape[axis] for shape in shape_set if shape[axis] is not None)
518 if len(unique_dims) > 1:
--> 519 raise ValueError(err_msg)
520
521 def _merge_function(self, inputs):
ValueError: A `Concatenate` layer requires inputs with matching shapes except for the concat axis. Got inputs shapes: [(None, 30, 30, 128), (None, 26, 26, 128)]
I think in Conv1D, concatenation is simple, in Conv2D, it is not known to me how to make concatenation. Does anyone know how to do this in keras? any idea?
update:
I also tried to treat each tensor as matrix and tried to concatenate them:
tf.concat(0, [[tensor_1], [tensor_2]])
but still, I ended up with a similar value error as follow:
ValueError: Dimension 2 in both shapes must be equal, but are 30 and
26. Shapes are [1,?,30,30,128] and [1,?,26,26,128]. From merging shape 0 with other shapes. for '{{node concat/concat_dim}} = Pack[N=2,
T=DT_INT32, axis=0](concat/concat_dim/0, concat/concat_dim/1)' with
input shapes: [1,?,30,30,128], [1,?,26,26,128].
desired output
I am not sure how multiple tensor can be concatenated, so I am okay with output as tensor with shape of (W, H, C). How should we do this in keras? any idea?
You can concatenate two tensors only when they have the same shapes but for axis along which concatenating.
x = np.arange(20).reshape(2, 2, 5)
y = np.arange(20, 30).reshape(2, 1, 5)
tf.keras.layers.concatenate([x, y], axis=1) # (2, 3, 5)
tf.keras.layers.concatenate
I recommend you rethink whole network so that you don't bother about two different shape tensors. If you can't, I'd say you
crop
x = np.arange(2*30*30*128).reshape(2, 30, 30, 128)
y = np.arange(2*26*26*128).reshape(2, 26, 26, 128)
x2 = tf.keras.layers.Cropping2D(cropping=((0, 0), (2, 2)))(x)
tf.keras.layers.concatenate([x2, y], axis=1) # (2, 56, 26, 128)
tf.keras.layers.Cropping2D
zeropad
x = np.arange(2*30*30*128).reshape(2, 30, 30, 128)
y = np.arange(2*26*26*128).reshape(2, 26, 26, 128)
y = tf.keras.layers.ZeroPadding2D(padding=((0, 0), (2, 2)))(y)
tf.keras.layers.concatenate([x, y2], axis=1) # (2, 56, 30, 128)
tf.keras.layers.ZeroPadding2D
Note: tf.keras.layers.concatenate and tf.keras.layers.Conatenate, which inherits from tf.keras.layers.Layer are different.
I intend to use the concepts of skip connection in my experiment. Basically, in my pipeline, features maps that comes after Conv2D are going to be stacked or concatenated. But, features maps are in different shape and try to stack them together into one tensor gave me error. Does anyone knows any possible way of doing this correctly in tensorflow? Any thoughts or ideas to make this happen? Thanks
idea flowchart
here is the pipeline flowchart I want to do it:
my case is little different because I got extra building block is used after Conv2D and its output now is feature maps of 15x15x64 and so on. I want to stack those features map into one then use it to Conv2D again.
my attempt:
this is my reproducible attempt:
import tensorflow as tf
from tensorflow.keras.layers import Dense, Dropout, Activation, Conv2D, Flatten, MaxPool2D, BatchNormalization
inputs = tf.keras.Input(shape=(32, 32, 3))
x = inputs
x = Conv2D(32, (3, 3), input_shape=(32,32,3))(x)
x = BatchNormalization(axis=-1)(x)
x = Activation('relu')(x)
fm1 = MaxPooling2D(pool_size=(2,2))(x)
x = Conv2D(32,(3, 3), input_shape=(15,15,32))(fm1)
x = BatchNormalization(axis=-1)(x)
x = Activation('relu')(x)
fm2 = MaxPooling2D(pool_size=(2,2))(x)
concatted = tf.keras.layers.Concatenate(axis=1)([fm1, fm2])
but this way I ended up with following error: ValueError: A Concatenate layer requires inputs with matching shapes except for the concat axis. Got inputs shapes: [(None, 15, 15, 32), (None, 6, 6, 32)]. I am not sure what would be correct way to stack features maps with different shape. How can we make this right? Any possible thoughts?
desired output
in my actual model, I got shape of features maps are TensorShape([None, 15, 15, 128]) and TensorShape([None, 6, 6, 128]). I need to find way to merge them or stack them into one. Ideally, shape of concatenated or stacked feature maps' shape would be: [None, 21,21,128]. Is there any way of stacking them into one? Any idea?
What you're trying to achieve doesn't work mathematically. Let me illustrate. Take the simple 1D problem (like 1D convolution). You have a (None, 64, 128) (fm1) sized output and a (None, 32, 128) (fm2) output that you want to concatenate. Then,
concatted = tf.keras.layers.Concatenate(axis=1)([fm1, fm2])
works totally fine, giving you an output of size (None, 96, 128).
Let's come to the 2D problem. Now you got two tensors (None, 15, 15, 128) and (None, 6, 6, 128) and want to end up with a (None, 21, 21, 128) sized output. Well the math doesn't work here. To understand why, reduce this to 1D format. Then you got
fm1 -> (None, 225, 128)
fm2 -> (None, 36, 128)
By concat you get,
concatted -> (None, 261, 128)
If the math works you should get (None, 441, 128) which is reshape-able to (None, 21, 21, 128). So this cannot be achieved unless you pad the edges of the smaller with 441-261 = 180 on the reshaped tensor. And then reshape it to the desired shape. Following is an example of how you can do it,
concatted = tf.keras.layers.Lambda(
lambda x: K.reshape(
K.concatenate(
[K.reshape(x[0], (-1, 225, 128)),
tf.pad(
K.reshape(x[1], (-1, 36, 128)), [(0,0), (0, 180), (0,0)]
)
], axis=1
), (-1, 21, 21, 128))
)([fm1, fm2])
Important: But I can't guaranttee the performance of your model this just solves your problem mathematically. In a machine learning perspective, I wouldn't advice this. Best way would be making sure the outputs are compatible in sizes for concatenation. Few ways would be,
Not reduce the size of convolution outputs (stride = 0 and padding='same')
Use transpose convolution operation to size-up the smaller one
The output of my conv2d function is a 4-D tensor [64, 32, 32, 64] = [batch_size, output_height, output_width, number of filters]. I want to make 3 rows on both sides, and 3 columns also on both sides to zero of output matrix. So, that's why i want to create a similar tensor (mask_tensor) of size [64, 26+3(zero_pad), 26+3 (zero_pad), 64] and multiply with original matrix. I thought of accessing [26, 26] and put zero padding on it...
How to access inner dimensions of tensor ?
import tensorflow as tf
drop_matrix = tf.constant(1, shape=[64, 26, 26, 64], dtype=tf.float32)
paddings = tf.constant([leave this dimension ,pick this dimension, pick this dimension, leave this dimension] )
t = tf.pad(drop_matrix_one_full, paddings, "CONSTANT")
You do not have to access the inner dimensions but can pad per dimension by providing a list of the size of the padding per dimension of the tensor:
>>> drop_matrix = tf.constant(1, shape=[64, 26, 26, 64], dtype=tf.float32)
>>> drop_matrix
<tf.Tensor 'Const:0' shape=(64, 26, 26, 64) dtype=float32>
>>> x = tf.pad(drop_matrix, [[0,0], [3,3], [3,3], [0,0]])
>>> x
<tf.Tensor 'Pad:0' shape=(64, 32, 32, 64) dtype=float32>
You can set the value to pad with to something using constant_values parameter, but the default is 0 so that shouldn't be necessary. For clarification, the 2 elements in each list are padding on either side of that dimension, so padding on just one side could be done by [0,3] for that dimension.
Edited example to be fitting to the situation
Try this to remove the first and last value from the array :
import tensorflow as tf
drop_matrix = tf.constant(1, shape=[64, 26, 26, 64], dtype=tf.float32)
paddings = tf.constant([leave this dimension ,pick this dimension, pick this dimension, leave this dimension][1:-1] )
t = tf.pad(drop_matrix_one_full, paddings, "CONSTANT")
By adding [1:-1] as accessors to your array, you will return an array without the first and last element, effectively accessing the middle values.
I have a variable :
network = input_data(shape=[None, 28, 28, 1])
I want to convert it to the above shape. I'm doing it with help of tflearn.reshape but it isn't working properly. Here is the code "
network = tflearn.reshape(network,(-1, 28, 28))
I don't want to redeclare it to shape [None,28,28] as I have elements in it. Any help on how to do it?
the problem is in the shape you put. it should be [-1,28,28] format.
So your code will be :
network = tflearn.reshape(network,[-1, 28, 28])
Regards