During debuging the Tensorflow code, I would like to output the shape of a tensor, say, print("mask's shape is: ",mask.get_shape()) However, the corresponding output is mask's shape is (?,?,?,1) How to explain this kind of output, is there anyway to know the exactly value of the first three dimensions of this tensor?
This output means that TensorFlow's shape inference has only been able to infer a partial shape for the mask tensor. It has been able to infer (i) that mask is a 4-D tensor, and (ii) its last dimension is 1; but it does not know statically the shape of the first three dimensions.
If you want to get the actual shape of the tensor, the main approaches are:
Compute mask_val = sess.run(mask) and print mask_val.shape.
Create a symbolic mask_shape = tf.shape(mask) tensor, compute mask_shape_val = sess.run(mask_shape) and print `mask_shape.
Shapes usually have unknown components if the shape depends on the data, or if the tensor is itself a function of some tensor(s) with a partially known shape. If you believe that the shape of the mask should be static, you can trace the source of the uncertainty by (recursively) looking at the inputs of the operation(s) that compute mask and finding out where the shape becomes partially known.
Related
I have to normalize a tensor of size size (b, c, h, w) with two tensors of size (c) which represent the respective mean and standard deviation.
I cannot manage to figure out how to multiply a tensor of shape, let say torch.Size([1, 3, 128, 128]) with a tensor of shape torch.Size([3]).
What I want to accomplish is: take the first element of the smaller tensor and multiply the first [128x128] part of the larger tensor with it. And do this for the second element and second [128x128] tensor etc.
def normalize(img, mean, std):
""" Normalizes an image tensor.
# Parameters:
#img, torch.tensor of size (b, c, h, w)
#mean, torch.tensor of size (c)
#std, torch.tensor of size (c)
# Returns the normalized image
"""
# TODO: 1. Implement normalization doing channel-wise z-score normalization.
img * mean #try1: this doesn't work
torch.mul(img.view(3,128,128), mean) #try2: this doesn't work
return img
Both of my attempts throw the same error: RuntimeError: The size of tensor a (128) must match the size of tensor b (3) at non-singleton dimension 3.
I imagine you could create a tensor of the needed size, fill it with the values necessary and multiply that, but I would image there is a better solution than that.
img * mean.reshape(1,3,1,1)
Will reshape the mean tensor so that torch can understand which dimensions you are trying to multiply together.
Edit for details:
Torch reads tensor sizes from lowest to highest dimension, so it can infer some of the higher dimensions (e.g. img * mean.reshape(3,1,1) also works in your case), however you must specify the lower dimensions to either be one, or match the tensor you are trying to multiply with.
I find it difficult to understand a notion about tensors.
For VGG (https://www.tensorflow.org/api_docs/python/tf/keras/applications/VGG16), we start from a batch of colour images (none,224,224,3) and apply 64 2D convolutional filters.
At the output we obtain a tensor of (none,224,224,64), we can see this by making a summary of the model.
However, a filter must treat all 3 colours and my intuition tells me that I should have an output tensor of (none,224,224,3,64).
Could one explain to me why my reasoning is wrong?
Thank you for your explanations.
All filters have shape
(kernel_height, kernel_width, input_channels)
When they pass on your input data with 'SAME' padding, the output shape result is
(input_height, input_width)
And that, for all filters, so
(input_height, input_width, n_filters)
I'm new to Keras and am trying to test out a model I've just trained.
I'm using Tensorflow backend and Python 3.
However, the shape my input has and the shape Keras says it has in an error are completely different. Here's my code:
testnote = np.zeros((3,))
testnote[0] = 70
testnote[1] = 70
print(testnote.shape)
pred = model.predict(testnote)
print(pred)
My consistent output is "(3,)" for the shape of testnote and then an error for my predict line: "ValueError: Error when checking input: expected dense_1_input to have shape (3,) but got array with shape (1,)"
How is it that Keras reads testnote as having shape (1,) when I've just confirmed that the shape is (3,)? Is it using some sort of different standard for what "shape" means? I've tried reshaping and adding brackets and a bunch of other things, but I don't really know what the problem is.
For additional context, the model takes in an array with 3 scalar input (representing pitch, velocity, and instrument class) and outputs an array with 1025 scalar outputs. I am carefully not using the word "dimension" since I think this is where I'm getting confused, and technically both are only 1 dimension. I'm sure there are many problems with my model which I will have to fix following this. However, I'd like to just get this prediction function working so I can understand what my output looks like.
Thanks in advance for any help.
A Keras Model implicitly expects that your data (passed as a np array) has a dimension for the batch size. Currently, your model is interpreting testnote as being 3 examples of shape 1. Try adding the batch dimension to 'testnote' as follows:
testnote = testnote.reshape(1,-1)
This will reshape testnote to shape (1, 3), so that you explicitly define the batch size to be 1.
I tried do some customized padding before feeding to a conv1D net as following.
x=tf.placeholder("float",[None,50,1])
padding=tf.constant([0,0],[5,0],[0,0])
y=tf.pad(x,padding)
However, after the above manipulation, y would be a tensor of shape (?,?,?), thus when feeding to tf.layers.conv1d, I get an error that "The channel dimension of the inputs should be defined. Found 'None'".
My question is why does pad result has None shape? It should not be hard to calculate the shape, my guess is this is only calculated in run time, but it is not convenient right? And can I use reshape before passing to conv1d?
I created a CNN whith Python and Keras which compresses 2D input of various length into a single output. All images have a height of 80 pixels, but different lenght, e.g. shape (80, lenght_of_image_i, 2), where 2 is the number of color channels.
I have 5000 images, the shape of the training data array X in numpy is (5000, 1) and the array has dtype object. This is because storing content with different shape is not possible in a single numpy array. Each object in the list has shape (80, lenght_of_image_i, 2).
With this said, when I call the model.fit(X,y) function of the sequential model, I get the following error:
ValueError: Error when checking input: expected conv2d_1_input to have 4
dimensions, but got array with shape (5000, 1)
Converting the numpy array to Python list of numpy arrays also doesn't work:
AttributeError: 'list' object has no attribute 'ndim'
Zero padding or transformations of my data to get all of my images to the same shape is not an option.
My Question now is: How can I call the model.fit(X,y) function when my data has not a fixed shape?
Thank you in advance!
Edit: Note that I do not have a problem with the architecture of my network (since I am not using dense layers). My problem is that I cannot call the fit function, due to problems with the shape of the numpy array.
My model is a replicate of this network: http://machine-listening.eecs.qmul.ac.uk/wp-content/uploads/sites/26/2017/01/sparrow.pdf
You need to pass "numpy arrays" to fit, of type "float". That is the only possibility.
So, you will probably have to group batches of images with the same length, or train each sample individually:
for image, output in zip(images,outputs):
model.train_on_batch(image.reshape((1,80,-1,2), outputs.reshape((1,)+outputs.shape, ....)