Develop Reference

Transform 2D embeddings into 3D embeddings

I want to fuse node embeddings from a GNN shaped {[N, 128], where N can vary, with hidden state from an LSTM shaped [2, 32, 128]. To do so, I wanted to add both feature tensors, but they need to be the same shape.
I have something like:
N = 75
t = torch.rand(N, 128)
t.unsqueeze_(0)
t = t.expand(2,N, 128)
I do not seem to figure out how to make dimension 1 equal to 32. I am looking for some combination of transforms and learnable layers that can help me make this shape conversion. Thanks in advance for any help you can give.

How to make a multidimensional array from one dimensional arrays?

I am preparing data for a convolutional neural network model. I am new to deep learning and want to start with seeing how LeNet 5 will work with my data (as it has few parameters).
I prepared 9 NumPy arrays, every array has dimension (10530, 32, 32, 1), 10530 images and each image is 32 x 32 pixel. I want to make one array has (10530, 32, 32, 9)
I tried np.concatenate but it is not working. Do you have a suggestion?

np.concatenate is the way:
np.concatenate(list_of_arrays, axis=-1)

Tensorflow Keras Conv2D error with 2D numpy array input

I would like to train a CNN using a 2D numpy array as input, but I am receiving this error: ValueError: Error when checking input: expected conv2d_input to have 4 dimensions, but got array with shape (21, 21).
My input is indeed a 21x21 numpy array of floats. The first layer of the network is defined as Conv2D(32, (3, 3), input_shape=(21, 21, 1)) to match the shape of the input array.
I have found some similar questions but none pertaining to a 2D input array, they mostly deal with images. According to the documentation, Conv2D is expecting an input of a 4D tensor containing (samples, channels, rows, cols), but I cannot find any documentation explaining the meaning of these values. Similar questions pertaining to image inputs suggest reshaping the input array using np.ndarray.reshape(), but when trying to do that I receive an input error.
How can I train a CNN on such an input array? Should input_shape be a different size tuple?

Your current numpy array has dimensions (21, 21). However, TensorFlow expects input tensors to have dimensions in the format (batch_size, height, width, channels) or BHWC implying that you need to convert your numpy input array to 4 dimensions (from the current 2 dimensions). One way to do so is as follows:
input = np.expand_dims(input, axis=0)
input = np.expand_dims(input, axis=-1)
Now, the numpy input array has dimensions: (1, 21, 21, 1) which can be passed to a TF Conv2D operation.
Hope this helps! :)

Call model.fit in Keras for inputs of different shapes?

I created a CNN whith Python and Keras which compresses 2D input of various length into a single output. All images have a height of 80 pixels, but different lenght, e.g. shape (80, lenght_of_image_i, 2), where 2 is the number of color channels.
I have 5000 images, the shape of the training data array X in numpy is (5000, 1) and the array has dtype object. This is because storing content with different shape is not possible in a single numpy array. Each object in the list has shape (80, lenght_of_image_i, 2).
With this said, when I call the model.fit(X,y) function of the sequential model, I get the following error:
ValueError: Error when checking input: expected conv2d_1_input to have 4
dimensions, but got array with shape (5000, 1)
Converting the numpy array to Python list of numpy arrays also doesn't work:
AttributeError: 'list' object has no attribute 'ndim'
Zero padding or transformations of my data to get all of my images to the same shape is not an option.
My Question now is: How can I call the model.fit(X,y) function when my data has not a fixed shape?
Thank you in advance!
Edit: Note that I do not have a problem with the architecture of my network (since I am not using dense layers). My problem is that I cannot call the fit function, due to problems with the shape of the numpy array.
My model is a replicate of this network: http://machine-listening.eecs.qmul.ac.uk/wp-content/uploads/sites/26/2017/01/sparrow.pdf

You need to pass "numpy arrays" to fit, of type "float". That is the only possibility.
So, you will probably have to group batches of images with the same length, or train each sample individually:
for image, output in zip(images,outputs):
model.train_on_batch(image.reshape((1,80,-1,2), outputs.reshape((1,)+outputs.shape, ....)

How to reshape the res5c layer of ResNet (3D to 2D)?

I extract the features of an image with ResNet of the 'res5c' layer, resulting of a numpy array of shape (2048, 14, 14)
I have trouble manipulating these dimensions. I understand there is 14*14 features of size 2048. I would like to iterate over to access every feature at a time.
Therefore, how I can reshape this to an array of (14*14, 2048) without mistakes and then easily iterate over it with a for loop?

You can read the features after net.forward():
feat = net.blobs['res5c'].data.cop() # copy to be on the safe side.
As you describe, feat is an np.array with shape = (2048, 14, 14).
You can reshape it:
feat.reshape((2048,-1)) # fix the first dimension to 2048, -1 set the number of features to match that of `feat`.
Now you can iterate over features:
for fi in xrange(feat.shape[1]):
f = feat[:,fi] # get the fi-th feature
# do somethinf to the feature f