I have an image dataset of a 1000 images, which I have created embeddings for. Each embeddings (512 embeddings for each image with a 256-d vector) is an ndarray of shape (512, 256), so the total array shape would be (1000, 512, 256).
Now, from each image (1000), I want to create a group of observation for the first embedding, of the 512 available, and collecting this embedding from each image. Then I want to do this for the second embedding, third, fourth, up to the 512th.
How would I go about creating these groups?
You can achieve that as follows:
groups = []
for i in range(512):
# Select the i-th embedding from each image
group = embeddings[:, i, :]
groups.append(group)
groups = np.array(groups)
Another optimized solution:
groups = np.array([embeddings[:, i, :] for i in range(512)])
groups = np.transpose(groups, (1, 0, 2))
Related
Imagine that you have a batch of two lists of embeddings of the same dimension 4. Each sequence has a different length. You want to compute a function of the two sequences for each pair in the batch. To this end, you write something like
import tensorflow as tf
# suppose the batch size is 1 for simplicity
a = tf.random.normal((1, 100, 4))
b = tf.random.normal((1, 57, 4))
batch = tf.ragged.stack((a, b), axis=1)
This works okay, yielding a tensor with shape batch.shape == TensorShape([1, None, None, 4]). Now, why is the following an IndexError?
batch[0][0]
I'm doing my project about machine learning and I need to merge (concatenate) two tensors that have different shapes.
For more details:
We're trying to concatenate an matrix of tokens with an one hot matrix. tokens pass through an embedding layer so we get a weights matrix with shape like (100, 10, 300).
Finally we need to merge one hot matrix and weights matrix like this:
(100, 300) and (100, 10, 300) to be (100, >11, 300)
This is, to append each of the 300 one hot vectors in matrix in the first position of each weight matrix like (1,300) + (1,10,300) to get a sample of merged values with shape (1,>11,300)
I actually reached this in a manual form through a loop but it takes too much time so I wanted to know if this is posible through keras or any other similar.
This is the function I wrote so here I reached what I wanted, but if it is possible to do in a better way that doesn't take too much time is the ideal.
def join_demo_sentence(X, Demo, embedding, max_length):
X = pad_sequences(X, maxlen=max_length, padding='post')
Demo = pad_sequences(Demo, maxlen=300, padding='post')
joined = []
for i, sequence in tqdm(enumerate(X), desc='Joining'):
demo = Demo[i]
sequence = embedding.get_weights()[0][sequence]
join = np.insert(sequence.T, 0, demo, axis=1)
joined.append(join.T)
X = np.asarray(joined)
return X
That function loops through the matrix to join demo one hot values and sentence tokens so for final result I get the sentences with the one hot of demographic in first position.
I'm learning about keras so I think there's a way with keras.layers.Concatenate
Is it what you need:
a = tf.random.uniform((100, 10, 300))
b = tf.random.uniform((100, 300))
b = b[:, tf.newaxis, :] # add the second axis
res = tf.concat((a, b), -2)
I have a bunch of images that are grouped into tensor of a following shape:
> images.shape
produces (2000, 1440, 1, 16) which have the following meaning (rows, cols, channels, images_count)
Now for the sake of explanation simplicity I need to perform a weighted sum of those images that would result with one image i.e. (2000, 1440, 1).
Actually there are multiple groups of weights (over 128) and this means that out of 16 input images I get 128 merged images instead of just one which judging by the image size is pretty heavy operation.
And so I'm looking for ways / ideas that would allow me to perform the operation fast and efficiently with minimal amount of temporaries and memory size consumed.
Are there any mechanisms in TF that would allow to perform this operation efficiently and fast?
Thank you in advance!
Suppose for simplicity that you have a 8 different groups of weights and the data is in format you have specified.
First we convert images to a conventional batch_size first form. Than we expand dimensions of images by adding second one to support broadcasting when we are doing element-wise multiplication between images and weights. Finally, we reduce the first dimension by computing the (already weighted) sum of images for each weight group.
import tensorflow as tf
import numpy as np
x = tf.placeholder(tf.float32, shape=(2000, 1440, 1, None))
w = tf.placeholder(tf.float32, shape=(None, 2000, 1440, 1))
xtransposed = tf.transpose(x, perm=[3, 0, 1, 2]) # n_samples first
xexpanded = tf.expand_dims(xtransposed, 1) # expand for broadcasting
multiplied = xexpanded * w
reduced = tf.reduce_sum(multiplied, axis=0) # weighted sum over all images
images = np.random.normal(size=(2000, 1440, 1, 16))
weights = np.random.normal(size=(8, 2000, 1440, 1))
with tf.Session() as sess:
res = sess.run(reduced, feed_dict={x:images, w:weights})
print(res.shape) # (8, 2000, 1440, 1)
res now stores weighted sums for 8 different weights groups in numpy format.
I am learning about deep learning and tensorflow in this website https://pythonprogramming.net/loading-custom-data-deep-learning-python-tensorflow-keras/
I want to know is there any rule of thumb to reshape the array of the datasets?
X = np.array(X).reshape(-1, IMG_SIZE, IMG_SIZE, 1)
For instance, from the code snippet above, I don't understand the meaning of the numbers passed to function reshape(). How to know the suitable numbers to be passed to the function?
The 4 dimensions in
X = np.array(X).reshape(-1, IMG_SIZE, IMG_SIZE, 1)
are the following:
dimension 0: images in the batch
dimension 1: rows
dimension 2: columns
dimension 3: number of channels in image
Since this is grayscale, the image has 1 channel, so the last number is 1 (if it was RGB image, the last number would have been 3, representing the 3 channels in RGB).
Dimensions 1 and 2 are the numbers of rows and columns in every image, which in your case is just IMG_SIZE. Dimension 0 is the batch size. You can specify there the number if you knowthe batch size, or you can leave it as -1, and it will be determined uniquely because all the other sizes (dimensions 1,2,3) are provided.
I extract the features of an image with ResNet of the 'res5c' layer, resulting of a numpy array of shape (2048, 14, 14)
I have trouble manipulating these dimensions. I understand there is 14*14 features of size 2048. I would like to iterate over to access every feature at a time.
Therefore, how I can reshape this to an array of (14*14, 2048) without mistakes and then easily iterate over it with a for loop?
You can read the features after net.forward():
feat = net.blobs['res5c'].data.cop() # copy to be on the safe side.
As you describe, feat is an np.array with shape = (2048, 14, 14).
You can reshape it:
feat.reshape((2048,-1)) # fix the first dimension to 2048, -1 set the number of features to match that of `feat`.
Now you can iterate over features:
for fi in xrange(feat.shape[1]):
f = feat[:,fi] # get the fi-th feature
# do somethinf to the feature f