I have a bunch of images that are grouped into tensor of a following shape:
> images.shape
produces (2000, 1440, 1, 16) which have the following meaning (rows, cols, channels, images_count)
Now for the sake of explanation simplicity I need to perform a weighted sum of those images that would result with one image i.e. (2000, 1440, 1).
Actually there are multiple groups of weights (over 128) and this means that out of 16 input images I get 128 merged images instead of just one which judging by the image size is pretty heavy operation.
And so I'm looking for ways / ideas that would allow me to perform the operation fast and efficiently with minimal amount of temporaries and memory size consumed.
Are there any mechanisms in TF that would allow to perform this operation efficiently and fast?
Thank you in advance!
Suppose for simplicity that you have a 8 different groups of weights and the data is in format you have specified.
First we convert images to a conventional batch_size first form. Than we expand dimensions of images by adding second one to support broadcasting when we are doing element-wise multiplication between images and weights. Finally, we reduce the first dimension by computing the (already weighted) sum of images for each weight group.
import tensorflow as tf
import numpy as np
x = tf.placeholder(tf.float32, shape=(2000, 1440, 1, None))
w = tf.placeholder(tf.float32, shape=(None, 2000, 1440, 1))
xtransposed = tf.transpose(x, perm=[3, 0, 1, 2]) # n_samples first
xexpanded = tf.expand_dims(xtransposed, 1) # expand for broadcasting
multiplied = xexpanded * w
reduced = tf.reduce_sum(multiplied, axis=0) # weighted sum over all images
images = np.random.normal(size=(2000, 1440, 1, 16))
weights = np.random.normal(size=(8, 2000, 1440, 1))
with tf.Session() as sess:
res = sess.run(reduced, feed_dict={x:images, w:weights})
print(res.shape) # (8, 2000, 1440, 1)
res now stores weighted sums for 8 different weights groups in numpy format.
Related
I have an image dataset of a 1000 images, which I have created embeddings for. Each embeddings (512 embeddings for each image with a 256-d vector) is an ndarray of shape (512, 256), so the total array shape would be (1000, 512, 256).
Now, from each image (1000), I want to create a group of observation for the first embedding, of the 512 available, and collecting this embedding from each image. Then I want to do this for the second embedding, third, fourth, up to the 512th.
How would I go about creating these groups?
You can achieve that as follows:
groups = []
for i in range(512):
# Select the i-th embedding from each image
group = embeddings[:, i, :]
groups.append(group)
groups = np.array(groups)
Another optimized solution:
groups = np.array([embeddings[:, i, :] for i in range(512)])
groups = np.transpose(groups, (1, 0, 2))
Imagine that you have a batch of two lists of embeddings of the same dimension 4. Each sequence has a different length. You want to compute a function of the two sequences for each pair in the batch. To this end, you write something like
import tensorflow as tf
# suppose the batch size is 1 for simplicity
a = tf.random.normal((1, 100, 4))
b = tf.random.normal((1, 57, 4))
batch = tf.ragged.stack((a, b), axis=1)
This works okay, yielding a tensor with shape batch.shape == TensorShape([1, None, None, 4]). Now, why is the following an IndexError?
batch[0][0]
I'm doing my project about machine learning and I need to merge (concatenate) two tensors that have different shapes.
For more details:
We're trying to concatenate an matrix of tokens with an one hot matrix. tokens pass through an embedding layer so we get a weights matrix with shape like (100, 10, 300).
Finally we need to merge one hot matrix and weights matrix like this:
(100, 300) and (100, 10, 300) to be (100, >11, 300)
This is, to append each of the 300 one hot vectors in matrix in the first position of each weight matrix like (1,300) + (1,10,300) to get a sample of merged values with shape (1,>11,300)
I actually reached this in a manual form through a loop but it takes too much time so I wanted to know if this is posible through keras or any other similar.
This is the function I wrote so here I reached what I wanted, but if it is possible to do in a better way that doesn't take too much time is the ideal.
def join_demo_sentence(X, Demo, embedding, max_length):
X = pad_sequences(X, maxlen=max_length, padding='post')
Demo = pad_sequences(Demo, maxlen=300, padding='post')
joined = []
for i, sequence in tqdm(enumerate(X), desc='Joining'):
demo = Demo[i]
sequence = embedding.get_weights()[0][sequence]
join = np.insert(sequence.T, 0, demo, axis=1)
joined.append(join.T)
X = np.asarray(joined)
return X
That function loops through the matrix to join demo one hot values and sentence tokens so for final result I get the sentences with the one hot of demographic in first position.
I'm learning about keras so I think there's a way with keras.layers.Concatenate
Is it what you need:
a = tf.random.uniform((100, 10, 300))
b = tf.random.uniform((100, 300))
b = b[:, tf.newaxis, :] # add the second axis
res = tf.concat((a, b), -2)
In my problem, I want to convolve two tensors in my neural network model.
The shape of two tensors is [None, 2, 1], [None, 3, 1] respectively. The axis with dimension None means the batch size of the input tensor. For each sample in batch, I want to convolve the two tensors with shape [2, 1] and [3, 1].
However, the tf.nn.conv1d in TensorFlow can only convolve the input with a fixed kernel. Is there any function that can support the convolution of two tensors according to the batch size axis, similar to the tf.multiply which can multiply two tensors for each sample or just elementwise multiplication.
The code I ran can be simplified as follows:
input_signal = Input(shape=(L, M), name='input_signal')
input_h = Input(shape=(N), name='input_h')
faded= Lambda(lambda x: tf.nn.conv1d(input, x))(input_h)
What I want to do is that the sample of input_signal can be convolved by the sample of input_h with the same index. However, it just shows my pure idea which can not be able to run in the env. My question is that how I can modify the code to enable the input tensor can be convolved with another input tensor for every sample in the batch.
According to the description of the kernel size arguments for Conv1D layer or any other layer mentioned in the documentation, you cannot add multiple filters with different Kernel size or strides.
Also, Convolutions with Kernels of different sizes will produce outputs of different height and width.
The general formula for output size assuming a symmetric kernel is given by
(X−K+2P)/S+1
Where X is the input Height / Width
K is the Kernel size
P is the zero-padding
S is the stride length
So assuming you are keeping zero paddings and stride same you cannot have multiple kernels with different sizes in ConvD layer.
You can, however, use the tf.keras.Model API to create Conv1D multiple times on the same input OR multiple Conv1D Layer for different inputs and kernel size respectively in your case and then either maxpool, crop or use zero paddings to match the dimensions of the different outputs before stacking them.
Example:
inputs = tf.keras.Input(shape=(n_timesteps,n_features))
x1 = tf.keras.layers.Conv1D(filters=32, kernel_size=2)(inputs)
x2 = tf.keras.layers.Conv1D(filters=16, kernel_size=3)(inputs)
#match dimensions (height and width) of x1 or x2 here
x3 = tf.keras.layers.Concatenate(axis=-1)[x1,x2]
You can use either Zeropadding1D or Cropping2D or Maxpool1D for matching the dimensions.
Reading the Tensorflow MNIST tutorial, I stumbled over the line
x_image = tf.reshape(x, [-1,28,28,1])
28, 28 comes from width, height, 1 comes from the number of channels. But why -1?
I guess this is related to mini-batch training, but I wondered why -1 and not 1 (which seems to give the same result in numpy).
(Probably related: Why does the reshape of numpy give the same results for -1,-2 and 1)?
-1 means that the length in that dimension is inferred. This is done based on the constraint that the number of elements in an ndarray or Tensor when reshaped must remain the same. In the tutorial, each image is a row vector (784 elements) and there are lots of such rows (let it be n, so there are 784n elements). So, when you write
x_image = tf.reshape(x, [-1, 28, 28, 1])
TensorFlow can infer that -1 is n.
In the MNIST tutorial that you are reading, the desired shape for your input layer : [batch_size, 28, 28, 1]
x_image = tf.reshape(x, [-1,28,28,1])
Here -1 for input x specifies that this dimension should be dynamically computed based on the number of input values in x, holding the size of all other dimensions constant. This allows us to treat batch_size(parameter with value -1) as a hyperparameter that we can tune.
−1 indicates that the length on the current axis needs to be automatically deduced according to the rule that the total elements of the tensor remain unchanged