I have a tensor of the shape T x B x N (training data for a RNN, T is max seq length, B is number of batches, and N number of features) and I'd like to flatten all the features across timesteps, such that I get a tensor of the shape B x TN. Haven't been able to figure out how to do this..
You need to permute your axes before flattening, like so:
t = t.swapdims(0,1) # (T,B,N) -> (B,T,N)
t = t.view(B,-1) # (B,T,N) -> (B,T*N) (equivalent to `t.view(B,T*N)`)
Related
For a time series prediction, I'm testing several models and size in order to evaluate which model/configuration is the more accurate for the datasets to learn.
A generic dataset start from a matrix n x m of examples and features.
With a common LSTM, conv1d, GRU,.. I reshape the matrix in 3d tensor n x q x m of examples, timestep and features as feed for the model. I then split the 3d tensor in 2 sub tensor of train and validation with n1 x... and n2 x .... where n1 + n2 = n.
....But for a convLSTM2d model, how can I reshape my 2d dataset to a 4d tensor in order to feed this new model ? I red several instruction on the web and in case of image processing, I could use its shape as 1st and 2nd element of the tensor. But in case of time series, which parameters I should use for obtain a coherent 4d tensor (suitable to then split in 2 coherent sub 4d tensors) ?
Thanks for help me
Let's say I have two tensors, whose shapes are [b, n] and [b, n, m] respectively. These can be interpreted as a batch of input vectors each of shape [n] and a batch of weight matrices each of shape [n, m], where the batch size is b. I would like to pair these up element-wise across the first dimension, so each input vector has a corresponding weight matrix, and then multiply each input by its weights, resulting in a tensor of shape [b, m].
In normal Python I suspect this would look something like
output_list = [matmul(w, i) for w, i in zip(weight_list, input_list)]
but haven't been able to find a Tensorflow analogue; is there a way of doing this?
tf.matmul can do a matmul over each training example in the batch. But you need to deal with some dimensions problem to achieve your goal.
import tensorflow as tf
b,n,m = 4,3,2
weight_list = tf.random.normal(shape=(b,n,m))
input_list = tf.random.normal(shape=(b,n))
result = tf.squeeze(tf.matmul(tf.expand_dims(input_list,axis=1),weight_list))
print(result.shape)
(4, 2)
I have the following neural network in Keras:
inp = layers.Input((3,))
#Middle layers omitted
out_prop = layers.Dense(units=3, activation='softmax')(inp)
out_value = layers.Dense(units=1, activation = 'linear')(inp)
Then I prepared a pseudo-input to test my network:
inpu = np.array([[1,2,3],[4,5,6],[7,8,9]])
When I try to predict, this happens:
In [45]:nn.network.predict(inpu)
Out[45]:
[array([[0.257513 , 0.41672954, 0.32575747],
[0.20175152, 0.4763418 , 0.32190666],
[0.15986516, 0.53449154, 0.30564335]], dtype=float32),
array([[-0.24281949],
[-0.10461146],
[ 0.11201331]], dtype=float32)]
So, as you can see above, I wanted two output: one should have been an array with size 3, the other should have been a normal value. Instead, I get a 3x3 matrix, and an array with 3 elements. What am I doing wrong?
You are passing three input samples to the network:
>>> inpu.shape
(3,3) # three samples of size 3
And you have two output layers: one of them outputs a vector of size 3 for each sample and the other outputs a vector of size one (i.e. scalar), again for each sample. As a result the output shapes would be (3, 3) and (3, 1).
Update: If you want your network to accept an input sample of shape (3,3) and outputs vectors of size 3 and 1, and you want to only use Dense layers in your network, then you must use a Flatten layer somewhere in the model. One possible option is to use it right after the input layer:
inp = layers.Input((3,3)) # don't forget to set the correct input shape
x = Flatten()(inp)
# pass x to other Dense layers
Alternatively, you could flatten your data to have a shape of (num_samples, 9) and then pass it to your network without using a Flatten layer.
Update 2: As #Mete correctly pointed out in the comments, make sure the input array have a shape of (num_samples, 3, 3) if each input sample has a shape of (3,3).
I want to create a shallow network that would take a vector and pass it through a network.
I have a vector that is of size 6. vec = [0,1,4,5,1,4,5]
My network:
vec_a = Input(shape=(6,))
x_1 = Convolution1D(nb_filter=10, filter_length=1, input_shape=(1, 6), activation='relu')(vec_a)
x_1 = Dense(16, activation='relu')(x_1)
But I keep getting:
ValueError: Input 0 is incompatible with layer conv1d_1: expected
ndim=3, found ndim=2
The shape of the training data to the fit function is:
(36400, 6)
You have to reshape the input data to have the correct input dimension, e.g.:
your_input_array.reshape(-1, 6, 1)
In addition your input layer should look like:
vec_a = Input(shape=(6,1))
The reason is that the 1D in Conv1D relates to the use of a sequence. But this sequence can have a vector of multiple values at each position. In your case it is the same, but you have "only" a vector of length 1 in the last dimension.
How is it that you would create a tensorflow vector from a tensorflow constant/variable etc?
For example I have a constant x and I want to create a vector which is [x].
I have tried the code below and it doesn't work.
Any help would be appreciated.
x = tf.placeholder_with_default(1.0,[], name="x")
nextdd = tf.constant([x], shape=[1], dtype=tf.float32)
First I'd like to define a tensor for you:
Tensors are n-dimensional matrices. A rank 0 tensor is a scalar, e.g. 42. a rank 1 tensor is a Vector, e.g. [1,2,3], a rank 2 tensor is a matrix, a rank 3 tensor might be an image of shape [640, 480, 3] (640x480 resolution, 3 color channels). a rank 4 tensor might be a batch of such images of shape [10, 640, 480, 3] (10 640x480 images), etc.
Second, you have basically 4 types of tensors in Tensorflow.
1) Placeholders - these are tensors that you pass into tensorflow when you call sess.run. For example: sess.run([nextdd], {x:[1,2,3]}) creates a rank 1 tensor out of x.
2) Constants - these are fixed values as the name suggests. E.g. tf.constant(42) and should be specified at compile time, not runtime (eluding to your primary mistake here).
3) Computed tensors - x = tf.add(a,b) is a computed tensor, it's computed from a,b. Its value is not stored after the computation is finished.
4) Variables - These are mutable tensors that are kept around after the computation is complete. For example the weights of a neural network.
Now to address your question explicitly. x is already a tensor. If you were passing in a vector then it's a rank 1 tensor (aka a vector). You can use it just like you'd use a constant, computed tensor, or variable. They all work the same in operations. There is no reason for the nextdd line at all.
Now, nextdd fails becuase you tried to create a constant from a variable term, which isn't a defined operation. tf.constant(42) is well defined, that's what a constant is.
You could just use x directly, as in:
x = tf.placeholder_with_default(1.0,[], name="x")
y = tf.add(x, x)
sess = tf.InteractiveSession()
y.eval()
Result:
2.0
From you description, it looks like you want to use tf.expand_dims:
# 't' is a tensor of shape [2]
tf.shape(tf.expand_dims(t, 0)) # [1, 2]
tf.shape(tf.expand_dims(t, 1)) # [2, 1]
tf.shape(tf.expand_dims(t, -1)) # [2, 1]