Related
Hi so what I exactly want is if we have matrix W and vector V such as:
V=[1,2,3,4]
W=[[1,1,1,1],[1,1,1,1],[1,1,1,1],[1,1,1,1]]
we should got the result:
result=[[1,1,1,1],[2,2,2,2],[3,3,3,3],[4,4,4,4]]
I found this method on the website:
V = tf.constant([1,2,4], dtype=tf.float32)
W = tf.constant([[1,2,3,4],[1,2,3,4],[1,2,3,4]], dtype=tf.float32)
tf.multiply(tf.expand_dims(V,1),W)
## produce: [[1,2,3,4],[2,4,6,8],[4,8,12,16]]
which is exactly what I want but when I implement this on my model it also include the batch size of the vector in which result in error such
with input shapes: [?,1,297], [?,297,300].
which I assume is the same error which this can produce
V = tf.constant([[1,2,4]], dtype=tf.float32)
W = tf.constant([[[1,2,3,4],[1,2,3,4],[1,2,3,4]]], dtype=tf.float32)
tf.multiply(tf.expand_dims(V,1),W)
I wanted to know what is the standard procedure to get each element from the softmax output vector and multiply them as weight for each vector in the feature tensor
I found that by using
V = tf.constant([[1,2,4]], dtype=tf.float32)
W = tf.constant([[[1,2,3,4],[1,2,3,4],[1,2,3,4]]], dtype=tf.float32)
h2=tf.keras.layers.multiply([W,tf.expand_dims(V,2)])
the keras layer will ignore the batch size part for us but we have to change the parameter of expand dim because we still have to consider the batch size of V before feed to the layer.
I've coded a custom TensorFlow model. However, when I pass in single tensor to it, it considers each element of that tensor as single input and therefore, gives a batch output.
For eg. my input tensor is of the shape [3,1] with values:
tf.Tensor(
[[0.7001484 ]
[0.2581525 ]
[0.04169908]], shape=(3, 1), dtype=float32)
Corresponding to this I should get a single vector of shape (3,). But what I'm getting is 3x3 tensor:
tf.Tensor(
[[0.31234854 0.3224371 0.36521438]
[0.32561225 0.3294511 0.3449366 ]
[0.33208787 0.33271718 0.33519495]], shape=(3, 3), dtype=float32)
My Model
class MAE_Model(tf.keras.Model):
def __init__(self):
super(MAE_Model, self).__init__()
self.h_fin = EnsembleBlock()
self.ipt = tf.keras.layers.InputLayer(input_shape=(3,1), batch_size=None)
self.fc_1 = tf.keras.layers.Dense(16, activation='relu')
self.fc_2 = tf.keras.layers.Dense(16, activation='relu')
self.classifier = tf.keras.layers.Dense(3, activation='softmax')
def call(self, inputs):
x = self.h_fin(inputs) # this returns a vector of shape [3] E.g. x = [1., 2., 3.]
x = tf.reshape(x, (3,1))
print(x)
x = self.ipt(x)
x = self.fc_1(x)
x = self.fc_2(x)
return self.classifier(x)
Here, print(x) prints a (3,1) tensor, similar to the example above. Is there any way to solve this? I want the model to consider the x as a single input (whole of it) and not as some batch input
The output of your model makes sense. The reason it gives a 3x3 tensor is because the last layer in your model outputs probabilities corresponding to 3 classes. So, given a batch of 3 inputs, your output will have probabilities for 3 classes for each element in your batch. If you want to find the predicted class, you want the classes with the highest probability. Tensorflow allows you to easily find the classes by doing
tf.argmax(predictions, axis=-1)
The corresponding output shape from this operation will be of shape (None,) where None is the batch size.
I have a Tensor as below:
y = tf.placeholder(tf.float32, [None, 3],name="output")
I want to multiply the last of the 3 dimension tensor.
I have tried this:
outputs_with_multiplier = y
outputs_with_multiplier[-1] = tf.multiply(outputs_with_multiplier[-1],tf.constant(2.0))
I received the following error:
outputs_with_multiplier[-1] = tf.multiply(outputs_with_multiplier[-1],tf.constant(2.0))
TypeError: 'Tensor' object does not support item assignment
I have check the following questions for reference, but I didn't found them helpful, may be because I didn't understood them.
1) Tensorflow - matmul of input matrix with batch data
2) Tensor multiplication in Tensorflow
Kindly, help me multiply the Tensors dimension so that it work smoothly.
For example if this is my y = [[1,2,3],[2,3,4],[3,4,5],[2,5,7],[8,9,10],[0,3,2]] So I want to make it outputs_with_multiplier = [[1,2,6],[2,3,8],[3,4,10],[2,5,14],[8,9,20],[0,3,4]]
Please let me know if there is any solution to this.
You can't do an item assignment but you can create a new Tensor. The key is to multiply the first 2 columns by 1 and the 3rd column by 2.
x = tf.placeholder(tf.float32, [None, 3], name="output")
y = tf.constant([[1.0, 1.0, 2.0]])
z = tf.multiply(x, y)
sess = tf.Session()
sess.run(z, feed_dict={x: [[1,2,3],[2,3,4],[3,4,5],[2,5,7],[8,9,10],[0,3,2]]})
I have two datasets, which is like:
input:
array([[[ 0.99309823],
...
[ 0. ]]])
shape : (1, 2501)
output:
array([[0, 0, 0, ..., 0, 0, 1],
...,
[0, 0, 0, ..., 0, 0, 0]])
shape : (2501, 9)
And I processed it with TFLearn; as
input_layer = tflearn.input_data(shape=[None,2501])
hidden1 = tflearn.fully_connected(input_layer,1205,activation='ReLU', regularizer='L2', weight_decay=0.001)
dropout1 = tflearn.dropout(hidden1,0.8)
hidden2 = tflearn.fully_connected(dropout1,1205,activation='ReLU', regularizer='L2', weight_decay=0.001)
dropout2 = tflearn.dropout(hidden2,0.8)
softmax = tflearn.fully_connected(dropout2,9,activation='softmax')
# Regression with SGD
sgd = tflearn.SGD(learning_rate=0.1,lr_decay=0.96, decay_step=1000)
top_k=tflearn.metrics.Top_k(3)
net = tflearn.regression(softmax,optimizer=sgd,metric=top_k,loss='categorical_crossentropy')
model = tflearn.DNN(net)
model.fit(input,output,n_epoch=10,show_metric=True, run_id='dense_model')
It works but not the way that I want. It's a DNN model. I want that when I enter 0.95, model must give me corresponding prediction for example [0,0,0,0,0,0,0,0,1]. However, when I want to enter 0.95, it says that,
ValueError: Cannot feed value of shape (1,) for Tensor 'InputData/X:0', which has shape '(?, 2501)'
When I tried to understand I realise that I need (1,2501) shaped data to predict for my wrong based model.
What i want is for every element in input, predict corresponding element in output. As you can see, in the instance dataset,
for [0.99309823], corresponding output is [0,0,0,0,0,0,0,0,1]. I want tflearn to train itself like this.
I may have wrong structured data, or model(probably dataset), I explained all the things, I need help I'm really out of my mind.
Your input data should be Nx1 (N = number of samples) dimensional to archive this transformation ([0.99309823] --> [0,0,0,0,0,0,0,0,1] ). According to your input data shape, it looks more likely including 1 sample with 2501 dimensions.
ValueError: Cannot feed value of shape (1,) for Tensor 'InputData/X:0', which has shape '(?, 2501)' This error means that tensorflow expecting you to provide a vector with shape (,2501), but you are feeding the network with a vector with shape (1,).
Example modified code with dummy data:
import numpy as np
import tflearn
#creating dummy data
input_data = np.random.rand(1, 2501)
input_data = np.transpose(input_data) # now shape is (2501,1)
output_data = np.random.randint(8, size=2501)
n_values = 9
output_data = np.eye(n_values)[output_data]
# checking the shapes
print input_data.shape #(2501,1)
print output_data.shape #(2501,9)
input_layer = tflearn.input_data(shape=[None,1]) # now network is expecting ( Nx1 )
hidden1 = tflearn.fully_connected(input_layer,1205,activation='ReLU', regularizer='L2', weight_decay=0.001)
dropout1 = tflearn.dropout(hidden1,0.8)
hidden2 = tflearn.fully_connected(dropout1,1205,activation='ReLU', regularizer='L2', weight_decay=0.001)
dropout2 = tflearn.dropout(hidden2,0.8)
softmax = tflearn.fully_connected(dropout2,9,activation='softmax')
# Regression with SGD
sgd = tflearn.SGD(learning_rate=0.1,lr_decay=0.96, decay_step=1000)
top_k=tflearn.metrics.Top_k(3)
net = tflearn.regression(softmax,optimizer=sgd,metric=top_k,loss='categorical_crossentropy')
model = tflearn.DNN(net)
model.fit(input_data, output_data, n_epoch=10,show_metric=True, run_id='dense_model')
Also my friend warned me about same thing as rcmalli. He says
reshape:
input = tf.reshape(input, (2501,1))
change
input_layer = tflearn.input_data(shape=[None,2501])
to
input_layer = tflearn.input_data(shape=[None, 1])
Variable dimension must be "None". In your wrong case, 2501 is the magnitude(or something else, I translated from another lang., but you got it) of your dataset. 1 is constant input magnitude.
I am implementing an RNN and contrarily to the examples I have found which minimize only the cost for the output in the last step
x = tf.placeholder ("float", [features_dimension, None, n_timesteps])
y = tf.placeholder ("float", [labels_dimension, None, n_timesteps])
# Define weights
weights = {'out': tf.Variable (tf.random_normal ([N_HIDDEN, labels_dimension]))}
biases = {'out': tf.Variable (tf.random_normal ([labels_dimension]))}
def RNN (x, weights, biases):
# Prepare data shape to match `rnn` function requirements
# Current data input shape: (features_dimension, BATCH_SIZE, n_timesteps)
# Required shape: `n_timesteps` tensors list of shape (BATCH_SIZE, features_dimension)
# We make a division of the data to split it in individual vectors that
# will be fed for each timestep
# Permuting features_dimension and n_timesteps
# Shape will be (n_timesteps, BATCH_SIZE, features_dimension)
x = tf.transpose (x, [2, 1, 0])
# Reshaping to (BATCH_SIZE*n_timesteps, features_dimension) (we are removing the depth dimension with this)
x = tf.reshape(x, [BATCH_SIZE*n_timesteps, features_dimension])
# Split the previous 2D tensor to get a list of `n_timesteps` tensors of
# shape (batch_size, features_dimension).
x = tf.split (x, n_timesteps, 0)
# Define a lstm cell with tensorflow
lstm_cell = rnn.BasicLSTMCell (N_HIDDEN, forget_bias=1.0)
# Get lstm cell output
outputs, states = rnn.static_rnn (lstm_cell, x, dtype=tf.float32)
# Linear activation; outputs contains the array of outputs for all the
# timesteps
pred = tf.matmul (outputs, weights['out']) + biases['out']
However, the object outputs is a list of Tensor with n_timesteps elements, so the pred = tf.matmul (outputs, weights['out']) + biases['out'] throws the error
ValueError: Shape must be rank 2 but is rank 3 for 'MatMul' (op:
'MatMul') with input shapes: [100,128,16], [16,1].
. How can I do this multiplication?
The solution is to tf.stack the list of tensors into a 3d tensor and then use tf.map_fn to apply the multiplication operation on each 2d tensor along dimension 0:
# Transform the list into a 3D tensor with dimensions (n_timesteps, batch_size, N_HIDDEN)
outputs = tf.stack(outputs)
def pred_fn(current_output):
return tf.matmul(current_output, weights['out']) + biases['out']
# Use tf.map_fn to apply pred_fn to each tensor in outputs, along dimension 0 (timestep dimension)
pred = tf.map_fn(pred_fn, outputs)