I have a script and I am trying to convert my math operations from NumPy operations to TensorFlow operations so it can get faster on GPU. And in my script I end up in a situation that I have an array with shape (260) and need to do matrix multiplication with another array with shape (260), illustrated by:
import numpy as np
x = np.array([2] * 260)
y = np.array([4] * 260)
r = np.matmul(x,y) #np.dot(x,y) also works
print(r) #2080
But the same operation in TensorFlow is not possible.
import tensorflow as tf
x = tf.Variable([2] * 260)
y = tf.Variable([4] * 260)
r = tf.matmul(x,y)
init = tf.initialize_all_variables()
sess = tf.Session()
sess.run(init)
result = sess.run(r)
print(result) # ERRROR
The TensorFlow error says:
ValueError: Shape must be rank 2 but is rank 1 for 'MatMul' (op:
'MatMul') with input shapes: [260], [260].
I have tried to reshape the inputs countless many ways, and none of those have worked, such as: x = tf.expand_dims(x,1).
Since both inputs are 1-dimensional, your matrix multiplication is the inner product,
tf.reduce_sum(tf.multiply(x, y))
or
tf.tensordot(x, y, 1)
Also see this answer for a few alternative ways of calculating the inner product.
Related
In follow-up to [this question], a few notes on what we're looking to accomplish:
We have two inputs X and Y of different sample sizes n and m, and a boolean vector z of size nm. Each element of z denotes whether two items in X and Y are some sort of match
We want to use embedding layers (perhaps the same, perhaps different) on X and Y before pairwise concatenating the output of these embedding layers to derive input to an output layer.
Here's one example of what a simple network could look like:
The linked answer and a few other resources helped get as far as this example, which builds a model but throws this error at fit time: ValueError: All input arrays (x) should have the same number of samples. Got array shapes: [(10, 2), (12, 2)].
Recent versions of Keras allow for skipping the dimension check, can this check be skipped in Tensorflow? I'd also be happy to use Keras, but I'm not sure how to perform the reshape and concatenate in Keras in the middle of a model.
Or, is this simply not possible in either? Is the only option to expand and pairwise concatenate prior to input?
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
import numpy as np
k = 2
N = 10
M = 12
x = np.random.randint(2, size = 2 * N).reshape((-1,2))
y = np.random.randint(2, size = 2 * M).reshape((-1,2))
x_rep = np.tile(x, (1, M)).reshape((-1,2))
y_rep = np.tile(y, (N, 1))
xy = np.concatenate((x_rep, y_rep), axis=1)
xy = xy.astype(bool)
z = (xy[:,0] == xy[:,2]) * (xy[:,1] ^ xy[:,3])
print(z[:20])
xy = xy.astype(int)
z = z.astype(int)
first = keras.Input(shape=(k,))
second = keras.Input(shape=(k,))
shared_dense = layers.Dense(k)
first_dense = shared_dense(first)
second_dense = shared_dense(second)
first_tiled = layers.Lambda(tf.tile, arguments={'multiples':[1, M]}, name='first_expanded' )(first_dense) #keras.backend.tile(first_dense, [1, M])
second_tiled = layers.Lambda(tf.tile, arguments={'multiples':[N,1]}, name='second_expanded')(second_dense) #keras.backend.tile(first_dense, [1, M])
first_reshaped = layers.Reshape((k,))(first_tiled)
concatenated = layers.Concatenate()([first_reshaped, second_tiled])
out = layers.Dense(1)(concatenated)
model = keras.Model([first, second], out)
keras.utils.plot_model(model, 'tf_nw.png', show_shapes=True)
model.compile('Adam', loss='binary_crossentropy', metrics=['accuracy'])
model.fit([x, y], z)
I have a Tensor as below:
y = tf.placeholder(tf.float32, [None, 3],name="output")
I want to multiply the last of the 3 dimension tensor.
I have tried this:
outputs_with_multiplier = y
outputs_with_multiplier[-1] = tf.multiply(outputs_with_multiplier[-1],tf.constant(2.0))
I received the following error:
outputs_with_multiplier[-1] = tf.multiply(outputs_with_multiplier[-1],tf.constant(2.0))
TypeError: 'Tensor' object does not support item assignment
I have check the following questions for reference, but I didn't found them helpful, may be because I didn't understood them.
1) Tensorflow - matmul of input matrix with batch data
2) Tensor multiplication in Tensorflow
Kindly, help me multiply the Tensors dimension so that it work smoothly.
For example if this is my y = [[1,2,3],[2,3,4],[3,4,5],[2,5,7],[8,9,10],[0,3,2]] So I want to make it outputs_with_multiplier = [[1,2,6],[2,3,8],[3,4,10],[2,5,14],[8,9,20],[0,3,4]]
Please let me know if there is any solution to this.
You can't do an item assignment but you can create a new Tensor. The key is to multiply the first 2 columns by 1 and the 3rd column by 2.
x = tf.placeholder(tf.float32, [None, 3], name="output")
y = tf.constant([[1.0, 1.0, 2.0]])
z = tf.multiply(x, y)
sess = tf.Session()
sess.run(z, feed_dict={x: [[1,2,3],[2,3,4],[3,4,5],[2,5,7],[8,9,10],[0,3,2]]})
I have 2 inputs into a lambda layer one size (2,3,) the other (3,) . The lambda layer should return an output of size 2, however when the multiply layer is executed the following error occurs:
tensorflow.python.framework.errors_impl.InvalidArgumentError: Dimensions must be equal, but are 2 and 3 for 'multiply_1/mul' (op: 'Mul') with input shapes: [?,2], [?,3].
The relevant code is below and any help would be much appreciated,thanks:
import numpy as np
from keras.models import Model
from keras import backend as K
from keras.engine.topology import Layer
from keras.layers import Dense,Input,concatenate,Lambda,multiply,add
import tensorflow as tf
import time
def weights_Fx(x):
j = x[0][:,0]
k = x[1][0]
y = j - k
return y
def sum_layer(x):
x = tf.reduce_sum(x)
return x
type1_2 = Dense(units=1, activation = 'relu',name = "one")
type1_3 = Dense(units=1,activation = 'relu',name = "two")
in1 = Input(shape=(1,))
in2 = Input(shape=(1,))
n1 = type1_2(in1)
n2 = type1_3(in2)
model = concatenate([n1,n2],axis=-1,name='merge_predicitions')
coords_in = Input(shape=(2,3,))
coords_target = Input(shape=(3,))
model2 = Lambda(weights_Fx,output_shape=(2,),name='weightsFx')([coords_in,coords_target])
model = multiply([model,model2])
model = Lambda(sum_layer)(model)
model = Model(inputs=[in1,in2,coords_in,coords_target],outputs=[model])
The issue was to do with how I was indexing the array. It is important to remember that though the data is of shape (2,3) keras will create a Tensor of shape (None,2,3) therefore to perform the operation as desired the following is needed:
y = x[0][:,:,0]-x[1][:,0]
Furthermore in "sum layer" in order to prevent the rank (number of dimensions) in the tensor being reduced by 1 the following is required:
y = K.sum(x,axis=1,keepdims=True)
Your Lambda layer is not returning output of shape 2 but it is returning output of shape 3.
Model2 shape is (,3) and not (,2) which is giving error in multiply of the model and model2
Take a look at your coords_in and coords_target shape.
I am trying to create a model graph where my input is tensorflow variable which I am inputting from my java program
In my code, I am using numpy methods where I need to convert my tensorflow variable input to numpy array input
Here, is my code snippet
import tensorflow as tf
import numpy as np
eps = np.finfo(float).eps
EXPORT_DIR = './model'
def standardize(x):
med0 = np.median(x)
mad0 = np.median(np.abs(x - med0))
x1 = (x - med0) / (mad0 + eps)
return x1
#tensorflow input variable
a = tf.placeholder(tf.float32, name="input")
with tf.Session() as session:
session.run(tf.global_variables_initializer())
#Converting the input variable to numpy array
tensor = a.eval()
#calling standardize method
numpyArray = standardize(tensor)
#converting numpy array to tf
tf.convert_to_tensor(numpyArray)
#creating graph
graph = tf.get_default_graph()
tf.train.write_graph(graph, EXPORT_DIR, 'model_graph.pb', as_text=False)
I am getting error: InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'input' with dtype float in line tensor = a.eval()
When I am giving constant value in place of placeholder then it's working and generating the graph. But I want to input from my java code.
Is there any way to do that or do I need to convert all my numpy methods to tensorflow methods
placeholder is just an empty variable in tensorflow, to which you can feed numpy values. Now, what you are trying to do does not make sense. You can not get value out of an empty variable.
If you want to standardize your tensor, why convert it to numpy var first? You can directly do this using tensorflow.
The following taken from this stackoverflow ans
def get_median(v):
v = tf.reshape(v, [-1])
m = v.get_shape()[0]//2
return tf.nn.top_k(v, m).values[m-1]
Now, you can implement your function as
def standardize(x):
med0 = get_median(x)
mad0 = get_median(tf.abs(x - med0))
x1 = (x - med0)/(mad0 + eps)
return x1
I get a mismatch of shapes between input and the feedholder even though i am pretty sure that the shapes in both the cases are same. Here's the code:
ex3data1.mat contains a 5000*400 matrix X.
import tensorflow as tf
import numpy as np
`import scipy.io as sio
theta1 = sio.loadmat('ex3weights.mat')['Theta1']
theta2 = sio.loadmat('ex3weights.mat')['Theta2']
x = tf.placeholder(tf.float64, shape=[1, 400])
x2 = tf.concat([[[1]] ,x], 1)
z2 = tf.matmul(x2,np.transpose(theta1))
h1 = tf.divide(1.0, (1.0 + tf.exp(-z1)))
h1= tf.concat([[[1]],h1], 1)
z2 = tf.matmul(h1, np.transpose(theta2))
max = tf.argmax(z2)
max = max+1
sess = tf.Session()
op = sio.loadmat('ex3data1.mat')['X'][1234]
op = np.reshape(op, [1, 400])
op.astype(np.float64)
m = {x:op}
sess.run(max,feed_dict=m)
I get the following error:
InvalidArgumentError: You must feed a value for placeholder tensor 'Placeholder_2' with dtype double and shape [1,400]
[[Node: Placeholder_2 = Placeholder[dtype=DT_DOUBLE, shape=[1,400], _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]
Why don't you
print(op.shape)
before assigning the dict to m? I suppose this is [5000, 400]. Reshape works when the number of elements does not change. But in case when you expect 'input_width' wide stream, you may define a placeholder this way:
x = tf.placeholder(tf.float64, [None, input_width], "network_input")
allowing the numer of cases to be flexible. Then you can feed it with any number of cases, like 5000 and the math will still work.