I am trying to create a model graph where my input is tensorflow variable which I am inputting from my java program
In my code, I am using numpy methods where I need to convert my tensorflow variable input to numpy array input
Here, is my code snippet
import tensorflow as tf
import numpy as np
eps = np.finfo(float).eps
EXPORT_DIR = './model'
def standardize(x):
med0 = np.median(x)
mad0 = np.median(np.abs(x - med0))
x1 = (x - med0) / (mad0 + eps)
return x1
#tensorflow input variable
a = tf.placeholder(tf.float32, name="input")
with tf.Session() as session:
session.run(tf.global_variables_initializer())
#Converting the input variable to numpy array
tensor = a.eval()
#calling standardize method
numpyArray = standardize(tensor)
#converting numpy array to tf
tf.convert_to_tensor(numpyArray)
#creating graph
graph = tf.get_default_graph()
tf.train.write_graph(graph, EXPORT_DIR, 'model_graph.pb', as_text=False)
I am getting error: InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'input' with dtype float in line tensor = a.eval()
When I am giving constant value in place of placeholder then it's working and generating the graph. But I want to input from my java code.
Is there any way to do that or do I need to convert all my numpy methods to tensorflow methods
placeholder is just an empty variable in tensorflow, to which you can feed numpy values. Now, what you are trying to do does not make sense. You can not get value out of an empty variable.
If you want to standardize your tensor, why convert it to numpy var first? You can directly do this using tensorflow.
The following taken from this stackoverflow ans
def get_median(v):
v = tf.reshape(v, [-1])
m = v.get_shape()[0]//2
return tf.nn.top_k(v, m).values[m-1]
Now, you can implement your function as
def standardize(x):
med0 = get_median(x)
mad0 = get_median(tf.abs(x - med0))
x1 = (x - med0)/(mad0 + eps)
return x1
Related
I need to convert a KerasTensor to a Tensor because when I try to use a contional (tf.cond()) it reports an error:
def custon_loss(self, input_tensor): # input type = <class 'tensorflow.python.keras.engine.keras_tensor.KerasTensor'>
def loss(y_actual, y_predicted):
mse = K.mean(K.sum(K.square(y_actual - y_predicted)))
mse = tf.reshape(mse, [1, 1])
y_actual = keras.layers.core.Reshape([1, 1])(y_actual)[0]
ax_input = tf.reshape(input_tensor[0][-1:][0][:1], [1, 1])
# convert here ax_input to Tensor
greater_equal = tf.reshape(tf.math.logical_and(tf.math.greater_equal(ax_input, y_actual), tf.math.greater_equal(ax_input, y_predicted))[0], [1, 1])
less_equal = tf.reshape(tf.math.logical_and(tf.math.less_equal(ax_input, y_actual), tf.math.less_equal(ax_input, y_predicted))[0], [1, 1])
logical_or = tf.reshape(tf.math.logical_or(greater_equal, less_equal)[0], [1, 1])
return tf.cond(logical_or, lambda: mse, lambda: tf.math.multiply(mse, 10))
return loss
Error caused in tf.cond:
TypeError: Cannot convert a symbolic Keras input/output to a numpy array. This error may indicate that you're trying to pass a symbolic value to a NumPy call, which is not supported. Or, you may be trying to pass Keras symbolic inputs/outputs to a TF API that does not register dispatching, preventing Keras from automatically converting the API call to a lambda layer in the Functional Model.
I believe that converting the tensor will not make the error.
Looks like Issue is with numpy =1.20 version. Downgrade your numpy version to 1.19.5.
W.R.T Keras tensor you can use below sample code
import tensorflow as tf
import numpy as np
np_var = np.array([1])
keras_var = tf.keras.backend.variable(np_var)
def f1(): return tf.multiply(np_var, 17)
def f2(): return tf.add(np_var, 23)
r = tf.cond(tf.less(np_var, np_var), f1, f2)
I get a mismatch of shapes between input and the feedholder even though i am pretty sure that the shapes in both the cases are same. Here's the code:
ex3data1.mat contains a 5000*400 matrix X.
import tensorflow as tf
import numpy as np
`import scipy.io as sio
theta1 = sio.loadmat('ex3weights.mat')['Theta1']
theta2 = sio.loadmat('ex3weights.mat')['Theta2']
x = tf.placeholder(tf.float64, shape=[1, 400])
x2 = tf.concat([[[1]] ,x], 1)
z2 = tf.matmul(x2,np.transpose(theta1))
h1 = tf.divide(1.0, (1.0 + tf.exp(-z1)))
h1= tf.concat([[[1]],h1], 1)
z2 = tf.matmul(h1, np.transpose(theta2))
max = tf.argmax(z2)
max = max+1
sess = tf.Session()
op = sio.loadmat('ex3data1.mat')['X'][1234]
op = np.reshape(op, [1, 400])
op.astype(np.float64)
m = {x:op}
sess.run(max,feed_dict=m)
I get the following error:
InvalidArgumentError: You must feed a value for placeholder tensor 'Placeholder_2' with dtype double and shape [1,400]
[[Node: Placeholder_2 = Placeholder[dtype=DT_DOUBLE, shape=[1,400], _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]
Why don't you
print(op.shape)
before assigning the dict to m? I suppose this is [5000, 400]. Reshape works when the number of elements does not change. But in case when you expect 'input_width' wide stream, you may define a placeholder this way:
x = tf.placeholder(tf.float64, [None, input_width], "network_input")
allowing the numer of cases to be flexible. Then you can feed it with any number of cases, like 5000 and the math will still work.
I have a script and I am trying to convert my math operations from NumPy operations to TensorFlow operations so it can get faster on GPU. And in my script I end up in a situation that I have an array with shape (260) and need to do matrix multiplication with another array with shape (260), illustrated by:
import numpy as np
x = np.array([2] * 260)
y = np.array([4] * 260)
r = np.matmul(x,y) #np.dot(x,y) also works
print(r) #2080
But the same operation in TensorFlow is not possible.
import tensorflow as tf
x = tf.Variable([2] * 260)
y = tf.Variable([4] * 260)
r = tf.matmul(x,y)
init = tf.initialize_all_variables()
sess = tf.Session()
sess.run(init)
result = sess.run(r)
print(result) # ERRROR
The TensorFlow error says:
ValueError: Shape must be rank 2 but is rank 1 for 'MatMul' (op:
'MatMul') with input shapes: [260], [260].
I have tried to reshape the inputs countless many ways, and none of those have worked, such as: x = tf.expand_dims(x,1).
Since both inputs are 1-dimensional, your matrix multiplication is the inner product,
tf.reduce_sum(tf.multiply(x, y))
or
tf.tensordot(x, y, 1)
Also see this answer for a few alternative ways of calculating the inner product.
I am trying to build a neural network using custom activation functions. I followed the solution given here, and it works when the input and output vectors have the same size, but not when using different sizes (like in a pooling function). Here is my problem so far:
I am trying to generalize this to the case when the input and the output have different sizes. In my code the input 'x' is of size (2,4), the output 'y' is of size (1,2), and the activation function MEX(.) does the mapping y = MEX(x). I have computed the gradient of MEX() as d_MEX(), where d_MEX(x) has the same size as 'x', that is (2,4). Nevertheless, I get this error
InvalidArgumentError (see above for traceback): Incompatible shapes: [1,2] vs. [2,4]
Shouldn't the gradient of MEX(x) be of the same size as x? Here is my complete code:
import tensorflow as tf
import numpy as np
# This is our target function
def MEX(x):
'''
:param x: is a row vector which is the concatenation of [input, beta]
:return MEX_{beta}(x): scalar output
'''
# lenx = np.size(x) # Number of columns (ROW vector)
lenx = x.shape[1]
N = x.shape[0]
out = np.zeros((1,N))
for ii in range(N):
c = x[ii,0:lenx-1]
beta = x[ii,lenx-1]
out[0,ii] = 1./beta * np.log( np.mean( np.exp(beta*c) ))
return np.array(out)
# Now we should write its derivative.
def d_MEX(x):
# lenx = np.size(x) # Number of
lenx = x.shape[1]
N = x.shape[0]
out = np.zeros((N,lenx))
for ii in range(N):
c = x[ii,0:lenx-1]
beta = x[ii,lenx-1]
d_beta = np.array([0.])
d_beta[0] = -1./beta*( MEX(np.array([x[ii,:]])) - np.mean( np.multiply( c, np.exp(beta*c)))/np.mean( np.exp(beta*c)) )
d_c = 1./lenx*np.exp(beta*c) /np.mean( np.exp(beta*c))
out[ii,:] = np.concatenate((d_c,d_beta), axis=0)
return out
# The first step is making it into a numpy function, this is easy:
np_MEX = np.vectorize(MEX, excluded=['x']) # IMPORTANT!! Otherwise np.vectorize() doesnt work
np_d_MEX = np.vectorize(d_MEX, excluded=['x']) # IMPORTANT!! Otherwise np.vectorize() doesnt work
# Now we make a tensforflow function
'''
Making a numpy fct to a tensorflow fct: We will start by making np_d_MEX_32 into a tensorflow function.
There is a function in tensorflow tf.py_func(func, inp, Tout, stateful=stateful, name=name) [doc]
which transforms any numpy function to a tensorflow function, so we can use it:
'''
np_d_MEX_32 = lambda x: np_d_MEX(x=x).astype(np.float32)
def tf_d_MEX(x,name=None):
with tf.name_scope(name, "d_MEX", [x]) as name:
y = tf.py_func(np_d_MEX_32,
[x],
[tf.float32],
name=name,
stateful=False)
return y[0]
'''
tf.py_func acts on lists of tensors (and returns a list of tensors), that is why we have [x] (and return y[0]).
The stateful option is to tell tensorflow whether the function always gives the same output for the same input (stateful = False)
in which case tensorflow can simply the tensorflow graph, this is our case and will probably be the case in most situations.
One thing to be careful of at this point is that numpy used float64 but tensorflow uses float32 so you need to convert
your function to use float32 before you can convert it to a tensorflow function otherwise tensorflow will complain.
This is why we need to make np_d_MEX_32 first.
What about the Gradients? The problem with only doing the above is that even though we now have tf_d_MEX which is the
tensorflow version of np_d_MEX, we couldn't use it as an activation function if we wanted to because tensorflow doesn't
know how to calculate the gradients of that function.
Hack to get Gradients: As explained in the sources mentioned above, there is a hack to define gradients of a function
using tf.RegisterGradient [doc] and tf.Graph.gradient_override_map [doc]. Copying the code from harpone we can modify
the tf.py_func function to make it define the gradient at the same time:
'''
def py_func(func, inp, Tout, stateful=True, name=None, grad=None):
# Need to generate a unique name to avoid duplicates:
rnd_name = 'PyFuncGrad' + str(np.random.randint(0, 1E+8))
tf.RegisterGradient(rnd_name)(grad) # see _MySquareGrad for grad example
g = tf.get_default_graph()
with g.gradient_override_map({"PyFunc": rnd_name}):
return tf.py_func(func, inp, Tout, stateful=stateful, name=name)
'''
Now we are almost done, the only thing is that the grad function we need to pass to the above py_func function needs to
take a special form. It needs to take in an operation, and the previous gradients before the operation and propagate
the gradients backward after the operation.
Gradient Function: So for our MEX activation function that is how we would do it:
'''
def MEXgrad(op, grad):
x = op.inputs[0]
# x = op
n_gr = tf_d_MEX(x)
return grad * n_gr
'''
The activation function has only one input, that is why x = op.inputs[0]. If the operation had many inputs, we would
need to return a tuple, one gradient for each input. For example if the operation was a-bthe gradient with respect to a
is +1 and with respect to b is -1 so we would have return +1*grad,-1*grad. Notice that we need to return tensorflow
functions of the input, that is why need tf_d_MEX, np_d_MEX would not have worked because it cannot act on
tensorflow tensors. Alternatively we could have written the derivative using tensorflow functions:
'''
# Combining it all together: Now that we have all the pieces, we can combine them all together:
np_MEX_32 = lambda x: np_MEX(x=x).astype(np.float32)
def tf_MEX(x, name=None):
with tf.name_scope(name, "MEX",[x]) as name:
y = py_func(np_MEX_32,
[x],
[tf.float32],
name=name,
grad=MEXgrad) # <-- here's the call to the gradient
return y[0]
with tf.Session() as sess:
x = tf.constant([[0.2,0.7,1.2,1.7],[0.2,0.7,1.2,1.7]])
y = tf_MEX(x)
tf.global_variables_initializer().run()
print(x.eval(), y.eval(), tf.gradients(y, [x])[0].eval())
In the console, I have checked that the variables have the "correct" shapes:
x.eval()
Out[9]:
array([[ 0.2 , 0.69999999, 1.20000005, 1.70000005],
[ 0.2 , 0.69999999, 1.20000005, 1.70000005]], dtype=float32)
y.eval()
Out[10]: array([[ 0.83393127, 0.83393127]], dtype=float32)
tf_d_MEX(x).eval()
Out[11]:
array([[ 0.0850958 , 0.19909413, 0.46581003, 0.07051659],
[ 0.0850958 , 0.19909413, 0.46581003, 0.07051659]], dtype=float32)
My bad, I just found the mistake.
Its here:
def MEXgrad(op, grad):
x = op.inputs[0]
# x = op
n_gr = tf_d_MEX(x)
return n_gr
I wonder if there is a typo here, where this mistake is also there.
I am trying to pass x_data as feed_dict but getting below error, I am not sure that what is wrong in the code.
InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'x_12' with dtype int32 and shape [1000]
[[Node: x_12 = Placeholder[dtype=DT_INT32, shape=[1000], _device="/job:localhost/replica:0/task:0/cpu:0"]()]]
My Code:
import tensorflow as tf
import numpy as np
model = tf.global_variables_initializer()
#define x and y
x = tf.placeholder(shape=[1000],dtype=tf.int32,name="x")
y = tf.Variable(5*x**2-3*x+15,name = "y")
x_data = tf.pack(np.random.randint(0,100,size=1000))
print(x_data)
print(x)
with tf.Session() as sess:
sess.run(model)
print(sess.run(y,feed_dict={x:x_data}))
I checked the shape of the x and x_data and it is same
Tensor("pack_8:0", shape=(1000,), dtype=int32)
Tensor("x_14:0", shape=(1000,), dtype=int32)
I am working with one dimensional data.
Any help is appreciated, Thanks!
To make it work I have changed two things, first I changed y to be a Tensor. And secondly I have not changed the x_data to Tensor, as commented here:
The optional feed_dict argument allows the caller to override the value of tensors in the graph. Each key in feed_dict can be one of the following types:
If the key is a Tensor, the value may be a Python scalar, string, list, or numpy ndarray that can be converted to the same dtype as that tensor. Additionally, if the key is a placeholder, the shape of the value will be checked for compatibility with the placeholder.
The changed code which works for me:
import tensorflow as tf
import numpy as np
model = tf.global_variables_initializer()
#define x and y
x = tf.placeholder(shape=[1000],dtype=tf.int32,name="x")
y = 5*x**2-3*x+15 # without tf.Variable, making it a tf.Tensor
x_data = np.random.randint(0,100,size=1000) # without tf.pack
print(x_data)
print(x)
with tf.Session() as sess:
sess.run(model)
print(sess.run(y,feed_dict={x:x_data}))