I would like to extract single values from a tensor and manipulate it while retaining backpropagation. My current implementation:
import keras
from keras import backend as K
from keras.models import Model
from keras.layers import Dense, Activation, Input
import tensorflow as tf
input = Input(shape=(100,1), dtype='float32')
x = Dense(100)(input)
x = Activation('relu')(x)
x = Dense(5)(x)
x = Activation('tanh')(x)
start_pad = 40.0 + 5.0 * x[0] # important line
# ...
zs = K.arange(0.0, 1000, step=1.0)
zs = K.relu( zs - start_pad )
# ...
out = zs # + ...
out = Reshape( (trace_length,1) )(out)
model = Model(inputs = input, outputs = out)
However, start_pad seems to be a tensor with dimensions of x. Running code above gives error:
ValueError: Dimensions must be equal, but are 1000 and 5 for 'sub' (op: 'Sub') with input shapes: [1000], [100,5].
where start_pad object is <tf.Tensor 'add_1:0' shape=(100, 5) dtype=float32>.
I would like to have scalar like value for start_pad and subtract from zs with broadcasting. How do I achive this with Tensorflow/Keras?
Ok, the solution i found is
x = tf.unstack(x, axis=1)
which returns a list of tf tensors
Related
I'm trying to draw learning curve on a small data set
Full code here
from keras.models import Sequential
from keras.layers import Dense
from keras.optimizers import Adam
import keras.backend as K
K.clear_session()
model = Sequential()
model.add(Dense(1, input_shape=(1,)))
model.compile(Adam(lr=0.2), "mean_squared_error")
model.fit(x,y,epochs=50)
iw = model.get_weights()
from keras.utils import to_categorical
yc= to_categorical(y)
from sklearn.model_selection import train_test_split
xtr, xts, ytr, yts = train_test_split(x,yc, test_size=0.3)
train_sizes = (len(xtr) * np.linspace(0.1, 0.99999999, 4)).astype(int)
test_scores = []
for i in train_sizes :
xtrfr, _, yrtfr, _ = train_test_split(xtr,ytr,train_size=i)
model.set_weights(iw)
res = model.fit(xtrfr, yrtfr, epochs=600)
e = model.evaluate(xts,yts)
test_scores.append(e[-1])
plt.plot(train_sizes, test_scores, label="Learning Curve")
plt.legend()
plt.show()
but I'm getting this error
ValueError: Error when checking target: expected dense_1 to have shape (1,) but got array with shape (270,)
I'm guessing there's something wrong with the to_categorical but I can't figure it out ":)
looking at the shapes of your x and y shows that they are a 1 dimensional array:
>>> x.shape
(10000,)
>>> y.shape
(10000,)
however your model expects an array with input_shape=(1,), so first you will have to reshape your data like so:
>>> x = np.array(x, np.float32).reshape((-1, 1))
>>> y = np.array(y, np.float32).reshape((-1, 1))
they will now have this shape:
>>> x.shape
(10000, 1)
>>> y.shape
(10000, 1)
>>> x
and look like this:
>>> x
array([[73.847015],
[68.781906],
[74.11011 ],
...,
[63.867992],
[69.03424 ],
[61.944244]], dtype=float32)
>>> y
array([[241.89357],
[162.31047],
[212.74086],
...,
[128.47531],
[163.85246],
[113.6491 ]], dtype=float32)
an array with an array that has only one element
I faced a problem when I try to fit the model, here is my build model and the shape of my train and test data:
import keras
def buildModel(dataLength, labelLength):
price=Input(shape=(dataLength, 51),name='price')
# price = Input(shape = (dataLength,1),name='price')
sentiment = Input(shape=(dataLength, 51),name='sentiment')
priceLayers = LSTM(64, return_sequences=False)(price)
sentimentLayers = LSTM(64, return_sequences=False)(sentiment)
output = keras.layers.concatenate(
[priceLayers,sentimentLayers,]
)
output = Dense(labelLength, activation='linear',name='output')(output)
model = Model(
inputs = [price,sentiment],
outputs=[output]
)
model.compile(optimizer='rmsprop',loss='mse')
return model
from keras.layers import Input, Embedding, LSTM, Dense
from keras.models import Model
lstm = buildModel(22234,1)
lstm.fit([trainX,trainS],[trainY],validation_data=(
[testX,testS],
[testY]),epochs = 10)
trainX.shape = (1, 22234, 51)
testX.shape = (1, 9500, 51)
trainY.shape = (22234,)
testY.shape = (9500,)
trainS.shape = (1, 22234, 51)
testS.shape = (1, 9500, 51)
Error shows:
ValueError Traceback (most recent call last)
<ipython-input-40-4d75b702c980> in <module>()
5 lstm.fit([trainX,trainS],[trainY],validation_data=(
6 [testX,testS],
----> 7 [testY]),epochs = 10
8 )
ValueError: Input arrays should have the same number of samples as target arrays. Found 1 input samples and 22234 target samples.
But I don't understand why it says my input and target samples have different size, is it because in X and S has 3 dimensions but Y only has 2D? My thought is: the input has to be 3D, so I reshape X and S; however, Y is the label, and it do not need to reshape
The batch dimension for inputs and targets needs to be the same. This needs to be dimension 0.
trainX.shape = (1, 22234, 51)
needs to be: (22234, 51, 1) when the outputs are shaped as (22234,).
Do not try to use 1 as time_step dimension since an LSTM with a single time step doesn't make sense.
The input dimensions should be (batch_size, time_steps, n_features). No need to specify batch_size when building the model. So that shapes you declare should be (time_steps, n_features). For a sequence of N measurements N is the time_steps and n_features is the number of values measured at a time.
I have 2 inputs into a lambda layer one size (2,3,) the other (3,) . The lambda layer should return an output of size 2, however when the multiply layer is executed the following error occurs:
tensorflow.python.framework.errors_impl.InvalidArgumentError: Dimensions must be equal, but are 2 and 3 for 'multiply_1/mul' (op: 'Mul') with input shapes: [?,2], [?,3].
The relevant code is below and any help would be much appreciated,thanks:
import numpy as np
from keras.models import Model
from keras import backend as K
from keras.engine.topology import Layer
from keras.layers import Dense,Input,concatenate,Lambda,multiply,add
import tensorflow as tf
import time
def weights_Fx(x):
j = x[0][:,0]
k = x[1][0]
y = j - k
return y
def sum_layer(x):
x = tf.reduce_sum(x)
return x
type1_2 = Dense(units=1, activation = 'relu',name = "one")
type1_3 = Dense(units=1,activation = 'relu',name = "two")
in1 = Input(shape=(1,))
in2 = Input(shape=(1,))
n1 = type1_2(in1)
n2 = type1_3(in2)
model = concatenate([n1,n2],axis=-1,name='merge_predicitions')
coords_in = Input(shape=(2,3,))
coords_target = Input(shape=(3,))
model2 = Lambda(weights_Fx,output_shape=(2,),name='weightsFx')([coords_in,coords_target])
model = multiply([model,model2])
model = Lambda(sum_layer)(model)
model = Model(inputs=[in1,in2,coords_in,coords_target],outputs=[model])
The issue was to do with how I was indexing the array. It is important to remember that though the data is of shape (2,3) keras will create a Tensor of shape (None,2,3) therefore to perform the operation as desired the following is needed:
y = x[0][:,:,0]-x[1][:,0]
Furthermore in "sum layer" in order to prevent the rank (number of dimensions) in the tensor being reduced by 1 the following is required:
y = K.sum(x,axis=1,keepdims=True)
Your Lambda layer is not returning output of shape 2 but it is returning output of shape 3.
Model2 shape is (,3) and not (,2) which is giving error in multiply of the model and model2
Take a look at your coords_in and coords_target shape.
I get a mismatch of shapes between input and the feedholder even though i am pretty sure that the shapes in both the cases are same. Here's the code:
ex3data1.mat contains a 5000*400 matrix X.
import tensorflow as tf
import numpy as np
`import scipy.io as sio
theta1 = sio.loadmat('ex3weights.mat')['Theta1']
theta2 = sio.loadmat('ex3weights.mat')['Theta2']
x = tf.placeholder(tf.float64, shape=[1, 400])
x2 = tf.concat([[[1]] ,x], 1)
z2 = tf.matmul(x2,np.transpose(theta1))
h1 = tf.divide(1.0, (1.0 + tf.exp(-z1)))
h1= tf.concat([[[1]],h1], 1)
z2 = tf.matmul(h1, np.transpose(theta2))
max = tf.argmax(z2)
max = max+1
sess = tf.Session()
op = sio.loadmat('ex3data1.mat')['X'][1234]
op = np.reshape(op, [1, 400])
op.astype(np.float64)
m = {x:op}
sess.run(max,feed_dict=m)
I get the following error:
InvalidArgumentError: You must feed a value for placeholder tensor 'Placeholder_2' with dtype double and shape [1,400]
[[Node: Placeholder_2 = Placeholder[dtype=DT_DOUBLE, shape=[1,400], _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]
Why don't you
print(op.shape)
before assigning the dict to m? I suppose this is [5000, 400]. Reshape works when the number of elements does not change. But in case when you expect 'input_width' wide stream, you may define a placeholder this way:
x = tf.placeholder(tf.float64, [None, input_width], "network_input")
allowing the numer of cases to be flexible. Then you can feed it with any number of cases, like 5000 and the math will still work.
i'm trying to use the tensorflow unique function (https://www.tensorflow.org/api_docs/python/tf/unique) in a keras lambda layer.
Code below:
def unique_idx(x):
output = tf.unique(x)
return output[1]
then
inp1 = Input(batch_shape(None, 1))
idx = Lambda(unique_idx)(inp1)
model = Model(inputs=inp1, outputs=idx)
when I now use **model.compile(optimizer='Adam', loss='mean_squared_error')**
I get the error:
ValueError: Tensor conversion requested dtype int32 for Tensor with
dtype float32: 'Tensor("lambda_9_sample_weights_1:0", shape=(?,),
dtype=float32)'
Does anybody know whats the error here or a different way of using the tensorflow function?
A keras model expects a float32 as output, but the indices returned from tf.unique is a int32. A casting fixes your problem.
Another issue is that unique expects a flatten array. reshape fixes this one.
import tensorflow as tf
from keras import Input
from keras.layers import Lambda
from keras.engine import Model
def unique_idx(x):
x = tf.reshape(x, [-1])
u, indices = tf.unique(x)
return tf.cast(indices, tf.float32)
x = Input(shape=(1,))
y = Lambda(unique_idx)(x)
model = Model(inputs=x, outputs=y)
model.compile(optimizer='adam', loss='mse')