I am starting to develop a model and am getting stuck with dimensions. My X_train and Y_train are numpy arrays of shape (65337, 19)
Input_1= Input(shape=(19,))
x = Dense(100, activation='relu')(Input_1)
out1 = Dense(1, activation='linear')(x)
out2 = Dense(1, activation='linear')(x)
...
out19 = Dense(1, activation = 'linear')(x)
model = Model(inputs=Input_1, outputs=[out1,out2,out3,out4,out5,out6,
out7,out8,out9,out10,out11,out12,
out13,out14,out15,out16,out17,out18,out19])
model.compile(optimizer = "rmsprop", loss = 'mse')
model.fit(X_train,y_train,epochs=5)
When I run this, I get the value error:
ValueError: Error when checking model target: the list of Numpy arrays
that you are passing to your model is not the size the model expected.
Expected to see 19 array(s), but instead got the following list of 1
arrays:
Looking at other questions here, it seems using .fit(np.array(X_train) , np.array(y_train) has helped some, but I get the same error (which makes sense, since it tells me I have an array).
You are expecting 19 differents outputs, so you need to feed to you network with the 19 slices of your array of labels :
model.fit(X_train,[y_train[:,0], y_train[:,1], y_train[:,2],[...], y_train[:, 18]] , epochs=5)
Related
I was wondering if you would be able to help me with an errors that I am getting in the code that I am writing.
I have 2 datasets as inputs and 1 other as taget dataset. All datasets are set of images in dimantion of (17, 20, 1).
I set a code as:
from tensorflow.keras.layers import Concatenate
from tensorflow.keras import Model
# define two sets of inputs
inputA = Input(shape=(17, 20, 1))
inputB = Input(shape=(17, 20, 1))
# the first branch operates on the first input
x = Sequential()(inputA)
x = Conv2D(filters=64, kernel_size=(3,3), activation='relu')(x)
x = Model(inputs=inputA, outputs=x)
# the second branch opreates on the second input
y = Sequential()(inputB)
y = Conv2D(filters=64, kernel_size=(3,3), activation='relu')(y)
y = Model(inputs=inputB, outputs=y)
# combine the output of the two branches
combined = Concatenate()([x.output, y.output])
# apply a FC layer and then a regression prediction on the
# combined outputs
z = Sequential()(combined)
z = Conv2D(filters=64, kernel_size=(3,3), activation='relu')(z)
# our model will accept the inputs of the two branches and
# then output a single value
model = Model(inputs=[x.input, y.input], outputs=z)
model.compile(loss="mean_absolute_percentage_error", optimizer='adam', metrics=['accuracy'])
history = model.fit([input1_train, input2_train], target_train, validation_data=([input1_test, input2_test], target_test), epochs=100, verbose=0)
then I get the error as:
ValueError: Dimensions must be equal, but are 17 and 13 for '{{node mean_absolute_percentage_error/sub}} = Sub[T=DT_FLOAT](IteratorGetNext:1, model_18/conv2d_18/Relu)' with input shapes: [?,17,20,1], [?,13,16,64].
I also test this code for images with shape (20, 20, 1), and get the code as:
ValueError: Dimensions must be equal, but are 20 and 16 for '{{node mean_absolute_percentage_error/sub}} = Sub[T=DT_FLOAT](IteratorGetNext:1, model_15/conv2d_15/Relu)' with input shapes: [?,20,20,1], [?,16,16,64].
but when I set the kernel size as (1,1) the code run with no problem.
Does anyone know where the problem comes from? And what should I do?
I'll very thanks if anyone can help me fix it.
I got an error while running a python code
I am writing a neural network to take the Mel frequency coefficients as inputs and then run the model. My dataset contains 100 samples - each sample is an array of 12 values corresponding to the coefficients. After splitting this data into train and test sets, I have created the X input corresponding to the array and the y input corresponding to the label.
Data array containing the coefficients
Here is a small sample of my data containing 5 elements in the X_train array:
['[107.59366 -14.153783 24.799461 -8.244417 20.95272\n -4.375943 12.77285 -0.92922235 3.9418116 7.3581047\n -0.30066165 5.441765 ]'
'[ 96.49664 2.0689797 21.557552 -32.827045 7.348135 -23.513977\n 7.9406714 -16.218931 10.594619 -21.4381 0.5903044 -10.569035 ]'
'[105.98041 -2.0483367 12.276348 -27.334534 6.8239 -23.019623\n 7.5176797 -21.884727 11.349695 -22.734652 3.0335162 -11.142375 ]'
'[ 7.73094559e+01 1.91073620e+00 6.72225571e+00 -2.74525508e-02\n 6.60858107e+00 5.99264860e-01 1.96265772e-01 -3.94772577e+00\n 7.46383286e+00 5.42239428e+00 1.21432066e-01 2.44894314e+00]']
When I create the Neural network, I want to use the 12 coefficients as an input for the network. In order to do this, I need to use each row of my X_train dataset that contains these arrays as the input. However, when I try to consider the array index as an input it gives me shape errors when trying to fit the model. My model is as follows:
def build_model_graph():
model = Sequential()
model.add(Input(shape=(12,)))
model.add(Dense(12))
model.add(Activation('relu'))
model.add(Dense(10))
model.add(Activation('relu'))
model.add(Dense(num_labels))
model.add(Activation('softmax'))
# Compile the model
model.compile(loss='categorical_crossentropy', metrics=['accuracy'], optimizer='adam')
return model
Here, I want to use every row of the X_train array as an input which would correspond to the shape(12,). When I use something like this:
num_epochs = 50
num_batch_size = 32
model.fit(x_train, y_train, batch_size=num_batch_size, epochs=num_epochs,
validation_data=(x_test, y_test), verbose=1)
I get an error for the shape which makes sense to me.
For reference, the error is as follows:
ValueError: Exception encountered when calling layer "sequential_20" (type Sequential).
Input 0 of layer "dense_54" is incompatible with the layer: expected min_ndim=2, found ndim=1. Full shape received: (None,)
But I am not exactly sure how I can extract the array of 12 coefficients present at each index of the X_train and then use it in the model input. Indexing the x_train and y_train did not work either. If anyone could point me in a relevant direction, it would be extremely helpful. Thanks!
Edit: My code for the dataframe is as follows:
clapdf = pd.read_csv("clapsdf.csv")
clapdf.drop('Unnamed: 0', inplace=True, axis=1)
clapdf.head()
nonclapdf = pd.read_csv("nonclapsdf.csv")
nonclapdf.drop('Unnamed: 0', inplace=True, axis=1)
sound_df = clapdf.append(nonclapdf)
sound_df.head()
d=sound_data.tolist()
df=pd.DataFrame(data=d)
data = df[0].to_numpy()
print("Before-->", data.shape)
dat = np.array([np.array(d) for d in data])
print('After-->', dat.shape)
Here, the shape remains the same as the values of each of the 80 samples are not in a comma separated format but instead in the form of a series.
If your data looks like this:
samples = 2
features = 12
x_train = tf.random.normal((samples, 1, features))
tf.Tensor(
[[[-2.5988803 -0.629626 -0.8306641 -0.78226614 0.88989156
-0.3851106 -0.66053045 1.0571191 -0.59061646 -1.1602987
0.69124466 -0.04354193]]
[[-0.86917496 2.2923143 -0.05498986 -0.09578358 0.85037625
-0.54679644 -1.2213608 -1.3766612 0.35416105 -0.57801914
-0.3699728 0.7884727 ]]], shape=(2, 1, 12), dtype=float32)
You will have to reshape it to (2, 12) in order to fit your model with the input shape (batch_size, 12):
import tensorflow as tf
def build_model_graph():
model = tf.keras.Sequential()
model.add(tf.keras.layers.Input(shape=(12,)))
model.add(tf.keras.layers.Dense(12))
model.add(tf.keras.layers.Activation('relu'))
model.add(tf.keras.layers.Dense(10))
model.add(tf.keras.layers.Activation('relu'))
model.add(tf.keras.layers.Dense(2))
model.add(tf.keras.layers.Activation('softmax'))
# Compile the model
model.compile(loss='categorical_crossentropy', metrics=['accuracy'], optimizer='adam')
return model
model = build_model_graph()
samples = 2
features = 12
x_train = tf.random.normal((samples, 1, features))
x_train = tf.reshape(x_train, (samples, features))
y = tf.random.uniform((samples, 1), maxval=2, dtype=tf.int32)
y_train = tf.keras.utils.to_categorical(y, 2)
model.fit(x_train, y_train, batch_size=1, epochs=2)
Also, you usually need to convert your labels to one-hot encoded vectors if you plan to use categorical_crossentropy.
y_train looks like this:
[[0. 1.]
[1. 0.]]
Update 1:
If your data is coming from a dataframe, try something like this:
import numpy as np
import pandas as pd
d = {'features': [[0.18525402, 0.92130125, 0.2296906, 0.75818471, 0.69813222, 0.47147329,
0.03560711, 0.06583931, 0.90921289, 0.76002148, 0.50413995, 0.36099004],
[0.18525402, 0.92130125, 0.2296906, 0.75818471, 0.69813222, 0.47147329,
0.03560711, 0.06583931, 0.90921289, 0.76002148, 0.50413995, 0.36099004]]}
df = pd.DataFrame(data=d)
data = df['features'].to_numpy()
print('Before -->', data.shape)
data = np.array([np.array(d) for d in data])
print('After -->', data.shape)
Before --> (2,)
After --> (2, 12)
I am facing some problems in training the following GRU model, which has to be stateful and output the hidden state.
import numpy as np
import tensorflow as tf #2.1.0
from tensorflow import keras
BATCH_SIZE = 1
nfeatures = 3
history = 30 # shapes input array
horizon = 5 # shapes output array
nodes = 32
input_layer = tf.keras.layers.Input(batch_shape=(1,30,3),name="INPUT")
output, state_h = tf.keras.layers.GRU(nodes,
return_sequences=True,
stateful=True,
return_state=True,
batch_input_shape=(1,history,3), name='GRU1')(input_layer)
output_layer = tf.keras.layers.GRU(nodes, activation='tanh', name='GRU2')(output, state_h)
output_dense = tf.keras.layers.Dense(5, name='DENSE')(output_layer)
model = tf.keras.Model(input_layer, [output_dense, state_h])
model.compile(optimizer=tf.keras.optimizers.Adam(clipvalue=2.0),
loss='mse',
metrics=['mean_absolute_error', 'mean_squared_error'])
As I need the model to output the hidden state, I do not use a Sequential model. (I had no problems training a stateful sequential model.)
The features fed to network are of shape np.shape(x)=(30,3) and the target np.shape(y)=(5,).
If I call model.predict(x), where x is a numpy array with the shape mentioned above, it throws an error, as expected, because the input shape doesn't match the expected input. Therefore, I reshape the input array to have an input shape of (1,30,3) by calling np.expand_dims(x,axis=0). After that, it works fine, i.e. I get an output.
The issues I am facing are when I try to train the model. Calling
model.fit(x, y,epochs=1,steps_per_epoch=STEPS_PER_EPOCH)
throws the same error, about the shape of the data
ValueError: Error when checking input: expected input to have 3 dimensions, but got array with shape (30, 3)
Reshapping the data as I did for the prediction didn't help
model.fit(np.expand_dims(x,axis=0), np.expand_dims(y,axis=0),epochs=1,steps_per_epoch=STEPS_PER_EPOCH)
ValueError: The number of samples 1 is not divisible by steps 30. Please change the number of steps to a value that can consume all the samples.
This was a new error, setting the steps_per_epoch=1 threw a new one
ValueError: Error when checking model target: the list of Numpy arrays that you are passing to your model is not the size the model expected. Expected to see 2 array(s), for inputs ['DENSE', 'GRU1'] but instead got the following list of 1 arrays: [array([[0.5124772 , 0.51047856, 0.509669 , 0.50830126, 0.5070507 ]],
dtype=float32)]...
Is the format of my data wrong or is the architecture of my layers missing something? I tried adding a Flatten layer after the input, but it didn't make much sense (in my head) and it didn't work either.
Thanks in advance.
Problem here is that the Number of Nodes should be equal to the Output Shape. Changing the value of Nodes from 32 to 5, along with other minor changes, will fix the Error.
Complete working code is shown below:
import numpy as np
import tensorflow as tf #2.1.0
from tensorflow import keras
BATCH_SIZE = 1
nfeatures = 3
history = 30 # shapes input array
horizon = 5 # shapes output array
nodes = 5
x = np.ones(shape = (30,3))
x = np.expand_dims(x, axis = 0)
y = np.ones(shape = (5,))
y = np.expand_dims(y, axis = 0)
print(x.shape) #(1, 30, 3)
print(y.shape) #(1, 5)
input_layer = tf.keras.layers.Input(batch_shape=(1,30,3),name="INPUT")
output, state_h = tf.keras.layers.GRU(nodes,
return_sequences=True,
stateful=True,
return_state=True,
batch_input_shape=(1,history,3), name='GRU1')(input_layer)
output_layer = tf.keras.layers.GRU(nodes, activation='tanh', name='GRU2')(output, state_h)
output_dense = tf.keras.layers.Dense(5, name='DENSE')(output_layer)
model = tf.keras.Model(input_layer, [output_dense, state_h])
model.compile(optimizer=tf.keras.optimizers.Adam(clipvalue=2.0),
loss='mse',
metrics=['mean_absolute_error', 'mean_squared_error'])
STEPS_PER_EPOCH = 1
model.fit(x, y,epochs=1,steps_per_epoch=STEPS_PER_EPOCH)
Output of the above code is:
(1, 30, 3)
(1, 5)
1/1 [==============================] - 0s 3ms/step - loss: 1.8172 - DENSE_loss: 1.1737 - GRU1_loss: 0.6435 - DENSE_mean_absolute_error: 1.0498 - DENSE_mean_squared_error: 1.1737 - GRU1_mean_absolute_error: 0.7157 - GRU1_mean_squared_error: 0.6435
<tensorflow.python.keras.callbacks.History at 0x7f698bf8ac50>
Hope this helps. Happy Learning!
I'm trying to implement a hybrid LSTM-DNN forecaster with multiple inputs using the code from Hvass-Labs Time Series tutorial #23. Basically I want to forecast day-ahead prices (just a 24 time step into the future for now) of electricity using sequential and non-sequential data. The model I'm using is two sets of inputs feeding an LSTM (for the sequential data) and Dense for the non-sequential data, with their outputs concatenated. It looks like this:
!https://imgur.com/a/x15FfIy
Basically whenever I try to fit the model after one epoch it shows this error:
UPDATE:
ValueError: Error when checking input: expected input_1 to have shape (168, 5) but got array with shape (5808, 5)
The changes I have implemented:
# Chop off x_test_scaled into two parts:
x_test1_scaled = x_test_scaled[:,0:5] # shape is (5808, 5)
x_test2_scaled = x_test_scaled[:,5:12] # shape is (5808, 7)
validation_data = [np.expand_dims(x_test1_scaled, axis=0), np.expand_dims(x_test2_scaled, axis=0)], np.expand_dims(y_test_scaled, axis=0)
I'm confused because I have indeed assigned the generator to the generator in the model.fit_generator, and I'm not passing the x_test1_scaled which does have the shape of (5808, 5). edit:(not validation_data)
%%time
model.fit_generator(generator=generator,
epochs=10,
steps_per_epoch=30,
validation_data=validation_data,
callbacks=callbacks)
If this helps, this is my model:
# first input model
input_1 = Input(shape=((168,5)))
dense_1 = Dense(50)(input_1)
# second input model
input_2 = Input(shape=((168,7)))
lstm_1 = LSTM(units=64, return_sequences=True, input_shape=(None, 7,))(input_2)
# merge input models
merge = concatenate([dense_1, lstm_1])
output = Dense(num_y_signals, activation='sigmoid')(merge)
model = Model(inputs=[input_1, input_2], outputs=output)
# summarize layers
print(model.summary())
EDIT: Cleared this problem, replaced with error on top.
Thus far I've managed everything up to actually fitting the model.
Whenever an epoch finishes however it goes into the error:
ValueError: Error when checking model input: the list of Numpy arrays that you are passing to your model is not the size the model expected. Expected to see 2 array(s), but instead got the following list of 1 arrays: [array([[[0.4 , 0.44444442, 0. , ..., 0.1734707 ,
0.07272629, 0.07110982],
[0.3904762 , 0.43434343, 0.04347826, ..., 0.1740398 ,
0.07282589, 0.06936309],
...
I have tried the solutions from other stackexchange posts of the same error message. They haven't been successful, but I was able to eventually isolate the problem array to that of the validation_data. I just don't know how to "reshape" it into the required 2 array.
The batch generator: I have included the two sets of inputs already. the x_batch_1 and x_batch_2
def batch_generator(batch_size, sequence_length):
"""
Generator function for creating random batches of training-data.
"""
# Infinite loop.
while True:
# Allocate a new array for the batch of input-signals.
x_shape = (batch_size, sequence_length, num_x_signals)
x_batch = np.zeros(shape=x_shape, dtype=np.float16)
# Allocate a new array for the batch of output-signals.
y_shape = (batch_size, sequence_length, num_y_signals)
y_batch = np.zeros(shape=y_shape, dtype=np.float16)
# Fill the batch with random sequences of data.
for i in range(batch_size):
# Get a random start-index.
# This points somewhere into the training-data.
idx = np.random.randint(num_train - sequence_length)
# Copy the sequences of data starting at this index.
x_batch[i] = x_train_scaled[idx:idx+sequence_length]
y_batch[i] = y_train_scaled[idx:idx+sequence_length]
x_batch_1 = x_batch[ :, :, 0:5]
x_batch_2 = x_batch[ :, :, 5:12]
yield ([x_batch_1, x_batch_2], y_batch)
batch_size = 32
sequence_length = 24 * 7
generator = batch_generator(batch_size=batch_size,
sequence_length=sequence_length)
Validation set:
validation_data = np.expand_dims(x_test_scaled, axis=0), np.expand_dims(y_test_scaled, axis=0)
And lastly the model fit:
%%time
model.fit_generator(generator=generator,
epochs=10,
steps_per_epoch=30,
validation_data=validation_data,
callbacks=callbacks)
ValueError: Error when checking model input: the list of Numpy arrays that you are passing to your model is not the size the model expected. Expected to see 2 array(s), but instead got the following list of 1 arrays: [array([[[0.4 , 0.44444442, 0. , ..., 0.1734707 ,
0.07272629, 0.07110982],
[0.3904762 , 0.43434343, 0.04347826, ..., 0.1740398 ,
0.07282589, 0.06936309],
...
The array is the same one as the validation_data. Another thing is that the error creeps up whenever the first epoch finishes which strengthens the case for the problem being the validation_data.
It's because your model need 2 sets of input, x_batch_1, x_batch_2 in your batch_generator. While your validation_data has only one array np.expand_dims(x_test_scaled, axis=0)
You need to make validation_data looks like the batch_generator, probably [np.expand_dims(x_test1_scaled, axis=0), np.expand_dims(x_test2_scaled, axis=0)], np.expand_dims(y_test_scaled, axis=0).
In case of you still don't understand, please provide information about x_test1_scaled, like it's shape or how you load it.
The weird point is that I got this "checking model input error: the list of Numpy arrays that you are passing to your model is not the size the model expected." only when I add validation dataset option (if I remove validation part in this code, it works).
Clearly, entered (to my knowledge) suitable validation data shape as well as training dataset. I just entered exact same form of validation dataset with training dataset.
training/validation input form is as follows.
partial_x_train / partial_estendo / partial_y_train : (7000,1) numpy array
x_val / estendo_val / y_val : (3000,1) numpy array
What I already tried(2 ways)
1) the other post in stackoverflow recommended to use "np.array()" for the case where keras model cannot recognize non-numpy array input. However, it doesn't work in my case. I don't think this is the solution for me.
2) I thought it is mainly due to improper parenthesis at first. so I just tried to alter that part. but Neither of them works. (Even when I enter the training dataset as validation dataset)
----------skip : just creating dataset -----------
def create_mean(inputs):
x = inputs
x = Dense(100, activation='relu')(x)
x = Dense(1, activation='linear', name='mean_output')(x)
return x
def create_std(inputs):
x = inputs
x = Dense(100, activation='relu')(x)
x = Dense(1, activation='softplus', name='var_output')(x)
return x
def generatesecmodel(name):
inputs = keras.Input(shape=(second_input,))
mean_branch = create_mean(inputs)
var_branch = create_std(inputs)
label_mean = Input(shape=(1,))
model2 = Model(inputs=[inputs, label_mean], outputs=[mean_branch, var_branch])
loss = 0.5 * K.log(var_branch + 1e-6) + 0.5 * K.square(label_mean - mean_branch) / (var_branch + 1e-6)
model2.add_loss(loss)
optimizer = optimizers.Adam(lr=0.1)
model2.compile(optimizer=optimizer, loss=None)
model2.fit([np.concatenate((partial_estendo, partial_x_train), axis=1), partial_y_train], epochs=num_epochs,
batch_size=batch_size, validation_data=([np.concatenate((estendo_val, x_val), axis=1), y_val]))
model = generatesecmodel('test')
Error message is as follows
ValueError: Error when checking model input: the list of Numpy arrays that you are passing to your model is not the size the model expected. Expected to see 2 array(s), but instead got the following list of 1 arrays: [array([[ 1.60455704e-04, -9.37487497e-01],
[ 9.99757528e-01, 1.45345668e-01],
[ 9.99548554e-01, 2.66093564e-02],
...,
[ 9.98771310e-01, -1.19436635e-01],
[ 6.7000...