I have a dataset of 3000 observations. Each observation consists of 3 timeseries of length 200 samples. As the output I have 5 class labels.
So I build train as test sets as follows:
test_split = round(num_samples * 3 / 4)
X_train = X_all[:test_split, :, :] # Start upto just before test_split
y_train = y_all[:test_split]
X_test = X_all[test_split:, :, :] # From test_split to end
y_test = y_all[test_split:]
# Print shapes and class labels
print(X_train.shape)
print(y_train.shape)
> (2250, 200, 3)
> (22250, 5)
I build my network using Keras functional API:
from keras.models import Model
from keras.layers import Dense, Activation, Input, Dropout, concatenate
from keras.layers.recurrent import LSTM
from keras.constraints import maxnorm
from keras.optimizers import SGD
from keras.callbacks import EarlyStopping
series_len = 200
num_RNN_neurons = 64
ch1 = Input(shape=(series_len, 1), name='ch1')
ch2 = Input(shape=(series_len, 1), name='ch2')
ch3 = Input(shape=(series_len, 1), name='ch3')
ch1_layer = LSTM(num_RNN_neurons, return_sequences=False)(ch1)
ch2_layer = LSTM(num_RNN_neurons, return_sequences=False)(ch2)
ch3_layer = LSTM(num_RNN_neurons, return_sequences=False)(ch3)
visible = concatenate([
ch1_layer,
ch2_layer,
ch3_layer])
hidden1 = Dense(30, activation='linear', name='weighted_average_channels')(visible)
output = Dense(num_classes, activation='softmax')(hidden1)
model = Model(inputs= [ch1, ch2, ch3], outputs=output)
# Compile model
model.compile(loss='categorical_crossentropy', optimizer=SGD(), metrics=['accuracy'])
monitor = EarlyStopping(monitor='val_loss', min_delta=1e-4, patience=5, verbose=1, mode='auto')
Then, I try to fit the model:
# Fit the model
model.fit(X_train, y_train,
epochs=epochs,
batch_size=batch_size,
validation_data=(X_test, y_test),
callbacks=[monitor],
verbose=1)
and I get the following error:
ValueError: Error when checking model input: the list of Numpy arrays that you are passing to your model is not the size the model expected. Expected to see 3 array(s), but instead got the following list of 1 arrays...
How should I reshape my data, to solve the issue?
You magically assume a single input with 3 time series X_train will split into 4 channels and be assigned to different inputs. Well this doesn't happen and that is what the error is complaining about. You have 1 input:
ch123_in = Input(shape=(series_len, 3), name='ch123')
latent = LSTM(num_RNN_neurons)(ch123_in)
hidden1 = Dense(30, activation='linear', name='weighted_average_channels')(latent)
By merging the series together into single LSTM, the model might pickup relations across time series as well. Now your target shape has to be y_train.shape == (2250, 5), the first dimension must match X_train.shape[0].
Another point is you have Dense layer with linear activation, that is almost useless as it doesn't provide any non-linearity. You might want to use a non-linear activation function like relu.
Related
i'm a deep learning enthusiasts, and i want finetune the keras.application deep model to train a new model, but something wrong. And the code is :
import keras.applications.resnet_v2
from keras.applications.resnet_v2 import ResNet50V2
from keras.applications.resnet_v2 import preprocess_input, decode_predictions
import numpy as np
import keras
import tensorflow as tf
from keras.layers import Dense, Input
from keras.models import Model
import os
mnist = tf.keras.datasets.mnist
(x_train, y_train), (x_test, y_test) = mnist.load_data()#train_szie :(60000, 28, 28)
print(x_train.shape)#(60000, 28, 28)
print(y_train.shape)#(60000,)
x_train = [cv2.cvtColor(cv2.resize(img,(28,28)),cv2.COLOR_GRAY2BGR) for img in x_train]
x_train = np.asarray(x_train)
x_test = [cv2.cvtColor(cv2.resize(img,(28,28)),cv2.COLOR_GRAY2BGR) for img in x_test]
x_test = np.asarray(x_test)
x_train, x_test = x_train / 255.0, x_test / 255.0
base_model = ResNet50V2( weights='imagenet', include_top = False)
x = base_model.get_layer("conv2_block1_preact_bn").output
x = Dense(1024, activation='relu')(x)
predictions = Dense(200, activation='softmax')(x)
model = Model(inputs=base_model.input, outputs=predictions)
model.compile(optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
model.fit(x_train, y_train, epochs=1)
when i run it, the code is error,named
"res = tf.nn.sparse_softmax_cross_entropy_with_logits(
Node: 'sparse_categorical_crossentropy/SparseSoftmaxCrossEntropyWithLogits/SparseSoftmaxCrossEntropyWithLogits'
logits and labels must have the same first dimension, got logits shape [1568,10] and labels shape [32]
[[{{node sparse_categorical_crossentropy/SparseSoftmaxCrossEntropyWithLogits/SparseSoftmaxCrossEntropyWithLogits}}]] [Op:__inference_train_function_4157]"
but when i write the code, is now running fine.
x = base_model.output
I don't know why,i need your help, thanks
Quite a few changes required in your code and has multiple issues.
You're missing a Flatten layer in the middle. Because of that, when computing loss (with a 1D vector of labels), tensorflow squash all of the dimensions except the last together giving [7 x 7 x32 = 1568, 200] sized set of logits. Here 7x7 is the output width and height for a 28x28 sized input by your last conv layer. 32 is default batch size when using model.fit(). This is not compatible with the labels which would simply be a 32 item 1d vector in one iteration.
base_model = ResNet50V2(weights='imagenet', include_top = False)
# Input the way you've defined has undefined width and heigh
# dimension. This is an important requirement for the Flatten layer downstream
input = Input(shape=(28,28,3))
x = input
# Getting only the layers we care about
for layer in base_model.layers:
x = layer(x)
if layer.name == "conv2_block1_preact_bn":
break
# Before feeding data into the dense layer, you need to flatten
x = Flatten()(x)
x = Dense(1024, activation='relu')(x)
predictions = Dense(200, activation='softmax')(x)
model = Model(inputs=input, outputs=predictions)
model.compile(optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy'])
One concern I have is that mnist has 10 classes, so your last layer should only need 10 not 200
I don't work with Keras or TF very often so just trying to understand how it works. For example, this is a bit confusing: we generate some points of sine plot and trying to predict the remainder:
import numpy as np
from tensorflow.keras import layers
from tensorflow.keras.models import Sequential
a = np.array([np.sin(i) for i in np.arange(0, 1000, 0.1)])
b = np.arange(0, 1000, 0.1)
x_train = a[:8000]
x_test = a[8000:]
y_train = b[:8000]
y_test = b[8000:]
model = Sequential(layers.Dense(20, activation='relu'))
model.compile(optimizer='adam', loss='mean_squared_error')
model.fit(x=x_train, y=y_train, epochs=200, validation_split=0.2)
Now if I generate predictions either by simply calling model(x_test) or by using predict(x_test) method, the array that I get has a shape (2000, 20).
Why is this happening? Why do I get multiple predictions? And how do I get just a 1-dimensional array of predictions?
It's because, in your model, you have 20 relu activated features in your last layer. That gave 20 features of a single instance in the inference time. All you need to do (as you requested) is to use a layer with 1 unit, place it as the last layer, and probably no activation.
Try this:
import numpy as np
from tensorflow.keras import layers
from tensorflow.keras.models import Sequential
a = np.array([np.sin(i) for i in np.arange(0, 1000, 0.1)])
b = np.arange(0, 1000, 0.1)
x_train = a[:8000]
x_test = a[8000:]
y_train = b[:8000]
y_test = b[8000:]
model = Sequential(
[
layers.Dense(20, activation='relu'),
layers.Dense(1, activation=None)
])
model.compile(optimizer='adam', loss='mean_squared_error')
model.fit(x=x_train, y=y_train, epochs=2, validation_split=0.2)
model.predict(x_test).shape
(2000, 1)
Okay so I'm pretty new to deep learning and have a very basic doubt. I have an input data with an array containing 255 data (Araay shape (255,)) in epochs_data and their corresponding labels in new_labels (Array shape (255,)).
I split the data using the following code:
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(epochs_data, new_labels, test_size = 0.2, random_state=30)
I'm using a sequential model:
from keras.models import Sequential
from keras import layers
from keras.layers import Dense, Activation, Flatten
model = Sequential()
I know how to code for the hidden layers and output layer:
model.add(Dense(500, activation='relu')) #Hidden Layer
model.add(Dense(2, activation='softmax')) #Output Layer
But I don't know how to code layer for input with the input_shape specified. The X_train is the input.It's an array of shape (180,). Also tell me how to code the model.fit() for the same. Any help is appreciated.
You have to copy this line before the hidden layer. You can add the activation function that you want. Finally, as you can see this line represent both the input layer and the 1° hidden layer (you have to choose the n° of neuron (I put 100) )
model.add(Dense(100, input_shape = (X_train.shape[1],))
EDIT:
Before fitting your model you have to configure your model with this line:
model.compile(loss = 'mse', optimizer = 'Adam', metrics = ['mse'])
So you have to choose a metric that in this case is Mean Squarred Error and an optimizer like Adam, Adamax, ect.
Then you can fit your model choosing the data (X,Y), n° epochs, val_split and the batch size.
history = model.fit(X_train, y_train, epochs = 200,
validation_split = 0.1, batch_size=250)
I have the following code that creates LSTM network using Keras with TensorFlow backend.
This code runs well.
import numpy as np
import pandas as pd
from sklearn import model_selection
from keras.models import Sequential
from keras.layers import Dense, Activation, Dropout
from keras.layers.recurrent import LSTM
from keras.utils import np_utils
flights = {
'flight_stage': [1,0,1,1,0,0,1],
'scheduled_hour': [16,16,17,17,17,18,18],
'delay_category': [1,0,2,2,1,0,2]
}
columns = ['flight_stage', 'scheduled_hour', 'delay_category']
df = pd.DataFrame(flights, columns=columns)
X = df.drop('delay_category',1)
y = df['delay_category']
X_train, X_test, y_train, y_test = model_selection.train_test_split(X, y, test_size=0.25, random_state=42)
nb_features = X_train.shape[1]
nb_classes = y.nunique()
hidden_neurons = 32
timestamps = X_train.shape[0]
# Reshape input data to 3D array
X_train = X_train.values.reshape(1, X_train.shape[0], X_train.shape[1])
X_test = X_test.values.reshape(1, X_test.shape[0], X_test.shape[1])
y_train = np_utils.to_categorical(y_train, nb_classes)
y_test = np_utils.to_categorical(y_test, nb_classes)
model = Sequential()
model.add(LSTM(
units=hidden_neurons,
return_sequences=True,
input_shape=(timestamps,nb_features)
)
)
model.add(Dropout(0.2))
model.add(Dense(activation='softmax', units=nb_classes))
model.compile(loss="categorical_crossentropy",
optimizer='adadelta')
But when I start training the model, it fails:
history = model.fit(X_train, y_train, validation_split=0.25, epochs=500, batch_size=2, shuffle=True, verbose=0)
Error:
ValueError: Error when checking target: expected dense_19 to have 3 dimensions, but got array with shape (5, 3)
This error refers to the final Dense layer. I used model.summary() to get exact dimensions. The output shape of a Dense layer is (None, 5, 3).
However I do not understand why does it have 3 dimensions and what None stands for (how did it appear in this last layer)?
3 is the number of units returned by the last layer. It is the number of classes for the softmax activation
5 is the number of units returned by the lstm which indicates the size of the sequences returned
None is the number of element by batch for the last layer. It simply means that the last layer can accept different size for each batches of tensor of shape [5, 3]
X_train shape: (1, 5, 2),
X_test shape: (1, 2, 2),
y_train shape: (5,3),
y_test shape: (2,3)
Looking at the data shape, there is clearly a mismatch between the batchsize of the features and that of the labels. The most left number should be equal between the features shape X and the labels shape y. It is the batchsize.
'1', 5, 2 => batch size of 1
'2', 3 => batch size of 2
There is a mismatch here.
Also to solve the issue between the output of the lstm layer and the input of the last layer, one can use a layer.flatten
nb_classes = 3
hidden_neurons = 32
model = Sequential()
model.add(LSTM(
units=hidden_neurons,
return_sequences=True,
input_shape=(5, 2)
)
)
model.add(Dropout(0.2))
model.add(Flatten())
model.add(Dense(activation='softmax', units=nb_classes))
model.compile(loss="categorical_crossentropy",
optimizer='adadelta')
model.compile(loss='categorical_crossentropy',
optimizer='adam')
live code
Hello I am currently making some simple NN but there are some problems that I don't know why.
The code looks like this
import csv
import numpy as np
np.random.seed(123) # for reproducibility
from keras.models import Sequential
from keras.layers import Dense, Dropout, Activation, Flatten
from keras.layers import Convolution2D, MaxPooling2D
from keras.utils import np_utils
from keras.datasets import mnist
f = open('training.csv', 'r', encoding='utf-8')
rdr = csv.reader(f)
X_train = list()
Y_train = list()
print(type(X_train))
for ele in rdr :
# print(type(ele))
# print(type(ele[0]))
X_train.append([float(ele[0])])
Y_train.append([float(ele[1])])
# reshaping the datas
X_train = np.asarray(X_train)
Y_train = np.asarray(Y_train)
print(X_train.shape)
X_train_1 = X_train.reshape(X_train.shape[0], 1,1)
print(X_train_1.shape)
# print(X_train.shape)
# defining models
model = Sequential()
# model.add(Flatten())
model.add(Dense(4, activation='relu',input_shape=((1, 1))))
print ("model.output_shape1: ", model.output_shape)
# model.add(Dropout(0.5))
# print("Hello")
# print(model.input_shape)
# model.add(Dense(4, activation='softmax'))
# model.add(Dense(1, activation='softmax'))
# print(model.input_shape)
# 8. Compile model
model.compile(loss='categorical_crossentropy',
optimizer='adam',
metrics=['accuracy'])
# 9. Fit model on training data
model.fit(X_train, Y_train,
batch_size=32, nb_epoch=10, verbose=0)
# 10. Evaluate model on test data
score = model.evaluate(X_train_1, Y_train, verbose=0)
# print (problem)
# print (answer)
f.close()
the error message was
expected dense_1_input to have 3 dimensions, but got array with shape
(863, 1)
the problem is I think I made my X_train_1 array into available shape before entering to NN what did I do wrong?
Also, if I increase the dimension 1 more time by
X_train_1 = X_train.reshape(X_train.shape[0], 1,1, 1)
then there are errors like
expected dense_1_input to have 3 dimensions, but got array with shape
(863, 1,1, 1)
It seems that I can't make 3 demension array What did I do wrong?
Your input dim id 1. First number in (863, 1) is a number of samples.
Error message
expected dense_1_input to have 3 dimensions, but got array with shape
(863, 1)
suggests that your input is a list of 863 float numbers shaped (1,) please try to change the input shape to input_shape=(1,) or input_dim=1
IF you would like to reshape your data please take a look at this code:
import numpy as np
from keras.models import Sequential
from keras.layers import Dense, Flatten
X_train = list()
Y_train = list()
for ele in range(0,10):
X_train.append([float(ele)])
Y_train.append([float(ele)])
# reshaping the data
X_train = np.asarray(X_train)
Y_train = np.asarray(Y_train)
X_train = X_train.reshape(X_train.shape[0], 1, 1)
print(X_train.shape)
model = Sequential()
model.add(Dense(4, activation='relu', input_shape=(1, 1)))
model.add(Flatten())
model.add(Dense(1, activation='softmax'))
# 8. Compile model
model.compile(loss='binary_crossentropy',
optimizer='adam',
metrics=['accuracy'])
#
# # 9. Fit model on training data
model.fit(X_train, Y_train,
batch_size=32, epochs=10, verbose=1)
I had to change it a bit but it shows the way you should reshape your data.
Please notice that I had to flatten the data in order to match the output shape. I also changed loss to binary_crossentropy as my output is a binary data. If you would like to classify to multiple classes you'll have to one hot your output.