I'm trying the get a hang of keras and I'm trying to get basic time series prediction working. My input is a list of random ints between 0 and 10 such as:[1,3,2,4,7,5,9,0] and my labels are the same as the input but delayed such as: [X,X,1,3,2,4,7,5] and I'm trying to have my model learn this relationship of remembering past data points.
My code is:
labels = keras.utils.to_categorical(output, num_keys)
model = keras.Sequential([
keras.layers.LSTM(10),
keras.layers.Dense(10, activation='relu'),
keras.layers.Dense(10, activation=tf.nn.softmax)
])
model.compile(optimizer=tf.train.AdamOptimizer(),
loss=tf.keras.losses.categorical_crossentropy,
metrics=['accuracy'])
model.fit(input, labels, epochs=30, verbose=2,shuffle=False)
and I get the error:ValueError: Please provide as model inputs either a single array or a list of arrays. You passed: x=[7, 6,...
I've tried reformating my input with:
input=numpy.array([[i,input[i]]for i in range(len(input))])
input=numpy.reshape(input,input.shape+(1,))
and adding input_shape=input.shape[1:] to my LSTM layer and that throws no errors but the accuracy is no better then just blind guessing
This seems like that kind of thing that could be trivial but I'm clearly missing something.
With keras.layers.LSTM(10), you need to include the input data shape: keras.layers.LSTM(10, input_shape = (input.shape[1], input.shape[2])).
Keras is expecting the input data shaped as [instances, time, predictors] and since you don't have any additional predictors, you may need to reshape your input data to input.reshape(input.shape[0], input.shape[1], 1).
Keras will infer the data shapes for the next layers, but the first layer needs the input shape defined.
Related
I'm working in a neural network using keras (from tensorflow) for a college project.
I'm pretty new to the library, so I don't really know how should I feed the data into the model in order for the training to work. I've been searching the internet for hours
and I can't find a proper tutorial / documentation on how to do it.
Here's the model I'm using, one of the simplest possible ones:
model = keras.Sequential([
keras.layers.Dense(20, input_dim=1,activation = activations.relu),
keras.layers.Dense(10, activation= activations.relu),
keras.layers.Dense(8, activation= activations.sigmoid)
])
model.compile(optimizer = "adam", loss = "sparse_categorical_crossentropy",
metrics = ["accuracy"])
The input to the network is a list of 20 floats, and the output a list of 8 floats ranged from 0 to 1 (confidence level), so I think this model is OK, please let me know if I'm wrong.
Here's a diagram of the model i'm trying to build and train:
Let's say I have:
10 input examples (10 lists of 20 floats) for the expected output [1,0,0,0,0,0,0,0]
10 input examples for the expect output [0,1,0,0,0,0,0,0,0,0]
...
10 input examples for the expected output [0,0,0,0,0,0,0,1]
How should I prepare this data in order to use it with
model.fit(training_inputs,expected_outputs,epochs = NUM_EPOCHS) ?
What should training_inputs exactly be? and expected_outputs?
Any help will be appreciated. Thank you for your time!
First of all, you have two issues in your model. According to your description, your input data is 20-dimensional, so in the first layer you should have input_dim=20. Then, you have a cross-entropy loss, so I'm assuming that you are training a 8-class classifier. If that's the case, then instead of keras.layers.Dense(8, activation= activations.sigmoid) you should use
keras.layers.Dense(8, activation=None),
keras.layers.Softmax()
as that ensures that you get a distribution over classes for each input data point.
Now regarding your input data question, training_inputs should a tensor (or numpy array, which will be readily converted) with shape (n_points, 20) in your case. Accordingly, expected_outputs should have shape (n_points, 8). So, just concatenate/stack your input data along the first dimension (axis=0), such that each row corresponds to your 20-dimensional data points. You do the same for expected_outputs, maybe something like,
expected_outputs = np.r_[
np.tile([[1,0,0,0,0,0,0,0]], (10, 1)),
np.tile([[0,1,0,0,0,0,0,0]], (10, 1)),
...
np.tile([[0,0,0,0,0,0,0,1]], (10, 1)),
]
Remember to set batch_size and shuffle!
I'm having some issues with the predict function predicting all 0's or all 1's from my model. Here is my model
model = keras.Sequential(
[
layers.BatchNormalization(),
layers.Dense(200, activation="relu"),
layers.Dense(500, activation="relu"),
layers.Dense(1300, activation="relu"),
layers.Dense(2000, activation="relu"),
layers.Dense(1320, activation="relu"),
layers.Dense(710, activation="relu"),
layers.Dense(150, activation="relu"),
layers.Dense(30, activation="relu"),
layers.BatchNormalization(),
layers.Dense(1, activation="sigmoid"),
]
)
model.compile(loss='binary_crossentropy', optimizer=keras.optimizers.Adam(learning_rate=(0.001)), metrics=[metrics])
history = model.fit(training, target, batch_size=2048, epochs=100, shuffle=True, validation_split=0.2)
I'm very new to deep learning and trying to create models to classify and get predictions. My answer is just based on a 0 or 1 which will say if a customer is going to leave or stay as a customer in the long run. I've tested the data for null and NaN.
I've looked at a lot of posts about what this could be, and for the most part it seems that people were using the wrong activation function for a classification instead of regression problem. And the answer was that if you're using binary crossentropy, you should be using sigmoid (Why does a binary Keras CNN always predict 1?). I thought the output of my network would be correct seeing that I am using ReLu and SigMoid with binary crossentropy but whenever I predict, it's persistent in being all 0's or all 1's. The layers might not make too much sense, I'm still very new at this and playing around to see how layers are affecting the results of when I train and evaluate.
Here is roughly how I am using predict with the data
data = pd.read_csv("judge.csv", skiprows=range(0,0))
samples_to_predict = data.drop(['Surname', 'CreditScore', 'Geography', 'Gender', 'Tenure', 'NumOfProducts', 'HasCrCard', 'EstimatedSalary'], axis=1)
prediction = loaded_model.predict(samples_to_predict.values)
print(prediction)
I've been trying to debug this for a while and any help as to which direction to error could be coming from would be welcomed. I've tried increasing my epoch to 1000, I tried lowering my learning_rate, I believe BatchNormalization might take care of not scaling my data(I might be misunderstanding that), tried lowering my batch_size, I tried simply using 3 Dense layers being two ReLu and one Sigmoid, checked that the data I'm predicting is a numpy array and they've all so far produced the same result of predict outputting all 0's or all 1's.
Turns out I was using predict with categories of the data that I had not trained the model with. For example I have a column titled CustomerID that I dropped to train the model, but when I was predicting, I had forgotten to drop that column which made my model predict all 0's or all 1's. After fixing that issue and making sure that I am using only the categories that I trained it with, to predict, got me predictions that were not all 0's or all 1's.
I have got a dataset which consists of 100,000 rows and 12 columns, where each column stands of a certain input to train a sequential GRU model to predict only 1 output. The following is the code for the model:
model = Sequential()
model.add(GRU(units=70, return_sequences=True, input_shape=(1,12),activity_regularizer=regularizers.l2(0.0001)))
model.add(GRU(units=50, return_sequences=True,dropout=0.1))
model.add(GRU(units=30, dropout=0.1))
model.add(Dense(units=5))
model.add(Dense(units=3))
model.add(Dense(units=1, activation='relu'))
model.compile(loss=['mae'], optimizer=Adam(lr=0.0001),metrics=['mse'])
model.summary()
history=model.fit(X_train, y_train, batch_size=1000,epochs=30,validation_split=0.1, verbose=1)
However, before that I had to transform the training dataset from 2D to 3D using x_train=x.reshape(-1,1,12) and the output from 1D to 2D using y_train=y.reshape(-1,1). That is the part I really don't understand, why not just keep them as they are?
You had to describe your data in order to be decisive.
But since each layers output is the input of the next layer, their shape must be equal. In the incomplete example you gave your labels need to be a single value for each sample and I think that's why reshape was used.
I hope that together we can solve my problem, I will try to summarise it and paste in some codes.
At our research we are using TensorFlow/Keras for our DNN, to classify images (convolutional network). It was a quite simple Sequential model, but now we are trying to add more inputs, so I started to change the network to the Functional API (honestly, this is the first time that I used it). I recreated the original convolutional network and everything was working fine.
So I generated the necessary additional data into simple text files (one for each image) and these will be the second input in our model. This is were something went wrong, because I've got an error and now I am stuck with the solution.
To recreate the error, I made a new simple python file without the convolution part, just to try out the model's text file part. The input is 5000 text files, containing only a float number. After the preprocessing, we will have 4000 for train and additional 1000 for testing, both are stored in a numpy array. The train and test arrays were split through sklearn.model_selection.train_test_split.
In the original multi-channel network, I tried to concatenate the convolutional part with the text file and after it some dense layers are coming. But no concatenate here, just the train for the data stored in the text files.
Here are the shapes of the input and label arrays:
X_train.shape
(4000,)
y_train.shape
(4000, 5)
The - very - simple network:
inputA = Input(shape=(X_train.shape[0],))
x = Flatten()(inputA)
x = Dense(256, activation='relu')(x)
x = Dense(256, activation='relu')(x)
x = Dense(64, activation='relu')(x)
x = Dense(5, activation='softmax')(x)
x = Model(inputs=inputA, outputs = x)
Compile:
x.compile(loss='categorical_crossentropy', optimizer='sgd', metrics=['accuracy'])
Model fit:
x.fit(X_train, y_train,
validation_data=(X_test, y_test),
batch_size=batch,
verbose=1,
epochs=epoch_number)
And the received error message:
ValueError: Error when checking input: expected input_8 to have shape (4000,) but got array with shape (1,)
The question is, what did I wrong? The previous codes are working just fine in the Sequential model, but here no. Could somebody help me solve this mystery?
Best wishes,
Tamas
It's because of this line:
inputA = Input(shape=(X_train.shape[0],))
Keras expects the number of features, not the number of samples. In your case, the input_shape=(1,).
I am working on a regression forecast where I have some complex 3D sequences and some features explaining some key characteristics of the sequences. They are held on two matrices of such shapes:
X1.shape, X2.shape
((9000, 300, 3), (9000, 106))
I want to feed them to a Model instance where the X1 matrix is dealt by a LSTM and the X2 matrix by a couple of dense layers. My plan is to merge them before the output layer.
I was planning to train by:
model.fit(zip(X1, X2), y, batch_size=BATCH, epochs=EPOCHS, validation_split=0.2)
How to build the model in order to receive the two matrices and deal them separately?
At the moment I just have my standard model for LSTM only:
def model(sample_size=300, axis=3):
inp=Input(shape=(sample_size, axis))
x=LSTM(50, return_sequences=True)(inp)
x=GlobalMaxPool1D(x)
x=Dense(1)(x)
model=Model(inputs=inp, ouputs=x)
model.compile(loss='mean_squared_error', optimizer='adam',
metrics= ['mae'])
return model
I think this should work
# First input
input1=Input(shape=(300,3))
x=LSTM(50, return_sequences=True)(input1)
x=GlobalMaxPool1D(x)
x=Dense(n)(x)
# Second Input
input2=Input(shape=(106))
y=Dense(n)(input2)
# Merge
merged=Concatenate([x,y])
merged=Dense(1)(merged)
# Define model with two inputs
model=Model(inputs=[input1,input2],outputs=merged)
Both the models should have same output space before merging. Then, you can pass a list of inputs and Keras will pass it at the appropriate places
model.fit([X1,X2],....)