I'm working with a time-series data, that has shape of 2000x1001, where 2000 is the number of cases, 1000 rows represent the data in time-domain, displacements in X direction during 1 sec period, meaning that the timestep is 0.001. The last column represents the speed, the output value that I need to predict based on the displacements during 1 sec. How the Input Data should be shaped for RNN in Keras? I've gone trough some tutorials, but still I'm cofused about Input Shape in RNN. Thanks in advance
#load data training data
dataset=loadtxt("Data.csv", delimiter=",")
x = dataset[:,:1000]
y = dataset[:,1000]
#Create train and test dataset with an 80:20 split
x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.2)
#input scaling
scaler = StandardScaler()
x_train_s =scaler.fit_transform(x_train)
x_test_s = scaler.transform(x_test)
num_samples = x_train_s.shape[0] ## Number of samples
num_vals = x_train_s.shape[1] # Number of elements in each sample
x_train_s = np.reshape(x_train_s, (num_samples, num_vals, 1))
#create model
model = Sequential()
model.add(LSTM(100, input_shape=(num_vals, 1)))
model.add(Dense(1, activation='relu'))
model.compile(loss='mae', optimizer='adam',metrics = ['mape'])
model.summary()
#training
history = model.fit(x_train_s, y_train,epochs=10, verbose = 1, batch_size =64)
look at this code:
it is trying to predict next 4 values based on previous 6 values.
follow the comments and see how very simple input is manipulated for using it
as input in rnn/lstm
follow the comments within code
from __future__ import absolute_import, division, print_function, unicode_literals
import tensorflow as tf
from tensorflow.keras import Model
import numpy as np
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from tensorflow.keras.layers import RNN, LSTM
"""
creating a toy dataset
lets use this below ```input_sequence``` as the sequence to make data points.
as per the question, we will use 6 points to predict next 4 points
"""
input_sequence = [1,2,3,4,5,6,7,8,9,10,1,2,3,4,5,6,7,8,9,10,1,2,3,4,5,6,7,8,9,10]
X_train = []
y_train = []
**#first 6 points will be our input data points and next 4 points will be data label.
#so on we will shift by 1 and make such data points and label pairs**
for i in range(len(input_sequence)-9):
X_train.append(input_sequence[i:i+6])
y_train.append(input_sequence[i+6:i+10])
X_train = np.array(X_train, dtype=np.float32)
y_train = np.array(y_train, dtype=np.int32)))
**#X_test for the predictions (contains 6 points)**
X_test = np.array([[8,9,10,1,2,3]],dtype=np.float32)
print(X_train.shape)
print(y_train.shape)
print(X_test.shape)
**#we will be using basic LSTM, which accepts input in ```[num_inputs, time_steps, data_points], therefore reshaping as per that```**
# so here:
# 1. num_inputs = how many sequence of 6 points you want to use i.e. rows (we use X_train.shape[0] )
# 2. time_steps = batches you can considered i.e. if you want to use 1 or 2 or 3 rows
# 3. data_points = number of points (for ex. in our case its 6 points we are using)
X_train = np.reshape(X_train, (X_train.shape[0], 1, X_train.shape[1]))
X_test = np.reshape(X_test, (X_test.shape[0], 1, X_test.shape[1]))
print(X_train.shape)
print(y_train.shape)
print(X_test.shape)
x_points = X_train.shape[-1]
print("one input contains {} points".format(x_points))
model = Sequential()
model.add(LSTM(4, input_shape=(1, x_points)))
model.add(Dense(4))
model.compile(loss='mean_squared_error', optimizer='adam')
model.summary()
model.fit(X_train, y_train, epochs=500, batch_size=5, verbose=2)
output = list(map(np.ceil, model.predict(X_test)))
print(output)
hope it helps. ask for any doubt pls.
Like explained in the doc, Keras expects the following shape for a RNN:
(batch_size, timesteps, input_dim)
batch_size is the umber of samples you feed before a backprop
timesteps is the number of timesteps for each sample
input_dim is the number of features for each timestep
EDIT more details:
In your case you should go for
batch_input_shape = (batch_size, timesteps, 1)
With batch_size and timesteps selected as you wish.
What about the timesteps?
Let's say you take one of your 2000 samples, and let's say that your sample has 10 elements instead of 1000, for example:
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
Then, if we chose timesteps=3, then you get a batch of length 8:
[[[0], [1], [2]],
[[1], [2], [3]],
[[2], [3], [4]],
[[3], [4], [5]],
[[4], [5], [6]],
[[5], [6], [7]],
[[6], [7], [8]],
[[7], [8], [9]]]
Related
I have 500 sample of housing data which I have converted all to numbers. It has 12 columns which are used to predict 1 price.
However, when I try to run the model, its loss is massive(14 digit loss). I have normalize the data but this had no effect. This is causing the programs predictions to be very off, x100 off. what can i do to fix this. Here is the code:
import numpy as np
from tensorflow.keras.layers import Input, Dense
from tensorflow.keras.models import Model
import pandas as pd
from sklearn.preprocessing import StandardScaler, MinMaxScaler
data = pd.read_csv('Housing.csv')
valdata = pd.read_csv('val.csv')
mapping1 = {'yes': 1, 'no': 0}
mapping2 = {'furnished': 2, 'semi-furnished': 1, "unfurnished": 0}
cols_to_convert = ['mainroad', 'guestroom', "basement", "hotwaterheating", "airconditioning", "prefarea"]
for col in cols_to_convert:
data[col] = data[col].map(mapping1)
valdata[col] = valdata[col].map(mapping1)
data["furnishingstatus"] = data["furnishingstatus"].map(mapping2)
valdata["furnishingstatus"] = valdata["furnishingstatus"].map(mapping2)
x_train = np.array(data.drop("price", axis=1))
y_train = np.array(data["price"])
scaler = MinMaxScaler()
x_train = scaler.fit_transform(x_train)
scaler = StandardScaler()
x_train = scaler.fit_transform(x_train)
input_shape = x_train[0].shape
inputs = Input(shape=input_shape)
# Define layers
dense1 = Dense(8, activation='relu')(inputs)
dense2 = Dense(1, activation='linear')(dense1)
model = Model(inputs=inputs, outputs=dense2)
model.compile(loss='mean_squared_error', optimizer='adam')
model.fit(x_train, y_train, epochs=100, validation_split=0.2)
x_new = np.array([[7420, 4, 2, 3, 1, 0, 0, 0, 1, 2, 1, 2]])
y_new = model.predict(x_new)
print(y_new)
And hers is roughly what the csv file looks like before mapping the strings to numbers.
Image of csv
You can try to improve your model by increasing the number of hidden layers (say from 1 to 2) or/and increasing the number of units in hidden Dense layers (say to 256 and 64).
But actually you might get better results when using other ML algorithms, like Random Forests or XGBoost instead of DNN. Please check the following article, which shows that Random Forests outperform DNN for tabular data like you have:
https://www.kdnuggets.com/2019/06/random-forest-vs-neural-network.html
I have seen that many others on stackoverflow have posted about this same problem, but I haven't been able to figure out how to apply those solutions to my example.
I have been working on creating a model to predict an outcome of either 0 or 1 based on a dataset which contains 16 features - Everything has seemed to work fine (accuracy evaluation, epoch completion, etc.).
As mentioned, my training features include 16 different variables, but when I pass in a list that contains 16 unique values separate from the training dataset in order to try and make an individual prediction (of either 0 or 1), I get this error:
ValueError: Layer sequential_11 expects 1 input(s), but it received 16 input tensors.
Here is my code -
y = datas.Result
X = datas.drop(columns = ['Date', 'home_team', 'away_team', 'home_pitcher', 'away_pitcher', 'Result'])
X = X.values.astype('float32')
y = y.values.astype('float32')
X_train, X_test, y_train, y_test = train_test_split(X,y, test_size = 0.2)
X_train, X_validation, y_train, y_validation = train_test_split(X, y, test_size = 0.2)
model=keras.Sequential([
keras.layers.Dense(32, input_shape = (16,)),
keras.layers.Dense(20,activation=tf.nn.relu),
keras.layers.Dense(2,activation='softmax')
])
model.compile(optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['acc'])
history = model.fit(X_train,y_train,epochs=20, validation_data=(X_validation, y_validation))
#all variables within features list are single values, ex: .351, 11, .991, etc.
features = [t1_pqm,t2_pqm,t1_elo,t2_elo,t1_era,t2_era,t1_bb9,t2_bb9,t1_fip,t2_fip,t1_ba,t2_ba,t1_ops,t2_ops,t1_so,t2_so]
prediction = model.predict(features)
The model expects an input of shape (None,16) but features has the shape (16,) (1D list). The easiest solution is to make it an numpy array with the right shape (1, 16):
features = np.array([[t1_pqm,t2_pqm,t1_elo,t2_elo,t1_era,t2_era,t1_bb9,t2_bb9,t1_fip,t2_fip,t1_ba,t2_ba,t1_ops,t2_ops,t1_so,t2_so]])
I'm using Tensorflow to train a network to predict the third item in a list of numbers.
When I train, the network appears to train quite well and do well on both the training and test set. However, when I evaluate its performance myself, it seems to be doing quite poorly.
For example, at the end of training, Tensorflow says that the validation loss is 2.1 x 10^(-5). However, when I compute it myself, I get 0.17 x 10^0. What am I doing wrong?
Here's code that can be run on Google Colab:
import numpy as np
import tensorflow as tf
from sklearn.model_selection import train_test_split
def create_dataset(k=5, n=2, example_amount=200):
'''Create a dataset of numbers where the goal is to always output the nth number'''
# UPGRADE: this could be done better with numpy to just generate all the examples at once
example_amount = 1000
x = []
y = []
ans = [x, y]
for i in range(example_amount):
example_x = np.random.rand(k)
example_y = example_x[n]
x.append(example_x)
y.append(example_y)
return ans
def tensorize(tensor_like) -> tf.Tensor:
'''Turn stuff into tensors'''
return tf.convert_to_tensor(tensor_like, dtype=tf.float32)
def split_dataset(dataset, train_split=0.8, random_state=42):
'''
Takes in a list (or tuple) where index 0 contains the inputs and index 1 contains the outputs
outputs x_train, x_test, y_train, y_test, train_indexes, test_indexes all as tf.Tensor
'''
indices = np.arange(len(dataset[0]))
return tuple([tensorize(data) for data in train_test_split(dataset[0], dataset[1], indices, train_size=train_split, random_state=random_state)])
# how many numbers in each example
K = 5
# the index of the solution
N = 2
# how many examples
EXAMPLE_AMOUNT = 20000
# what percentage of the examples are in the training set
TRAIN_SPLIT = 0.5
# how long to train for
epochs = 50
dataset = create_dataset(K, N, EXAMPLE_AMOUNT)
x_train, x_test, y_train, y_test, train_indexes, test_indexes = split_dataset(dataset, train_split=TRAIN_SPLIT)
model_input = tf.keras.layers.Input(shape=(K,), name="input")
model_dense1 = tf.keras.layers.Dense(10, name="dense1")(model_input)
model_dense2 = tf.keras.layers.Dense(10, name="dense2")(model_dense1)
model_output = tf.keras.layers.Dense(1, name="output")(model_dense2)
model = tf.keras.Model(inputs=model_input, outputs=model_output)
model.compile(optimizer=tf.keras.optimizers.Adam(), loss="mse")
history = model.fit(x=x_train, y=y_train, validation_data=(x_test, y_test), epochs=epochs)
# the validation loss as Tensorflow computes it
print(history.history["val_loss"][-1]) # 2.1036579710198566e-05
# the validation loss as I compute it
val_loss = tf.math.reduce_mean(tf.keras.losses.MSE(y_test, model.predict(x_test))).numpy()
print(val_loss) # 0.1655631
What you miss is that the shape of y_test.
y_test.numpy().shape
(500,) <-- causing the behaviour
Simply reshape it like:
val_loss = tf.math.reduce_mean(tf.keras.losses.MSE(y_test.numpy().reshape(-1,1), model.predict(x_test))).numpy()
print(val_loss) # 1.1548506e-05
Also:
history.history["val_loss"][-1] # 1.1548506336112041e-05
Or you can flatten() both of the data while calculating it:
val_loss = tf.math.reduce_mean(tf.keras.losses.MSE(y_test.numpy().flatten(), model.predict(x_test).flatten())).numpy()
print(val_loss) # 1.1548506e-05
So, I have this little project going on about predicting the nba 2019 champion but it seems that my code is not clear enough to make keras understand what I want. I have passed a list of past champions on my dataset and made it the output class to get the current champion.
I'm using a dataset for teams stats from 2014 to 2018 regular seasons and I'm assuming that I should have the 2019 stats to do it. I have made my dataset very well encoded for my NN to understand by providing one hot encoding in every feature I think it's useful.
x = pd.concat([df.drop(['Unnamed: 0','Team','Game','Date','Opponent','LastSeasonChamp'], axis = 1), df_ohc], axis = 1)
y = df['LastSeasonChamp']
x_train, x_test, y_train, y_test = train_test_split(x, y, test_size = 0.35)
x_train = tf.keras.utils.normalize(x_train.values, axis = 1)
x_test = tf.keras.utils.normalize(x_test.values, axis = 1)
n_classes = 30
model = tf.keras.Sequential()
model.add(tf.keras.layers.Flatten())
model.add(tf.keras.layers.Dense(x_train.shape[1], input_shape = (x_train.shape[0],x_train.shape[1]), activation = tf.nn.relu))
model.add(tf.keras.layers.Dense(np.mean([x_train.shape[1], n_classes], dtype = int), activation = tf.nn.relu))
model.add(tf.keras.layers.Dense(n_classes, activation = tf.nn.softmax))
model.compile(optimizer = 'adagrad' , loss = 'sparse_categorical_crossentropy', metrics=['accuracy'])
model.fit(x_train, y_train.values, epochs = 3)
model.evaluate(x_test, y_test)
model.save('nba_champ_2019')
new_model = tf.keras.models.load_model('nba_champ_2019')
pred = new_model.predict(x_test)
y_pred = to_categorical(pred)
So, I could expect my y_pred to be a column with 0 and 1 and but all I get is a column full of 1.
The to_categorical function is used to convert a list of class IDs to an one-hot matrix. You don't need it here. You should get the output you expect by removing in this case.
If I have a set of targets a.k.a y's as [1,0,9,9,7,5,4,0,4,1] and I use model.predict(X) Keras returns a 6 item array for each of the 10 samples. It returns 6 items because there are 6 possible targets (0,1,4,5,7,9) and keras returns a decimal/float (for each label) representing likelihood of any one of those being the correct target. For the first sample, for example - where y=1 Keras returns an array that looks like this: [.1, .4,.003,.001,.5,.003].
I want to know which value matches to which target (does .1 refer to 1 because it's first in the dataset or 0 because it's the lowest number or 9 because it's the last number, etc). How does Keras order it's predictions? The documentation does not seem to articulate this; it only says
"Generates output predictions for the input samples."
So I'm not sure how to match the labels to the prediction results.
EDIT:
Here is my model and training code:
X_train, X_test, y_train, y_test = train_test_split(data, labels, test_size=0.25, random_state=42)
Y_train = to_categorical(y_train)
Y_test = to_categorical(y_test)
sequence_input = Input(shape=(MAX_SEQUENCE_LENGTH,), dtype='int32')
embedded_sequences = embedding_layer(sequence_input)
x = Conv1D(64, 5, activation='relu')(embedded_sequences)
x = MaxPooling1D(4)(x)
x = Conv1D(64, 5, activation='relu')(x)
x = MaxPooling1D(4)(x)
x = Conv1D(64, 5, activation='relu')(x)
x = MaxPooling1D(4)(x) # global max pooling
x = Flatten()(x)
x = Dense(64, activation='relu')(x)
preds = Dense(labels_Index, activation='softmax')(x)
model = Model(sequence_input, preds)
model.fit(X_train, Y_train, epochs=10, verbose = 1)
Keras doesn't order anything, it all depend on how the classes in the data you used to train the model are defined and one-hot encoded.
You can usually recover the integer class label by taking the argmax of the class probability array for each sample.
From your example, 0.1 is class 0, 0.4 is class 1, 0.003 is class 2, 0.001 is class 3, 0.5 is class 4, and 0.003 is class 5 (6 classes in total).