So I have created a predictive model using Keras, which has accuracy about 60%-65%.
So the data we pass is xtst,xtrn,ytst,ytrn to train_test_split, test_split=.3 and so on, to train and test on supervised data. Now after all these I have a new set of data say xnew.
How do i use this data to predict the y values for this new data?
Where should i feed this xnew data for it to give me y?
The model:
model = Sequential()
model.add(Dense(10,input_shape=(4,),activation = 'relu')
model.add(Dense(32,activation = 'relu'))
model.add(Dense(101,activation = 'softmax'))
from keras import optimizers
model.compile(Adam(lr=.01),loss='sparse_categorical_crossentropy',metrics=['accuracy'])
model.fit(x_train,y_train,batch_size=20,epochs=40,shuffle=True,verbose=2)
pred = model.predict(x_test,batch_size = 10,verbose = 2)
for i in pred:
print(i)
When you have trained your model you can use model.save(your_model_name.h5) to save your model. Then you can load it again using model = load_model(your_model_name.h5). From there you can use model.predict(xnew) or perhaps model.predict_classes(xnew) if you have made a classifier. I suggest that you look at the Model API also.
for a multi-category classifier model with a softmax output layer you can train the model using those vectors
x => [....] #some vector
y => [0,0,0,1,0,0,0,...]
where y is a vector indicating the probabilities of each category
to predict given some x like y = model.predict(x) you will get a probabilities vector like this [0.1, 0.05, 0.5, ....] you simply need to find the index with the max probability, you can use category = numpy.argmax(y)
Related
I want to create a model which can predict two outputs. I did some research and I found that there's a way to do it by creating two branches (for predicting two outputs) using functional API in Tensorflow Keras but I have a another approach in my mind which looks like this :
i.e. given a input, first I want to predict output1 and then based on that I want to predict output2.
So how can this can be done in Tensorflow ?
Please let me know how the training will be done as well i.e. how I'll be to pass labels for each output1 and output2 and then calculate the loss as well.
Thank you
You can do it with functional API of tensorflow. I write it in some sort of pseudo code:
Inputs = your_input
x = hidden_layers()(Inputs)
Output1 = Dense()(x)
x = hidden_layers()(Output1)
Output2 = Dense()(x)
So you can separate it to two models if it is what you desired:
model1 = tf.keras.models.Model(inputs=[Input], outputs=[Output1])
model2 = tf.keras.models.Model(inputs=[Input], outputs=[Output2])
Or have everything in one model:
model = tf.keras.models.Model(inputs=[Input], outputs=[Output2])
Output1_pred = model.get_layer('Output1').output
UPDATE:
In order to training model with two outputs, you can separate model to two parts and train each part separately as follow:
model1 = tf.keras.models.Model(inputs=[Input], outputs=[Output1])
model2 = tf.keras.models.Model(inputs=[model1.get_layer('Output1').output], outputs=[Output2])
model1.cmpile(...)
model1.fit(...)
for layer in model1.layers:
layer.trainable = False
model2.compile(...)
model2.fit(...)
You can actually modify the great answer by #Mohammad to compose a unique model with two outputs.
Inputs = your_input
x = hidden_layers()(Inputs)
Output1 = Dense()(x)
x = hidden_layers()(Output1)
Output2 = Dense()(x)
model = tf.keras.models.Model(inputs=[Inputs], outputs=[Output1, Output2])
model.compile(loss=[loss_1, loss_2], loss_weights=[0.5, 0.5], optimizer=sgd, metrics=['accuracy'])
of course you can change weights, optimiser and metric according to your case.
Then the model has to be trained on data like (X, y1, y2) where (y1, y2) are output1 and output2 labels respectively.
I have a tensorflow keras model trained with tensorflow 2.3. The model takes as input an image, however the model was trained with scaled inputs and therefore we have to scale the image by 255 before inputting them into the model.
As we use this model across a variety of platforms, I am trying to simplify this by modifying the model to simply insert a rescale layer at the start of the keras model (i.e. immediately after the input). Therefore any future consumption of this model can simply pass an image without having to scale them.
I am having a lot of trouble getting this to work. I understand I need to use the following function to create a rescaling layer;
tf.keras.layers.experimental.preprocessing.Rescaling(255, 0.0, "rescaling")
But I am unsure how to insert this to the start of the model.
Thank you in advance
you can insert this layer at the top of your trained model. below an example where first we train a model manual scaling the input and the we using the same trained model but adding at the top a Rescaling layer
from tensorflow.keras.layers.experimental.preprocessing import Rescaling
# generate dummy data
input_dim = (28,28,3)
n_sample = 10
X = np.random.randint(0,255, (n_sample,)+input_dim)
y = np.random.uniform(0,1, (n_sample,))
# create base model
inp = Input(input_dim)
x = Conv2D(8, (3,3))(inp)
x = Flatten()(x)
out = Dense(1)(x)
# fit base model with manual scaling
model = Model(inp, out)
model.compile('adam', 'mse')
model.fit(X/255, y, epochs=3)
# create new model with pretrained weight + rescaling at the top
inp = Input(input_dim)
scaled_input = Rescaling(1/255, 0.0, "rescaling")(inp)
out = model(scaled_input)
scaled_model = Model(inp, out)
# compare prediction with manual scaling vs layer scaling
pred = model.predict(X/255)
pred_scaled = scaled_model.predict(X)
(pred.round(5) == pred_scaled.round(5)).all() # True
Rescaling the images is part of data preprocessing, also rescaling images is called image normalization, this process is useful for providing a uniform scale for the dataset or numerical values you are using before building your model.In keras you can do this in many ways using one of the following according to your target:
If you are training using an Artificial neural network model you can use:-
"Batch normalization layer" or "Layer Normalization" or by the rescale method of keras you mentioned. You can look at this resource for more information about normalization .
https://machinelearningknowledge.ai/keras-normalization-layers-explained-for-beginners-batch-normalization-vs-layer-normalization/
to use the rescale method you mentioned:
#importing you libraries 1st
import tensorflow as tf
from tensorflow.keras.layers import BatchNormalization
#if your are using dataset from directory
import pathlib
then import your Dataset:
Dataset_Dir = '/Dataset/ path'
image size = (256,256) #the image size in your dataset
image shape = (96,96,3) #The shape you wish for your images in your network
Then divide your dataset to train-test I use 70-30 percent
Training_set = tf.keras.preprocessing.image_dataset_from_directory(Dataset_Dir,batch_size= 32,
image_size= image_size,
validation_split= 0.3,subset = "training",seed =123)
Test set
Testing_set = tf.keras.preprocessing.image_dataset_from_directory(Dataset_Dir,image_size= image_size,
validation_split=0.3,seed=123,subset ="validation")
normalization layer:
normalization_layer = tf.keras.layers.experimental.preprocessing.Rescaling(1./255)
normalized_training_set = Training_set.map(lambda x, y: (normalization_layer(x), y))
training_image_batch,training_labels_batch = next(iter(normalized_training_set))
for more about this method too:
look at tensorflow tutorial:
https://www.tensorflow.org/tutorials/images/classification
I want to create a custom loss function for a Keras deep learning regression model. For the custom loss function, I want to use a feature that is in the dataset but I am not using that particular feature as an input to the model.
My data looks like this:
X | Y | feature
---|-----|--------
x1 | y1 | f1
x2 | y2 | f2
The input to the model is X and I want to predict Y using the model. I want something like the following as the loss function:
def custom_loss(feature):
def loss(y_true, y_pred):
root_mean__square(y_true - y_pred) + std(y_pred - feature)
return loss
I can't use a wrapper function as above, because the feature values depends on the training and test batches, thus cannot be passed to the custom loss function at the model compile time. How can I use the additional feature in the dataset to create a custom loss function?
EDIT:
I did the following based on an answer on this thread. When I make predictions using this model, does it make predictions for 'Y' or a combination of Y and the additional feature? I want to make sure because model.fit( ) takes both 'Y' and 'feature' as y to train but model.predict( ) only gives the one output. If the predictions are a combination of Y and the additional feature, how can I extract only Y?
def custom_loss(data, y_pred):
y_true = data[:, 0]
feature = data[:, 1]
return K.mean(K.square((y_pred - y_true) + K.std(y__pred - feature)))
def create_model():
# create model
model = Sequential()
model.add(Dense(5, input_dim=1, activation="relu"))
model.add(Dense(1, activation="linear"))
(train, test) = train_test_split(df, test_size=0.3, random_state=42)
model = models.create_model(train["X"].shape[1])
opt = Adam(learning_rate=1e-2, decay=1e-3/200)
model.compile(loss=custom_loss, optimizer=opt)
model.fit(train["X"], train[["Y", "feature"]], validation_data=(test["X"], test[["Y", "feature"]]), batch_size = 8, epochs=90)
predY = model.predict(test["X"]) # what does the model predict here?
First check the data structure of your input Y in fit function see if it have same structure as the answer in that thread you following, if you does thing exactly right then it should solve your problem.
When I make predictions using this model, does it make predictions for 'Y' or a combination of Y and the additional feature?
The model will have same output shape exactly like what you defined, in your case because model output is Dense(1, activation="linear"), so it have output shape y_pred.shape == (batchsize, 1), nothing more, you can be sure about that, print it out using tf.print(y_pred) to see for yourself
also i don't know if it's your typing error, last line of your custom_loss function should be :
return K.mean(K.square((y_pred - y_true) + K.std(y_pred - feature)))
instead of
return K.mean(K.square((y_pred - y_true) + K.std(y__pred - feature)))
You can also use .add_loss with a simple mse loss the following way:
input = Input(size)
output = YourLayers(input)
model = Model(input, output)
model.add_loss(std(tf.gather(input, feature_idx, axis=1) - output))
model.compile(loss='mse', optimizer=opt)
BTW, it is strange that your regularizer is a square of variance, while your loss is mse. Maybe you would prefer them to be on the same squared scale (variance and mse), as people usually do (consider any L2 shrinkage, e.g. Ridge regression).
I have trained a regression model that approximates the weights for the equation :
Y = R+B+G
For this, I provide pre-determined values of R, B and G and Y, as training data and after training the model, the model is successfully able to predict the value of Y for given values of R, B and G. I used a neural network with 3 inputs, 1 dense layer (hidden layer) with 2 neurons and the output layer (output) with a single neuron.
hidden = tf.keras.layers.Dense(units=2, input_shape=[3])
output = tf.keras.layers.Dense(units=1)
But, I need to implement the inverse of this. i.e., I need to train a model that takes in value of Y and predicts values of R, B and G that corrspond to that value of Y.
I have just learnt that regression is all about a single output. So, I am unable to think of a solution and the path to it.
Kindly Help.
(P.S Is it possible to use the model that I have already trained, to do this? I mean, once, the weights have been determined for R, B and G, is it possible to manipulate the model to use these weights to map Y to R, B and G?)
Here is an example to start solving your problem using neural network in tensorflow.
import numpy as np
from tensorflow.python.keras.layers import Input, Dense
from tensorflow.python.keras.models import Model
X=np.random.random(size=(100,1))
y=np.random.randint(0,100,size=(100,3)).astype(float) #Regression
input1 = Input(shape=(1,))
l1 = Dense(10, activation='relu')(input1)
l2 = Dense(50, activation='relu')(l1)
l3 = Dense(50, activation='relu')(l2)
out = Dense(3)(l3)
model = Model(inputs=input1, outputs=[out])
model.compile(
optimizer='adam',
loss=['mean_squared_error']
)
history = model.fit(X, [y], epochs=10, batch_size=64)
I'm currently working with a time series dataset of 46 lines about meteorological measurements on approximately each 3 hours by day during one week. My explanatory variables (X) is composed of 26 variables and some variable has different units of measurement (degree, minimeters, g/m3 etc.). My variable to explain (y) is composed of only one variable temperature.
My goal is to predict temperature (y) on a slot of 12h-24h with the ensemble of variables (X)
For that I used Keras Tensorflow and Python, with MLP regressor model :
X = df_forcast_cap.loc[:, ~df_forcast_cap.columns.str.startswith('l')]
X = X.drop(['temperature_Y'],axis=1)
y = df_forcast_cap['temperature_Y']
y = pd.DataFrame(data=y)
# normalize the dataset X
scaler = MinMaxScaler(feature_range=(0, 1))
scaler.fit_transform(X)
normalized = scaler.transform(X)
# normalize the dataset y
scaler = MinMaxScaler(feature_range=(0, 1))
scaler.fit_transform(y)
normalized = scaler.transform(y)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)
# define base model
def norm_model():
# create model
model = Sequential()
model.add(Dense(26, input_dim=26, kernel_initializer='normal', activation='relu'))# 30 is then number of neurons
#model.add(Dense(6, kernel_initializer='normal', activation='relu'))
model.add(Dense(1, kernel_initializer='normal'))
# Compile model
model.compile(loss='mean_squared_error', optimizer='adam')
return model
# fix random seed for reproducibility
seed = 7
numpy.random.seed(seed)
# evaluate model with standardized dataset
estimator = KerasRegressor(build_fn=norm_model, epochs=(100), batch_size=5, verbose=1)
kfold = KFold(n_splits=10, random_state=seed)
results = cross_val_score(estimator, X, y, cv=kfold)
print(results)
[-0.00454741 -0.00323181 -0.00345096 -0.00847261 -0.00390925 -0.00334816
-0.00239754 -0.00681044 -0.02098541 -0.00140129]
# invert predictions
X_train = scaler.inverse_transform(X_train)
y_train = scaler.inverse_transform(y_train)
X_test = scaler.inverse_transform(X_test)
y_test = scaler.inverse_transform(y_test)
results = scaler.inverse_transform(results)
print("Results: %.2f (%.2f) MSE" % (results.mean(), results.std()))
Results: -0.01 (0.01) MSE
(1) I read that cross-validation is not adapted for time series prediction. So, I'm wondering which others techniques exist and which one is more adapted to time-series.
(2) In a second place, I decided to normalize my data because my X dataset is composed of different metrics (degree, minimeters, g/m3 etc.) and my variable to explain y is in degree. In this way, I know that have to deal with a more complicated interpretation of the MSE because its result won't be in the same unity that my y variable. But for the next step of my study I need to save the result of the y predicted (made by the MLP model) and I need that these values be in degree. So, I tried to inverse the normalization but without success, when I print my results, the predicted values are still in normalized format (see in my code above). Does anyone see my mistake.s ?
The model that you present above is looking at a single instance of 26 measurements to make a prediction. From your description it seems that you would like to make predictions from a sequence of these measurements. I'm not sure if I fully understood the description but I'll assume that you have a sequence of 46 measurements, each with 26 values that you believe should be good predictors of the temperature. If that is the case, the input shape of your model should be (46, 26,). The 46 here is called time_steps, 26 is the number of features.
For a time series you need to select a model design. There are 2 approaches: a recurrent network or a convolutional network (or a mixture of the 2nd). A convolutional network is typically used to detect patterns in the input data which may be located somewhere in the data. For instance, suppose you want to detect a given shape in an image. Convolutional Networks are a good starting point. Recurrent networks, update their internal state after each time step. They can detect patterns as well as a convolutional network, but you can think of them as being less position independent.
Simple example of a convolutional approach.
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras.layers import *
from tensorflow.keras.models import Sequential, Model
average_tmp = 0.0
model = Sequential([
InputLayer(input_shape=(46,26,)),
Conv1D(16, 4),
Conv1D(32, 4),
Conv1D(64, 2),
Conv1D(128, 4),
MaxPooling1D(),
Flatten(),
Dense(256, activation='relu'),
Dense(1, bias_initializer=keras.initializers.Constant(average_tmp)),
])
model.compile('adam', 'mse')
model.summary()
A mixed approach, would replace the ```Flatten`` layer above with an LSTM node. That would probably be a reasonable starting point to start experimenting.
(1) I read that cross-validation is not adapted for time series prediction. So, I'm wondering which others techniques exist and which one is more adapted to time-series.
cross validation is a technique that is very well suited for this problem. If you try the example model above, I can almost guarantee that it will overfit your dataset very significantly. cross-validation can help you determine the right regularisation parameters for your model in order to avoid overfitting.
Examples of regularisation techniques that you probably want to consider:
Saving the model weights at the epoch with lower validation score.
Dropout and/or BatchNormalization.
kernel regularisation.
(2) In a second place, I decided to normalize my data because my X dataset is composed of different metrics (degree, minimeters, g/m3 etc.) and my variable to explain y is in degree.
Good call. It will avoid training cycles of your model trying to discover the bias at very high values from the random initialisation.
In this way, I know that have to deal with a more complicated interpretation of the MSE because its result won't be in the same unity that my y variable.
This is orthogonal. The inputs are not assumed to be in the same unit as y. We assume in a DNN that we can create a combination of linear transformation of weights (plus non-linear activations). That has no implicit assumption of units.
But for the next step of my study I need to save the result of the y predicted (made by the MLP model) and I need that these values be in degree. So, I tried to inverse the normalization but without success, when I print my results, the predicted values are still in normalized format (see in my code above). Does anyone see my mistake.s ?
scaler.inverse_transform(results) should do the trick.
It doesn't make sense to inverse transform the inputs X_ and Y_. And it would probably help you keep your code straight to not use the same variable name for both the X and Y scalers.
It is also possible to refrain from scaling Y. If you choose to do so, I'd suggest that you initialise the output layer bias with the mean of the Ys.