I've built a Sequential model like this:
model=Sequential()
model.add(Dense(40, activation='relu',input_dim=12))
model.add(Dense(60, activation='relu'))
model.add(Dense(units=3, activation='softmax'))
opt=tf.keras.optimizers.Adam(lr=0.001)
model.compile(loss="mse", optimizer=opt)
model.summary()
I would like to create a second model and then change its weights according to a rule made by me, so I''ve written this code
model2=model
w1=model.get_weights()
w2=model2.get_weights()
for i in range(len(w1)):
j=np.random.random(1)
w1[i]=w2[i]*j
model.set_weights(w1)
model2.set_weights(w2)
After the for cycle w1 is different from w2, but after I set the weights of both models and then recall the get_weights() functions, they are still the same. Why this happens?
create a copy of your model with tf.keras.models.clone_model
model=Sequential()
model.add(Dense(40, activation='relu',input_dim=12))
model.add(Dense(60, activation='relu'))
model.add(Dense(units=3, activation='softmax'))
opt=tf.keras.optimizers.Adam(lr=0.001)
model.compile(loss="mse", optimizer=opt)
model.summary()
model2 = tf.keras.models.clone_model(model) # make a copy
w1 = model.get_weights()
w2 = model2.get_weights()
for i in range(len(w1)):
j=np.random.random(1)
w1[i]=w2[i]*j
model.set_weights(w1)
model2.set_weights(w2)
Related
I am trying to train a sequential model using the LSTM layer.
The size of sequence data for learning is as follows:
x = np.array(sequences)
y = to_categorical(labels).astype(int)
x.shape => (1800, 34, 48)
y.shape => (1800, 20)
After that, I make a sequential model and try to stack the LSTM layer and the dense layer, but I don't know how much to do that.
First, I did something like this:
model = Sequential()
model.add(LSTM(64, return_sequences=True, activation='relu', input_shape=x_train.shape[1:3]))
model.add(LSTM(128, return_sequences=True, activation='relu'))
model.add(LSTM(64, return_sequences=False, activation='relu'))
model.add(Dense(64, activation='relu'))
model.add(Dense(32, activation='relu'))
model.add(Dense(actions.shape[0], activation='softmax'))
model.compile(optimizer='Adam', loss='categorical_crossentropy', metrics=['acc'])
model.summary()
However, this doesn't seem to fit my case as I followed someone else's code.
How many layers should I stack in a sequential model?
I am studying deep learning and as an assignment, I am doing a classification project, which has 17k records with 14 features and a target variable that have 11 classes.
I tried to train a simple neural network
# define the keras model
model1 = keras.Sequential()
model1.add(keras.layers.Dense(64, input_dim=14, activation='relu'))
model1.add(keras.layers.Dense(128, activation='relu'))
model1.add(keras.layers.Dense(64, activation='relu'))
model1.add(keras.layers.Dense(1, activation='softmax'))
# compile the keras model
model1.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
# fit the keras model on the dataset
performance1 = model1.fit(x_train, y_train, epochs=100, validation_split=0.2)
But the problem here is I am getting the same accuracy for each epoch, it doesn't seem that model is even learning.
I tried to research this problem and found some similar problems on StackOverflow like this question and tried following things
Applied StandardScaler
Increased/Decreased the hidden layer and neurons
Added dropout layer
Changed the optimizers, loss, and activation function
I also tried to batch_size
But none of them worked, of course, the accuracy was different in the different trials (but has the same problem).
Few of trials are as follows:
# define the keras model
model1 = keras.Sequential()
model1.add(keras.layers.Dense(64, input_dim=14, activation='sigmoid'))
model1.add(keras.layers.Dense(128, activation='sigmoid'))
model1.add(keras.layers.Dense(64, activation='sigmoid'))
model1.add(keras.layers.Dense(1, activation='softmax'))
sgd = keras.optimizers.SGD(lr=0.01)
# compile the keras model
model1.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])
# define the keras model
model1 = keras.Sequential()
model1.add(keras.layers.Dense(64, input_dim=14, activation='relu'))
model1.add(keras.layers.Dense(128, activation='relu'))
model1.add(keras.layers.Dropout(0.2))
model1.add(keras.layers.Dense(64, activation='relu'))
model1.add(keras.layers.Dropout(0.2))
model1.add(keras.layers.Dense(1, activation='softmax'))
# compile the keras model
model1.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
I don't know what's the problem here. Please let me know if you require more details to process this. And please don't close this question I know this question stands a chance to marked as a duplicate question but trust me I tried many things which I can understand as a beginner.
The problem is that the softmax should be applied on an output array to get probabilities and that output array from the model should represent the logits for each target class. hence you would have to change this line
model1.add(keras.layers.Dense(1, activation='softmax'))
# TO
model1.add(keras.layers.Dense(df['Class'].nunique(), activation='softmax'))
EDIT:
# Let's say you have 11 unique values in your class then you last layer will become
model1.add(keras.layers.Dense(11, activation='softmax'))
# Now your loss will be
model1.compile(loss=tf.keras.loss.SparseCategoricalCrossentropy(), optimizer='adam', metrics=[tf.keras.metrics.SparseCategoricalAccuracy()])
I am building an auto-encoder and training the model so the targeted output is the same as the input.
I am using a sequential Keras model. When I use model.predict I would like it to export the array from a specific layer (Dense256) not the output.
This is my current model:
model = Sequential()
model.add(Dense(4096, input_dim = x.shape[1], activation = 'relu'))
model.add(Dense(2048, activation='relu'))
model.add(Dense(1024, activation='relu'))
model.add(Dense(512, activation='relu'))
model.add(Dense(256, activation='relu'))
model.add(Dense(512, activation='relu'))
model.add(Dense(1024, activation='relu'))
model.add(Dense(2048, activation='relu'))
model.add(Dense(4096, activation='relu'))
model.add(Dense(x.shape[1], activation ='sigmoid'))
model.compile(loss = 'mean_squared_error', optimizer = 'adam')
history = model.fit(data_train,data_train,
verbose=1,
epochs=10,
batch_size=256,
shuffle=True,
validation_data=(data_test, data_test))
After training, create a new model (model2) from your trained model (model) ending in your desired layer.
You can do so either with layer name:
(In model.summary(), your dense's layer 'name' with 256 neurons is dense_5)
from keras.models import Model
model2= Model(model.input,model.get_layer('dense_5').output)
Or with layer order:
(your dense layer with 256 neurons is fifth in model.summary())
from keras.models import Model
model2= Model(model.input,model.layers[4].output)
Then you can use predict
preds=model2.predict(x)
layer.get_weights() returns the weights of a layer as a numpy array which can then be saved, for example with np.save.
To set the weights from a numpy array, layer.set_weights(weights) can be used.
You can access your layer either by name (model.get_layer(LAYER_NAME) or by its number (model.layers[LAYER_INDEX]).
I have created with keras a neural network for predicting addition.
I have 2 inputs and 1 output (result of adding the 2 inputs).
I trained my neural network with tensorflow and then I tried to predict addition but the program returns 0 or 1 value not 3,4,5,etc.
This is my code :
from keras.models import Sequential
from keras.layers import Dense
import numpy
# fix random seed for reproducibility
seed = 7
numpy.random.seed(seed)
# load dataset
dataset = numpy.loadtxt("data.csv", delimiter=",")
# split into input (X) and output (Y) variables
X = dataset[:,0:2]
Y = dataset[:,2]
# create model
model = Sequential()
model.add(Dense(12, input_dim=2, init='uniform', activation='relu'))
model.add(Dense(2, init='uniform', activation='relu'))
model.add(Dense(1, init='uniform', activation='sigmoid'))
# Compile model
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
# Fit the model
model.fit(X, Y, epochs=150, batch_size=10, verbose=2)
# calculate predictions
predictions = model.predict(X)
# round predictions
rounded = [round(x[0]) for x in predictions]
print(rounded)
And my file data.csv:
1,2,3
3,3,6
4,5,9
10,8,18
1,3,4
5,3,8
For example:
1+2=3
3+3=6
4+5=9
...etc.
But I get this as output : 0,1,0,0,1,0,1...
Why didn't I get the output as 3,6,9...?
i updated code for use other loss function but i have same error :
from keras.models import Sequential
from keras.layers import Dense
import numpy
# fix random seed for reproducibility
seed = 7
numpy.random.seed(seed)
# load pima indians dataset
dataset = numpy.loadtxt("data.csv", delimiter=",")
# split into input (X) and output (Y) variables
X = dataset[:,0:2]
Y = dataset[:,2]
# create model
model = Sequential()
model.add(Dense(12, input_dim=2, init='uniform', activation='relu'))
model.add(Dense(2, init='uniform', activation='relu'))
#model.add(Dense(1, init='uniform', activation='sigmoid'))
model.add(Dense(1, input_dim=2, init='uniform', activation='linear'))
# Compile model
#model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
model.compile(loss='mean_squared_error', optimizer='adam', metrics=['accuracy'])
# Fit the model
model.fit(X, Y, epochs=150, batch_size=10, verbose=2)
# calculate predictions
predictions = model.predict(X)
# round predictions
rounded = [round(x[0]) for x in predictions]
print(rounded)
outout=1,1,1,3,1,1,...etc
As #ebeneditos mentioned, you need to change your activation function in the last layer to something other than sigmoid. You can try changing it to linear.
model.add(Dense(1, init='uniform', activation='linear'))
You should also change your loss function to something like mean squared error, as your problem is more of a regression problem than a classification problem (binary_crossentropy is used as a loss function for binary classification problems)
model.compile(loss='mean_squared_error', optimizer='adam', metrics=['accuracy'])
This is due to the Sigmoid function you have in the last layer. As it is defined:
It can only take values from 0 to 1. You should change last layer's activation function.
You can try this instead (with Dense(8) instead of Dense(2)):
# Create model
model = Sequential()
model.add(Dense(12, input_dim=2, init='uniform', activation='relu'))
model.add(Dense(8, init='uniform', activation='relu'))
model.add(Dense(1, init='uniform', activation='linear'))
# Compile model
model.compile(loss='mean_squared_error', optimizer='adam', metrics=['accuracy'])
# Fit the model
model.fit(X, Y, epochs=150, batch_size=10, verbose=2)
I have a trained neural network that works really well, but I scaled the data beforehand [sklearn preprocessing.scale(X)]. This works great, but what do I do when I save the model and want to input new data into it? How do I make sure the scaling is the same as when it was trained?
df = pd.read_csv("Trimmed Training Data.csv", delimiter=",")
X = np.array(df.drop(['PredictThis'],1))
y = np.array(df['PredictThis'])
X = preprocessing.scale(X)
# create NN model
model = Sequential()
# 2 inputs, 10 neurons in 1 hidden layer, with tanh activation and dropout
model.add(Dropout(0.2, input_shape=(15,)))
model.add(Dense(100, init='uniform', input_shape=(15,), activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(100, init='uniform', input_shape=(100,), activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(5, init='uniform', input_shape=(100,), activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(1, init='uniform', activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='rmsprop')
# Fit the model
model.fit(X, y, nb_epoch=25, batch_size=15)
What is usually done is saving a scale object as you may read here.
Here is how you could save scale object:
from sklearn.externals import joblib
joblib.dump(scale, 'filename.pkl')
and here is how you could reload it:
scale = joblib.load('filename.pkl')
When predicting new data - it's the best to reload scaler - scale data accordingly and then to use your model. You could even create a new class with scaler kept and saved like keras.model.