I have the below code.
I would like to see how the weights and bias changes during training.
Ideally I would like to see it in tensorboard.
Would someone be able to show me how to do this.
from time import time
import numpy as np
import matplotlib.pyplot as plt
import keras
import tensorflow as tf
from keras.callbacks import TensorBoard
from sklearn.preprocessing import MinMaxScaler
scaler = MinMaxScaler()
x = scaler.fit_transform(np.array([[1965.0], [1980.0]])).reshape(-1,1)
y = scaler.fit_transform(np.array([[320.0], [345.0]])).reshape(-1,1)
tensorboard = TensorBoard(log_dir='logs/{}'.format(time()), write_grads=True)
model = keras.Sequential([keras.layers.Dense(1, activation='linear')])
model.compile(optimizer='sgd',
loss="mean_squared_error")
model.fit(x=x, y=y, epochs=1000, callbacks=[tensorboard])
yHat = model.predict(x)
Based on the Keras documentation, all you need to do maybe is just run the command line:
tensorboard --logdir=logs
Notice that the logdir setting is pointing to the root of your log directory.
Related
I am trying to create an MPLClassifier with predefined weights and biases so that I can save them to a file and then
If I train the network like this:
import numpy as np
from sklearn.neural_network import MLPClassifier
data = np.load("data.npy")
labels = np.load("labels.npy")
clf = MLPClassifier()
clf.fit(data, labels)
np.save("weights.npy", clf.coefs_)
np.save("biases.npy", clf.intercepts_)
and then access the weights an biases like this:
import numpy as np
from sklearn.neural_network import MLPClassifier
weights = np.load("weights.npy")
biases = np.load("biases.npy")
I want to be able to create a new network like:
clf = MLPClassifier(weights=weights, biases=biases)
As #Plagon commented, you can't create an MLPClassifier from weights and biases.
Instead, you should import pickle and use it like:
with open("network.pkl", "wb") as network:
pickle.dump(clf, network)
and access it like:
with open("network.pkl", "wb") as network:
clf = pickle.load(network)
For more information on pickle you can go its documentation at https://docs.python.org/3/library/pickle.html.
My sample data are as follows.
I have a bigger dataset.
In this dataset I want to predict the next date text.
The codes I want to run are like this.
import pandas as pd
import numpy as np
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from sklearn.model_selection import train_test_split,GridSearchCV,cross_val_score
from sklearn.metrics import mean_squared_error,r2_score
from sklearn import model_selection
from sklearn.neighbors import KNeighborsRegressor
from warnings import filterwarnings
from sklearn.metrics import mean_squared_error, mean_absolute_error
from tensorflow.keras.callbacks import EarlyStopping
filterwarnings('ignore')
data = pd.read_excel("datasets.xlsx")
data.head()
x_train, x_test, y_train, y_test = train_test_split(x,y,test_size=0.2,random_state=10)
model = Sequential()
model.add(Dense(30,activation="relu"))
model.add(Dense(15,activation="relu"))
model.add(Dense(15,activation="relu"))
model.add(Dense(15,activation="relu"))
model.add(Dense(1))
model.compile(optimizer="adam",loss="mse")
model.fit(x=x_train, y = y_train,validation_data=(x_test,y_test),batch_size=50,epochs=300)
I don't know exactly how to separate x and y here.I tried separating x and y as columns before, but
I was not successful in this dataset.
In this dataset, I need to separate x and y as rows.
As you can see I could only create the model
Also, do I need to use a tokenizer here? Anyone who can help with a sample code?
I try to run SHAP with my basic model LSTM with 3D input.
import shap
import tensorflow.keras.backend
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
from tensorflow.keras.models import load_model
import shap
tf.compat.v1.disable_eager_execution()
regressor = load_model(Model_name)
pred_x = regressor.predict_classes(X_Training)
#explainer = shap.DeepExplainer((regressor.layers[0].input, regressor.layers[-1].output),data)
background = X_Training[np.random.choice(X_Training.shape[0], 200, replace=False)]
explainer = shap.DeepExplainer(regressor,background)
test = X_Training[np.random.choice(X_Training.shape[0], 20, replace=False)]
shap_val = explainer.shap_values(test,check_additivity=False)
shap_val = np.array(shap_val)
shap_val = np.reshape(shap_val,(int(shap_val.shape[1]),int(shap_val.shape[2]),int(shap_val.shape[3])))
shap_abs = np.absolute(shap_val)
sum_0 = np.sum(shap_abs,axis=0)
f_names = ['A','B','C','D','E','F','G','H','K','L','M','Results']
shap.initjs()
shap.summary_plot(
shap_val[0],
#X_Testing,
feature_names=f_names[:-1],
max_display=50,
plot_type='bar')
The output is as picture below.
The model summary()
I checked the shap_values and all values are zeros.
The versions of shap and tensorflow are 0.37.0 and 2.3.0 respectively.
I try to build an android apps to predict text classification using AverageWordVecModelSpec that have been provided by Tensorflow Lite Model Maker.
I'm using books content to test if my apps works. There are 3 books I've provided for this experiment. Here's the code:
!pip install git+https://github.com/tensorflow/examples.git#egg=tensorflow-examples[model_maker]
import numpy as np
import os
import tensorflow as tf
assert tf.__version__.startswith('2')
from tensorflow_examples.lite.model_maker.core.data_util.text_dataloader import TextClassifierDataLoader
from tensorflow_examples.lite.model_maker.core.task.model_spec import AverageWordVecModelSpec
from tensorflow_examples.lite.model_maker.core.task import text_classifier
data_path = '/content/drive/My Drive/datasetps'
model_spec = AverageWordVecModelSpec()
train_data = TextClassifierDataLoader.from_folder(os.path.join(data_path, 'train'), model_spec=model_spec, class_labels=['categorya', 'categoryb'])
test_data = TextClassifierDataLoader.from_folder(os.path.join(data_path, 'test'), model_spec=model_spec, is_training=False, shuffle=False)
model = text_classifier.create(train_data, model_spec=model_spec)
loss, acc = model.evaluate(test_data)
model.export(export_dir='.')
It works when i only use 2 classes/books (same as examples provided by tensorflow team):
it works normal even though it has small acurracy-- because i only takes 20 sample page per book as dataset actually
You can see that i have rational loss value here,
But i have a problem when i've try to add the 3rd class:
train_data = TextClassifierDataLoader.from_folder(os.path.join(data_path, 'train'), model_spec=model_spec, class_labels=['categorya', 'categoryb', 'categoryc'])
test_data = TextClassifierDataLoader.from_folder(os.path.join(data_path, 'test'), model_spec=model_spec, is_training=False, shuffle=False)
Here's the training result involving 3rd class:
enter image description here
You can see that it's not rational for having loss value more than 1.
I've tried to find which line of code (from Tensorflow Model Maker) that i should change to solve it and ended up to this question in this forum.
So is it possible to have multiclass model for textclassifier using
AverageWordVecModelSpec TFlite model maker?
It is possible. I would suggest to encode your labels first and then follow the workflow:
from tflite_model_maker import model_spec
from tflite_model_maker import text_classifier
from tflite_model_maker import TextClassifierDataLoader
from tflite_model_maker import ExportFormat
from sklearn.model_selection import train_test_split
import pandas as pd
df = pd.read_excel('data_set.xls')
col = ['sentence', 'your_label']
df = df[col]
# Encoding happens here
df.your_label = pd.Categorical(df.your_label)
df['label'] = df.book_label.cat.codes
train, test = train_test_split(df, test_size=0.2)
train.to_csv('train.csv', index=False)
test.to_csv('test.csv', index=False)
spec = model_spec.get('average_word_vec')
train_data = TextClassifierDataLoader.from_csv(
filename='train.csv',
text_column='sentence',
label_column='label',
model_spec=spec,
delimiter=',',
is_training=True)
test_data = TextClassifierDataLoader.from_csv(
filename='test.csv',
text_column='sentence',
label_column='label',
model_spec=spec,
delimiter=',',
is_training=False)
model = text_classifier.create(train_data, model_spec=spec, batch_size=5, epochs=4)
config = configs.QuantizationConfig.create_dynamic_range_quantization(optimizations=[tf.lite.Optimize.OPTIMIZE_FOR_LATENCY])
model.export(export_dir='average_word_vec/', export_format=[ExportFormat.LABEL, ExportFormat.VOCAB])
So I build a GRU model and I'm comparing 3 different datasets on the same model. I was just running the first dataset and set the number of epochs to 25, but I have noticed that my validation loss is increasing just after the 6th epoch, doesn't that indicate overfitting, am I doing something wrong?
import pandas as pd
import tensorflow as tf
from keras.layers.core import Dense
from keras.layers.recurrent import GRU
from keras.models import Sequential
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import MinMaxScaler
from google.colab import files
from tensorboardcolab import TensorBoardColab, TensorBoardColabCallback
tbc=TensorBoardColab() # Tensorboard
df10=pd.read_csv('/content/drive/My Drive/Isolation Forest/IF 10 PERCENT.csv',index_col=None)
df2_10= pd.read_csv('/content/drive/My Drive/2019 Dataframe/2019 10minutes IF 10 PERCENT.csv',index_col=None)
X10_train= df10[['WindSpeed_mps','AmbTemp_DegC','RotorSpeed_rpm','RotorSpeedAve','NacelleOrientation_Deg','MeasuredYawError','Pitch_Deg','WindSpeed1','WindSpeed2','WindSpeed3','GeneratorTemperature_DegC','GearBoxTemperature_DegC']]
X10_train=X10_train.values
y10_train= df10['Power_kW']
y10_train=y10_train.values
X10_test= df2_10[['WindSpeed_mps','AmbTemp_DegC','RotorSpeed_rpm','RotorSpeedAve','NacelleOrientation_Deg','MeasuredYawError','Pitch_Deg','WindSpeed1','WindSpeed2','WindSpeed3','GeneratorTemperature_DegC','GearBoxTemperature_DegC']]
X10_test=X10_test.values
y10_test= df2_10['Power_kW']
y10_test=y10_test.values
# scaling values for model
x_scale = MinMaxScaler()
y_scale = MinMaxScaler()
X10_train= x_scale.fit_transform(X10_train)
y10_train= y_scale.fit_transform(y10_train.reshape(-1,1))
X10_test= x_scale.fit_transform(X10_test)
y10_test= y_scale.fit_transform(y10_test.reshape(-1,1))
X10_train = X10_train.reshape((-1,1,12))
X10_test = X10_test.reshape((-1,1,12))
# creating model using Keras
model10 = Sequential()
model10.add(GRU(units=512, return_sequences=True, input_shape=(1,12)))
model10.add(GRU(units=256, return_sequences=True))
model10.add(GRU(units=256))
model10.add(Dense(units=1, activation='sigmoid'))
model10.compile(loss=['mse'], optimizer='adam',metrics=['mse'])
model10.summary()
history10=model10.fit(X10_train, y10_train, batch_size=256, epochs=25,validation_split=0.20, verbose=1, callbacks=[TensorBoardColabCallback(tbc)])
score = model10.evaluate(X10_test, y10_test)
print('Score: {}'.format(score))
y10_predicted = model10.predict(X10_test)
y10_predicted = y_scale.inverse_transform(y10_predicted)
y10_test = y_scale.inverse_transform(y10_test)
plt.plot( y10_predicted, label='Predicted')
plt.plot( y10_test, label='Measurements')
plt.legend()
plt.savefig('/content/drive/My Drive/Figures/Power Prediction 10 Percent.png')
plt.show()
LSTMs(and also GRUs in spite of their lighter construction) are notorious for easily overfitting.
Reduce the number of units(the output size) in each of the layers(32(layer1)-64(layer2); you could also eliminate the last layer altogether.
The second of all, you are using the activation 'sigmoid', but your loss function + metric is mse.
Ensure that your problem is either a regression or a classification one. If it is indeed a regression, then the activation function should be 'linear' at the last step. If it is a classification one, you should change your loss_function to binary_crossentropy and your metric to 'accuracy'.
Therefore, the plot displayed is just misleading for the moment. If you modify like I suggested and you still get such a train-val loss plot, then we can state for sure that you have an overfitting case.