I'm trying to prune a pre-trained model: MobileNetV2 and I got this error. Tried searching online and couldn't understand. I'm running on Google Colab.
These are my imports.
import tensorflow as tf
import tensorflow_model_optimization as tfmot
import tensorflow_datasets as tfds
from tensorflow import keras
import os
import numpy as np
import matplotlib.pyplot as plt
import tempfile
import zipfile
This is my code.
model_1 = keras.Sequential([
basemodel,
keras.layers.GlobalAveragePooling2D(),
keras.layers.Dense(1)
])
model_1.compile(optimizer='adam',
loss=keras.losses.BinaryCrossentropy(from_logits=True),
metrics=['accuracy'])
model_1.fit(train_batches,
epochs=5,
validation_data=valid_batches)
prune_low_magnitude = tfmot.sparsity.keras.prune_low_magnitude
pruning_params = {
'pruning_schedule': tfmot.sparsity.keras.PolynomialDecay(initial_sparsity=0.50,
final_sparsity=0.80,
begin_step=0,
end_step=end_step)
}
model_2 = prune_low_magnitude(model_1, **pruning_params)
model_2.compile(optmizer='adam',
loss=keres.losses.BinaryCrossentropy(from_logits=True),
metrics=['accuracy'])
This is the error i get.
---> 12 model_2 = prune_low_magnitude(model, **pruning_params)
ValueError: Please initialize `Prune` with a supported layer. Layers should either be a `PrunableLayer` instance, or should be supported by the PruneRegistry. You passed: <class 'tensorflow.python.keras.engine.training.Model'>
I believe you are following Pruning in Keras Example and jumped into Fine-tune pre-trained model with pruning section without setting your prunable layers. You have to reinstantiate model and set layers you wish to set as prunable. Follow this guide for further information on how to set prunable layers.
https://www.tensorflow.org/model_optimization/guide/pruning/comprehensive_guide.md
I faced the same issue with:
tensorflow version: 2.2.0
Just updating the version of tensorflow to 2.3.0 solved the issue, I think Tensorflow added support to this feature in 2.3.0.
One thing I found is that the experimental preprocessing I added to my model was throwing this error. I had this at the beginning of my model to help add some more training samples but the keras pruning code doesn't like subclassed models like this. Similarly, the code doesn't like the experimental preprocessing like I have with centering of the image. Removing the preprocessing from the model solved the issue for me.
def classificationModel(trainImgs, testImgs):
L2_lambda = 0.01
data_augmentation = tf.keras.Sequential(
[ layers.experimental.preprocessing.RandomFlip("horizontal", input_shape=IM_DIMS),
layers.experimental.preprocessing.RandomRotation(0.1),
layers.experimental.preprocessing.RandomZoom(0.1),])
model = tf.keras.Sequential()
model.add(data_augmentation)
model.add(layers.experimental.preprocessing.Rescaling(1./255, input_shape=IM_DIMS))
...
Saving the model as below and reloading worked for me.
_, keras_file = tempfile.mkstemp('.h5')
tf.keras.models.save_model(model, keras_file, include_optimizer=False)
print('Saved baseline model to:', keras_file)
Had the same problem today, its the following error.
If you don't want the layer to be pruned or don't care for it, you can use this code to only prune the prunable layers in a model:
from tensorflow_model_optimization.python.core.sparsity.keras import prunable_layer
from tensorflow_model_optimization.python.core.sparsity.keras import prune_registry
def apply_pruning_to_prunable_layers(layer):
if isinstance(layer, prunable_layer.PrunableLayer) or hasattr(layer, 'get_prunable_weights') or prune_registry.PruneRegistry.supports(layer):
return tfmot.sparsity.keras.prune_low_magnitude(layer)
print("Not Prunable: ", layer)
return layer
model_for_pruning = tf.keras.models.clone_model(
base_model,
clone_function=apply_pruning_to_pruneable_layers
)
Related
I have a model trained in Keras with 1.10 Tensorflow backend and I want to make inferences using Tensorflow 2.4.
I converted the .h5 model to SavedModel format:
import tensorflow as tf
from tensorflow.keras.models import load_model
from tensorflow.python.saved_model import builder
from tensorflow.python.saved_model.signature_def_utils import predict_signature_def
from tensorflow.python.saved_model import tag_constants
def export_h5_to_pb(path_to_h5, export_path):
if tf.executing_eagerly():
tf.compat.v1.disable_eager_execution()
loaded_model = load_model(path_to_h5)
b = builder.SavedModelBuilder(export_path)
signature = predict_signature_def(inputs={"inputs": loaded_model.input},
outputs={"score": loaded_model.output})
session = tf.compat.v1.Session()
init_op = tf.compat.v1.global_variables_initializer()
session.run(init_op)
b.add_meta_graph_and_variables(
sess=session, tags=[tag_constants.SERVING], signature_def_map={"serving_default": signature})
b.save()
export_h5_to_pb('./trained_nework_VGG3_5comp.h5', './export/Servo/1')
Tensorflow prediction gives me:
import tensorflow as tf
imported = tf.saved_model.load('./export/Servo/1', tags='serve')
f = imported.signatures["serving_default"]
f(inputs=tf.constant(test_payload))
> {'score': <tf.Tensor: shape=(1, 6), dtype=float32, numpy=array([[0.16693498, 0.16678475, 0.16666655, 0.16653116, 0.16678214,
0.16630043]], dtype=float32)>}
While the original (correct) Keras prediction gives:
from keras.models import load_model
model = load_model('./trained_nework_VGG3_5comp.h5')
model.predict(test_payload)
> array([[1.0000000e+00, 3.0078113e-09, 2.0143587e-10, 5.7580127e-09, 1.9100479e-09, 4.1776910e-10]], dtype=float32)
What am I doing wrong?
I had a very similar problem, which you replied to here, and I'll share what worked for me. If you are doing this for Sagemaker/AWS (which by the file directory path and the use of the word "payload" I assume you are?), then the problem is caused by a discrepancy in TensorFlow versions.
In all of the blogs I found (e.g. this one), they used framework_version 1.12 when loading their model using TensorflowModel. Because of this, I reinstalled TensorFlow in the Sagemaker Jupyter instance to version 1.12, retrained my model using 1.12, and changed the framework_version to 1.12, and this worked for me. If you aren't using the model for AWS this very well may not apply, but if you are then this is a potential solution. Good luck!
I have completed a udacity nanodegree for NLP. I used the udacity platform for the project, but I am now trying to use my own local machine to train models etc.
I've finally gotten my GPU/tensorflow issues worked out(I think), but I'm running into some problems that I believe are related to the versions of tensorflow that udacity was using.
I am currently using TensorFlow 2.2
Specifically, I am getting an error from a validation step the project uses to list the loss function.
def _test_model(model, input_shape, output_sequence_length, french_vocab_size):
if isinstance(model, Sequential):
model = model.model
print(model.loss_functions)
When this is called I get the "'Model' object has no attribute 'loss_functions'" error.
The model is built with the below code.
def simple_model(input_shape, output_sequence_length, english_vocab_size, french_vocab_size):
"""
Build and train a basic RNN on x and y
:param input_shape: Tuple of input shape
:param output_sequence_length: Length of output sequence
:param english_vocab_size: Number of unique English words in the dataset
:param french_vocab_size: Number of unique French words in the dataset
:return: Keras model built, but not trained
"""
# TODO: Build the layers
learning_rate = 0.01
#Config Model
inputs = Input(shape=input_shape[1:])
hidden_layer = GRU(output_sequence_length, return_sequences=True)(inputs)
outputs = TimeDistributed(Dense(french_vocab_size, activation='softmax'))(hidden_layer)
#Create Model from parameters defined above
model = keras.Model(inputs=inputs, outputs=outputs)
#loss_function = 'sparse_categorical_crossentropy'
loss_fn = keras.losses.SparseCategoricalCrossentropy()
model.compile(loss=loss_fn,optimizer=Adam(learning_rate),metrics=['accuracy'])
I am using the below libraries along the way
import tensorflow as tf
from tensorflow.keras.losses import sparse_categorical_crossentropy
from tensorflow.keras.optimizers import Adam
from tensorflow import keras
import collections
import helper
import numpy as np
import project_tests as tests
from tensorflow.keras.preprocessing.text import Tokenizer
from tensorflow.keras.preprocessing.sequence import pad_sequences
from tensorflow.keras.models import Model, Sequential
from tensorflow.keras.layers import GRU, Input, Dense, TimeDistributed, Activation, RepeatVector, Bidirectional, Dropout
from tensorflow.keras.layers.embeddings import Embedding
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.losses import sparse_categorical_crossentropy
I can just comment out this check for the loss functions, but I would really like to understand what happened.
Thanks
I think the API changed in Tensorflow 2, does the following work:
model.compiled_loss._get_loss_object(model.compiled_loss._losses).fn
I am trying to save my ANN model using SavedModel format. The command that I used was:
model.save("my_model")
It supposed to give me a folder namely "my_model" that contains all saved_model.pb, variables and asset, instead it gives me an HDF file namely my_model. I am using keras v.2.3.1 and tensorflow v.2.2.0
Here is a bit of my code:
from keras import optimizers
from keras import backend
from keras.models import Sequential
from keras.layers import Dense
from keras.activations import relu,tanh,sigmoid
network_layout = []
for i in range(3):
network_layout.append(8)
model = Sequential()
#Adding input layer and first hidden layer
model.add(Dense(network_layout[0],
name = "Input",
input_dim=inputdim,
kernel_initializer='he_normal',
activation=activation))
#Adding the rest of hidden layer
for numneurons in network_layout[1:]:
model.add(Dense(numneurons,
kernel_initializer = 'he_normal',
activation=activation))
#Adding the output layer
model.add(Dense(outputdim,
name="Output",
kernel_initializer="he_normal",
activation="relu"))
#Compiling the model
model.compile(optimizer=opt,loss='mse',metrics=['mse','mae','mape'])
model.summary()
#Training the model
history = model.fit(x=Xtrain,y=ytrain,validation_data=(Xtest,ytest),batch_size=32,epochs=epochs)
model.save('my_model')
I have read the API documentation in the tensorflow website and I did what it said to use model.save("my_model") without any file extension, but I can't get it right.
Your help will be very appreciated. Thanks a bunch!
If you would like to use tensorflow saved model format, then use:
tms_model = tf.saved_model.save(model,"export/1")
This will create a folder export and a subfolder 1 inside that. Inside the 1 folder you can see the assets, variables and .pb file.
Hope this will help you out.
Make sure to change your imports like this
from tensorflow.keras import optimizers
So I think tensorflow.keras and the independant keras packages are in conflict and I can't load my model, which I have made with transfer learning.
Import in the CNN ipynb:
!pip install tensorflow-gpu==2.0.0b1
import tensorflow as tf
from tensorflow import keras
print(tf.__version__)
Loading this pretrained model
base_model = keras.applications.xception.Xception(weights="imagenet",
include_top=False)
avg = keras.layers.GlobalAveragePooling2D()(base_model.output)
output = keras.layers.Dense(n_classes, activation="softmax")(avg)
model = keras.models.Model(inputs=base_model.input, outputs=output)
Saving with:
model.save('Leavesnet Model 2.h5')
Then in the new ipynb for the already trained model (the imports are the same as in the CNN ipynb:
from keras.models import load_model
model =load_model('Leavesnet Model.h5')
I get the error:
AttributeError Traceback (most recent call last)
<ipython-input-4-77ca5a1f5f24> in <module>()
2 from keras.models import load_model
3
----> 4 model =load_model('Leavesnet Model.h5')
13 frames
/usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py in placeholder(shape, ndim, dtype, sparse, name)
539 x = tf.sparse_placeholder(dtype, shape=shape, name=name)
540 else:
--> 541 x = tf.placeholder(dtype, shape=shape, name=name)
542 x._keras_shape = shape
543 x._uses_learning_phase = False
AttributeError: module 'tensorflow' has no attribute 'placeholder'
I think there might be a conflict between tf.keras and the independant keras, can someone help me out?
Yes, there is a conflict between tf.keras and keras packages, you trained the model using tf.keras but then you are loading it with the keras package. That is not supported, you should use only one version of this package.
The specific problem is that you are using TensorFlow 2.0, but the standalone keras package does not support TensorFlow 2.0 yet.
Try to replace
from keras.models import load_model
model =load_model('Leavesnet Model.h5')
with
model = tf.keras.models.load_model(model_path)
It works for me, and I am using:
tensorflow version: 2.0.0
keras version: 2.3.1
You can check the following:
https://www.tensorflow.org/api_docs/python/tf/keras/models/load_model?version=stable
I think downgrading your keras or tensorflow will not be that much suitable because you need to retrain your model for various dependencies. Why not try to load the weights instead of loading the model?
here is a piece of code
import tensorflow as tf
{your code here}
#save the weights
model.save_weights('model_checkpoint')
#initialise the model again (example - MobileNetv2)
encoder = MobileNetV2(input_tensor=inputs, weights="imagenet", include_top=False, alpha=0.35)
#load the weights
encoder.load_weights('model_checkpoint')
and you are good to go
I don't understand how to use tensor board to visualize the training step of my keras network.
I already launch tensor board with the command line : tensorboard --logdir=/run1
But he raise this error :
No dashboards are active for the current data set. Probable causes:You
haven’t written any data to your event files. TensorBoard can’t find
your event files.
import tensorflow as tf
from tensorflow import keras
import numpy as np
# Create the array of data
train_data = [[1.0,2.0,3.0],[4.0,5.0,6.0]]
train_data_np = np.asarray(train_data)
train_label = [[1,2,3],[4,5,6]]
train_label_np = np.asarray(train_data)
### Build the model
model = keras.Sequential([
keras.layers.Dense(3,input_shape =(3,2)),
keras.layers.Dense(3,activation=tf.nn.sigmoid)
])
model.compile(optimizer='sgd',loss='sparse_categorical_crossentropy',metrics=['accuracy'])
#Train the model
tensorboard = TensorBoard(log_dir="run1")
model.fit(train_data_np,train_label_np,epochs=10,callbacks=tensorboard)
#test the model
restest = model.evaluate(test_data_np,test_label_np)
Adding formal answer here; looks like there is a typo in the tensorboard logdir parameter. You need to remove the slash at the beginning of the directory
tensorboard --logdir=run1