Develop Reference

Why is the fully connected layer + category outputs missing from tensorflow's pre-trained mobilenet v2?

I'm trying to use tensorflow's MobileNet v2.
I don't understand why, but it seems that the last fully connected layers, with the output categories (dimensionality 1000) layer is missing and I'm left with what seems to be just the embeddings after some convolutional layer.
Any idea on why this is happening? How can I add, or where can I find the pre-trained fully connected layers block?
Here is the code:
image = np.array(PIL.Image.open("amsterdam.jpg"))
image = np.expand_dims(image,0)
IMG_SIZE = image.shape[1:3]
IMG_SHAPE = IMG_SIZE + (3,)
base_model = tf.keras.applications.MobileNetV2(input_shape=IMG_SHAPE,
include_top=False,
weights='imagenet')
tf.keras.utils.plot_model(base_model,to_file='model.png', show_shapes=True)
Here you can see the structure of the neural network as I plotted it with tf.keras.utils.plot_model:
Any idea on how to fix this?

include_top=False: return the model without dense layers for classification. You can add your own dense layers.
include_top=True: return the entire model.
If you want to get also dense layers for classification, use include_top=True as the default is. When you set include_top=False the model will not return the dense layers, in order to let you make your own dense layers and make your own classification to suit your needs.

How to use Deep Learning Models from Keras for a problem that does not fit imagenet dataset?

I followed a blog on how to implement a vgg16-model from scratch and want to do the same with the pretrained model from Keras. I looked up some other blogs but can't find a fitting solution I think. My task is to classify integrated circuit images into defect or non defects.
I have seen on a paper that they used pretrained imagenet model of vgg16 for fabric defect detection, where they freezed the first seven layers and fine tuned the last nine for their own problem.
(Source: https://journals.sagepub.com/doi/full/10.1177/1558925019897396)
I have already seen examples on how to freeze all layers except the fully connected layers, but how can I try the example with freezing first x layers and fine tune the others for my problem?
The VGG16 is fairly easy to implement from scratch but for models like resnet or xception it is getting a little trickier.

It is not necessary to implement a model from scratch to freeze a few layers. You can do this on pre-trained models as well. In keras, you'd use trainable = False.
For example, let's say you want to use the pre-trained Xception model from keras and want to freeze the first x layers:
#In your includes
from keras.applications import Xception
#Since you're using the model for a different task, you'd want to remove the top
base_model = Xception(weights='imagenet', include_top=False)
#Freeze layers 0 to x
for layer in base_model.layers[0:x]:
layer.trainable = False
#To see all the layers in detail and to check trainable parameters
base_model.summary()
Ideally you'd want to add another layer on top of this model with the output as your classes. For more details, you can check this keras guide: https://keras.io/guides/transfer_learning/
A lot of times the pre-trained weights can be very useful in other classification tasks but in case you want to train a model from scratch on your dataset, you can load the model without the imagenet weights. Or better, load the weights but don't freeze any layers. This will retrain every layer taking imagenet weights as an initialization.
I hope I've answered your question.

Issue with Keras using pretrained Inceptionv3

I am using InceptionV3 with imagenet weights in Keras. The version of Keras I am using is 2.2.4 and Keras-applications is 1.0.8. The tensorflow version is 1.14.0. I am following the standard way of using InceptionV3 for transfer learning, as outlined here. I am getting this error ValueError: Input 0 is incompatible with layer global_average_pooling2d_3: expected ndim=4, found ndim=2. I found a GitHub post where the user was facing the same issue. I followed the suggestion which fixed the issue on the GitHub post, but I have had no such luck. MWE is below
from keras.layers import Input, Dense, Activation, GlobalAveragePooling2D
from keras.models import Model
from keras.applications.inception_v3 import InceptionV3
base_model = InceptionV3(weights='imagenet', include_top='False')
x = base_model.output
x = GlobalAveragePooling2D()(x) # Error appears here
x = Dense(1024, activation='relu')(x)
predictions = Dense(3, activation='softmax')(x)
model = Model(inputs=base_model.input, outputs=predictions)

The reason is that you passed the string 'False' to include_top. Non-empty strings evaluate to True, so what you thought was the topless model was, in fact, fully adorned with the dimensionality-reducing average pooling and fully-connected layers.
Accordingly, one way to solve your problem would be to change 'False' to False. I would add, however, that you can just specify pooling='avg', so you only have to add the last Dense layer...

How to remove (pop) initial layers of Keras InceptionV3 pre-trained model?

I am trying to use pre-trained InceptionV3 model. However, I want to remove initial five layers and add my custom layers. How can I do that? I tried model.layers.pop(0), but that alone will not solve the problem.
Edit:
tf.keras does not help either as mentioned in the first answer:

model.layers.pop() doesn't work in the same way in tf.keras as it doesn in Keras. In tf.keras, model.layers is a view of the model. You can't remove the layers but what you can do is define the layer for which you want the output. For example,
base_model = InceptionV3(shape=shape, weights="imagenet", include_top=True)
# you don't want the last five layers:
base_model_output = base_model.layers[-6].output
# new layers
outputs = Dense(....)(base_model_output)
model = Model(base_model.input, outputs)

Since the first few layers starting from the input are changed, then the pretrained weights cannot be used. So, the architecture can be directly taken from here and modified accordingly instead of trying complex surgeries.
https://github.com/keras-team/keras-applications/blob/master/keras_applications/inception_v3.py

Keras : Why does Sequential and Model give different outputs?

I am using Keras for computing a simple sequence classification neural network. I played with the different module and I found that there are two way to create Sequential neural network.
The first way is to use Sequential API. This is the most common way which I found in a lot of tutorial/documentation.
Here is the code :
# Sequential Neural Network using Sequential()
model = Sequential()
model.add(Conv1D(filters=32, kernel_size=3, padding='same', activation='relu', input_shape=(27 , 300,)))
model.add(MaxPooling1D(pool_size=2))
model.add(LSTM(100))
model.add(Dense(len(7, activation='softmax'))
model.summary()
The second ways is to build de sequential neural network from "scratch" with the Model API. Here is the code.
# Sequential neural network using Model()
inputs = Input(shape=(27 , 300))
x = Conv1D(filters=32, kernel_size=3, padding='same', activation='relu')(inputs)
x = MaxPooling1D(pool_size=2)(x)
x = LSTM(100)(x)
predictions = Dense(7, activation='softmax')(x)
model = Model(inputs=inputs, outputs=predictions)
model.summary()
I trained it both with a fixed seed (np.random.seed(1337)), with the same training data and my output are different...
Knowing that the only difference in the summary is the first layer of inputs with the Model API.
Is there anyone that knows why this neural network are different ?
And if there are not, why did i get different results ?
Thanks

You setup the random seed only in numpy and not in tensorflow (in case it's the backend of keras in your case). Try to add this in your code:
from numpy.random import seed
seed(1337)
from tensorflow import set_random_seed
set_random_seed(1337)
the detailed article about this topic here

tf.keras.backend.clear_session()
tf.random.set_seed(seed_value)
You can use above code block and run the loaded model for some iterations and check if the error still persists .
I was facing the same issue for reproducibiity,it worked for me .
As mentioned by andrey, over and above these 2 seed setter, you need to setup the Python Hash Environment
import os
os.environ['PYTHONHASHSEED']=str(seed_value)
you can still add one more block to force TensorFlow to use single thread.
( if you are using multicore)
Multiple threads are a potential source of non-reproducible results.
session_conf = tf.ConfigProto(
intra_op_parallelism_threads=1,
inter_op_parallelism_threads=1)
sess = tf.Session(config=session_conf)