I'm new to Deep Learning and I can't find anywhere how to do the bottleneck in my AE with convolutional and dense layers. The code below is the specific part where I'm struggling:
...
encoded = Conv2D(8, (3, 3), activation='relu', padding='same')(encoded)
# encoded = Dense(2)(encoded) # Linear activation function at the bottleneck
decoded = Conv2D(8, (3, 3), activation='relu', padding='same')(decoded)
...
I tried some solutions, like flatten and reshape, but nothing seems to work here. The point is that I need the latent space to be a dense layer of 2 because I need to sample points [x,y] from it. I did it with MLP following this link (https://www.kaggle.com/code/apapiu/manifold-learning-and-autoencoders/notebook) and it worked, but I can't manage to do the same with my structure.
Thanks in advice, and best regards!
Convolution2D takes the input of a 4+ Dimension tensor, hence you need to reshape the input before passing it to Convolution2D layer. You can use a model like below.
input_img = Input(shape=(784,))
input_img1 = Reshape(target_shape=(28,28,1))(input_img)
encoded = Convolution2D(8, (3, 3), activation='relu', padding='same')(input_img1)
encoded = Dense(2)(encoded)
decoded1 = Convolution2D(8, (3, 3), activation='relu', padding='same')(encoded)
decoded2 = Flatten()(decoded1)
decoded = Dense(784,)(decoded2)
Please refer to this gist for complete code with random data.
Related
I would like to train an autoencoder by using only specific PARTS of a layer (the layer named FEATURES in the autoencoder example at the bottom of this question).
In my case, NOK pictures for a new product are very rare, but needed for training. The aim is generate NOK pictures from OK pictures (all examples I found did the opposite). The idea is to force learning OK-picture structure in features[0:n-x] and learning NOK-picture structure (maybe from a similiar product) in features[n-x:n] in order to use the NOK-features as parameters to generate NOK-pictures from OK-pictures.
Two ideas came to my mind using a non-random dropout
(1) keras.layers.Dropout(rate, noise_shape=None, seed=None) has the noise_shape argument, but I am not sure if it helps me as it only describes the shape. It would be perfect to be able to provide a mask consisting of {0,1} to apply on the layer in order to switch on/off specific nodes
(2) creating a custom layer (named MaskLayer below) which performs masking specific nodes of the layer e.g. as a tuple of {0,1}.
I have read this, but I do not think it applies (generate a layer by concatenating layers which can be freezed separately).
def autoEncGenerate0( imgSizeX=28, imgSizeY=28, imgDepth=1): ####:
''' keras blog autoencoder'''
input_img = Input(shape=(imgSizeX, imgSizeY, imgDepth))
x = Conv2D(32, (3, 3), activation='relu', padding='same')(input_img)
x = MaxPooling2D((4, 4), padding='same')(x)
x = Conv2D(32, (3, 3), activation='relu', padding='same')(x)
encoded0 = MaxPooling2D((8, 8), padding='same', name="FEATURES")(x)
encoded1 = MaskLayer(mask)(encoded0) # TO BE DONE (B2) masking layer parts
x = Conv2D(32, (3, 3), activation='relu', padding='same')(encoded1)
x = UpSampling2D((8, 8))(x)
x = Conv2D(32, (3, 3), activation='relu', padding='same')(x)
x = UpSampling2D((4, 4))(x)
decoded = Conv2D( imgDepth, (3, 3), activation='sigmoid', padding='same')(x)
autoencoder = Model(input_img, decoded)
autoencoder.compile(optimizer='adadelta', loss='binary_crossentropy')
return( autoencoder)
Thanks for hints.
There is trainable attribute that each instance of tf.keras.layer.Layer has which disables training of the variables of that layer. UpSampling2D doesn't have any variables so you CAN'T train it. What you want is to train the variables of the convolutional layer that comes before that upsampling layer.
You could do it like this:
# define architecture here
autoencoder = Model(input_img, decoded)
layers_names = [l.name for l in autoencoder.layers]
trainable_layer_index = layers_names.index('FEATURES') - 1
for i in range(len(autoencoder.layers)):
if i != trainable_layer_index:
autoencoder.layers[i].trainable = False
# compile here
NOTE that you compile the model AFTER you set layers to trainable/non-trainable.
I have the following convolutional neural network to apply to images:
classifier = Sequential()
classifier.add(Convolution2D(128, (3, 3), input_shape = (128, 128, 3), activation = 'relu'))
classifier.add(MaxPooling2D(pool_size = (2, 2)))
classifier.add(Convolution2D(64, (3, 3), activation = 'relu'))
classifier.add(MaxPooling2D(pool_size = (2, 2)))
classifier.add(Flatten())
After applying the convolutional and maxpooling layers, I flatten the results and want to store only that result (later I want to work with this result using unsupervised methods). How do I do that? The only examples I have continue the proccess to fit the model and I never store the flatten layers.
This is covered in the Keras documentation for pretrained models. See the examples about feature extraction, https://keras.io/applications/#extract-features-with-vgg16
Once you have your model, you just do:
features = model.predict(x)
I'm training an autoencoder in Keras right now, below is the network structure.
input_img = Input(shape=(target_size[0], target_size[1], 3))
x = Conv2D(8, (3, 3), activation = 'relu', padding = 'same')(input_img)
x = MaxPooling2D((2, 2), padding = 'same')(x)
x = Conv2D(16, (3, 3), activation = 'relu', padding = 'same')(x)
x = MaxPooling2D((2, 2), padding = 'same')(x)
x = Conv2D(32, (3, 3), activation = 'relu', padding = 'same')(x)
encoded = MaxPooling2D((2, 2), padding = 'same')(x)
x = Conv2D(32, (3, 3), padding = 'same')(encoded)
x = UpSampling2D((2, 2))(x)
x = Conv2D(16, (3, 3), activation = 'relu', padding = 'same')(x)
x = UpSampling2D((2, 2))(x)
x = Conv2D(8, (3, 3), activation = 'relu', padding = 'same')(x)
x = UpSampling2D((2, 2))(x)
decoded = Conv2D(3, (3, 3), activation = 'sigmoid', padding = 'same')(x)
I'm having trouble thinking of a way to:
Insert an input layer between where "encoded" is defined and the Conv2D layer after it - the intention is to get the encoding of two different images, create a bunch of iterative "steps" between their encoding, then input these encodings into the "decoder" half of this network to generate an image for every step. I want to make a gif of the output "morphing" from one image to the other.
I wanted to try inserting another Conv2D(...) and MaxPooling2D(...) pair right after the input layer, and a corresponding UpSampling2D(...) and Conv2D(...) at the end. I had this idea from NVidia's "Progressive Growing of GANs for Improved Quality, Stability, and Variation" paper, where they trained their GAN to generate really good images at low resolutions, then progressively added more layers at the beginning and end of their network and trained the whole network with the new layers.
Does this make sense? Please let me know if I can clarify anything, I feel like this is very specific problem that's hard to explain over text all at once.
Thanks!
I have an input shape of 64x60x4 for reinforcement learning an agent to play Mario.
The problem is, it seems very "if screen looks like this then do that", which isn't very good for this problem.
I want to add an LSTM layer after 3 conv2D layers in Keras (TensorFlow) but it complains that it expects 5 dimensions, but received 4. When I play with the layers, it then becomes 6 and 5.
So how do I get an LSTM layer into the following model with input_shape 64x60x4 (the 4 being the last 4 frames for helping learn acceleration and direction of objects):
image_input = Input(shape=input_shape)
out = Conv2D(filters=32, kernel_size=8, strides=(4, 4), padding=padding, activation='relu')(image_input)
out = Conv2D(filters=64, kernel_size=4, strides=(2, 2), padding=padding, activation='relu')(out)
out = Conv2D(filters=64, kernel_size=4, strides=(1, 1), padding=padding, activation='relu')(out)
out = MaxPooling2D(pool_size=(2, 2))(out)
out = Flatten()(out)
out = Dense(256, activation='relu')(out)
### LSTM should go here ###
q_value = Dense(num_actions, activation='linear')(out)
Any other suggestions/pointers for this would be welcome.
I would suggest something like this, after your MaxPooling Layer)
out = Reshape((64, -1))(out)
out = LSTM(...)(out)
out = Flatten...
Also I don't recommend starting with 32 filters then going up, I suggest starting with 64 then going down, but hey, you do you.
Also I would suggest separate CNN layers for different aspects, like score, time...etc. Other than that, all is set.
I am working on a CNN model and want to add a new categorical feature before the Dense layer. I tried to concatenate the feature to the flattened output of CNN layer but looks like the concatenate function in Keras requires input of tensors and not arrays. How should I go about it? Here is the code that I have tried so far:
model = Sequential()
model.add(Conv2D(128, (6, 6), padding='same'))
model.add(Activation('relu'))
model.add(Conv2D(128, (6, 6)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.25))
model.add(Flatten())
I am trying to use Concatenate function but it can join tensors, where as my feature is a numpy array of shape (1, 3). Any help would be appreciated.
You should create a new model on side of your actual model.
This second model will take in input your numpy array and does nothing else.
Then you concatenate them.
Like this ->
m1 = Sequential()
m1.add(Conv2D(128, (6, 6), padding='same'))
m1.add(Activation('relu'))
m1.add(Conv2D(128, (6, 6)))
m1.add(Activation('relu'))
m1.add(MaxPooling2D(pool_size=(2, 2)))
m1.add(Dropout(0.25))
m1.add(Flatten())
m2 = Sequential()
m2.add(Input()) # Put needed infos to input your numpy array
#Don't forget to flatten it if needed ?
model = Sequential()
model.add(Merge([m1,m2], mode='concat'))
#Then add your final layer.
#To train it, in place of the normal var X_train, you'll use [X_train,yournumpyarray] in model.train method