Let's begin by creating a very basic deep neural network in MXNet Gluon (inspired by this tutorial):
import mxnet as mx
from mxnet import gluon
ctx = mx.cpu()
net = gluon.nn.Sequential()
with net.name_scope():
net.add(gluon.nn.Conv2D(channels=20, kernel_size=5, activation='relu'))
net.add(gluon.nn.MaxPool2D(pool_size=2, strides=2))
Now, if we want to print out the dimensions of a layer, all we have to do is...
print(net[0])
# prints: Conv2D(None -> 20, kernel_size=(5, 5), stride=(1, 1), Activation(relu))
print(net[1])
# prints: MaxPool2D(size=(2, 2), stride=(2, 2), padding=(0, 0), ceil_mode=False)
However, instead of printing it out, what if we want to programmatically inspect the padding of net[1]?
When I try net[1].padding, I get the error AttributeError: 'MaxPool2D' object has no attribute 'padding'.
When I try net[1]['padding'], I get the error TypeError: 'MaxPool2D' object is not subscriptable.
So, what's the right way to programmatically access the dimensions of a neural network layer in MXNet Gluon?
print(net[1]._kwargs["pad"])
Try getting them from kwargs dictionary. Look for other keys at this source.
This is the Colab link for the code.
Other keys are kernel for kernel size, stride for stride, .
For getting all the keys and values:
for k, v in net[1]._kwargs.items():
print(k, v)
Related
I am using Keras tuner. For the simple following code:
import keras_tuner as kt
from tensorflow.keras.regularizers import l1, l2
from tensorflow.keras.models import Sequential
# x: 100 x 20
# y: 1 x 100
tuner = kt.Hyperband(
self.build_auto_encoder_model,
objective='val_loss',
max_epochs=30,
hyperband_iterations=20)
tuner.search(x[0:80], y[0:80], epochs=30, validation_data=(x[80:], y[80:]))
best_model = tuner.get_best_models(1)[0]
def build_auto_encoder_model(hp):
model = Sequential()
regulizer_rate = hp.Choice("regulizer_rate", [1e-1, 1e-2, 1e-3, 1e-4, 1e-5])
model.add(Dense(18, input_dim=20, activation='relu', kernel_regularizer=l1(regulizer_rate)))
model.add(Dense(12, activation='relu', kernel_regularizer=l1(regulizer_rate)))
model.add(Dense(10, activation='relu', kernel_regularizer=l1(regulizer_rate)))
model.compile(optimizer=Adam(1e-2), loss='mse')
I have already tried for the different number of Dense layers, I am getting the following error:
Tensor's shape (20, 18) is not compatible with supplied shape (20, 15)
However, when totally, created a new project, it works. What was the reason?
The reason is because of some previous errors in the function code, an object has been created and loaded for any future trials, due to overwrite variable of the turner is False by default. Also, in the last version of the created object, the first layer was 15 that has been changed to 18 in your example.
A simple solution to resolve the problem (instead of creating a new project) is to make the overwrite variable to True to prevent reloading previously incompatible object with new changes, like the following:
# ...
tuner = kt.Hyperband(
self.build_auto_encoder_model,
objective='val_loss',
max_epochs=30,
hyperband_iterations=20,
overwrite = True) # here
# ...
I’m new to Python and trying to do some manipulations with filters in PyTorch.
I’m struggling re how to apply a Conv2d. I’ve got the following code which creates a 3x3 moving average filter:
resized_image4D = np.reshape(image_noisy, (1, 1, image_noisy.shape[0], image_noisy.shape[1]))
t = torch.from_numpy(resized_image4D)
conv = torch.nn.Conv2d(in_channels=1, out_channels=1, kernel_size=3, padding=1, bias=False)
conv.weight = torch.nn.Parameter(torch.ones((1,1,3, 3))/9.0)
Normally in NumPy I’d just call filtered_image = convolve2d(image, kernel), but I’ve not been able to figure out what the PyTorch equivalent is after days of searching.
I think you are looking for torch.nn.functional.conv2d.
Hence, your snippets becomes:
resized_image4D = np.reshape(image_noisy, (1, 1, image_noisy.shape[0], image_noisy.shape[1]))
t = torch.from_numpy(resized_image4D)
conv = torch.nn.functional.conv2d(in_channels=1, out_channels=1, kernel_size=3, padding=1, bias=False)
conv.weight = torch.nn.Parameter(torch.ones((1,1,3, 3))/9.0)
I want to implement this neural network using Keras:
When trying to run the following commands:
model_All_CNN = Sequential()
# input: 126*126 images with 3 channels -> (126, 126, 3) tensors.
# this applies 320 convolution filters of size 2*2 each.
act = keras.layers.advanced_activations.LeakyReLU(alpha=0.3)
# This returns a tensor
#input_img = Input(shape=(3 ,126, 126))
conv1 = model_All_CNN.add(Conv2D(320, (2, 2), strides=(1, 1), activation=act,input_shape= (128,128,3)))
model_All_CNN.add(Activation(act))
I am getting the following error:
File "/Users/M.I.T/anaconda/envs/Mulimediaassignment/lib/python3.5/site-packages/keras/backend/tensorflow_backend.py",
line 2856, in conv2d
x = tf.nn.convolution(
AttributeError: module 'tensorflow.python.ops.nn' has no attribute 'convolution'
I'm using Keras with TF backend. Recently, when using the functional API to make "hybrid" models, it seemed to me that Keras requires me to feed values that it shouldn't need.
As a background, I am trying to implement a conditional GAN in Keras. My implementation has a generator and a discriminator. As an example, the generator accepts (20, 20, 1) inputs and returns (20, 20, 1) outputs. These are stacked by channel to produce a (20, 20, 2) input to the discriminator. The discriminator is supposed to decide whether it is seeing a ground-truth translation of the original (20, 20, 1) image or a translation by the generator. This is represented by 0=fake, 1=real.
By itself, the discriminator is just a CNN for binary classification. Therefore, it can be trained by feeding data points with inputs of shape (20, 20, 2) and outputs in {0,1}. Therefore, if I write something like:
# <disc> is the discriminator
arbitrary_input = np.full(shape=(5, 20, 20, 2), fill_value=0.5)
arbitrary_labels = np.array([1, 1, 0, 0, 1])
disc.fit(arbitrary_input, arbitrary_labels, epochs=5)
training will proceed without errors (obviously this is a useless dataset, though).
However, when I insert the discriminator into the generator-discriminator stack:
# <disc> is the discriminator, <gen> is the generator
input = Input(shape=(20, 20, 1), name='stack_input')
gen_output = gen(input)
pair = Concatenate(axis=FEATURES_AXIS)([input, gen_output])
disc_output = disc(gen_output)
stack = Model(input, disc_output)
stack.compile(optimizer='adam', loss='binary_crossentropy')
arbitrary_input = np.full(shape=(5, 20, 20, 2), fill_value=0.5)
arbitrary_labels = np.array([1, 1, 0, 0, 1])
disc.fit(arbitrary_input, arbitrary_labels, epochs=5)
suddenly I need to feed an extra placeholder. I get this error message on disc.fit():
InvalidArgumentError (see above for traceback): You must feed a value for placeholder tensor 'stack_input' with dtype float
[[Node: stack_input = Placeholder[dtype=DT_FLOAT, shape=[], _device="/job:localhost/replica:0/task:0/cpu:0"]()]]
As you can see by the name, this is the input to the hybrid/stacked model. I haven't changed the discriminator at all, I have only included it in another model. Therefore disc.fit() should still work, right?
There's a workaround available by freezing the weights of the generator and using fit() on the full stack, I think, but I do not understand why the method above doesn't work.
Is it perhaps some issue with scoping?
Edit: The discriminator is really just a simple CNN. It is initialized with disc = pix2pix_discriminator(input_shape=(20, 20, 2), n_filters=(32, 64)). The function in question is:
def pix2pix_discriminator(input_shape, n_filters, kernel_size=4, strides=2, padding='same', alpha=0.2):
x = Input(shape=input_shape, name='disc_input')
# first layer
h = Conv2D(filters=n_filters[0],
kernel_size=kernel_size,
strides=strides,
padding=padding,
data_format=DATA_FORMAT)(x)
# no BatchNorm
h = LeakyReLU(alpha=alpha)(h)
for i in range(1, len(n_filters)):
h = Conv2D(filters=n_filters[i],
kernel_size=kernel_size,
strides=strides,
padding=padding,
data_format=DATA_FORMAT)(h)
h = BatchNorm(axis=FEATURES_AXIS)(h)
h = LeakyReLU(alpha=alpha)(h)
h_flatten = Flatten()(h) # required for the upcoming Dense layer
y_pred = Dense(units=1, activation='sigmoid')(h_flatten) # binary output
discriminator = Model(inputs=x, outputs=y_pred)
discriminator.compile(optimizer='adam',
loss='binary_crossentropy',
metrics=['accuracy'])
return discriminator
I want to convert a Lasagne model specification to Keras. What is the equivalent layer in Keras to this in Lasagne:
nn = Conv3DDNNLayer(nn, 8, 3) # Lasagne layers
The Convolution3D layer specification for Keras is:
keras.layers.convolutional.Convolution3D(nb_filter, kernel_dim1, kernel_dim2, kernel_dim3, init='glorot_uniform', activation=None, weights=None, border_mode='valid', subsample=(1, 1, 1), dim_ordering='default', W_regularizer=None, b_regularizer=None, activity_regularizer=None, W_constraint=None, b_constraint=None, bias=True)
...and for Lasagne:
class lasagne.layers.dnn.Conv3DDNNLayer(incoming, num_filters, filter_size, stride=(1, 1, 1), pad=0, untie_biases=False, W=lasagne.init.GlorotUniform(), b=lasagne.init.Constant(0.), nonlinearity=lasagne.nonlinearities.rectify, flip_filters=False, **kwargs)
So, in the above example, the Lasagne layer has 'nn' incoming, 8 filters, and filter size 3.
However, Keras requires that each kernel_dim be specified. Are they all just 3?
Thank you.
As you may read here:
filter_size : int or iterable of int
An integer or a 3-element tuple specifying the size of the filters.
nn = Conv3DDNNLayer(nn, 8, 3)
is equivalent to:
model.add(Convolution3D(8, 3, 3, 3, ...)
or
conv_3d_output = Convolution3D(8, 3, 3, 3, ...)(conv_3d_input)
depending on which Keras.API you are using.