When I run my code, I get a value error with the following message:
ValueError: Input dimension mis-match. (input[0].shape[1] = 1, input[2].shape[1] = 20)
Apply node that caused the error: Elemwise{Composite{((i0 + i1) - i2)}}[(0, 0)](Dot22.0, InplaceDimShuffle{x,0}.0, InplaceDimShuffle{x,0}.0)
Toposort index: 18
Inputs types: [TensorType(float64, matrix), TensorType(float64, row), TensorType(float64, row)]
Inputs shapes: [(20, 1), (1, 1), (1, 20)]
Inputs strides: [(8, 8), (8, 8), (160, 8)]
Inputs values: ['not shown', array([[ 0.]]), 'not shown']
Outputs clients: [[Elemwise{Composite{((i0 * i1) / i2)}}(TensorConstant{(1, 1) of 2.0}, Elemwise{Composite{((i0 + i1) - i2)}}[(0, 0)].0, Elemwise{mul,no_inplace}.0), Elemwise{Sqr}[(0, 0)](Elemwise{Composite{((i0 + i1) - i2)}}[(0, 0)].0)]]
My training data is a matrix of entries such as ..
[ 815.257786 320.447 310.841]
And the batches I'm inputting to my training function have a shape of (BATCH_SIZE, 3) and type TensorType(float64, matrix)
My neural net is very simple:
self.inpt = T.dmatrix('inpt')
self.out = T.dvector('out')
self.network_in = nnet.layers.InputLayer(shape=(BATCH_SIZE, 3), input_var=self.inpt)
self.l0 = nnet.layers.DenseLayer(self.network_in, num_units=40,
nonlinearity=nnet.nonlinearities.rectify,
)
self.network = nnet.layers.DenseLayer(self.l0, num_units=1,
nonlinearity=nnet.nonlinearities.linear
)
My loss function is:
pred = nnet.layers.get_output(self.network)
loss = nnet.objectives.squared_error(pred, self.out)
loss = loss.mean()
I'm a bit confused as to why I'm getting a dimension mismatch. I'm passing in the correct input and label types (as per my symbolic variables), and the shape of my input data corresponds to the expected 'shape' parameter that I'm giving my InputLayer. I believe it's a problem with how I'm specifying the batch size, as when I use a batch size of 1 then my network can train without any problem, and the input[2].shape[1] value from the error message is my batch size. I'm quite new to machine learning, and any help would be greatly appreciated!
Turns out the problem was that my labels had the wrong dimensionality.
My data had shapes:
x_train.shape == (batch_size, 3)
y_train.shape == (batch_size,)
And the symbolic inputs to my net were:
self.inpt = T.dmatrix('inpt')
self.out = T.dvector('out')
I was able to solve my problem by reshaping y_train. I then changed the symbolic output variable to a matrix to account for these changes.
y_train = np.reshape(y_train, y_train.shape + (1,))
# y_train.shape == (batch_size, 1)
self.out = T.dmatrix('out')
Related
I know this question has been asked a bunch of times but I just can't resolve this error using any of the answers at SO. I am trying to build embeddings and my data is shaped: (20, 7, 12) i.e. 20 training samples, that have 7 words each, with one-hot encoded to 12 dimensions.
When I fit my model using the below specs, I get the error:
Error when checking input: expected embedding_24_input to have 2
dimensions, but got array with shape (20, 7, 12)
embedding_dims = 10
model = Sequential()
model.add(Embedding(12, embedding_dims,input_length=7))
I then tried to Flatten before Embedding, but that failed complaining that "input_length" is 7, but received input has shape (None, 84)". I then changed the input_length on the embedding layer to match that but no luck with that either:
Error when checking target: expected embedding_26 to have 3
dimensions, but got array with shape (20, 12)
model = Sequential()
model.add(Flatten())
model.add(Embedding(12, embedding_dims,input_length=84))
I would really appreciate any help with some explanation, please!
Embedding layer don't expect the one hot encoding in input data, you should using function like numpy.argmax or tf.argmax to convert data to integer matrix of size (batch, input_length) before feed data in Embedding layer.
Following example working without any error:
import tensorflow as tf
import numpy as np
embedding_dims = 10
batch_size = 20
vocabulary_size = 12
# The model will take as input an integer matrix of size (batch, input_length),
# and the largest integer (i.e. word index) in the input should be no larger than 11 (vocabulary size - 1)
model = tf.keras.Sequential()
model.add(tf.keras.layers.Embedding(vocabulary_size, embedding_dims, input_length=7))
model.compile('rmsprop', 'mse')
raw_input_array = np.random.randint(2, size=(batch_size, 7, vocabulary_size))
input_array = np.argmax(raw_input_array, axis=-1)
output_array = model.predict(input_array)
print("One hot encodeing data shape: {}\ninput integer matrix shape: {}\noutput shape: {}\n".format(raw_input_array.shape, input_array.shape, output_array.shape))
Outputs:
One hot encodeing data shape: (20, 7, 12)
input integer matrix shape: (20, 7)
output shape: (20, 7, 10)
I am trying to construct a model that looks like this.
Notice that the output shape of the padding layer is 1 * 48 * 48 * 32. The input shape to padding layer is 1 * 48 * 48 * 16. Which type of padding operation does that?
My code:
prelu3 = tf.keras.layers.PReLU(shared_axes = [1, 2])(add2)
deptconv3 = tf.keras.layers.DepthwiseConv2D(3, strides=(2, 2), padding='same')(prelu3)
conv4 = tf.keras.layers.Conv2D(32, 1, strides=(1, 1), padding='same')(deptconv3)
maxpool1 = tf.keras.layers.MaxPool2D()(prelu3)
pad1 = tf.keras.layers.ZeroPadding2D(padding=(1, 1))(maxpool1) # This is the padding layer where problem lies.
This is the part of code that is trying to replicate that block. However, I get model that looks like this.
Am I missing something here or am I using the wrong layer?
By default, keras maxpool2d takes in:
Input shape : 4D tensor with shape (batch_size, rows, cols, channels).
Output shape : (batch_size, padded_rows, padded_cols, chamels)
PLease have a look here zero_padding2d layer docs in keras.
In that respect you are trying to double what is getting treated as a channel here.
Your input looks more like (batch, x, y, z) and you want to have a (batch, x, y, 2*z)
Why do you want to have a zeropadding to double your z? I would rather suggest you to use a dense layer like
tf.keras.layers.Dense(32)(maxpool1)
That would increase z shape from 16 to 32.
Edited:
I got something which can help you.
tf.keras.layers.ZeroPadding2D(
padding=(0, 8), data_format="channels_first"
)(maxpool1)
What this does is treats your y, z as (x, y) and x as channel and pads (0, 8) around (y, z) to give (y, 32)
Demo:
import tensorflow as tf
input_shape = (4, 28, 28, 3)
x = tf.keras.layers.Input(shape=input_shape[1:])
y = tf.keras.layers.Conv2D(16, 3, activation='relu', dilation_rate=2, input_shape=input_shape[1:])(x)
x=tf.keras.layers.ZeroPadding2D(
padding=(0, 8), data_format="channels_first"
)(y)
print(y.shape, x.shape)
(None, 24, 24, 16) (None, 24, 24, 32)
I want to build a neural network using neupy.
Therefore I consturcted the following architecture:
network = layers.join(
layers.Input(10),
layers.Linear(500),
layers.Relu(),
layers.Linear(300),
layers.Relu(),
layers.Linear(10),
layers.Softmax(),
)
My data is shaped as follwoing:
x_train.shape = (32589,10)
y_train.shape = (32589,1)
When I try to train this network using:
model.train(x_train, y_trian)
I get the follwoing error:
ValueError: Input dimension mis-match. (input[0].shape[1] = 10, input[1].shape[1] = 1)
Apply node that caused the error: Elemwise{sub,no_inplace}(SoftmaxWithBias.0, algo:network/var:network-output)
Toposort index: 26
Inputs types: [TensorType(float64, matrix), TensorType(float64, matrix)]
Inputs shapes: [(32589, 10), (32589, 1)]
Inputs strides: [(80, 8), (8, 8)]
Inputs values: ['not shown', 'not shown']
Outputs clients: [[Elemwise{Composite{((i0 * i1) / i2)}}(TensorConstant{(1, 1) of 2.0}, Elemwise{sub,no_inplace}.0, Elemwise{mul,no_inplace}.0), Elemwise{Sqr}[(0, 0)](Elemwise{sub,no_inplace}.0)]]
How do I have to edit my network to map this kind of data?
Thank you a lot!
Your architecture has 10 outputs instead of 1. I assume that your y_train function is a 0-1 class identifier. If so, than you need to change your structure to this:
network = layers.join(
layers.Input(10),
layers.Linear(500),
layers.Relu(),
layers.Linear(300),
layers.Relu(),
layers.Linear(1), # Single output
layers.Sigmoid(), # Sigmoid works better for 2-class classification
)
You can make it even simpler
network = layers.join(
layers.Input(10),
layers.Relu(500),
layers.Relu(300),
layers.Sigmoid(1),
)
The reason why it works is because layers.Liner(10) > layers.Relu() is the same as layers.Relu(10). You can learn more in official documentation: http://neupy.com/docs/layers/basics.html#mutlilayer-perceptron-mlp
self.model = Sequential()
self.model.add(Dense(units=20, input_dim=9))
self.model.add(Activation('relu'))
self.model.add(Dense(units=len(labels)))
self.model.add(Activation('softmax'))
self.model.compile(optimizer='sgd', # rmsprop
loss='categorical_crossentropy',
metrics=['accuracy'])
x = np.array([[0] * 9])
print('x {} {}'.format(x.shape, x))
a = self.model.predict(x)
that gives
TypeError: only integer scalar arrays can be converted to a scalar index
It does not make sense at all.
x = np.array([0] * 9)
ValueError: Error when checking : expected dense_1_input to have shape (None, 9) but got array with shape (9, 1)
Please help the welp
The error message is a bit confusing in my opinion but the solution is simple. If your input array has the shape (x, y, z), the expected input shape for predict is (n, x, y, z) where n is the number of samples (1 in your case).
Just use
self.model.predict(x.reshape((1, ) + x.shape))
I'm trying to use Theano's autoencoders to discover context-specific features from two different types of data.
The first type has 13 features and the second one has 60.
n_ins=[13,60],
n_hiddens=[20, 20, 20],
Both have their own independent stack of autoencoders.
I merge the outputs of the topmost layers and feed these into a regression layer for supervised training.
self.logLayer = LogisticRegression(
input=(self.sigmoid_layers[0][-1].output+self.sigmoid_layers[1][-1].output),
n_in=self.n_modes*n_hiddens[-1],
n_out=n_outs
)
Pre-training for each context seems to work correctly, however I hit a snag during finetuning using the standard training function in the tutorials.
train_fn = theano.function(
inputs=[index],
outputs=self.finetune_cost,
updates=updates,
givens={
self.x: train_set_x[
index * batch_size: (index + 1) * batch_size
],
self.y: train_set_y[
index * batch_size: (index + 1) * batch_size
]
},
name='train'
)
I get the following error:
ValueError: dimension mismatch in args to gemm (5,73)x(13,20)->(5,20)
Apply node that caused the error: GpuDot22(GpuSubtensor{int64:int64:}.0, W)
Inputs types: [CudaNdarrayType(float32, matrix), CudaNdarrayType(float32, matrix)]
Inputs shapes: [(5, 73), (13, 20)]
Inputs strides: [(73, 1), (20, 1)]
Inputs values: ['not shown', 'not shown']
I believe this has something to do with how the Theano nodes are processed during training. Seems like the training batch (5, 73), is being applied straight into the output nodes separately starting with the 1st context (13, 20).