I want to create a shallow network that would take a vector and pass it through a network.
I have a vector that is of size 6. vec = [0,1,4,5,1,4,5]
My network:
vec_a = Input(shape=(6,))
x_1 = Convolution1D(nb_filter=10, filter_length=1, input_shape=(1, 6), activation='relu')(vec_a)
x_1 = Dense(16, activation='relu')(x_1)
But I keep getting:
ValueError: Input 0 is incompatible with layer conv1d_1: expected
ndim=3, found ndim=2
The shape of the training data to the fit function is:
(36400, 6)
You have to reshape the input data to have the correct input dimension, e.g.:
your_input_array.reshape(-1, 6, 1)
In addition your input layer should look like:
vec_a = Input(shape=(6,1))
The reason is that the 1D in Conv1D relates to the use of a sequence. But this sequence can have a vector of multiple values at each position. In your case it is the same, but you have "only" a vector of length 1 in the last dimension.
Related
I am using tf 1.14, and following the official guide to pass my own sparse tensor that has a dense shape of 2000 x 2000 as input to a model, like so:
input = layers.Input((2000,), sparse=True)
print(input.get_shape().as_list())
output = some_layers(input)
model = models.Model(inputs=input, outputs=output)
However when I print the input shape, it returns [None, None], and I get the error:
ValueError: Error when checking input: expected input_10 to have 2 dimensions, but got array with shape (None, None, None)
If I then specify a batch input = layers.Input((2000,), batch_size=1, sparse=True), the size is (1, 2000) and it runs without error.
And if I specific input is 2d like so: input = layers.Input((2000,2000), batch_size=1, sparse=True) then i return a size (1,2000,2000), and again it runs without error
Which approach is correct? Ultimately, I want to use the sparse tensor as an adjacency matrix for a GCN layer. Therefore I want it without the batch dimension.
When passing the output of my embedding layer to the LSTM layer I'm running into a ValueError that I cannot figure out. My model is:
def lstm_mod(self, n_cells,batch_size):
input = tf.keras.Input((self.n_seq, self.n_features))
embedding = tf.keras.layers.Embedding(batch_size,self.n_seq,input_length=self.n_clusters)(input)
x= tf.keras.layers.LSTM(n_cells)(embedding)
out = tf.keras.layers.Dense(1)(x)
model = tf.keras.Model(input, out,name="LSTM")
model.compile(loss='mse', optimizer='Adam')
return model
The error is:
ValueError: Input 0 of layer lstm is incompatible with the layer: expected ndim=3, found ndim=4. Full shape received: [None, 128, 7, 128]
Given that the dimensions passed to the model input and the embedding layer are consistent through the arguments of the model I'm puzzled by this. Any guidance is appreciated.
Keras adds an additional dimension (None) when you feed your data through your model because it processes your data in batches.
In this line :
input = tf.keras.Input((self.n_seq, self.n_features))
You've defined a 2-dimensional input, and Keras adds a 3rd dimension (the batch), hence expected ndim=3.
However, the data that is being passed to the input layer is 4-dimensional, which means that your actual input data shape is 3-dimensional + the batch dimension, not 2-dimensional + batch.
To fix this you need to either re-shape your 3-D input to 2-D, or add an additional dimension to the input shape.
Print out the values for self.n_seq and self.n_features and find out what is missing from the shape 128, 7, 128 and that should guide you as to what you need to add.
I want to use a convolutional neural network, but I have a 2D array for the input, not an image. I am trying to evaluate a board game state where shapes are important.
The board is 5x5 and the values can be between -1 and 1, stored as a list of lists ex:
[[-1,1.0,-1,1,-1],[0,1,0,0,0],[0,0,0,0,0],[0,0,0,0,0],[-1,0.6,-1,-1,1]]
the first layer of the model is
tf.keras.layers.Conv2D(32, (3,3), input_shape=(5,5,1))
I convert the board to a numpy array
np.array([[-1,1.0,-1,1,-1],[0,1,0,0,0],[0,0,0,0,0],[0,0,0,0,0],[-1,0.6,-1,-1,1]])
I gather the boards into a list.
Then I convert the list into an array of arrays to fit
model.fit(np.array(x_train_l), y_train, epochs=10)
I get the following error:
ValueError: Input 0 of layer sequential is incompatible with the layer: : expected min_ndim=4, found ndim=3. Full shape received: [None, 5, 5]
Just reshape your numpy array to have shape (5,5,1). Currently it is with shape (5,5).
np.array([[-1,1.0,-1,1,-1],[0,1,0,0,0],[0,0,0,0,0],[0,0,0,0,0],[-1,0.6,-1,-1,1]]).reshape(5,5,1)
I have the following neural network in Keras:
inp = layers.Input((3,))
#Middle layers omitted
out_prop = layers.Dense(units=3, activation='softmax')(inp)
out_value = layers.Dense(units=1, activation = 'linear')(inp)
Then I prepared a pseudo-input to test my network:
inpu = np.array([[1,2,3],[4,5,6],[7,8,9]])
When I try to predict, this happens:
In [45]:nn.network.predict(inpu)
Out[45]:
[array([[0.257513 , 0.41672954, 0.32575747],
[0.20175152, 0.4763418 , 0.32190666],
[0.15986516, 0.53449154, 0.30564335]], dtype=float32),
array([[-0.24281949],
[-0.10461146],
[ 0.11201331]], dtype=float32)]
So, as you can see above, I wanted two output: one should have been an array with size 3, the other should have been a normal value. Instead, I get a 3x3 matrix, and an array with 3 elements. What am I doing wrong?
You are passing three input samples to the network:
>>> inpu.shape
(3,3) # three samples of size 3
And you have two output layers: one of them outputs a vector of size 3 for each sample and the other outputs a vector of size one (i.e. scalar), again for each sample. As a result the output shapes would be (3, 3) and (3, 1).
Update: If you want your network to accept an input sample of shape (3,3) and outputs vectors of size 3 and 1, and you want to only use Dense layers in your network, then you must use a Flatten layer somewhere in the model. One possible option is to use it right after the input layer:
inp = layers.Input((3,3)) # don't forget to set the correct input shape
x = Flatten()(inp)
# pass x to other Dense layers
Alternatively, you could flatten your data to have a shape of (num_samples, 9) and then pass it to your network without using a Flatten layer.
Update 2: As #Mete correctly pointed out in the comments, make sure the input array have a shape of (num_samples, 3, 3) if each input sample has a shape of (3,3).
I have the following idea to implement:
Input -> CNN-> LSTM -> Dense -> Output
The Input has 100 time steps, each step has a 64-dimensional feature vector
A Conv1D layer will extract features at each time step. The CNN layer contains 64 filters, each has length 16 taps. Then, a maxpooling layer will extract the single maximum value of each convolutional output, so a total of 64 features will be extracted at each time step.
Then, the output of the CNN layer will be fed into an LSTM layer with 64 neurons. Number of recurrence is the same as time step of input, which is 100 time steps. The LSTM layer should return a sequence of 64-dimensional output (the length of sequence == number of time steps == 100, so there should be 100*64=6400 numbers).
input = Input(shape=(100,64), dtype='float', name='mfcc_input')
CNN_out = TimeDistributed(Conv1D(64, 16, activation='relu'))(mfcc_input)
CNN_out = BatchNormalization(axis=-1, momentum=0.99, epsilon=0.001, center=True, scale=True)(CNN_out)
CNN_out = TimeDistributed(MaxPooling1D(pool_size=(64-16+1), strides=None, padding='valid'))(CNN_out)
LSTM_out = LSTM(64,return_sequences=True)(CNN_out)
... (more code) ...
But this doesn't work. The second line reports "list index out of range" and I don't understand what's going on.
I'm new to Keras, so I appreciate sincerely if anyone could help me with it.
This picture explains how CNN should be applied to EACH TIME STEP
The problem is with your input. Your input is of shape (100, 64) in which the first dimension is the timesteps. So ignoring that, your input is of shape (64) to a Conv1D.
Now, refer to the Keras Conv1D documentation, which states that the input should be a 3D tensor (batch_size, steps, input_dim). Ignoring the batch_size, your input should be a 2D tensor (steps, input_dim).
So, you are providing 1D tensor input, where the expected size of the input is a 2D tensor. For example, if you are providing Natural Language input to the Conv1D in form of words, then there are 64 words in your sentence and supposing each word is encoded with a vector of length 50, your input should be (64, 50).
Also, make sure that you are feeding the right input to LSTM as given in the code below.
So, the correct code should be
embedding_size = 50 # Set this accordingingly
mfcc_input = Input(shape=(100, 64, embedding_size), dtype='float', name='mfcc_input')
CNN_out = TimeDistributed(Conv1D(64, 16, activation='relu'))(mfcc_input)
CNN_out = BatchNormalization(axis=-1, momentum=0.99, epsilon=0.001, center=True, scale=True)(CNN_out)
CNN_out = TimeDistributed(MaxPooling1D(pool_size=(64-16+1), strides=None, padding='valid'))(CNN_out)
# Directly feeding CNN_out to LSTM will also raise Error, since the 3rd dimension is 1, you need to purge it as
CNN_out = Reshape((int(CNN_out.shape[1]), int(CNN_out.shape[3])))(CNN_out)
LSTM_out = LSTM(64,return_sequences=True)(CNN_out)
... (more code) ...