I have sample data in the form: Data[n][31][31][5][2] with:
"[n]" being the sample
"[31][31]" being the array of data points
"[5]" being the number of bits within that data point
and "[2]" being one-hot encoding of the bits (eg a bit of 1 would be [1, 0] and a zero [0, 1])
The output is intended to either be a [5][2] or a [10] array of values which is validated against another [5][2] or [10] array.
When trying to build the model, I get the following error:
"ValueError: Shapes (None, 5, 2) and (None, 10) are incompatible"
The model code looks like this:
(with train_m[n][31][31][5][2], tr_m[5][2], check_m[n][31][31][5][2], cr_m[5][2] being training data and expected output followed by validation data and expected output.)
model = Sequential([
Conv2D(num_filters, filter_size, input_shape=(31, 31, 5, 2)),
Flatten(),
Dense(10, activation='relu'),
])
model.compile(
'adam',
loss='categorical_crossentropy',
metrics=['accuracy'],
)
model.summary()
model.fit(
train_m,
tr_m,
epochs=(100),
validation_data=(check_m, cr_m),
verbose=0
)
As the [5][2] outputs are one-hotted, I'm uncertain if they can be made to a [10] matrix while still being interpreted correctly. Further, would there be any way to make the dense layer to a [5][2]?
The full error can be seen here. I felt it would be awfully long to include in rawtext here.
If there's anything more that's needed, please let me know - I'm still very new to working with TensorFlow.
Your label shapes are (5,2) but network output is (10,) so this is confusing. Both output shape and label shape should be the same.
use:
tf.keras.layers.Reshape((5,2))
after the Dense layer. you'll be fine
Related
I am getting an error relating to shapes whilst defining a very simple network using Tensorflow 2.
My code is:
import tensorflow as tf
import pandas as pd
data = pd.read_csv('data.csv')
target = data.pop('result')
target = tf.keras.utils.to_categorical(target.values, num_classes=3)
data_set = tf.data.Dataset.from_tensor_slices((data.values, target))
model = tf.keras.Sequential([
tf.keras.layers.Input(shape=data.shape[1:]),
tf.keras.layers.Dense(12, activation='relu'),
tf.keras.layers.Dense(3, activation='softmax')
])
model.compile(optimizer='adam', loss='categorical_crossentropy')
model.fit(data_set, epochs=5)
The call to fit() throws the following error:
ValueError: Input 0 of layer sequential is incompatible with the layer: expected axis -1 of input shape to have value 12 but received input with shape [12, 1]
Walking through the code:
The input CSV file has thirteen columns - with the last being the label
This is converted to a 3 bit one-hot encoding
The Dataset is constructed of two Tensors - one of shape (12,) and the other of shape (3,)
The network Input layer defines it's expected shape as be the value data shape ignoring the first axis which is the batch size
I am stumped about why there is mismatch between the shape of the data and the expected data shape for the network - especially as the latter is defined by reference to the former.
Add .batch() at the end of the dataset:
data_set = tf.data.Dataset.from_tensor_slices((data.values, target)).batch(8)
I have a simple network:
input_layer = Input(1)
inner_layer = Dense(4, activation='relu')(input_layer)
output_layer = Dense(1, activation='linear')(inner_layer)
model = Model(input_layer, output_layer)
optimizer = Adam(learning_rate=0.01)
model.compile(optimizer=optimizer, loss='mse')
Intuitively, inference for input 0 would simply be model.predict(0). However this generates this error: expected input_2 to have 2 dimensions, but got array with shape ()
I understand it expects the input (which is a single number) to be two dimensional, but I don't understand what Tensorflow accepts as valid input. I tried many different combinations of inputs, some work and some don't, it seems quite inconsistent and the warnings/error are usually not useful:
When calling model.predict():
model.predict(0) - Throws
model.predict([0]) - Works
model.predict([[0]]) - Works
When calling model() (I saw here that's needed to get the gradients):
model(0) - Throws
model([0]) - Throws
model([[0]]) - Throws
When using np.reshape:
model(np.reshape(0,[1,1])) - Works
model(np.reshape([0],[1,1])) - Works
model(np.reshape([[0]],[1,1])) - Works
What seems to be working consistently is using numpy's reshape function. it always works both for model.predict() and for model() on all the inputs as long as they're reshaped to a [1,1] shape.
My questions:
What are the guidelines to feeding inputs into tensorflow models in regards to inputs shapes/types?
What does "shape ()" mean?
What does "(None, 1)" mean?
Why does reshape work but [[0]] does not? Both create a 2-dimensional collection.
Why when calling model(0)/model([0])/model([[0]]) does this warning show: WARNING:tensorflow:Model was constructed with shape Tensor("input_1:0", shape=(None, 1), dtype=float32) for input (None, 1), but it was re-called on a Tensor with incompatible shape ()?
The shape of the tensor inputs = tf.keras.layers.Input(1) is (None, 1) (run inputs.get_shape().as_list()). The None means any size that is determined dynamically (batch size). The 1 is the shape of your data point. For example, this is a tensor of shape (3, 1):
[[1], [2], [1]]
This is a tensor of shape (3,)
[1, 2, 1]
If you define a tensor of shape (None, 1) you must feed the data of same shape.
The [[0]] has correct shape (1, 1) and won't throw any error or warning if you pass it as numpy array of expected data type:
import tensorflow as tf
import numpy as np
input_layer = tf.keras.layers.Input(1)
inner_layer = tf.keras.layers.Dense(4, activation='relu')(input_layer)
output_layer = tf.keras.layers.Dense(1, activation='linear')(inner_layer)
model = tf.keras.models.Model(input_layer, output_layer)
optimizer = tf.keras.optimizers.Adam(learning_rate=0.01)
model.compile(optimizer=optimizer, loss='mse')
print(model(np.array([[0.]], dtype=np.float32)).numpy()) # [[0.]]
print(model.predict(np.array([[0.], [1]], dtype=np.float32))) # [[0. ]
# [0.08964952]]
np.reshape() works because it automatically converts your list to numpy array. For more about np.reshape refer to the official documentation.
The model.predict() also expects the same shape as model.__call__(), but can perform automatic reshaping (expands dimension on the left, i.e [1] -- > [[1]]).
I have a dataset consisting of time series of different lengths. For instance, consider this
ts1 = np.random.rand(230, 4)
ts2 = np.random.rand(12309, 4)
I have 200 sequences in the form of list of arrays
input_x = [ts1, ts2, ..., ts200]
These time series have labels 1 if good and 0 if not. Hence my labels will be something like
labels = [0, 0, 1, 0, 1, ....]
I am building a keras model as follows:
model = keras.Sequential([
keras.layers.Conv1D(64, 3, activation='relu', input_shape=(None, 4)),
keras.layers.MaxPool1D(3),
keras.layers.Conv1D(160, 10, activation='relu'),
keras.layers.GlobalAveragePooling1D(),
keras.layers.Dropout(0.5),
keras.layers.Dense(128, activation='relu'),
keras.layers.Dense(2, activation='softmax')
])
The 4 in the input shape of the first convolution layer corresponds to the number of columns in each time series which is constant (think of it as having 4 sensors returning measurements for different operations). The objective is to classify if a time series is good or bad (0 or 1) however I am unable to figure out how to train this using keras.
Running this line
model.fit(input_x, labels, epochs=5, batch_size=1)
Returns an error
Error when checking model input: the list of Numpy arrays that you are passing to your model is not the size the model expected. Expected to see 1 array(s), but instead got the following list of 200 arrays
Even using np.array(input_x) gives an error. How can I train this model with sequences of variable lengths? I know padding is an option but that's not what I am looking for. Also, I don't want to use an RNN with a sliding window. I am really looking into a solution with 1D CNN that works with sequences of variable lengths. Any help would be so much appreciated!
When working with a time series you want to define the input to the NN as (batch_size, sequence_length, features).
Which corresponds to a input_shape=(sequence_length, 4,) in your case. You will have to decide upon a maximum sequence length that you will process for the purposes of training and generating predictions.
The inputs to the NN also need to be in the shape (batch_size, sequence_length, features).
I am using Keras for building Conv Net for the first time. My layers are as follows:
layers = [
Conv2D(8,kernel_size=(4,4),padding='same',input_shape=( 200, 180,3),kernel_initializer="glorot_normal",data_format="channels_first"),
Activation("relu"),
MaxPooling2D(pool_size=(8,8),padding='same',data_format='channels_first'),
Conv2D(16,(2,2),padding='same',kernel_initializer="glorot_normal"),
Activation("relu"),
MaxPooling2D(pool_size=(4,4),padding='same',data_format='channels_first'),
Conv2D(4,(3,3),padding='same',kernel_initializer="glorot_normal"),
Activation("relu"),
MaxPooling2D(pool_size=(2,2),padding='same',data_format='channels_first'),
Flatten(),
Dense(2,input_shape=(48,)),
Softmax(axis=-1)
]
#Edit, here is the part for compiling the model and fitting it
model = Sequential(layers)
model.compile(optimizer="adam",loss="sparse_categorical_crossentropy"
metrics=["accuracy"])
trainHistory = model.fit(x=X_train,y=Y_train,batch_size=3,epochs=1000)
My labels array is of shape (,2). But when I try to use fit on the model, it gives me the error that softmax_1 expected to have shape (1,). But I have clearly mentioned units of Dense as 2 and softmax returns output of the same dimension as the input.
So where did the 1, came from? I tried to use dummy label array of 1 dimension and it runs. So what am I doing wrong? How do I use 2 dimensional array that I have?
The problem is that you are using sparse_categorical_crossentropy as the loss function. This loss function is used when the given labels (i.e. Y_train) are encoded as integers (i.e. 0, 1, 2, ...). However, If the labels are one-hot encoded, which seems to be the case in your code, you need to use categorical_crossentropy as the loss function instead.
I'm trying the get a hang of keras and I'm trying to get basic time series prediction working. My input is a list of random ints between 0 and 10 such as:[1,3,2,4,7,5,9,0] and my labels are the same as the input but delayed such as: [X,X,1,3,2,4,7,5] and I'm trying to have my model learn this relationship of remembering past data points.
My code is:
labels = keras.utils.to_categorical(output, num_keys)
model = keras.Sequential([
keras.layers.LSTM(10),
keras.layers.Dense(10, activation='relu'),
keras.layers.Dense(10, activation=tf.nn.softmax)
])
model.compile(optimizer=tf.train.AdamOptimizer(),
loss=tf.keras.losses.categorical_crossentropy,
metrics=['accuracy'])
model.fit(input, labels, epochs=30, verbose=2,shuffle=False)
and I get the error:ValueError: Please provide as model inputs either a single array or a list of arrays. You passed: x=[7, 6,...
I've tried reformating my input with:
input=numpy.array([[i,input[i]]for i in range(len(input))])
input=numpy.reshape(input,input.shape+(1,))
and adding input_shape=input.shape[1:] to my LSTM layer and that throws no errors but the accuracy is no better then just blind guessing
This seems like that kind of thing that could be trivial but I'm clearly missing something.
With keras.layers.LSTM(10), you need to include the input data shape: keras.layers.LSTM(10, input_shape = (input.shape[1], input.shape[2])).
Keras is expecting the input data shaped as [instances, time, predictors] and since you don't have any additional predictors, you may need to reshape your input data to input.reshape(input.shape[0], input.shape[1], 1).
Keras will infer the data shapes for the next layers, but the first layer needs the input shape defined.