I'm trying to reshape a Tensorflow model's input along the batch dimension. I want to combine some of the batch samples into a time-series so I can feed it into an LSTM layer.
Specifically, I have 1024 samples and I'd like to put them into groups of 64 timesteps with the result being 16 batches of 64 timesteps, each timestep having the original 24 features.
#input tensor is (1024, 24)
inputLayer = Input(shape=(24,))
#I want it to be (16, 64, 24)
reshapedLayer = layers.Reshape([64, 24])(inputLayer)
lstmLayer = layers.LSTM(128, activation='relu')(reshapedLayer)
This compiles but throws a runtime error
tensorflow.python.framework.errors_impl.InvalidArgumentError:
Input to reshape is a tensor with 24576 values, but the requested shape has 1572864
I understand what the error is telling me, but I'm not sure the right way to go about fixing it.
Perhaps this could work for you:
import tensorflow as tf
inputs = tf.keras.layers.Input(shape=(24,))
x = tf.reshape(inputs, (16, 64, 24))
x = tf.keras.layers.LSTM(128, activation='relu')(x)
model = tf.keras.Model(inputs=inputs, outputs=x)
# dummy data
inputs = tf.random.uniform(shape=(1024, 24))
outputs = model(inputs)
Replacing the Reshape layer with tf.reshape.
Related
I am facing some problems in training the following GRU model, which has to be stateful and output the hidden state.
import numpy as np
import tensorflow as tf #2.1.0
from tensorflow import keras
BATCH_SIZE = 1
nfeatures = 3
history = 30 # shapes input array
horizon = 5 # shapes output array
nodes = 32
input_layer = tf.keras.layers.Input(batch_shape=(1,30,3),name="INPUT")
output, state_h = tf.keras.layers.GRU(nodes,
return_sequences=True,
stateful=True,
return_state=True,
batch_input_shape=(1,history,3), name='GRU1')(input_layer)
output_layer = tf.keras.layers.GRU(nodes, activation='tanh', name='GRU2')(output, state_h)
output_dense = tf.keras.layers.Dense(5, name='DENSE')(output_layer)
model = tf.keras.Model(input_layer, [output_dense, state_h])
model.compile(optimizer=tf.keras.optimizers.Adam(clipvalue=2.0),
loss='mse',
metrics=['mean_absolute_error', 'mean_squared_error'])
As I need the model to output the hidden state, I do not use a Sequential model. (I had no problems training a stateful sequential model.)
The features fed to network are of shape np.shape(x)=(30,3) and the target np.shape(y)=(5,).
If I call model.predict(x), where x is a numpy array with the shape mentioned above, it throws an error, as expected, because the input shape doesn't match the expected input. Therefore, I reshape the input array to have an input shape of (1,30,3) by calling np.expand_dims(x,axis=0). After that, it works fine, i.e. I get an output.
The issues I am facing are when I try to train the model. Calling
model.fit(x, y,epochs=1,steps_per_epoch=STEPS_PER_EPOCH)
throws the same error, about the shape of the data
ValueError: Error when checking input: expected input to have 3 dimensions, but got array with shape (30, 3)
Reshapping the data as I did for the prediction didn't help
model.fit(np.expand_dims(x,axis=0), np.expand_dims(y,axis=0),epochs=1,steps_per_epoch=STEPS_PER_EPOCH)
ValueError: The number of samples 1 is not divisible by steps 30. Please change the number of steps to a value that can consume all the samples.
This was a new error, setting the steps_per_epoch=1 threw a new one
ValueError: Error when checking model target: the list of Numpy arrays that you are passing to your model is not the size the model expected. Expected to see 2 array(s), for inputs ['DENSE', 'GRU1'] but instead got the following list of 1 arrays: [array([[0.5124772 , 0.51047856, 0.509669 , 0.50830126, 0.5070507 ]],
dtype=float32)]...
Is the format of my data wrong or is the architecture of my layers missing something? I tried adding a Flatten layer after the input, but it didn't make much sense (in my head) and it didn't work either.
Thanks in advance.
Problem here is that the Number of Nodes should be equal to the Output Shape. Changing the value of Nodes from 32 to 5, along with other minor changes, will fix the Error.
Complete working code is shown below:
import numpy as np
import tensorflow as tf #2.1.0
from tensorflow import keras
BATCH_SIZE = 1
nfeatures = 3
history = 30 # shapes input array
horizon = 5 # shapes output array
nodes = 5
x = np.ones(shape = (30,3))
x = np.expand_dims(x, axis = 0)
y = np.ones(shape = (5,))
y = np.expand_dims(y, axis = 0)
print(x.shape) #(1, 30, 3)
print(y.shape) #(1, 5)
input_layer = tf.keras.layers.Input(batch_shape=(1,30,3),name="INPUT")
output, state_h = tf.keras.layers.GRU(nodes,
return_sequences=True,
stateful=True,
return_state=True,
batch_input_shape=(1,history,3), name='GRU1')(input_layer)
output_layer = tf.keras.layers.GRU(nodes, activation='tanh', name='GRU2')(output, state_h)
output_dense = tf.keras.layers.Dense(5, name='DENSE')(output_layer)
model = tf.keras.Model(input_layer, [output_dense, state_h])
model.compile(optimizer=tf.keras.optimizers.Adam(clipvalue=2.0),
loss='mse',
metrics=['mean_absolute_error', 'mean_squared_error'])
STEPS_PER_EPOCH = 1
model.fit(x, y,epochs=1,steps_per_epoch=STEPS_PER_EPOCH)
Output of the above code is:
(1, 30, 3)
(1, 5)
1/1 [==============================] - 0s 3ms/step - loss: 1.8172 - DENSE_loss: 1.1737 - GRU1_loss: 0.6435 - DENSE_mean_absolute_error: 1.0498 - DENSE_mean_squared_error: 1.1737 - GRU1_mean_absolute_error: 0.7157 - GRU1_mean_squared_error: 0.6435
<tensorflow.python.keras.callbacks.History at 0x7f698bf8ac50>
Hope this helps. Happy Learning!
I have a dataset where I have images and per image I have 10 questions and thus 10 answers. I have successfully trained and checkpointed the model. The model consists of two inputs, one input as image to a CNN and the other input as question to the LSTM. So for every image I would feed in 10 questions. The results of both would then be concatenated and input to the FC layers.
Considering I have a batch size of 64, I would feed in 64 images and 640 questions. At the time of concatenation I would need to have the dimension at axis:0 to be equal, to avoid concatenation error due to different dimensions. Thus, I flatten my CNN network output and repeat it 10 times and then concatenate it to the LSTM output.
Inside the model_rn.py, I do following:
class Model:
def __init__(self):
self.img = tf.placeholder(
name='img',
dtype=tf.float32,
shape=[self.batch_size, self.img_size, self.img_size, 3]
)
self.q = tf.placeholder(
name='ques',
dtype=tf.float32,
shape=[self.batch_size * 10, self.ques_dim]
)
self.ans = tf.placeholder(
name='ans',
dtype=tf.float32,
shape=[self.batch_size * 10, self.ans_dim]
)
# and some more class variables
self.build()
def build(self):
def cnn(img, q, scope):
# some Conv2D and BatchNormalization
flat = Flatten(name='flatten')(bn_4) # layer where data is flattened before concatenate
flat = tf.keras.backend.repeat_elements(flat, 10, axis=0) # repeat 10 times
# some statements to feed data into LSTM and CNN
Then I load my model and try to run it on the test dataset which contains 20 images, 200 questions and thus 200 answers. But then I get the error:
ValueError: Cannot feed value of shape (20, 128, 128, 3) for Tensor 'img_1:0', which has shape '(640, 128, 128, 3)'
To feed the batches from the test dataset I have used padded_batch.
dataset_img = Dataset.from_tensor_slices((images)).padded_batch(
64, padded_shapes=(128, 128, 3)
)
dataset_ques = Dataset.from_tensor_slices((questions)).padded_batch(
64 * 10, padded_shapes=(14)
)
dataset_ans = Dataset.from_tensor_slices((answers)).padded_batch(
64 * 10, padded_shapes=(22)
)
Can someone help me with this?
Thank you!
you have constrained your placeholder to take exactly "batch_size" number of rows. To take any rows you can create placeholder like this
self.img = tf.placeholder(
name='img',
dtype=tf.float32,
shape=[None, self.img_size, self.img_size, 3]
)
similarly for self.q and self.ans
I am running a CNN for classification of medical scans using Keras and transfer learning with imagenet and InceptionV3. I am building the model with some practice data of size X_train = (624, 128, 128, 1) and Y_train = (624, 2).
I am trying to resize the input_tensor to suit the shape of my images (128 x 128 x 1) using the below code.
input_tensor = Input(shape=(128, 128, 1))
base_model = InceptionV3(input_tensor=input_tensor,weights='imagenet',include_top=False)
Doing this I get a value error:
ValueError: Dimension 0 in both shapes must be equal, but are 3 and 32. Shapes
are [3,3,1,32] and [32,3,3,3]. for 'Assign_753' (op: 'Assign') with input
shapes: [3,3,1,32], [32,3,3,3]
Is there a way to allow this model to accept my images in their format?
Edit:
For what its worth, here is the code to generate the training data.
X = []
Y = []
for subj, subj_slice in slices.items():
# X.extend([s[:, :, np.newaxis, np.newaxis] for s in slice])
subj_slice_norm = [((imageArray - np.min(imageArray)) / np.ptp(imageArray)) for imageArray in subj_slice]
X.extend([s[ :, :, np.newaxis] for s in subj_slice_norm])
subj_status = labels_df['deadstatus.event'][labels_df['PatientID'] == subj]
subj_status = np.asanyarray(subj_status)
#print(subj_status)
Y.extend([subj_status] * len(subj_slice))
X = np.stack(X, axis=0)
Y = to_categorical(np.stack(Y, axis=0))]
n_samp_train = int(X.shape[0]*0.8)
X_train, Y_train = X[:n_samp_train], Y[:n_samp_train]
Edit2:
I think the other alternative would be to take my X which is shape (780, 128, 128, 1), clone each of the 780 images and append two as dummies. Is this possible? Resulting in (780, 128, 128, 3).
We can use the existing keras layers to convert the existing image shape to the expected shape for the pre-trained model rather than using the numpy for replicating channels. As replicating channels before training may consume 3x the memory, but integrating this processing at runtime will save up a lot of memory.
You can proceed this way.
Step 1: Create a Keras Model that converts your input images to the shape that can be fed as the input for the base_model as follows:
from keras.models import Model
from keras.layers import RepeatVector, Input, Reshape
inputs = Input(shape=(128, 128, 1))
reshaped1 = Reshape(target_shape=((128 * 128 * 1,)))(inputs)
repeated = RepeatVector(n=3)(reshaped1)
reshaped2 = Reshape(target_shape=(3, 128, 128))(repeated)
input_model = Model(inputs=inputs, outputs=reshaped2)
Step 2: Define pre-trained model InceptionV3 as follows:
base_model = InceptionV3(input_tensor=input_model.output, weights='imagenet', include_top=False)
Step 3: Combine both the models as follows:
combined_model = Model(inputs=input_model.input, outputs=base_model.output)
The advantage of this method is that the keras model itself will take care of the image processing stuff like channel replication at runtime. Thus, we need not replicate the image channels by ourselves with numpy and the results will be memory efficient.
I'm attempting to follow along on what I'm thinking is the 5th or 6th simple introductory tutorial for keras that almost but never quite works.
Stripping everything out, I appear to come down to a problem with the format of my input. I read in an array of images, and extract two types, images of sign language ones and images of sign language zeros. I then set up an array of ones and zeros to correspond to what the images actually are, then make sure of sizes and types.
import numpy as np
from subprocess import check_output
print(check_output(["ls", "../data/keras/"]).decode("utf8"))
## load dataset of images of sign language numbers
x = np.load('../data/keras/npy_dataset/X.npy')
# Get the zeros and ones, construct a list of known values (Y)
X = np.concatenate((x[204:409], x[822:1027] ), axis=0) # from 0 to 204 is zero sign and from 205 to 410 is one sign
Y = np.concatenate((np.zeros(205), np.ones(205)), axis=0).reshape(X.shape[0],1)
# test shape and type
print("X shape: " , X.shape)
print("X class: " , type(X))
print("Y shape: " , Y.shape)
print("Y type: " , type(Y))
This gives me:
X shape: (410, 64, 64)
X class: <class 'numpy.ndarray'>
Y shape: (410, 1)
Y type: <class 'numpy.ndarray'>
which is all good. I then load the relevant bits from Keras, using Tensorflow as the backend and try to construct a classifier.
# get the relevant keras bits.
from keras.models import Sequential
from keras.layers import Convolution2D
# construct a classifier
classifier = Sequential() # initialize neural network
classifier.add(Convolution2D(32, (3, 3), input_shape=(410, 64, 64), activation="relu", data_format="channels_last"))
classifier.compile(optimizer = 'adam', loss = 'categorical_crossentropy', metrics = ['accuracy'])
classifier.fit(X, Y, batch_size=32, epochs=10, verbose=1)
This results in:
ValueError: Error when checking input: expected conv2d_1_input to have 4 dimensions, but got array with shape (410, 64, 64)
This SO question, I think, suggests that my input shape needs to be altered to have a 4th dimension added to it - though it also says it's the output shape that needs to altered, I haven't been able to find anywhere to specify an output shape, so I'm assuming it is meant that I should alter the input shape to input_shape=(1, 64, 64, 1).
If I change my input shape however, then I immeadiately get this:
ValueError: Input 0 is incompatible with layer conv2d_1: expected ndim=4, found ndim=5
Which this github issue suggests is because I no longer need to specify the number of samples. So I'm left with the situation of using one input shape and getting one error, or changing it and getting another error.
Reading this and this made me think I might need to reshape my data to include information about the channels in X, but if I add in
X = X.reshape(X.shape[0], 64, 64, 1)
print(X.shape)
Then I get
ValueError: Error when checking target: expected conv2d_1 to have 4 dimensions, but got array with shape (410, 1)
If I change the reshape to anything else, i.e.
X = X.reshape(X.shape[0], 64, 64, 2)
Then I get a message saying it's unable to reshape the data, so I'm obviously doing something wrong with that, if that is, indeed, the problem.
I have read the suggested Conv2d docs which shed exactly zero light on the matter for me. Anyone else able to?
At first I used the following data sets (similar to your case):
import numpy as np
import keras
X = np.random.randint(256, size=(410, 64, 64))
Y = np.random.randint(10, size=(410, 1))
x_train = X[:, :, :, np.newaxis]
y_train = keras.utils.to_categorical(Y, num_classes=10)
And then modified your code as follows to work:
from keras.models import Sequential
from keras.layers import Convolution2D, Flatten, Dense
classifier = Sequential() # initialize neural network
classifier.add(Convolution2D(32, (3, 3), input_shape=(64, 64, 1), activation="relu", data_format="channels_last"))
classifier.add(Flatten())
classifier.add(Dense(10, activation='softmax'))
classifier.compile(optimizer = 'adam', loss = 'categorical_crossentropy', metrics = ['accuracy'])
classifier.fit(x_train, y_train, batch_size=32, epochs=10, verbose=1)
Changed the shape of X from 410 x 64 x 64 to 410 x 64 x 64 x 1 (with channel 1).
input_shape be the shape of a sample data, that is, 64 x 64 x 1.
Changed the shape of Y using keras.utils.to_categorical() (one-hot encoding with num_classes=10).
Before compiling, Flatten() and Dense() were applied because you want categorical_crossentropy.
The full error message is like this:
ValueError: Shapes (2, 1) and (50, 1) are incompatible
It occurs when my model is trained. The mistake either is in my input_fn:
train_input_fn = tf.estimator.inputs.numpy_input_fn(
x = {"x" : training_data},
y = training_labels,
batch_size = 50,
num_epochs = None,
shuffle = True)
in my logits and loss function:
dense = tf.layers.dense(inputs = pool2_flat, units = 1024, activation = tf.nn.relu)
dropout = tf.layers.dropout(inputs = dense, rate = 0.4, training = mode == tf.estimator.ModeKeys.TRAIN)
logits = tf.layers.dense(inputs = dropout, units = 1)
loss = tf.losses.softmax_cross_entropy(labels = labels, logits = logits)
or in my dataset. I can only print out the shape of my dataset for you to take a look at it.
#shape of the dataset
train_data.shape
(1196,2,1)
train_data[0].shape
(2,1)
#this is the data
train_data[0][0].shape
(1,)
train_data[0][0][0].shape
(20,50,50)
#this is the labels
train_data[0][1].shape
(1,)
The problem seems to be the shape of the logits. They are supposed to be [batch_size, num_classes] in this case [50,1] but are [2,1]. The shape of the labels is correctly [50,1]
I have made a github gist if you want to take a look at the whole code.
https://gist.github.com/hjkhjk1999/38f358a53da84a94bf5a59f44050aad5
In your code, you are stating that the inputs to your model will be feed in batches of 50 samples per batch with one variable. But it looks like your are feeding actually a batch of 2 samples with 1 variable (shape=[2, 1]) despite feeding labels with shape [50, 1].
That's the problem, you are giving 50 'questions' and two 'answers'.
Also, your dataset is shaped in a really weird way. I see you named your github gist 3D Conv. If you are indeed trying to do a 3D convolution you might want to reshape your dataset into a tensor (numpy array) of this shape shape = [samples, width, height, deepth]