I have a tf.data.Dataset that looks like this:
<BatchDataset shapes: ((None, 256, 256, 3), (None,)), types: (tf.float32, tf.int32)>
The 2nd element (1st if zero indexing) corresponds with a label. I want to cast the 2nd term (labels) to tf.uint8.
How can one use tf.cast when dealing with td.data.Dataset?
Similar Questions
How to convert tf.int64 to tf.float32? is very similar, but is not for a tf.data.Dataset.
Repro
From Image classification from scratch:
curl -O https://download.microsoft.com/download/3/E/1/3E1C3F21-ECDB-4869-8368-6DEBA77B919F/kagglecatsanddogs_5340.zip
unzip kagglecatsanddogs_5340.zip
Then in Python with tensorflow~=2.4:
import tensorflow as tf
ds = tf.keras.preprocessing.image_dataset_from_directory(
"PetImages", batch_size=32
)
print(ds)
A map function may help
a = tf.data.Dataset.from_tensor_slices(np.empty((2,5,3)))
b = tf.data.Dataset.range(5, 8)
c = tf.data.Dataset.zip((a,b))
d = c.batch(1)
d
<BatchDataset shapes: ((None, 5, 3), (None,)), types: (tf.float64, tf.int64)>
# change the dtype of the 2nd arg in the batch from int64 to int8
e = d.map(lambda x,y:(x,tf.cast(y, tf.int8)))
<MapDataset shapes: ((None, 5, 3), (None,)), types: (tf.float64, tf.int8)>
Related
I know this question has been asked a bunch of times but I just can't resolve this error using any of the answers at SO. I am trying to build embeddings and my data is shaped: (20, 7, 12) i.e. 20 training samples, that have 7 words each, with one-hot encoded to 12 dimensions.
When I fit my model using the below specs, I get the error:
Error when checking input: expected embedding_24_input to have 2
dimensions, but got array with shape (20, 7, 12)
embedding_dims = 10
model = Sequential()
model.add(Embedding(12, embedding_dims,input_length=7))
I then tried to Flatten before Embedding, but that failed complaining that "input_length" is 7, but received input has shape (None, 84)". I then changed the input_length on the embedding layer to match that but no luck with that either:
Error when checking target: expected embedding_26 to have 3
dimensions, but got array with shape (20, 12)
model = Sequential()
model.add(Flatten())
model.add(Embedding(12, embedding_dims,input_length=84))
I would really appreciate any help with some explanation, please!
Embedding layer don't expect the one hot encoding in input data, you should using function like numpy.argmax or tf.argmax to convert data to integer matrix of size (batch, input_length) before feed data in Embedding layer.
Following example working without any error:
import tensorflow as tf
import numpy as np
embedding_dims = 10
batch_size = 20
vocabulary_size = 12
# The model will take as input an integer matrix of size (batch, input_length),
# and the largest integer (i.e. word index) in the input should be no larger than 11 (vocabulary size - 1)
model = tf.keras.Sequential()
model.add(tf.keras.layers.Embedding(vocabulary_size, embedding_dims, input_length=7))
model.compile('rmsprop', 'mse')
raw_input_array = np.random.randint(2, size=(batch_size, 7, vocabulary_size))
input_array = np.argmax(raw_input_array, axis=-1)
output_array = model.predict(input_array)
print("One hot encodeing data shape: {}\ninput integer matrix shape: {}\noutput shape: {}\n".format(raw_input_array.shape, input_array.shape, output_array.shape))
Outputs:
One hot encodeing data shape: (20, 7, 12)
input integer matrix shape: (20, 7)
output shape: (20, 7, 10)
I have created my own custom dataset (with 2 classes) with the following code:
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
from tensorflow.keras.preprocessing.image import ImageDataGenerator
import matplotlib.pyplot as plt
ds_train = tf.keras.preprocessing.image_dataset_from_directory(
'C:/Users/mydir/Source_Images/',
labels = 'inferred', # from subfolders in alphabetical order
label_mode = "int",
class_names = ["CVS", "No_CVS"],
color_mode = 'rgb',
batch_size = 2,
image_size = (224, 224),
shuffle = True, # randomized order of images
seed = 123, #set the seed if train, valid images are the same when you run again
validation_split = 0.1,
subset = "training"
)
df_train results in:
<BatchDataset shapes: ((None, 224, 224, 3), (None,)), types: (tf.float32, tf.int32)>
Now, I want to visualize my data by looking at 9 images:
for i, (image, label) in enumerate(ds_train.take(9)):
ax = plt.subplot(3, 3, i + 1)
plt.imshow(image.numpy().astype("uint8"))
plt.axis("off")
However, I get the following error:
line 61, in
plt.imshow(image.numpy().astype("uint8"))
TypeError: Invalid shape (2, 224, 224, 3) for image data
I'm looking for a way to resolve this, and be able to plot my images with matplotlib.
EDIT:
More importantly, it seems that the data of the data cannot be used when training the model either as I get this error:
ValueError: Input 0 is incompatible with layer EfficientNet: expected shape=(None, 224, 224, 3), found shape=(2, None, 224, 224, 3)
After running the Keras example code I found here (where I created ds_train with the image_dataset_from_directory instead of the tdsf.load() function).
So I think there is something going wrong in the way I created the ds_train. Any resolutions are very welcome.
It seems like you are leaving the batch_size in, when you do:
plt.imshow(image.numpy().astype("uint8"))
With your original code you still won't be able to see 9 images because of your batch_size. I think it will be fine if you do it like:
No errors should be thrown like TypeError: Invalid shape...:
plt.imshow(image[i].numpy().astype("uint8"))
Furthermore you can do following to see batch_size:
for img_batch_size, labels_batch_size in train_df:
print(img_batch_size.shape)
print(labels_batch_size.shape)
For your case img_batch_size.shape should print (2,224,224,3) where this tuple corresponds to image tensor.
For input_shape problem, you need to add your model so we can see what's wrong with input_shape.
I am trying to use the tensorflow.keras.layers.Flatten layer outside of a model to flatten a 4x4 tensor. I can't figure out why the Flatten layer isn't actually flattening my array.
Here is my code:
import tensorflow as tf
import numpy as np
flayer = tf.keras.layers.Flatten()
X = tf.constant(np.random.random((4,4)),dtype=tf.float32)
Xf = flatten_layer(X)
print(Xf)
and print(Xf) shows
tf.Tensor(
[[0.9866459 0.52488756 0.86211777 0.06254051]
[0.32552275 0.23201537 0.8646714 0.80754006]
[0.55823076 0.51929855 0.538077 0.4111973 ]
[0.95845264 0.14468837 0.30223057 0.09648433]], shape=(4, 4), dtype=float32)
Why doesn't my flatten layer output a 16x1 tensor?
That's because the Flatten() layer assumes that the first dimension is the number of samples, so it returns 4 flattened rows. You have 4 observations, and 1D input for each of these already. It would behave differently if you had data with shape (32, 28, 28, 1), for example, which has a higher dimensionality for each row.
import tensorflow as tf
import numpy as np
flayer = tf.keras.layers.Flatten()
X = tf.constant(np.random.random((32, 28, 28, 1)),dtype=tf.float32)
Xf = flayer(X)
print(Xf.shape)
(32, 784)
If you meant to flatten one observation with shape (4, 4), you should add a batch dimension for it to work:
X = tf.constant(np.random.random((1, 4, 4)),dtype=tf.float32)
Xf = flayer(X)
print(Xf.shape)
(1, 16)
I want to build a neural network using neupy.
Therefore I consturcted the following architecture:
network = layers.join(
layers.Input(10),
layers.Linear(500),
layers.Relu(),
layers.Linear(300),
layers.Relu(),
layers.Linear(10),
layers.Softmax(),
)
My data is shaped as follwoing:
x_train.shape = (32589,10)
y_train.shape = (32589,1)
When I try to train this network using:
model.train(x_train, y_trian)
I get the follwoing error:
ValueError: Input dimension mis-match. (input[0].shape[1] = 10, input[1].shape[1] = 1)
Apply node that caused the error: Elemwise{sub,no_inplace}(SoftmaxWithBias.0, algo:network/var:network-output)
Toposort index: 26
Inputs types: [TensorType(float64, matrix), TensorType(float64, matrix)]
Inputs shapes: [(32589, 10), (32589, 1)]
Inputs strides: [(80, 8), (8, 8)]
Inputs values: ['not shown', 'not shown']
Outputs clients: [[Elemwise{Composite{((i0 * i1) / i2)}}(TensorConstant{(1, 1) of 2.0}, Elemwise{sub,no_inplace}.0, Elemwise{mul,no_inplace}.0), Elemwise{Sqr}[(0, 0)](Elemwise{sub,no_inplace}.0)]]
How do I have to edit my network to map this kind of data?
Thank you a lot!
Your architecture has 10 outputs instead of 1. I assume that your y_train function is a 0-1 class identifier. If so, than you need to change your structure to this:
network = layers.join(
layers.Input(10),
layers.Linear(500),
layers.Relu(),
layers.Linear(300),
layers.Relu(),
layers.Linear(1), # Single output
layers.Sigmoid(), # Sigmoid works better for 2-class classification
)
You can make it even simpler
network = layers.join(
layers.Input(10),
layers.Relu(500),
layers.Relu(300),
layers.Sigmoid(1),
)
The reason why it works is because layers.Liner(10) > layers.Relu() is the same as layers.Relu(10). You can learn more in official documentation: http://neupy.com/docs/layers/basics.html#mutlilayer-perceptron-mlp
When I run my code, I get a value error with the following message:
ValueError: Input dimension mis-match. (input[0].shape[1] = 1, input[2].shape[1] = 20)
Apply node that caused the error: Elemwise{Composite{((i0 + i1) - i2)}}[(0, 0)](Dot22.0, InplaceDimShuffle{x,0}.0, InplaceDimShuffle{x,0}.0)
Toposort index: 18
Inputs types: [TensorType(float64, matrix), TensorType(float64, row), TensorType(float64, row)]
Inputs shapes: [(20, 1), (1, 1), (1, 20)]
Inputs strides: [(8, 8), (8, 8), (160, 8)]
Inputs values: ['not shown', array([[ 0.]]), 'not shown']
Outputs clients: [[Elemwise{Composite{((i0 * i1) / i2)}}(TensorConstant{(1, 1) of 2.0}, Elemwise{Composite{((i0 + i1) - i2)}}[(0, 0)].0, Elemwise{mul,no_inplace}.0), Elemwise{Sqr}[(0, 0)](Elemwise{Composite{((i0 + i1) - i2)}}[(0, 0)].0)]]
My training data is a matrix of entries such as ..
[ 815.257786 320.447 310.841]
And the batches I'm inputting to my training function have a shape of (BATCH_SIZE, 3) and type TensorType(float64, matrix)
My neural net is very simple:
self.inpt = T.dmatrix('inpt')
self.out = T.dvector('out')
self.network_in = nnet.layers.InputLayer(shape=(BATCH_SIZE, 3), input_var=self.inpt)
self.l0 = nnet.layers.DenseLayer(self.network_in, num_units=40,
nonlinearity=nnet.nonlinearities.rectify,
)
self.network = nnet.layers.DenseLayer(self.l0, num_units=1,
nonlinearity=nnet.nonlinearities.linear
)
My loss function is:
pred = nnet.layers.get_output(self.network)
loss = nnet.objectives.squared_error(pred, self.out)
loss = loss.mean()
I'm a bit confused as to why I'm getting a dimension mismatch. I'm passing in the correct input and label types (as per my symbolic variables), and the shape of my input data corresponds to the expected 'shape' parameter that I'm giving my InputLayer. I believe it's a problem with how I'm specifying the batch size, as when I use a batch size of 1 then my network can train without any problem, and the input[2].shape[1] value from the error message is my batch size. I'm quite new to machine learning, and any help would be greatly appreciated!
Turns out the problem was that my labels had the wrong dimensionality.
My data had shapes:
x_train.shape == (batch_size, 3)
y_train.shape == (batch_size,)
And the symbolic inputs to my net were:
self.inpt = T.dmatrix('inpt')
self.out = T.dvector('out')
I was able to solve my problem by reshaping y_train. I then changed the symbolic output variable to a matrix to account for these changes.
y_train = np.reshape(y_train, y_train.shape + (1,))
# y_train.shape == (batch_size, 1)
self.out = T.dmatrix('out')