Develop Reference

Tensorflow : Manually selecting the batch when training deep neural networks

# x_train.shape[0] = 54000
model.fit(
x_train, y_train,
batch_size = 128,
epochs = 12,
validation_data = (x_val, y_val)
)
When I am using this fit() method to train a neural network:
batch_size = 128 means that I randomly pick 54000 // 128 batches of size 128 in my training dataset every epoch.
Are those batches chosen with replacement? I suspect from the docs they're not but I'd like confirmation.
Can I manually choose my batches? I would like to focus on specific images and not others for a given batch, by choosing them personally instead of letting randomness choose for me.

Are those batches chosen with replacement?
In each individual epoch, no. Of course the entire dataset is used again in the next epoch.
Can I manually choose my batches? I would like to focus on specific images and not others for a given batch, by choosing them personally instead of letting randomness choose for me.
You should create a custom dataset for this, and leave the rest of the training loop (data loader, model etc.) unchanged.
But be aware that the samples in a minibatch are supposed to be random.

Tensorflow input generator ran out of data

So I was trying to implement this kaggle code into my Jupyter to test the performance of my laptop.
There were some modifications of the code to fit the my version of environments:
#from scipy.ndimage import imread
from imageio import imread
Upon block[11], i received the error as below
Any help or suggestions are appreciated.

You have specified step_per_epoch incorrectly.
The steps_per_epoch should be equal to
steps_per_epoch = ceil(number_of_samples / batch_size)
For your case
steps_per_epoch = ceil(1161 / 16) = ceil(72.56) = 73
Try specifying steps_per_epoch = 73
As you can your entire data is exhausted in 73 steps. Now, if you specify steps_per_epoch any higher than 73 ie 74
There is no data available. Therefore you get input generator ran out of data
More Information:
Model training comprises of two parts forward pass and backward pass.
1 train step = 1 forward pass + 1 backward pass
A single train step(1 forward pass + 1 backward pass) is calculated on a single batch.
So if you have 100 samples and your batch size is 10.
Your model will have 10 train steps.
Epoch: Epoch is defined as complete iteration over the dataset.
Therefore, for your model to completely iterate over the dataset of 100 samples, it should undergo 10 train steps.
This train step is nothing but steps_per_epoch.
The steps_per_epoch argument is usually specified when you give infinite data generator to your fit() command and does not need to be specified if you have finite data.

Steps per Epoch/Validation steps in Matterport-Mask RCNN

I am following Matterport Mask RCNN model but I have a doubt regarding setting Steps_per_epoch and Validation_steps for training. I am trying to train a custom dataset. I have 2500 training samples,1500 validation samples and 1000 test samples. If I set the value Steps_per_epoch=1000 and Validation_steps=100 then how many training samples and validation samples are used during one epoch?

Please look at https://keras.rstudio.com/reference/fit_generator.html in regard to questions about fit_generator.
Step per epochs overrides the length of an epoch to be X batches.
Validation_steps only matter if your validation_data is a generator, it will validate x batches. Else it should use all the validation data provided.

STEPS_PER_EPOCH should be the number of instances (training examples) divided by (GPU_COUNT*IMAGES_PER_GPU),
For small enough images and given a single GPU, if you can fit 8in its memory then:
STEPS_PER_EPOCH = 2500/8 # 312
VALIDATION_STEPS = 1500/8 # 187
Keep in mind that the more steps the slower it gets You may overwrite these with your own choices to accelerate matters. Usually one sets
STEPS_PER_EPOCH = 100
VALIDATION_STEPS = 50

tensorflow 2.0, model.fit() : Your input ran out of data

I am absolutely new to TensorFlow and Keras, and I am trying to make my way around trying out some code that I am finding online.
In particular I am using the fashion-MNIST - consisting of 60000 examples and test set of 10000 examples. Each of them is a 28x28 grayscale image.
I am following this tutorial "https://towardsdatascience.com/building-your-first-neural-network-in-tensorflow-2-tensorflow-for-hackers-part-i-e1e2f1dfe7a0", and I have no problem until the definition of
history = model.fit(
train_dataset.repeat(),
epochs=10,
steps_per_epoch=500,
validation_data=val_dataset.repeat(),
validation_steps=2)
As long as I understood, I need to use train_dataset.repeat() as input dataset because otherwise I won't have enough training example using those values for the hyperparameters (epochs, steps_per_epochs).
My question is: how can I avoid to have to use .repeat()?
How do I need to change the hyperparameters?
I am coping the code here, for simplicity:
def preprocess(x,y):
x = tf.cast(x,tf.float32) / 255.0
y = tf.cast(y, tf.float32)
return x,y
def create_dataset(xs, ys, n_classes=10):
ys = tf.one_hot(ys, depth=n_classes)
return tf.data.Dataset.from_tensor_slices((xs, ys)).map(preprocess).shuffle(len(ys)).batch(128)
model.compile(optimizer = 'adam', loss =tf.losses.CategoricalCrossentropy(from_logits= True), metrics =['accuracy'])
history1 = model.fit(train_dataset.repeat(),
epochs=10,
steps_per_epoch=500,
validation_data=val_dataset.repeat(),
validation_steps=2)
Thanks!

If you don't want to use .repeat() you need to have your model passing thought your entire data only one time per epoch.
In order to do that you need to calculate how many steps it will take for your model to pass throught the entire dataset, the calcul is easy :
steps_per_epoch = len(train_dataset) // batch_size
So with a train_dataset of 60 000 sample and a batch_size of 128, you need to have 468 steps per epoch.
By setting this parameter like that you make sure that you do not exceed the size of your dataset.

I encountered the same problem and here is what I found.
Documentation of tf.keras.Model.fit: "If x is a tf.data dataset, and 'steps_per_epoch' is None, the epoch will run until the input dataset is exhausted."
In other words, we don't need to specify 'steps_per_epoch' if we use the tf.data.dataset as the training data, and tf will figure out how many steps are there. Meanwhile, tf will automatically repeat the dataset when the next epoch begins, so you can specify any 'epoch'.
When passing an infinitely repeating dataset (e.g. dataset.repeat()), you must specify the steps_per_epoch argument.

Keras LSTM Batch Size and model.fit()

I am training an LSTM in Keras. As per documentation, my training data and labels have shape (20, 20, 1) representing 20 samples with 20 time steps and one feature. When I use model.fit() to train my model, do I need to specify batch size or will all 20 samples be sent as one batch by default?

According to Keras's fit documentation
batch_size Integer or NULL. Number of samples per gradient update. If unspecified, batch_size will default to 32.