I have coded mini_batch creator for miniBatchGradientDescent
The code is here:
# function to create a list containing mini-batches
def create_mini_batches(X,y, batch_size):
print(X.shape, y.shape) # gives (280, 34) (280,)
splitData=[]
splitDataResults=[]
batchCount=X.shape[0] // batch_size #using floor division for getting indexes integer form
for i in range(batchCount):
splitData.append(X[(i) * batch_size : (i+1) * batch_size, :])
splitDataResults.append(y[(i) * batch_size : (i+1) * batch_size, :]) # GIVES ERROR
splitData=np.asarray(splitData)
splitDataResults=np.asarray(splitDataResults)
return splitData, splitDataResults, batchCount
the error says:
splitDataResults.append(y[(i) * batch_size : (i+1) * batch_size, :])
IndexError: too many indices for array: array is 1-dimensional, but 2 were indexed
I am sure that the shape is correct but it gives me an error. What is wrong?
try reshaping y:
print(X.shape, y.shape) # gives (280, 34) (280,)
y = y.reshape(-1, 1)
this should fix your problem, since y will become 2 dimentional
Related
I stuck in convert list to numpy. Convert list size is (33, n, 428). N is randomly difference that I don't know how numbers are consist. Here is error.
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
C:\Users\HILAB_~1\AppData\Local\Temp/ipykernel_22960/872733971.py in <module>
----> 1 X_train = np.array(X_train, dtype=np.float64)
2
3 for epoch in range(EPOCH):
4 X_train_ten, y_train_ten = Variable(torch.from_numpy(X_train)), Variable(torch.tensor(y_train, dtype=torch.float32, requires_grad=True))
5 print(X_train_ten.size())
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (33,) + inhomogeneous part.
and problem code is here.
X_train, X_test, y_train, y_test = train_test_split(
X, y, test_size=0.3, random_state=42, shuffle=True
)
print("[SIZE]\t\tTrain X size : {}, Train y size : {}\n\t\tTest X size : {}, Test y size : {}"\
.format(len(X_train), len(y_train), len(X_test), len(y_test)))
train_dataloadloader = DataLoader(X_train)
test_dataloader = DataLoader(X_test)
X_train = np.array(X_train, dtype=np.float64)
I can't understand what does error means. Please help. thanks :D
It means that whatever sequences X contains, they are not of the same length. You can check {len(e) for e in X); this is the set of all different lengths found in X.
Consider the following example:
>>> import numpy as np
>>> x = [[1, 2], [2, 3, 4]]
>>> np.array(x, dtype=np.float64)
[...]
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
Here, the list x contains two other lists, one of length 2 and the other of length 3. They can't be combined into one array since the "column" dimension doesn't match.
I use np.random.choice to get random index but run into the following error.
TypeError: only integer scalar arrays can be converted to a scalar index. Any suggestion would be much appreciated.
x_train = [0,1,2,3,4]
train_size = 4
batch_size =3
batch_mask = np.random.choice(train_size, batch_size)
print(batch_mask)
x_batch = x_train[batch_mask]
print(x_batch)
x_train is a simply python list.
You're trying to use a NumPy array as an index; this is not legal.
Make x_train a numpy array, instead.
x_train = np.array([0,1,2,3,4])
train_size = 4
batch_size =3
batch_mask = np.random.choice(train_size, batch_size)
print(batch_mask)
x_batch = x_train[batch_mask]
print(x_batch)
Output:
[3 3 2]
[13 13 12]
My model expects to receive two input arrays when making a predict. (value, label)
train_model = models.Model(inputs=[x, y], outputs=[out_seg, shared_decoder(masked_by_y, conv_cap_2_1, x )]) # [x:image,y: mask] // [out_seg:length, reconstruction output]
So I made the following class using Sequence.
class No_decoder_sequence(keras.utils.Sequence):
def __init__(self, x_set, y_set, batch_size):
self.x, self.y = x_set, y_set
self.batch_size = batch_size
def __len__(self):
return int(np.ceil(len(self.x) / float(self.batch_size)))
def __getitem__(self, idx):
batch_x = self.x[idx * self.batch_size:(idx + 1) * self.batch_size]
batch_x = np.expand_dims(batch_x ,-1)
batch_y = self.y[idx * self.batch_size:(idx + 1) * self.batch_size]
return np.array(batch_x), np.array(batch_y)
However, an error occurs when I attempt to use the class to predict as follows:
no_decoder_generator_1 = No_decoder_sequence(X_val_1, y_val_1, batch_size=1)
y_predict_1, x_predict_1 = model.predict_generator(generator=no_decoder_generator_1, steps=len(X_val_1))
y_predict_1 = np.asarray(y_predict_1)
x_predict_1 = np.asarray(x_predict_1)
ValueError: Error when checking model input: the list of Numpy arrays
that you are passing to your model is not the size the model expected.
Expected to see 2 array(s), but instead got the following list of 1
arrays: [array([[[[ 0.9287956 ],.
Why is my No_decoder_sequence returning only one array value?
And how do I check the values sequence class returns?
I'm trying to use the following (custom) loss function to train a keras neural network:
y_pred and y_true are arrays of length 40.
Say y_true is 0 everywhere except on the jth component where it is equal to 1,
write y and z for y_true and y_pred resp. then:
blank">
{i<40}(|i-j|+1)\cdot(y_i-z_i)^2" title="boostSquare(y,z)=\sum_{i<40}(|i-j|+1)\cdot(y_i-z_i)^2" />
Here's the code I intended to use :
import keras.backend as K
def boost_square(y_true, y_pred):
w = K.constant(np.array([[np.abs(i - j) + 1 for i in range(40)] for j in
range(40)]), dtype=np.float64)
return K.sum(K.transpose(w * y_true) * K.square(y_true - y_pred))
Running this works and prints 2.25 as expected :
y_true = np.array([int(i == 2) for i in range(40)])
y_pred = np.array([0.5 * int(i < 2) for i in range(40)])
print(K.eval(boost_square(y_true, y_pred)
Yet, this fails to compile with the following error message :
from keras.layers import Input, Dense
input_layer = Input(shape=(40,), name='input_layer')
output_layer = Dense(units=40, name='output_layer')(input_layer)
model = Model([input_layer], [output_layer])
model.compile(optimizer='adam', loss=boost_square,
metrics=['accuracy'])
TypeError: Input 'y' of 'Mul' Op has type float32 that does not match type float64 of argument 'x'.
Since I'm stubborn, I also tried this, which didn't fix anything and might hinder performance :
def boost_square_bis(y_true, y_pred):
z_true = K.cast(y_true, np.float64)
z_pred = K.cast(y_pred, np.float64)
w = K.constant(np.array([[np.abs(i - j) + 1 for i in range(40)] for j in
range(40)]), dtype=np.float64)
boost = K.transpose(w * z_true)
boost = K.cast(boost, dtype=np.float64)
square = K.square(z_true - z_pred)
square = K.cast(square, np.float64)
ret = K.sum(boost * square)
return K.cast(ret, dtype=np.float64)
What am I missing? Where does this error come from?
Solution 1
Credits to AnnaKrogager : the dtype of w wasn't compatible with the model. The
model compiles when one defines :
def boost_square(y_true, y_pred):
w = K.constant(np.array([[np.abs(i - j) + 1 for i in range(40)] for j in
range(40)]), dtype=np.float64)
return K.sum(K.transpose(w * y_true) * K.square(y_true - y_pred))
Iteration 1
Now, the model compiles but won't fit, I get this error message (128 is the batch_size) :
ValueError: Dimensions must be equal, but are 40 and 128 for 'mul_2' (op: 'Mul') with input shapes: [40,40], [128,40].
My custom loss function behaves oddly with respect to this first axis indeed,
this code will raise the very same error :
fake_input = np.random.rand(128,40)
fake_output = np.random.rand(128,40)
print(K.eval(boost_square(fake_intput,fake_output)))
Iteration 2
As AnnaKrogager pointed out, it is more consistent to use a proper np.dot than * followed by a transposition (that messes with batch axis). So I came up with this new definition of boost_square :
def boost_square(y_true, y_pred):
w = K.constant(np.array([[np.abs(i - j) + 1 for i in range(40)] for j in
range(40)]), dtype=np.float32)
return K.sum(K.dot(w, y_true) * K.square(y_true - y_pred))
But this triggers following when I try to fit the model :
AttributeError: 'numpy.ndarray' object has no attribute 'get_shape'
Hence, I tried
def boost_square(y_true, y_pred):
w = K.constant(np.array([[np.abs(i - j) + 1 for i in range(40)] for j in
range(40)]), dtype=np.float32)
return K.sum(K.dot(K.dot(w, y_true), K.square(y_true - y_pred)))
And got a brand new error message \o/ :
Matrix size-incompatible: In[0]: [40,40], In[1]: [32,40]
Definitive Solution
Credits to AnnaKrogager
Ingredients
Use proper matrice product K.dot ratter than * .
Though w was meant to be applied to y_true, don't use K.dot(w,y_true) since
it messes with the batch axis. Ratter, use K.dot(y_true,w) and transpose to have matching shapes.
If you want to test the loss function with np.arrays, say y_true and y_pred, make sure you recast them as K.constant.
Here's the code :
def boost_square(y_true, y_pred):
w = K.constant(np.array([[np.abs(i - j) + 1 for i in range(40)] for j in
range(40)]), dtype=np.float32)
return K.sum(K.dot(K.dot(y_true, w), K.transpose(K.square(y_true -
y_pred))))
And for the test :
y_true = K.constant(np.array([[int(i == 2) for i in range(40)]],
dtype=np.float32))
y_pred = K.constant(np.array([[0.5 * int(i < 2) for i in range(40)]],
dtype=np.float32))
print(K.eval(boost_square(y_true,y_pred)))
>>2.25
The problem is that your model outputs float32 whereas the constant w inside your loss function is of type float64. You can fix this by simply changing the data type of w:
def boost_square(y_true, y_pred):
w = K.constant(np.array([[np.abs(i - j) + 1 for i in range(40)] for j in
range(40)]), dtype=np.float32)
return K.sum(K.transpose(w * y_true) * K.square(y_pred))
Answer to your second question: If you multiply tensors in Keras it means that the tensors get multiplied element wise, hence they must have the same shape. What you want is the matrix product so you should use K.dot(y, w) instead of w * y.
self.model = Sequential()
self.model.add(Dense(units=20, input_dim=9))
self.model.add(Activation('relu'))
self.model.add(Dense(units=len(labels)))
self.model.add(Activation('softmax'))
self.model.compile(optimizer='sgd', # rmsprop
loss='categorical_crossentropy',
metrics=['accuracy'])
x = np.array([[0] * 9])
print('x {} {}'.format(x.shape, x))
a = self.model.predict(x)
that gives
TypeError: only integer scalar arrays can be converted to a scalar index
It does not make sense at all.
x = np.array([0] * 9)
ValueError: Error when checking : expected dense_1_input to have shape (None, 9) but got array with shape (9, 1)
Please help the welp
The error message is a bit confusing in my opinion but the solution is simple. If your input array has the shape (x, y, z), the expected input shape for predict is (n, x, y, z) where n is the number of samples (1 in your case).
Just use
self.model.predict(x.reshape((1, ) + x.shape))