I was trying to use a simple NN on a matrix of chemical compositions of 24 elements a total of 270 analysis (25X270 including the label). However when I am running the gradient descent it shows the error 'numpy.float64' object cannot be interpreted as an integer.
data = np.array(data)
m, n = data.shape
np.random.shuffle(data)
data_dev = data[0:60].T
Y_dev = data_dev[0]
Y_dev = np.array(Y_dev, dtype=np.float_)
X_dev = data_dev[1:n]
X_dev = preprocessing.normalize(X_dev)
data_train = data[60:m].T
Y_train = data_train[0]
Y_train = np.array(Y_train, dtype=np.float_)
X_train = data_train[1:n]
X_train = preprocessing.normalize(X_train)
_,m_train = X_train.shape
enter image description here
enter image description here
This is the error that I got:
enter image description here
I´ve tried the code with the MNIST dataset (Changing the structure of the first W1 to 10,784 and dividing each pixel data by 255 to avoid any conflict) and it works well. I also check the dtype of the arrays and it is the same (float64) I don't know why is there a problem with my data.
What the type of Y array?
If it is a float64 then the problem is that Y.max() returns float value but you need integers when you define the sizes of arrays. so you need to change:
Y = np.array([1, 2, 3], np.float64)
one_hot_Y = np.zeros((Y.size, Y.max() + 1)) # will not work
to
one_hot_Y = np.zeros((Y.size, int(Y.max()) + 1))
Or convert the type of Y to int: Y.astype(int)
Related
I want to make a prediction using knn and I have following lines of code:
def knn(trainImages, trainLabels, testImages, testLabels):
max = 0
for i in range(len(trainImages)):
if len(trainImages[i]) > max:
max = len(trainImages[i])
for i in range(len(trainImages)):
aux = np.array(trainImages[i])
aux.resize(max)
trainImages[i] = aux
max = 0
for i in range(len(testImages)):
if len(testImages[i]) > max:
max = len(testImages[i])
for i in range(len(testImages)):
aux = np.array(testImages[i])
aux.resize(max)
testImages[i] = aux
scaler = StandardScaler()
scaler.fit(list(trainImages))
trainImages = scaler.transform(list(trainImages))
testImages = scaler.transform(list(testImages))
classifier = KNeighborsClassifier(n_neighbors=5)
classifier.fit(trainImages, trainLabels)
pred = classifier.predict(testImages)
print(classification_report(testLabels, pred))
I got the error at testImages = scaler.transform(list(testImages)). I understand that its a difference between arrays number. How can I solve it?
scaler in scikit-learn expects input shape as (n_samples, n_features).
If your second dimension in train and test set is not equal, then not only in sklearn it is incorrect and cause to raise error, but also in theory it does not make sense. n_features dimension of test and train set should be equal, but first dimension can be different, since it show number of samples and you can have any number of samples in train and test sets.
When you execute scaler.transform(test) it expects test have the same feature numbers as where you executed scaler.fit(train). So, all your images should be in the same size.
For example, if you have 100 images, train_images shape should be something like (90,224,224,3) and test_images shape should be like (10,224,224,3) (only first dimension is different).
So, try to resize your images like this:
import cv2
resized_image = cv2.resize(image, (224,224)) #don't include channel dimension
Im creating a model that uses Keras's functional API, this model takes 2 inputs, hence im using
video_input = Input(shape=(16, 112, 112, 3))
image_input = Input(shape=(112, 112, 3))
Model(inputs=[video_input, image_input], outputs=merge_model)
So as you can see, this means that the model expects an array with the first element being of shape (16, 112, 112, 3) and second of shape (112, 112, 3).
I'm using a class that i created which inherits Keras.util.sequence class to provide generated batches of data.
the problem comes after generating batches of data when tensorflow attempts to feed the model with the input the input is changed from being array of 2 to be array 1 and this 1 element consists of 2 for example it should expect
[array(...), array(...)] instead it receives [array(array[...],array[...])]
ValueError: Error when checking model input: the list of Numpy arrays that you are passing to your model is not the size the model expected. Expected to see 2 array(s), but instead got the following list of 1 arrays: [array([[array([[[[-76.87925 , -81.45539 , -82.91122 ],
[-76.90526 , -81.45103 , -83.00473 ],
[-76.77082 , -81.259674, -82.92529 ],
...,
[-76.17821 , -80.61866 , -8...
i tried to make the data holder in the sequence generator as python array where i append data then convert it to numpy array but got the error above.
somehow keras wraps it into 1 array before it returns it to the model.
this is the data generation method
def __data_generation(self, list_IDs_temp):
'Generates data containing batch_size samples' # X : (n_samples, *dim, n_channels)
# Initialization
X = []
y = np.empty((self.batch_size), dtype=int)
# Generate data
for i, ID in enumerate(list_IDs_temp):
# Store sample
print(ID)
frame_data = input_data.get_frames_data(
self.work_directory + ID, self.num_of_frames, self.crop_size)
image_index = random.randint(0, len(frame_data) - 1)
im = frame_data[image_index]
X.append([frame_data, im])
# Store class
y[i] = self.labels[ID]
return np.array(X), keras.utils.to_categorical(
y, num_classes=self.n_classes)
edited function that works
def __data_generation(self, list_IDs_temp):
'Generates data containing batch_size samples' # X : (n_samples, *dim, n_channels)
# Initialization
vX = np.empty((self.batch_size, *self.c3d_dim))
iX = np.empty((self.batch_size, *self.static_dim))
y = np.empty((self.batch_size), dtype=int)
# Generate data
for i, ID in enumerate(list_IDs_temp):
# Store sample
print(ID)
frame_data = input_data.get_frames_data(
self.work_directory + ID, self.num_of_frames, self.crop_size)
image_index = random.randint(0, len(frame_data) - 1)
im = frame_data[image_index]
vX[i, ] = frame_data
iX[i, ] = im
# Store class
y[i] = self.labels[ID]
return vX, iX, keras.utils.to_categorical(
y, num_classes=self.n_classes)
As I remember you should feed each input as independent array. For example you have 2 input images, you should not have array of type [[image_1, image_2], [image_3, image_4],[image_5, image_6] ..] but instead you should have something like [[image_1, image_3,image_5 ..], [image_2, image_4, image_6 ..]] as you see, first array is input for first image and second array is input for second image. This applies to your case as well. Just store inputs in different arrays and combine them when you apply fit. Should be something like [video_frames, images]
Hope it helps.
I'm trying to train a classifier via PyTorch. However, I am experiencing problems with training when I feed the model with training data.
I get this error on y_pred = model(X_trainTensor):
RuntimeError: Expected object of scalar type Float but got scalar type Double for argument #4 'mat1'
Here are key parts of my code:
# Hyper-parameters
D_in = 47 # there are 47 parameters I investigate
H = 33
D_out = 2 # output should be either 1 or 0
# Format and load the data
y = np.array( df['target'] )
X = np.array( df.drop(columns = ['target'], axis = 1) )
X_train, X_test, y_train, y_test = train_test_split(X, y, train_size = 0.8) # split training/test data
X_trainTensor = torch.from_numpy(X_train) # convert to tensors
y_trainTensor = torch.from_numpy(y_train)
X_testTensor = torch.from_numpy(X_test)
y_testTensor = torch.from_numpy(y_test)
# Define the model
model = torch.nn.Sequential(
torch.nn.Linear(D_in, H),
torch.nn.ReLU(),
torch.nn.Linear(H, D_out),
nn.LogSoftmax(dim = 1)
)
# Define the loss function
loss_fn = torch.nn.NLLLoss()
for i in range(50):
y_pred = model(X_trainTensor)
loss = loss_fn(y_pred, y_trainTensor)
model.zero_grad()
loss.backward()
with torch.no_grad():
for param in model.parameters():
param -= learning_rate * param.grad
Reference is from this github issue.
When the error is RuntimeError: Expected object of scalar type Float but got scalar type Double for argument #4 'mat1', you would need to use the .float() function since it says Expected object of scalar type Float.
Therefore, the solution is changing y_pred = model(X_trainTensor) to y_pred = model(X_trainTensor.float()).
Likewise, when you get another error for loss = loss_fn(y_pred, y_trainTensor), you need y_trainTensor.long() since the error message says Expected object of scalar type Long.
You could also do model.double(), as suggested by #Paddy
.
Before converting to Tensor, try this
X_train = X_train.astype(np.float32)
The issue can be fixed by setting the datatype of input to Double i.e torch.float32
I hope the issue came because your datatype is torch.float64.
You can avoid such situations either while setting the data, as explained in one of other answers or make the model type also to the same as of your data. i.e use either float64 or float32.
For debug, print obj.dtype and check for consistency.
Let's do that:
df['target'] = df['target'].astype(np.float32)
and for x features too
This issue can also occur if the wrong loss function is selected. For example, if you have regression problem, but you are trying to use cross entropy loss. Then it will be fixed by changing your loss function on MSE
try to use:
target = target.float() # target is the name of error
New to PyTorch. For some reason, calling torch.set_default_dtype() with the needed datatype was what worked for me on Google Colab. network.double()/network.float() and tensor.double()/tensor.float() didn’t have any effect, for some reason.
Try this example:
from sentence_transformers import SentenceTransformer, util
import numpy as np
import torch
a = np.array([0, 1,2])
b = [[0, 1,2], [4, 5,6], [7,8,9]]
bb = np.zeros((3,3))
for i in range(0, len(b)):
bb[i,:] = np.array(b[i])
a = torch.from_numpy(a)
b = torch.from_numpy(bb)
a= a.float()
b = b.float()
cosine_scores = util.pytorch_cos_sim(b, a)
print(cosine_scores)
I'm getting a runtime error:
RuntimeError: Dimension out of range (expected to be in range of [-1, 0], but got 1)`
and can't figure out how to fix it.
The error appears to refer to the line:
i_enc = F.normalize(input =i_batch, p=2, dim=1, eps=1e-12) # (batch, K, feat_dim)
I'm trying to encode image features (batch x 36 x 2038) by applying a L2 norm. Below is the full code for the section.
def forward(self, q_batch, i_batch):
# batch size = 512
# q -> 512(batch)x14(length)
# i -> 512(batch)x36(K)x2048(f_dim)
# one-hot -> glove
emb = self.embed(q_batch)
output, hn = self.gru(emb.permute(1, 0, 2))
q_enc = hn.view(-1,self.h_dim)
# image encoding with l2 norm
i_enc = F.normalize(input =i_batch, p=2, dim=1, eps=1e-12) # (batch, K, feat_dim)
q_enc_copy = q_enc.repeat(1, self.K).view(-1, self.K, self.h_dim)
q_i_concat = torch.cat((i_enc, q_enc_copy), -1)
q_i_concat = self.non_linear(q_i_concat, self.td_W, self.td_W2 )#512 x 36 x 512
i_attention = self.att_w(q_i_concat) #512x36x1
i_attention = F.softmax(i_attention.squeeze(),1)
#weighted sum
i_enc = torch.bmm(i_attention.unsqueeze(1), i_enc).squeeze() # (batch, feat_dim)
# element-wise multiplication
q = self.non_linear(q_enc, self.q_W, self.q_W2)
i = self.non_linear(i_enc, self.i_W, self.i_W2)
h = torch.mul(q, i) # (batch, hid_dim)
# output classifier
# BCE with logitsloss
score = self.c_Wo(self.non_linear(h, self.c_W, self.c_W2))
return score
I would appreciate any help.
Thanks
I would suggest to check the shape of i_batch (e.g. print(i_batch.shape)), as I suspect i_batch has only 1 dimension (e.g. of shape [N]).
This would explain why PyTorch is complaining you can normalize only over the dimension #0; while you are asking for the operation to be done over a dimension #1 (c.f. dim=1).
Getting output classification with Lasagne/Theano
I am migrating my code from pure Theano to Lasagne.
I had this certain code from a tutorial to get the result of a prediction with a certain data and I would generate a csv file to send to kaggle.
But with lasagne, it doesn't work.
I have tried several things but they all give errors.
I would love if anyone could help me figure what's wrong!
I pasted the whole code here :
http://pastebin.com/e7ry3280
test_data = np.loadtxt("../inputData/test.csv", dtype=np.uint8, delimiter=',', skiprows=1)
# The inputs are vectors now, we reshape them to monochrome 2D images,
# following the shape convention: (examples, channels, rows, columns)
data = data.reshape(-1, 1, 28, 28)
test_data = test_data.reshape(-1, 1, 28, 28)
index = T.lscalar() # index to a [mini]batch
preds = []
for it in range(len(test_data)):
test_data = test_data[it]
N = len(test_data)
# print "N : ", N
test_data = theano.shared(np.asarray(test_data, dtype=theano.config.floatX))
test_labels = T.cast(theano.shared(np.asarray(np.zeros(batch_size), dtype=theano.config.floatX)),'uint8')
###target_var
#y = T.ivector('y') # the labels are presented as 1D vector of [int] labels
#index = T.lscalar() # index to a [mini]batch
ppm = theano.function([index],lasagne.layers.get_output(network, deterministic=True),
givens={
input_var: test_data[index * batch_size: (index + 1) * batch_size],
target_var: test_labels
}, on_unused_input='warn')
p = [ppm(ii) for ii in range(N // batch_size)]
p = np.array(p).reshape((N, 10))
print (p)
p = np.argmax(p, axis=1)
p = p.astype(int)
preds.append(p)
subm = np.empty((len(preds), 2))
subm[:, 0] = np.arange(1, len(preds) + 1)
subm[:, 1] = preds
np.savetxt('submission.csv', subm, fmt='%d', delimiter=',',header='ImageId,Label', comments='')
return preds
The code fails on the line that starts with ppm = theano.function...:
TypeError: Cannot convert Type TensorType(float32, 3D) (of Variable Subtensor{int64:int64:}.0) into Type TensorType(float32, 4D). You can try to manually convert Subtensor{int64:int64:}.0 into a TensorType(float32, 4D).
I'm just trying to input the test data to the CNN and get the results to a CSV file. How can I do it? I know I must use minibatches because the whole test data wont fit on the GPU.
As pointed out by the error message and Daniel Renshaw in the comments, the problem is a mismatch of dimensions between test_data and input_var. On the first line on the loop, you write:
test_data = test_data[it]
Which turns the 4D array test_data into a 3D array with the same name (that is why using the same variable name for different types is never recommended :) ). After that you encapsulate it in a shared variable which doesn't change the dimension, and then you slice it to assign it to input_var, which again doesn't change the dimension.
If I understand your code, I think you should just remove that first line. That way test_data remains a list of examples, and you can slice it to make a batch.