How to fix "ValueError: Expected 2D array, got 1D array instead"? - python

When running:
scaler = StandardScaler().fit(train)
I am getting this error:
enter image description here
After that I tried:
train = train.array.reshape(-1, 1)
And I got:
AttributeError: 'list' object has no attribute 'array'
How can I reshape by data to fix the value error?

Try train = np.array(train).reshape(-1, 1).
It appears train is originally a list. So you must convert it into an array using np.array(). Then, you can use the .reshape() method to change the dimensions. .reshape(-1, 1) asks numpy to create a 2-dimensional matrix with a single column. numpy can infer the number of rows in this matrix since you only want one column. the -1 parameter asks numpy to infer the length of the first dimension. Please see this answer for a more complete understanding.

Related

How to make numpy array containing multidimensional arrays of different shape? [duplicate]

I am trying to generate a numpy array with elements as two other numpy arrays, as below.
W1b1 = np.zeros((256, 161))
W2b2 = np.zeros((256, 257))
Wx = np.array([W1b1, W2b2], dtype=np.object)
this gives an error:
ValueError: could not broadcast input array from shape (256,161) into shape (256).
However, if I take entirely different dimensions for of W1b1 and W2b2 then I do not get an error, as below.
A1 = np.zeros((256, 161))
A2 = np.zeros((257, 257))
A3 = np.array([A1, A2], dtype=np.object)
I do not get what is wrong in the first code and why is numpy array trying to broadcast one of the input arrays.
I have tried on below versions (Python 2.7.6, Numpy 1.13.1) and (Python 3.6.4, Numpy 1.14.1).
Don't count on np.array(..., object) making the right object array. At the moment we don't have control over how many dimensions it makes. Conceivably it could make a (2,) array, or (2, 256) (with 1d contents). Sometimes it works, sometimes raises an error. There's something of a pattern, but I haven't seen an analysis of the code that shows exactly what is happening.
For now it is safer to allocate the array, and fill it:
In [57]: arr = np.empty(2, object)
In [58]: arr[:] = [W1b1, W2b2]
np.array([np.zeros((3,2)),np.ones((3,4))], object) also raises this error. So the error arises when the first dimensions match, but the later ones don't. Now that I think about, I've seen this error before.
Earlier SO questions on the topic
numpy array 1.9.2 getting ValueError: could not broadcast input array from shape (4,2) into shape (4)
Creation of array of arrays fails, when first size of first dimension matches
Creating array of arrays in numpy with different dimensions

How can I do same calculation in numpy array list at once?

I want to normalize my list containing 2D NumPy arrays of different sizes with axis=1.
My NumPy array list is like below:
First, I tried to get mean and std using numpy.hstack.
An error occurred in the following operation that was successfully obtained:
mean,std = get_mean_std(uttrSpectro) # get_mean_std : my function which returns mean and std
uttrSpectro = (uttrSpectroList-mean)/std
Error Message : could not broadcast input array from shape (801,284) into shape (801)
How can I do this calculation without for-loop?

Broadcasting error when forming numpy array with elements as two other numpy arrays

I am trying to generate a numpy array with elements as two other numpy arrays, as below.
W1b1 = np.zeros((256, 161))
W2b2 = np.zeros((256, 257))
Wx = np.array([W1b1, W2b2], dtype=np.object)
this gives an error:
ValueError: could not broadcast input array from shape (256,161) into shape (256).
However, if I take entirely different dimensions for of W1b1 and W2b2 then I do not get an error, as below.
A1 = np.zeros((256, 161))
A2 = np.zeros((257, 257))
A3 = np.array([A1, A2], dtype=np.object)
I do not get what is wrong in the first code and why is numpy array trying to broadcast one of the input arrays.
I have tried on below versions (Python 2.7.6, Numpy 1.13.1) and (Python 3.6.4, Numpy 1.14.1).
Don't count on np.array(..., object) making the right object array. At the moment we don't have control over how many dimensions it makes. Conceivably it could make a (2,) array, or (2, 256) (with 1d contents). Sometimes it works, sometimes raises an error. There's something of a pattern, but I haven't seen an analysis of the code that shows exactly what is happening.
For now it is safer to allocate the array, and fill it:
In [57]: arr = np.empty(2, object)
In [58]: arr[:] = [W1b1, W2b2]
np.array([np.zeros((3,2)),np.ones((3,4))], object) also raises this error. So the error arises when the first dimensions match, but the later ones don't. Now that I think about, I've seen this error before.
Earlier SO questions on the topic
numpy array 1.9.2 getting ValueError: could not broadcast input array from shape (4,2) into shape (4)
Creation of array of arrays fails, when first size of first dimension matches
Creating array of arrays in numpy with different dimensions

regarding proper way of broadcasting input array from one shape into another shape

There is a function, I got the following message while calling this function
The function is to resize a given image set, and put the transformed ones into the new set imgs_p.
For instance, the input imgs has shape (5635,1,420,580). I want to transform it (5635,64,80,1). This is what I did as follows, but i got the error message as ValueError: could not broadcast input array from shape (80,64) into shape (80,1)
How to solve this problem? Thanks.
def preprocess(imgs):
imgs_p = np.ndarray((imgs.shape[0],img_rows, img_cols,imgs.shape[1]), dtype=np.uint8)
print('imgs_p: ',imgs_p.shape)
for i in range(imgs.shape[0]):
print('imgs[i,0]: ',imgs[i,0].shape)
imgs_p[i,0]=resize(imgs[i,0],(img_rows,img_cols))
return imgs_p
I presume you want to roll the "1" dimension to the correct position:
z = np.moveaxis(z, 1, -1).shape
whereafter you can run a for-loop over each image and reshape, either using skimage or scipy.ndimage.
Be careful with downsampling! You probably want to apply a Gaussian blur first to make sure all the data is taken into account.

TypeError from theano While using 3D numpy array

I am trying something similar to code below
datax=theano.shared(value=rng.rand(5,500,45))
x=T.dmatrix('x')
i=T.lscalar('i')
W=theano.shared(value=rng.rand(90,45,500))
Hb=theano.shared(value=np.zeros(90))
w_v_bias=T.dot(W,x).sum(axis=2).sum(axis=1)+Hb
z=theano.function([i],w_v_bias,givens={x:datax[i*5:(i+1)*5]})
z(0)
Theano is giving me a TypeError with msg:
Cannot convert Type TensorType(float64, 3D) (of Variable Subtensor{int64:int64:}.0) into Type TensorType(float64, matrix). You can try to manually convert Subtensor{int64:int64:}.0 into a TensorType(float64, matrix)
What I am doing wrong here?
Edit
As mentioned by daniel changing x to dtensor3 will result in another error.
ValueError: Input dimension mis-match. (input[0].shape[1] = 5, input[1].shape[1] = 90)
Apply node that caused the error: Elemwise{add,no_inplace}(Sum{axis=[1], acc_dtype=float64}.0, DimShuffle{x,0}.0)
Another way is to modify my train function but then I won't be able to do batch learning.
z=theano.function([x],w_v_bias)
z(datax[0])
I am trying to implement RBM with integer values for visible units.
The problem is that datax is a 3D tensor and datax[index*5:(index+1)*5] is also a 3D tensor but you're trying to assign that to x which is a 2D tensor (i.e. a matrix).
Changing
x = T.dmatrix('x')
to
x = T.dtensor3('x')
solves this problem but creates a new one because the dimensions of W and x don't match up to perform the dot product. It's unclear what the desired outcome is.
Solved it after few hit and trials.
What I needed was to change
x=T.dmatrix('x')
w_v_bias=T.dot(W,x).sum(axis=2).sum(axis=1)+Hb
to
x=T.dtensor3('x')
w_v_bias=T.dot(x,W).sum(axis=3).sum(axis=1)+Hb
Now it produces (5,90) array after adding Hb elementwise to each of the five vectors of dot product.

Categories

Resources