I have an array, which is of shape (210000,64). I would like to transfer its shape to (210000,64,1) and moreover, I want to generate a new array, which is a random permutation of the original array along the dimension of 210000.
Related
I have a two-dimensional np.array, where cells are filled with floats or 1d arrays.
In the two-dimensional array, the first dimension are samples, the second dimension are sample descriptions from different sources. Each cell is a string, represented as an ASCII-encoded array or floats.
Example:
array([[3.2, array([1,2,5,1]), array([1,6,9]), array([1,2])],
[2.1, array([1,2,9]), array([8,3,5,8]), array([1,3])],
[1.2, array([1,1]), array([4,2,6,4,5]), array([2,2,4])]])
The first three columns are my inputs, the fourth is my output.
I want to feed a seq2seq LSTM in TensorFlow with this data.
As first approach, I've tried to convert each 1d array in cells to a Tensor but I get an error:
ValueError: Failed to convert a NumPy array to a Tensor (Unsupported object
type tensorflow.python.framework.ops.EagerTensor).
I'm wondering if it is necessary to unpack the 1d arrays in cells to a new dimension. How can that be done, considering 1d arrays in cells have different lenghts?
Somewhere, I've read that using batch_size=1 is it possible to feed LSTM with arrays of different dimensions. Does someone have experience with that?
Thanks for your help.
I have 2 arrays
The first one has this shape
(4133,10000,12)
and the second one has this shape:
(4133,2)
I want to combine those two arrays so I get this shape
(4133,10000,12,2)
Shape of an array along a dimension is NOT the total number of elements. It is the number of elements PER corresponding dimension. Thus, you cannot concatenate arrays of shapes (4133,10000,12) and (4133,2) to have an array of shape (4133,10000,12,2). An easier example to think of is two matrices of shapes (m,n) and (m,k). You cannot concatenate them to have an array of shape (m,n,k).
I don't know what you mean by combine but you can reshape the arrays then let broadcasting kick in. For example:
x = np.empty((10,8,4))
y = np.empty((10,2))
combined = x.reshape((10,8,4,1))*y.reshape((10,1,1,2))
print(combined.shape)
# (10,8,4,2)
I've got a numpy_array of size (3275412, 50, 22) which represents my data reshaped for LSTM purposes and I have got a target vector of shape (3275412,).
I want to balance my data so that there is approximately the same number of data with target 0 and 1.
The way I prepared the data makes that I can not do this balancing operation before reshaping.
Firstly, I wanted to apply make_imbalance function (see this link for details) but I can't apply it on a 2-D array (got an error).
My question is : what's the most efficient way to do it for a 3D array ?
My thoughts: I thought about firstly "flatten" my 3-D array to a 2-D array by "concatenating" the second and third dimension (but don't know how so please tell me ??) then apply make_imbalance and then reshape the result to a 3-D array (again, don't know how to do). It seems a little bit tricky however...
So any help would be appreciated, either for an other imbalancing method or for help about reshaping 3D->2D or vice-versa
You can use np.reshape with -1 for unknown dimension size.
data2d = data3d.reshape(data3d.shape[0], -1)
will give you a 2d array of shape (n_samples, n_features)
with the second and the third dimensions merged.
data2d_new, y_new = make_imbalance(data2d, y)
After make_imbalance call, you will get a 2d array with a shape (n_samples_new, n_features), where the number of rows is "unknown" but you know your other two 'feature' dimensions of the original 3d array, so
data3d_new = data2d.reshape(-1, data3d.shape[1], data3d.shape[2])
will give you back the balanced 3d dataset.
I have two images , image 1 of dimension (32,43,3) and image2 of dimension (67,86,3) . How can i store this in a numpy array , Whenever i try to append the array
image=cv2.imread(image1,0)
image=cv2.resize(image,(32,43))
x_train=np.array(image.flatten())
x_train=x_train.reshape(-1,3,32,43)
X_train =np.append(X_train,x_train) #X_train is my array
image=cv2.imread(image2,0)
image=cv2.resize(image,(67,86))
x_train=np.array(image.flatten())
x_train=x_train.reshape(-1,3,67,86)
X_train =np.append(X_train,x_train)
Value Error: total size of new array must be unchanged.
i want the X_train in shape (-1,depth,height,width).So that i can feed it into my neural network. Is there any way to store images of different dimension in array and feed into neural network ?
Don't use np.append. If you must join arrays, start with np.concatenate. It'll force you to pay more attention to the compatibility of dimensions.
You can't join 2 arrays with shapes (32,43,3) (67,86,3) to make a larger array of some compatible shape. The only dimension they share is the last.
These reshapes don't make sense either: (-1,3,32,43), (-1,3,67,86).
It works, but it also messes up the 'image'. You aren't just adding a 4th dimension. It looks like you want to do some axis swapping or transpose as well. Practice with some small arrays so you can see what's happening, e.g. (2,4,3).
What final shape do you expect for Xtrain?
You can put these two images in a object dtype array, which is basically the same as the list [image1, image2]. But I doubt if your neuralnet can do anything practical with that.
If you reshaped the (32,43,3) array to (16,86,3) you could concatenate that with (67,86,3) on axis=0 to produce a (83,86,3) array. If you needed the 3 to be first, I'd use np.transpose(..., (2,0,1)).
Conversely reshape (67,86,3) to (2*67,43,3).
Passing the (32,43,3) to (32,86,3) is another option.
Joining them on a new 4th dimension, requires that the number of 'rows' match as well as the number of 'columns'.
I m working on audio stuff with python and I have the data into a 2d array with left and right channels in two different dimensions
so my array shape looks like this (exemple) :
(360448, 2)
I'm trying to find a way to reshape the array so the data fits into a 3d array where each cell represent each sample in this way : [left_sample,right_sample,0] so the final array will have shape (X,X,3)
is there a way to reshape the array( without iterating and assigning the values) ?