Develop Reference

Numpy flatten a nested array using concatenate

I have a numpy array with subarrays of different shapes. I was trying to use an iterator to flatten them into a 1D array. Below is the code:
import numpy as np
a=np.array([np.random.rand(1,2),np.random.rand(2,2),np.random.rand(1,4)],dtype=object)
b=np.concatenate(x.ravel for x in a)
This returns an error:
TypeError: The first input argument needs to be a sequence
I am not quite sure what I am doing incorrectly. It works fine when I create a for loop with the same logic and keep concatenating my array recursively. Any help appreciated.
The goal is to flatten the array into a 1D array. (Note that hstack doesn't work because the arrays are of different shapes. flatten doesn't work because it is already a 1D array (of arrays).)

b=np.concatenate([x.ravel() for x in a])
print(b)
array([0.0928126 , 0.26396728, 0.37416516, 0.86079876, 0.3070049 ,
0.86714361, 0.67955231, 0.11715076, 0.34659847, 0.17392114])

Check if numpy array has a normal shape

How do I check if a numpy array has a regular shape.
In the example below x is a *2 by 3* matrix. However y is not regular in the sense that it can't be represented as a proper matrix.
Given that I have a numpy array, is there a method (preferably in-built) that I can use to check that the numpy array is an actual matrix
In [9]: import numpy as np
In [10]: x = np.array([[1,2,3],[4,5,6]])
In [11]: x.shape
Out[11]: (2, 3)
In [12]: y = np.array([[1,2,3],[4,5]])
In [13]: y.shape
Out[13]: (2,)

Both are arrays and those are valid shapes. But, with normal, think you meant that each element has the same shape and length across it. For that, a better way would be to check for the datatype. For the variable length case, it would be object. So, we can check for that condition and call out accordingly. Hence, simply do -
def is_normal_arr(a): # a is input array to be tested
return a.dtype is not np.dtype('object')

I think the .shape method is capable of checking it.
If you input an array which can form a matrix it returns it's actual shape, (2, 3) in your case. If you input an incorrect matrix it returns something like (2,), which says something's wrong with the second dimension, so it can't form a matrix.

Here y is a one-dimensional array and the size of y is 2. y contains 2 list values.
AND x is our actual matrix in a proper format.
check the dimensions by y.ndim AND x.ndim.

Broadcasting error when forming numpy array with elements as two other numpy arrays

I am trying to generate a numpy array with elements as two other numpy arrays, as below.
W1b1 = np.zeros((256, 161))
W2b2 = np.zeros((256, 257))
Wx = np.array([W1b1, W2b2], dtype=np.object)
this gives an error:
ValueError: could not broadcast input array from shape (256,161) into shape (256).
However, if I take entirely different dimensions for of W1b1 and W2b2 then I do not get an error, as below.
A1 = np.zeros((256, 161))
A2 = np.zeros((257, 257))
A3 = np.array([A1, A2], dtype=np.object)
I do not get what is wrong in the first code and why is numpy array trying to broadcast one of the input arrays.
I have tried on below versions (Python 2.7.6, Numpy 1.13.1) and (Python 3.6.4, Numpy 1.14.1).

Don't count on np.array(..., object) making the right object array. At the moment we don't have control over how many dimensions it makes. Conceivably it could make a (2,) array, or (2, 256) (with 1d contents). Sometimes it works, sometimes raises an error. There's something of a pattern, but I haven't seen an analysis of the code that shows exactly what is happening.
For now it is safer to allocate the array, and fill it:
In [57]: arr = np.empty(2, object)
In [58]: arr[:] = [W1b1, W2b2]
np.array([np.zeros((3,2)),np.ones((3,4))], object) also raises this error. So the error arises when the first dimensions match, but the later ones don't. Now that I think about, I've seen this error before.
Earlier SO questions on the topic
numpy array 1.9.2 getting ValueError: could not broadcast input array from shape (4,2) into shape (4)
Creation of array of arrays fails, when first size of first dimension matches
Creating array of arrays in numpy with different dimensions

Element-wise Multiplication of (x,y,1) * (x,y)

matrixADimensions = matrixA.shape # returns [901,1249,1]
matrixBDimensions = matrixB.shape # returns [901,1249]
I am trying to get the element-wise multiplication of matrixA and matrixB but I am getting the error ValueError: operands could not be broadcast together with shapes (901,1249,1) (901,1249).
I believe it has something to do with the dimensions of both matrices since they are not the same. Actually, technically they are the same since [901,1249,1] is the same thing as [901,1249] but Python does not seem to know this.
How can I multiply matrixA with matrixB?

You can use numpy.squeeze to remove single-dimensional entries from the shape of your array. So in your case, you would do:
import numpy as np
np.squeeze(matrixA) * matrixB
This has the advantage of not needing to know the position of your single-dimensional entry in your array shape (unlike taking an indexing approach such as matrixA[:,:,0]).

why unable to concatenate two arrays in python?

I have two arrays
>>> array1.shape
(97, 195)
>>> array2.shape
(195,)
>>> array1 = numpy.concatenate((array1, array2), axis=0)
when I perform concatenate operation it shows an error
ValueError: all the input arrays must have same number of dimensions
is that the second array shape (195,) creating problem?

Just make both have the same dimensions and the same size except along the axis to be concatenated:
np.concatenate((array1, array2[np.newaxis,...]), axis=0)

In order for this to work, you need array2 to actually be 2d.
array1 = numpy.concatenate((array1, array2.reshape((1,195)))
should work

Another easy way to achieve the array concatenation that you’re looking for is to use Numpy’s vstack function as follows:
array1 = np.vstack([array1, array2])