Develop Reference

How to vectorize passing a function to two numpy arrays: 3D and 2D?

I have two multidimensional numpy arrays: x is 3D and y is 2D.
If I have a function foo(a, b), which takes as inputs two 2D arrays, how can I pass to foo my multidimensional arrays and iterate over x's 3rd dimension in a vectorized way in order to get a list of foo's results?
I have been trying to do this with np.vectorize, but it iterates through the rows of the arrays and yields an error, so I am stuck.

You can specify the function's signature using the signature keyword. This will, however, try to use the last dimensions of each input, so you'd have to manually transpose. Example
F = np.vectorize(np.matmul, signature='(m,n),(n,l)->(m,l)', otypes=(float,))
A = np.arange(12).reshape(2, 2, 3)
B = np.diag((1.5, 2.5))
# F(A.transpose(2,0,1), B)
# array([[[ 0. , 7.5],
# [ 9. , 22.5]],
#
# [[ 1.5, 10. ],
# [10.5, 25. ]],
#
# [[ 3. , 12.5],
# [12. , 27.5]]])
As pointed out by #hpaulj in the comments vectorize is a convenience function, not a performance enhancer.

Matplotlib 4D data in a 2D array

I am trying to plot a 4D array using as 4th dimension the color. Here is a sample of my matrix:
[[ 4.216 0. 1. 0. ]
[ 5.36 0. 1. 0. ]
[ 5.374 0. 2. 0. ]
...,
[ 0.294 0. 1. 0. ]
[ 0.314 0. 2. 0. ]
[ 0.304 0. 1. 0. ]]
4th column only contains values 0, 1 and 2.
So when I try to plot it using this script:
fig = plt.figure()
ax = fig.add_subplot(111, projection='3d')
ax.scatter(data[:,0],data[:,1],data[:,2], c=data[:,3], cmap=plt.hot())
plt.show()
I am getting this error:
TypeError: can't multiply sequence by non-int of type 'float'

This isn't a 4D array. It's a 2D array with 4 columns (the 2 dimensions could be referred to as "rows" and "columns"). But I see what you're trying to say—each row could be interpreted as describing a point in 4-dimensional space, with the fourth "dimension" being colour.
Two-dimensionality is actually the key to the problem. I suspect your data variable is a numpy.matrix rather than a vanilla numpy.array. A matrix is particular class of 2D-array that has various special properties, including the fact that a slice of it (for example, data[:, 0]) is still a 2-dimensional matrix object, whereas .scatter() expects each argument to be a 1-D array.
The fix is to say:
data = numpy.asarray(data)
to convert your data from a matrix to a normal array whose column slices will be 1-dimensional.
BTW: you probably meant to say cmap='hot'. The call to plt.hot() sets the default colormap (so your figure may look right, but there's a side effect) but it actually returns None.

Numpy matrix multiplication behaviour

I have a problem to understand the matrix multiplication in numpy.
For example I have the following matrix (2d numpy array):
a = [ [ 1. 1. ]
[ 1. 2. ]
[ 1. 3. ] ]
And the following row vector theta:
theta = [ 1. 1. ]
The only way to multiply a with theta would be to transform
theta in a column vector first and then I would get the result:
result = [ [ 2. ]
[ 3. ]
[ 4. ] ]
When I multiply the matrix and the row vector (without transforming)
result = np.dot(a,theta)
I get this:
result = [ 2. 3. 4. ]
How is this even possible? I mean, I didn't transform the matrix.
Can you please tell me how this numpy multiplication works?
Thank you for your attention.

No, you're multiplying numpy array with another numpy array (not a matrix with a vector), although it looks like that. This is because, in essence, numpy arrays are not the same as matrices. And the dot product treats it that way as well.
If you write out the array and multiply, then you will see why. It's just the dot product (element-wise multiplication) of each row in the array 'a' with the vector 'theta'.
PS: (matrices are 2-D while arrays are not limited to any dimension)
Also, please take a look at this answer and this excellent answer

Meaning of the return of np.shape()

I have a program in numpy utf8, which allows me to calculate the coordinates of a parabolic shot from the ground. I need to create a function which returns the coordinates (#1), create the different arrays of values to work with (#2), and finally use the function to generate the different coordinates for each pack of values
#1
def coordenadas(v,a,t,g=9.81):
rad=deg2rad(a)
x=v*cos(a)*t
y=v*sin(a)*t-(1./2.)*g*(t**2)
xy=array([x,y]).T
return xy
#2
v=arange(50,100,10) #m/s
adegree=arange(45,90,5) #degrees
a=deg2rad(adegree) #rads
t=linspace(0,10,50) #segundos
#3
v.shape=(5,1,1)
a.shape=(1,9,1)
t.shape=(1,1,50)
#5
XY=coordenadas(v,a,t,g=9.81)
print shape(XY)
print XY
#4
My question is that shape(XY) returns
(50L, 9L, 5L, 2L)
And XY (only a bit, is too long)
[[[[ 0. 0. ]
[ 0. 0. ]
[ 0. 0. ]
[ 0. 0. ]
[ 0. 0. ]]
And more boxes of this shape
What this really means(big boxes, boxes, small boxes, rows, columns) ???

(50L, 9L, 5L, 2L) means a 4D array.
You can visualize as a 50x9 matrix and each cell of this matrix contains a 5x2 matrix

Numpy arrays are basically matrices, where each box [] represents the start of a new dimension. As an easy example the matrix
11
23
could be written in numpy as:
a = numpy.array([[1,1],[2,3]])
which then would be printed as
array([[1, 1],
[2, 3]])
As this is a two-dimensional matrix, the outer "box" marks the edges of the matrix, whereas the inner boxes are the rows of the matrix with the , separating the entries. Calling a.shape without an argument gives the shape of the 2x2 matrix:
(2, 2)
Calling the shape method with argument reshapes the matrix given to the shape defined in the argument. But to further help you with the code:
1
Your function definition seems to be totally fine, except I don't see a reason, why you export x and y in an array, rather than just returning two different values.
2
The initialization of your arrays seem to be fine as well.
3
There is totally no reason to reshape the arrays you just created, just leave them as they are.
4
You have to call the function separately with each set of values to create the coordinates. Do that by using an itteration over the arrays you just created.

Reading 2d arrays into a 3d array in python

I searched stackoverflow but could not find an answer to this specific question. Sorry if it is a naive question, I am a newbie to python.
I have several 2d arrays (or lists) that I would like to read into a 3d array (list) in python. In Matlab, I can simply do
for i=1:N
# read 2d array "a"
newarray(:,:,i)=a(:,:)
end
so newarray is a 3d array with "a" being the 2d slices arranged along the 3rd dimension.
Is there a simple way to do this in python?
Edit: I am currently trying the following:
for file in files:
img=mpimg.imread(file)
newarray=np.array(0.289*cropimg[:,:,0]+0.5870*cropimg[:,:,1]+0.1140*cropimg[:,:,2])
i=i+1
I tried newarray[:,:,i] and it gives me an error
NameError: name 'newarray' is not defined
Seems like I have to define newarray as a numpy array? Not sure.
Thanks!

If you're familiar with MATLAB, translating that into using NumPy is fairly straightforward.
Lets say you have a couple arrays
a = np.eye(3)
b = np.arange(9).reshape((3, 3))
print(a)
# [[ 1. 0. 0.]
# [ 0. 1. 0.]
# [ 0. 0. 1.]]
print(b)
# [[0 1 2]
# [3 4 5]
# [6 7 8]]
If you simply want to put them into another dimension, pass them both to the array constructor in an iterable (e.g. a list) like so:
x = np.array([a, b])
print(x)
# [[[ 1. 0. 0.]
# [ 0. 1. 0.]
# [ 0. 0. 1.]]
#
# [[ 0. 1. 2.]
# [ 3. 4. 5.]
# [ 6. 7. 8.]]]
Numpy is smart enough to recognize the arrays are all the same size and creates a new dimension to hold it all.
print(x.shape)
# (2, 3, 3)
You can loop through it, but if you want to apply the same operations to it across some dimensions, I would strongly suggest you use broadcasting so that NumPy can vectorize the operation and it runs a whole lot faster.
For example, across one dimension, lets multiply one slice by 2, another by 3. (If it's not a pure scalar, we need to reshape the array to the same number of dimensions to broadcast, then the size on each needs to either match the array or be 1). Note that I'm working along the 0th axis, your image is probably different. I don't have a handy image to load up to toy with
y = x * np.array([2, 3]).reshape((2, 1, 1))
print(y)
#[[[ 2. 0. 0.]
# [ 0. 2. 0.]
# [ 0. 0. 2.]]
#
# [[ 0. 3. 6.]
# [ 9. 12. 15.]
# [ 18. 21. 24.]]]
Then we can add them up
z = np.sum(y, axis=0)
print(z)
#[[ 2. 3. 6.]
# [ 9. 14. 15.]
# [ 18. 21. 26.]]

If you're using NumPy arrays, you can translate almost directly from Matlab:
for i in range(1, N+1):
# read 2d array "a"
newarray[:, :, i] = a[:, :]
Of course you'd probably want to use range(N), because arrays use 0-based indexing. And obviously you're going to need to pre-create newarray in some way, just as you'd have to in Matlab, but you can translate that pretty directly too. (Look up the zeros function if you're not sure how.)
If you're using lists, you can't do this directly—but you probably don't want to anyway. A better solution would be to build up a list of 2D lists on the fly:
newarray = []
for i in range(N):
# read 2d list of lists "a"
newarray.append(a)
Or, more simply:
newarray = [read_next_2d_list_of_lists() for i in range(N)]
Or, even better, make that read function a generator, then just:
newarray = list(read_next_2d_list_of_lists())
If you want to transpose the order of the axes, you can use the zip function for that.