Develop Reference

vectorize a function on a 3D numpy array using a specific signature

I'd like to apply a function f(x, y) on a numpy array a of shape (N,M,2), whose last axis (2) contains the variables x and y to give in input to f.
Example.
a = np.array([[[1, 1],
[2, 1],
[3, 1]],
[[1, 2],
[2, 2],
[3, 2]],
[[1, 3],
[2, 3],
[3, 3]]])
def function_to_vectorize(x, y):
# the function body is totaly random and not important
if x>2 and y-x>0:
sum = 0
for i in range(y):
sum+=i
return sum
else:
sum = y
for i in range(x):
sum-=i
return sum
I'd like to apply function_to_vectorize in this way:
[[function_to_vectorize(element[0], element[1]) for element in vector] for vector in a]
#array([[ 1, 0, -2],
# [ 2, 1, -1],
# [ 3, 2, 3]])
How can I vectorize this function with np.vectorize?

With that function, the np.vectorize result will also expect 2 arguments. 'signature' is determined by the function, not by the array(s) you expect to supply.
In [184]: f = np.vectorize(function_to_vectorize)
In [185]: f(1,2)
Out[185]: array(2)
In [186]: a = np.array([[[1, 1],
...: [2, 1],
...: [3, 1]],
...:
...: [[1, 2],
...: [2, 2],
...: [3, 2]],
...:
...: [[1, 3],
...: [2, 3],
...: [3, 3]]])
Just supply the 2 columns of a:
In [187]: f(a[:,:,0],a[:,:,1])
Out[187]:
array([[ 1, 0, -2],
[ 2, 1, -1],
[ 3, 2, 0]])

Using Numpy to write a function that returns all rows of A that have completely distinct entries

For example:
For
A = np.array([
[1, 2, 3],
[4, 4, 4],
[5, 6, 6]])
I want to get the output
array([[1, 2, 3]]).
For A = np.arange(9).reshape(3, 3),
I want to get
array([[0, 1, 2],
[3, 4, 5],
[6, 7, 8]])
WITHOUT using loops

You can use:
B = np.sort(A, axis=1)
out = A[(B[:,:-1] != B[:, 1:]).all(1)]
Or using pandas:
import pandas as pd
out = A[pd.DataFrame(A).nunique(axis=1).eq(A.shape[1])]
Output:
array([[1, 2, 3]])

Reference to ndarray rows in ndarray

is it possible to store references of specific rows of an numpy array in another numpy array?
I have an array of 2D nodes, e.g.
nodes = np.array([[1, 2], [2, 3], [3, 4], [4, 5], [5, 6]])
Now I want to select only a few of them and store a reference in another numpy array:
nn = np.array([nodes[0], nodes[3]])
If I modify a entry in nn the array nodes remains unchanged. Is there a way to store a reference to nodes in the ndarray nn?

If the reference can be created with basic indexing/slicing, then you get a view (an array that does not own its data, but refers to another array’s data instead) of the initial array where changes propagate:
>>> nn = nodes[0:4:3] # reference array for rows 0 and 3
>>> nn[0][0] = 0
>>> nodes
array([[0, 2],
[2, 3],
[3, 4],
[4, 5],
[5, 6]])
Otherwise, you get a copy from the original array as in your code, and updates do not propagate to the initial array.

You can store an index to the rows you want in a numpy array:
ref = np.array([0, 3])
You can use the reference in an indexing expression to access the nodes you want:
>>> nn = nodes[ref]
>>> nn
array([[1, 2],
[4, 5]])
nn will be a deep copy with no connection to the original in this case. While nn[foo] = bar won't affect the original array, you can use ref directly:
>>> nodes[ref, 1] = [17, 18]
>>> nodes
array([[ 1, 17],
[ 2, 3],
[ 3, 4],
[ 4, 18],
[ 5, 6]])
Alternatively, you can use a mask for ref:
>>> ref2 = np.zeros(nodes.shape[0], dtype=np.bool)
>>> ref2[ref] = True
>>> ref2
array([ True, False, False, True, False], dtype=bool)
You can do almost all the same operations:
>>> nn2 = nodes[ref2]
>>> nn2
array([[1, 2],
[4, 5]])
Modifications work too:
>>> nodes[ref2, 1] = [19, 23]
>>> nodes
array([[ 1, 19],
[ 2, 3],
[ 3, 4],
[ 4, 23],
[ 5, 6]])
The only thing that is more convenient with an array of indices is selecting a particular node from within the selection:
>>> nodes[ref[0], 0]
1

In Numpy, you can get a view of an array that can be edited. In your example, you can do this:
import numpy as np
nodes = np.array([[1, 2], [2, 3], [3, 4], [4, 5], [5, 6]])
node_idx = np.array([0, 3])
nodes[node_idx] = np.array([[1, 5], [2, 5]])
nodes
Output:
array([[1, 5],
[2, 3],
[3, 4],
[2, 5],
[5, 6]])
You can also replace it with boolean arrays:
import numpy as np
nodes = np.array([[1, 2], [2, 3], [3, 4], [4, 5], [5, 6]])
node_mask = np.array([True, False, False, True, False])
nodes[node_mask] = np.array([[1, 5], [2, 5]])
nodes
Which produces the same result. Of course, this means you can do magic like this:
import numpy as np
nodes = np.array([[1, 2], [2, 3], [3, 4], [4, 5], [5, 6]])
nodes[nodes[:, 0] == 3] = [1, 5]
nodes
Which replaces all rows with the first element equal to 3 with [1, 5]. Output:
array([[1, 2],
[2, 3],
[1, 5],
[4, 5],
[5, 6]])

Method 1
First, initialize a Numpy array of None with dtype=object. (It don't have to be None. My guess it that you just cannot put references at initialization as Numpy somehow just creates an deep copy of it.)
Then, put the reference into the array.
nodes = np.array([[1, 2], [2, 3], [3, 4], [4, 5], [5, 6]])
# nn = np.array([nodes[0], nodes[1]],dtype=object) would not work
nn = np.array([None, None], dtype=object)
nn[0] = nodes[0]
nn[1] = nodes[3]
# Now do some modification.
nn[0][1] = 100
Output of nodes:
array([[ 1, 100],
[ 2, 3],
[ 3, 4],
[ 4, 5],
[ 5, 6]])
# make it a function
def make_ref(old_array, indeces):
ret = np.array([None for _ in range(len(indeces))])
for i in range(len(indeces)):
ret[i] = old_array[indeces[i]]
return ret
nn = make_ref(nodes, [0, 3])
Method 2
If you don't need to put it in Numpy arrays, just use a list to host the references.
nn = [nodes[0], nodes[1]]

Numpy matrix minus a vector with repeated row/column indices

>>> a
array([[1, 2],
[3, 4],
[5, 6]])
>>>b
array([[1, 2],
[4, 3]])
What I want is doing
b[0,:]-=a[0,:], b[1,:]-=a[1,:], b[0,:]-=a[2,:]
But the following doesn't work:
>>> b[[0,1,0],:]-=a
>>> b
array([[-4, -4], # want [-5, -6]
[ 1, -1]])
How can this be achieved without using a for loop?

Use np.subtract.at -
np.subtract.at(b,[0,1,0],a)
Sample run -
In [15]: a
Out[15]:
array([[1, 2],
[3, 4],
[5, 6]])
In [16]: b
Out[16]:
array([[1, 2],
[4, 3]])
In [17]: np.subtract.at(b,[0,1,0],a)
In [18]: b
Out[18]:
array([[-5, -6],
[ 1, -1]])

Python - Matrix outer product

Given two matrices
A: m * r
B: n * r
I want to generate another matrix C: m * n, with each entry C_ij being a matrix calculated by the outer product of A_i and B_j.
For example,
A: [[1, 2],
[3, 4]]
B: [[3, 1],
[1, 2]]
gives
C: [[[3, 1], [[1 ,2],
[6, 2]], [2 ,4]],
[9, 3], [[3, 6],
[12,4]], [4, 8]]]
I can do it using for loops, like
for i in range (A.shape(0)):
for j in range (B.shape(0)):
C_ij = np.outer(A_i, B_j)
I wonder If there is a vectorised way of doing this calculation to speed it up?

The Einstein notation expresses this problem nicely
In [85]: np.einsum('ac,bd->abcd',A,B)
Out[85]:
array([[[[ 3, 1],
[ 6, 2]],
[[ 1, 2],
[ 2, 4]]],
[[[ 9, 3],
[12, 4]],
[[ 3, 6],
[ 4, 8]]]])

temp = numpy.multiply.outer(A, B)
C = numpy.swapaxes(temp, 1, 2)
NumPy ufuncs, such as multiply, have an outer method that almost does what you want. The following:
temp = numpy.multiply.outer(A, B)
produces a result such that temp[a, b, c, d] == A[a, b] * B[c, d]. You want C[a, b, c, d] == A[a, c] * B[b, d]. The swapaxes call rearranges temp to put it in the order you want.

Simple Solution with Numpy Array Broadcasting
Since, you want C_ij = A_i * B_j, this can be achieved simply by numpy broadcasting on element-wise-product of column-vector-A and row-vector-B, as shown below:
# import numpy as np
# A = [[1, 2], [3, 4]]
# B = [[3, 1], [1, 2]]
A, B = np.array(A), np.array(B)
C = A.reshape(-1,1) * B.reshape(1,-1)
# same as:
# C = np.einsum('i,j->ij', A.flatten(), B.flatten())
print(C)
Output:
array([[ 3, 1, 1, 2],
[ 6, 2, 2, 4],
[ 9, 3, 3, 6],
[12, 4, 4, 8]])
You could then get your desired four sub-matrices by using numpy.dsplit() or numpy.array_split() as follows:
np.dsplit(C.reshape(2, 2, 4), 2)
# same as:
# np.array_split(C.reshape(2,2,4), 2, axis=2)
Output:
[array([[[ 3, 1],
[ 6, 2]],
[[ 9, 3],
[12, 4]]]),
array([[[1, 2],
[2, 4]],
[[3, 6],
[4, 8]]])]

Use numpy;
In [1]: import numpy as np
In [2]: A = np.array([[1, 2], [3, 4]])
In [3]: B = np.array([[3, 1], [1, 2]])
In [4]: C = np.outer(A, B)
In [5]: C
Out[5]:
array([[ 3, 1, 1, 2],
[ 6, 2, 2, 4],
[ 9, 3, 3, 6],
[12, 4, 4, 8]])
Once you have the desired result, you can use numpy.reshape() to mold it in almost any shape you want;
In [6]: C.reshape([4,2,2])
Out[6]:
array([[[ 3, 1],
[ 1, 2]],
[[ 6, 2],
[ 2, 4]],
[[ 9, 3],
[ 3, 6]],
[[12, 4],
[ 4, 8]]])