Develop Reference

Python Multiply Matrix by Vector of Matrices

Does anybody know of a way (preferably using numpy or something similar) to multiply a matrix by a vector of matrices and obtain the desired product shown below? Basically the idea is to follow the normal rules of matrix multplication of a matrix and a vector, only the elements of the vector are matrices themselves and not numbers.

If I understand the question correctly, you can try this:
import numpy as np
A = np.arange(3*3*3).reshape(3, 3, 3)
b = np.arange(9).reshape(3, 3)
print(f"A=\n{A}\n\nb=\n{b}")
It gives:
A=
[[[ 0 1 2]
[ 3 4 5]
[ 6 7 8]]
[[ 9 10 11]
[12 13 14]
[15 16 17]]
[[18 19 20]
[21 22 23]
[24 25 26]]]
b=
[[0 1 2]
[3 4 5]
[6 7 8]]
Then:
out = (b#A.transpose(2, 0, 1)).transpose(1, 2, 0)
print(out)
which gives:
[[[ 45 48 51]
[ 54 57 60]
[ 63 66 69]]
[[126 138 150]
[162 174 186]
[198 210 222]]
[[207 228 249]
[270 291 312]
[333 354 375]]]
The matrix out[0] is equal to 0*A[0] + 1*A[1] + 2*A[2], out[1] is equal to 3*A[0] + 4*A[1] + 5*A[2] etc.

Is this what you want to calculate:
# Define two matrices
A = np.arange(9).reshape(3, 3)
B = np.arange(9, 18).reshape(3, 3)
# First calculate the desired result:
rows = []
for i in range(3):
rows.append([A[i, j] * B for j in range(3)])
result = np.stack(rows).sum(axis=1)
assert(result.shape == (3, 3, 3))
print(result)
[[[ 27 30 33]
[ 36 39 42]
[ 45 48 51]]
[[108 120 132]
[144 156 168]
[180 192 204]]
[[189 210 231]
[252 273 294]
[315 336 357]]]
Is this correct?
If so, then here is the same calculation using numpy's einsum function:
C = np.array([B] * 3) # shape (3, 3, 3)
result = np.einsum("ij,jkl->ikl", A, C)

Boolean Indexing numpy Array with or logical operator

I was trying to do an or boolean logical indexing on a Numpy array but I cannot find a good way.
The and operator & works properly like:
X = np.arange(25).reshape(5, 5)
# We print X
print()
print('Original X = \n', X)
print()
X[(X > 10) & (X < 17)] = -1
# We print X
print()
print('X = \n', X)
print()
Original X =
[[ 0 1 2 3 4]
[ 5 6 7 8 9]
[10 11 12 13 14]
[15 16 17 18 19]
[20 21 22 23 24]]
X =
[[ 0 1 2 3 4]
[ 5 6 7 8 9]
[10 -1 -1 -1 -1]
[-1 -1 17 18 19]
[20 21 22 23 24]]
But when I try with:
X = np.arange(25).reshape(5, 5)
# We use Boolean indexing to assign the elements that are between 10 and 17 the value of -1
X[ (X < 10) or (X > 20) ] = 0 # No or condition possible!?!
I got the error:
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
Does exist any good way to use the or logic operator?

You can use numpy.logical_or for that task following way:
import numpy as np
X = np.arange(25).reshape(5,5)
X[np.logical_or(X<10,X>20)] = 0
print(X)
Output:
[[ 0 0 0 0 0]
[ 0 0 0 0 0]
[10 11 12 13 14]
[15 16 17 18 19]
[20 0 0 0 0]]
There is also numpy.logical_and, numpy.logical_xor and numpy.logical_not

I would use something with np.logical_and and np.where.
For your given example, I believe this would work.
X = np.arange(25).reshape(5, 5)
i = np.where(np.logical_and(X > 10 , X < 17))
X[i] = -1
This is not a very pythonic answer. But it's pretty clear

Deleting certain elements from a matrix

I have the following problem:
I have a matrix. Now, I want to delete one entry in each row of the matrix: In rows that contain a certain number (say 4) I want to delete the entry with that number, and in other rows I simply want to delete the last element.
E.g. if I have the matrix
matrix=np.zeros((2,2))
matrix[0,0]=2
matrix[1,0]=4
matrix
which gives
2 0
4 0
after the deletion it should simply be
2
0
thanks for your help!

so, assuming there's maximum only one 4 in a row, what you want to do is:
iterate all rows, and if there's a four use roll so it becomes the last element
delete the last column
in rows that have 4, it will delete this 4 and shift the remaining values that come after it,
in rows that don't have 4, it will delete the last element.
(I took the liberty of trying with a little bigger matrix just to make sure output is as expected)
try this:
import numpy as np
# Actual solution
def remove_in_rows(mat, num):
for i, row in enumerate(mat):
if num in row.tolist():
index = row.tolist().index(num)
mat[i][index:] = np.roll(row[index:], -1)
return np.delete(mat, -1, 1)
# Just some example to demonstrate it works
matrix = np.array([[10 * y + x for x in range(6)] for y in range(6)])
matrix[1, 2] = 4
matrix[3, 3] = 4
matrix[4, 0] = 4
print("BEFORE:")
print(matrix)
matrix = remove_in_rows(matrix, 4)
print("AFTER:")
print(matrix)
Output:
BEFORE:
[[ 0 1 2 3 4 5]
[10 11 4 13 14 15]
[20 21 22 23 24 25]
[30 31 32 4 34 35]
[ 4 41 42 43 44 45]
[50 51 52 53 54 55]]
AFTER:
[[ 0 1 2 3 5]
[10 11 13 14 15]
[20 21 22 23 24]
[30 31 32 34 35]
[41 42 43 44 45]
[50 51 52 53 54]]

Creating a dictionary column using combination of two columns in dataframe, and then computing ratio of values of two columns with common keys

I have a dataframe of the following format:
Id Name_prev Weight_prev Name_now Weight_now
1 [1,3,4,5] [10,34,67,37] [1,3,5] [45,76,12]
2 [10,3,40,5] [100,134,627,347] [10,40,5] [34,56,78]
3 [1,30,4,50] [11,22,45,67] [1,30,50] [12,45,78]
4 [1,7,8,9] [32,54,76,98] [7,8,9] [34,12,32]
I want to create two new variables:
Union of Name_prev and Name_now : This is intersection of Name_prev and Name_now fields and can be done using set operation on the two columns, I am able to compute the same.
Ratio of Name_prev and Name_now : This is ratio of values (Weight_prev and Weight_now) corresponding to common names in (Name_prev and Name_now).
Expected Output:
Id Union of Name_prev and Name_now Ratio of Name_prev and Name_now
1 [1,3,5] [10/45, 34/76,37/12]
2 [10,40,5] [100/34,627/56,347/78]
3 [1,30,50] [11/12,22/45,67/78]
4 [7,8,9] [54/34,76/12,98/32]
I am trying to create a dictionary like structure by combining Name_prev and Weigth_prev as key, value pair and doing the same for Name_now and Weight_now and then taking ratio for common keys, but am stuck...

Use:
a, b = [],[]
for n1, n2, w1, w2 in zip(df['Name_prev'], df['Name_now'],
df['Weight_prev'], df['Weight_now']):
#get intersection of lists
n = [val for val in n1 if val in n2]
#get indices by enumerate and select weights
w3 = [w1[i] for i, val in enumerate(n1) if val in n2]
w4 = [w2[i] for i, val in enumerate(n2) if val in n1]
#divide each value in list
w = [i/j for i, j in zip(w3, w4)]
a.append(n)
b.append(w)
df = df.assign(name=a, weight=b)
print (df)
Id Name_prev Weight_prev Name_now Weight_now \
0 1 [1, 3, 4, 5] [10, 34, 67, 37] [1, 3, 5] [45, 76, 12]
1 2 [10, 3, 40, 5] [100, 134, 627, 347] [10, 40, 5] [34, 56, 78]
2 3 [1, 30, 4, 50] [11, 22, 45, 67] [1, 30, 50] [12, 45, 78]
3 4 [1, 7, 8, 9] [32, 54, 76, 98] [7, 8, 9] [34, 12, 32]
name weight
0 [1, 3, 5] [0.2222222222222222, 0.4473684210526316, 3.083...
1 [10, 40, 5] [2.9411764705882355, 11.196428571428571, 4.448...
2 [1, 30, 50] [0.9166666666666666, 0.4888888888888889, 0.858...
3 [7, 8, 9] [1.588235294117647, 6.333333333333333, 3.0625]
If need remove original columns use DataFrame.pop:
a, b = [],[]
for n1, n2, w1, w2 in zip(df.pop('Name_prev'), df.pop('Name_now'),
df.pop('Weight_prev'), df.pop('Weight_now')):
n = [val for val in n1 if val in n2]
w3 = [w1[i] for i, val in enumerate(n1) if val in n2]
w4 = [w2[i] for i, val in enumerate(n2) if val in n1]
w = [i/j for i, j in zip(w3, w4)]
a.append(n)
b.append(w)
df = df.assign(name=a, weight=b)
print (df)
Id name weight
0 1 [1, 3, 5] [0.2222222222222222, 0.4473684210526316, 3.083...
1 2 [10, 40, 5] [2.9411764705882355, 11.196428571428571, 4.448...
2 3 [1, 30, 50] [0.9166666666666666, 0.4888888888888889, 0.858...
3 4 [7, 8, 9] [1.588235294117647, 6.333333333333333, 3.0625]
EDIT:
Working with lists in pandas is always not vectorized, so better is flatten lists first, merge and if necessary aggregate lists:
from itertools import chain
df_prev = pd.DataFrame({
'Name' : list(chain.from_iterable(df['Name_prev'].values.tolist())),
'Weight_prev' : list(chain.from_iterable(df['Weight_prev'].values.tolist())),
'Id' : df['Id'].values.repeat(df['Name_prev'].str.len())
})
print (df_prev)
Name Weight_prev Id
0 1 10 1
1 3 34 1
2 4 67 1
3 5 37 1
4 10 100 2
5 3 134 2
6 40 627 2
7 5 347 2
8 1 11 3
9 30 22 3
10 4 45 3
11 50 67 3
12 1 32 4
13 7 54 4
14 8 76 4
15 9 98 4
df_now = pd.DataFrame({
'Name' : list(chain.from_iterable(df['Name_now'].values.tolist())),
'Weight_now' : list(chain.from_iterable(df['Weight_now'].values.tolist())),
'Id' : df['Id'].values.repeat(df['Name_now'].str.len())
})
print (df_now)
Name Weight_now Id
0 1 45 1
1 3 76 1
2 5 12 1
3 10 34 2
4 40 56 2
5 5 78 2
6 1 12 3
7 30 45 3
8 50 78 3
9 7 34 4
10 8 12 4
11 9 32 4
df = df_prev.merge(df_now, on=['Id','Name'])
df['Weight'] = df['Weight_prev'] / df['Weight_now']
print (df)
Name Weight_prev Id Weight_now Weight
0 1 10 1 45 0.222222
1 3 34 1 76 0.447368
2 5 37 1 12 3.083333
3 10 100 2 34 2.941176
4 40 627 2 56 11.196429
5 5 347 2 78 4.448718
6 1 11 3 12 0.916667
7 30 22 3 45 0.488889
8 50 67 3 78 0.858974
9 7 54 4 34 1.588235
10 8 76 4 12 6.333333
11 9 98 4 32 3.062500
df = df.groupby('Id')['Name','Weight'].agg(list).reset_index()
print (df)
Id Name Weight
0 1 [1, 3, 5] [0.2222222222222222, 0.4473684210526316, 3.083...
1 2 [10, 40, 5] [2.9411764705882355, 11.196428571428571, 4.448...
2 3 [1, 30, 50] [0.9166666666666666, 0.4888888888888889, 0.858...
3 4 [7, 8, 9] [1.588235294117647, 6.333333333333333, 3.0625]

Numpy array operation using another indices array

I want to do a multidimensional array operation using numpy on three arrays, of which one is an index array, e.g.:
a = numpy.arange(20).reshape((5, 4))
# a = [[ 0 1 2 3] [ 4 5 6 7] [ 8 9 10 11] [12 13 14 15] [16 17 18 19]]
b = numpy.arange(24).reshape(((3, 2, 4)))
# b = [[[ 0 1 2 3] [ 4 5 6 7]] [[ 8 9 10 11] [12 13 14 15]] [[16 17 18 19] [20 21 22 23]]]
c = numpy.array([0,0,1,1,2])
# c = [0 0 1 1 2]
now, what I want is:
d = a * b[&] + b[&&]
where & is the second element of second dimension of b (e.g. [ 4 5 6 7]) and && is the first element of second dimension (e.g. [ 0 1 2 3]) related to i-th item of the first dimension of b, where i is from array c (e.g. c[0]=0 for the first element of first dimension of array b). d has same dimension as a.
Edit: Answer for the above example is:
# d = [[0 6 14 24] [16 26 38 52] [104 126 150 176] [152 178 206 236] [336 374 414 456]]
Thanks

>>> a * b[c,1,:] + b[c,0,:]
array([[ 0, 6, 14, 24],
[ 16, 26, 38, 52],
[104, 126, 150, 176],
[152, 178, 206, 236],
[336, 374, 414, 456]])