Develop Reference

In python numpy, how to replace some rows in array A with array B if we know the index

In python numpy, how to replace some rows in array A with array B if we know the index.
For example
we have
a = np.array([[1,2],[3,4],[5,6]])
b = np.array([[10,10],[1000, 1000]])
index = [0,2]
I want to change a to
a = np.array([[10,10],[3,4],[1000,1000]])
I have considered the funtion np.where but it need to create the bool condition, not very convenient,

I would do it following way
import numpy as np
a = np.array([[1,2],[3,4],[5,6]])
b = np.array([[10,10],[1000, 1000]])
index = [0,2]
a[index] = b
print(a)
gives output
[[ 10 10]
[ 3 4]
[1000 1000]]

You can use :
a[index] = b
For example :
import numpy as np
a = np.array([[1,2],[3,4],[5,6]])
b = np.array([[10,10],[1000, 1000]])
index = [0,2]
a[index] = b
print(a)
Result :
[[ 10 10]
[ 3 4]
[1000 1000]]

In Python's NumPy library, you can use the numpy.put() method to replace some rows in array A with array B if you know the index. Here's an example:
import numpy as np
# Initialize array A
A = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
# Initialize array B
B = np.array([[10, 20, 30], [40, 50, 60]])
# Indices of the rows to be replaced in array A
indices = [0, 1]
# Replace rows in array A with rows in array B
np.put(A, indices, B)
print(A)
In this example, the first two rows in array A are replaced with the first two rows in array B, so the output will be
[[10 20 30]
[40 50 60]
[ 7 8 9]]

Simply a[indices] = b or if you want to be more fancy np.put(a, indices, b)

Numpy python - calculating sum of columns from irregular dimension

I have a multi-dimensional array for scores, and for which, I need to get sum of each columns at 3rd level in Python. I am using Numpy to achieve this.
import numpy as np
Data is something like:
score_list = [
[[1,1,3], [1,2,5]],
[[2,7,5], [4,1,3]]
]
This should return:
[[3 8 8] [5 3 8]]
Which is happening correctly using this:
sum_array = np_array.sum(axis=0)
print(sum_array)
However, if I have irregular shape like this:
score_list = [
[[1,1], [1,2,5]],
[[2,7], [4,1,3]]
]
I expect it to return:
[[3 8] [5 3 8]]
However, it comes up with warning and the return value is:
[list([1, 1, 2, 7]) list([1, 2, 5, 4, 1, 3])]
How can I get expected result?

numpy will try to cast it into an nd array which will fail, instead consider passing each sublist individually using zip.
score_list = [
[[1,1], [1,2,5]],
[[2,7], [4,1,3]]
]
import numpy as np
res = [np.sum(x,axis=0) for x in zip(*score_list)]
print(res)
[array([3, 8]), array([5, 3, 8])]

Here is one solution for doing this, keep in mind that it doesn't use numpy and will be very inefficient for larger matrices (but for smaller matrices runs just fine).
# Create matrix
score_list = [
[[1,1,3], [1,2,5]],
[[2,7,5], [4,1,3]]
]
# Get each row
for i in range(1, len(score_list)):
# Get each list within the row
for j in range(len(score_list[i])):
# Get each value in each list
for k in range(len(score_list[i][j])):
# Add current value to the same index
# on the first row
score_list[0][j][k] += score_list[i][j][k]
print(score_list[0])
There is bound to be a better solution but this is a temporary fix for you :)
Edit. Made more efficient

A possible solution:
a = np.vstack([np.array(score_list[x], dtype='object')
for x in range(len(score_list))])
[np.add(*[x for x in a[:, i]]) for i in range(a.shape[1])]
Another possible solution:
a = sum(score_list, [])
b = [a[x] for x in range(0,len(a),2)]
c = [a[x] for x in range(1,len(a),2)]
[np.add(x[0], x[1]) for x in [b, c]]
Output:
[array([3, 8]), array([5, 3, 8])]

Efficient ways to aggregate and replicate values in a numpy matrix

In my work I often need to aggregate and expand matrices of various quantities, and I am looking for the most efficient ways to do these actions. E.g. I'll have an NxN matrix that I want to aggregate from NxN into PxP where P < N. This is done using a correspondence between the larger dimensions and the smaller dimensions. Usually, P will be around 100 or so.
For example, I'll have a hypothetical 4x4 matrix like this (though in practice, my matrices will be much larger, around 1000x1000)
m=np.array([[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12], [13, 14, 15, 16]])
>>> m
array([[ 1, 2, 3, 4],
[ 5, 6, 7, 8],
[ 9, 10, 11, 12],
[13, 14, 15, 16]])
and a correspondence like this (schematically):
0 -> 0
1 -> 1
2 -> 0
3 -> 1
that I usually store in a dictionary. This means that indices 0 and 2 (for rows and columns) both get allocated to new index 0 and indices 1 and 3 (for rows and columns) both get allocated to new index 1. The matrix could be anything at all, but the correspondence is always many-to-one when I want to compress.
If the input matrix is A and the output matrix is B, then cell B[0, 0] would be the sum of A[0, 0] + A[0, 2] + A[2, 0] + A[2, 2] because new index 0 is made up of original indices 0 and 2.
The aggregation process here would lead to:
array([[ 1+3+9+11, 2+4+10+12 ],
[ 5+7+13+15, 6+8+14+16 ]])
= array([[ 24, 28 ],
[ 40, 44 ]])
I can do this by making an empty matrix of the right size and looping over all 4x4=16 cells of the initial matrix and accumulating in nested loops, but this seems to be inefficient and the vectorised nature of numpy is always emphasised by people. I have also done it by using np.ix_ to make sets of indices and use m[row_indices, col_indices].sum(), but I am wondering what the most efficient numpy-like way to do it is.
Conversely, what is the sensible and efficient way to expand a matrix using the correspondence the other way? For example with the same correspondence but in reverse I would go from:
array([[ 1, 2 ],
[ 3, 4 ]])
to
array([[ 1, 2, 1, 2 ],
[ 3, 4, 3, 4 ],
[ 1, 2, 1, 2 ],
[ 3, 4, 3, 4 ]])
where the values simply get replicated into the new cells.
In my attempts so far for the aggregation, I have used approaches with pandas methods with groupby on index and columns and then extracting the final matrix with, e.g. df.values. However, I don't know the equivalent way to expand a matrix, without using a lot of things like unstack and join and so on. And I see people often say that using pandas is not time-efficient.
Edit 1: I was asked in a comment about exactly how the aggregation should be done. This is how it would be done if I were using nested loops and a dictionary lookup between the original dimensions and the new dimensions:
>>> m=np.array([[1,2,3,4],[5,6,7,8],[9,10,11,12],[13,14,15,16]])
>>> mnew=np.zeros((2,2))
>>> big2small={0:0, 1:1, 2:0, 3:1}
>>> for i in range(4):
... inew = big2small[i]
... for j in range(4):
... jnew = big2small[j]
... mnew[inew, jnew] += m[i, j]
...
>>> mnew
array([[24., 28.],
[40., 44.]])
Edit 2: Another comment asked for the aggregation example towards the start to be made more explicit, so I have done so.

Assuming you don't your indices don't have a regular structure I would do it try sparse matrices.
import scipy.sparse as ss
import numpy as np
# your current array of indices
g=np.array([[0,0],[1,1],[2,0],[3,1]])
# a sparse matrix of (data=ones, (row_ind=g[:,0], col_ind=g[:,1]))
# it is one for every pair (g[i,0], g[i,1]), zero elsewhere
u=ss.csr_matrix((np.ones(len(g)), (g[:,0], g[:,1])))
Aggregate
m=np.array([[1,2,3,4],[5,6,7,8],[9,10,11,12],[13,14,15,16]])
u.T # m # u
Expand
m2 = np.array([[1,2],[3,4]])
u # m2 # u.T

Remove and append middle column from a 3D numpy array

Suppose I have a 3D numpy array A, say given below:
A = np.array( [[[1,2,3], [4,5,6]] , [[7,8,9] , [10,11,12], [13,14,15]] ] , ndmin = 3 )
The only thing given about A is that it is a 3D arrays which is an array of arbitrary number of 2D arrays, where each 2D array is an array of arbitrary number of 1D arrays, and each 1D array has exactly 3 elements.
I want to remove the middle element from each 1D array from this 3D array, basically get the new array A1, and the removed column as X given below:
A1 = np.array( [[[1,3], [4,6]] , [[7,9] , [10,12], [13,15]] ] , ndmin = 3 )
X = np.array( [ [[2],[5]], [[8],[11],[14]] ], ndmin = 3 )
I want to write a function that given A it outputs (A1, X) and another function which given (A1, X) outputs A. I believe it should be possible to write the first function via array slicing, but I am not able to do so. Also how do I write the second function.

For you ragged array, it is better to store in a list of np.arrays with shape n by 3:
A = [np.array([[1,2,3],
[4,5,6]]) ,
np.array([[7,8,9],
[10,11,12],
[13,14,15]])]
Now you could:
def remove_middle(arr):
x = [a[:, 1] for a in arr]
arr_new = [np.delete(a, 1, axis = 1) for a in arr]
return arr_new, x
def insert_middle(arr, x):
return [np.concatenate([a[:, :1], xx.reshape(-1, 1), a[:, 1:]], axis = 1) for a, xx in zip(arr, x)]
remove_middle(A)
([array([[1, 3],
[4, 6]]),
array([[ 7, 9],
[10, 12],
[13, 15]])],
[array([2, 5]), array([ 8, 11, 14])])
insert_middle(*remove_middle(A))
# gets back the original A
[array([[1, 2, 3],
[4, 5, 6]]),
array([[ 7, 8, 9],
[10, 11, 12],
[13, 14, 15]])]

Without the ndmin=3 argument I can solve your answer using two nested list comprehensions in which the first indexes the middle argument and the second one deletes the middle argument of the inner arrays.
import numpy
A = np.array([[[1,2,3], [4,5,6]] , [[7,8,9] , [10,11,12], [13,14,15]]])
middle = [[array1d[1] for array1d in array2d] for array2d in A]
without_middle = [[np.delete(array1d, 1) for array1d in array2d] for array2d in A]

With your strange data, this absurd-looking quintuple-nested list-comprehension is the best I could come up with :P
A1 = [[[[[e for i, e in enumerate(d) if i != 1] for d in c] for c in b] for b in a] for a in A]
X = [[[[[e for i, e in enumerate(d) if i == 1] for d in c] for c in b] for b in a] for a in A]
Output:
>>> A1
[[[[[1, 3], [4, 6]], [[7, 9], [10, 12], [13, 15]]]]]
>>> X
[[[[[2], [5]], [[8], [11], [14]]]]]

Here is a solution. Note that lists are returned as raw python list which you can use as you want.
I changed your definition of A to a more suitable object.
import numpy as np
def f(A):
A1 = A.tolist()
X = []
for i in range(len(A1)):
temp = []
for j in range(len(A1[i])):
temp.append([A1[i][j].pop(1)])
X.append(temp)
return (A1, X)
def g(A1, X):
A = A1
for i in range(len(A1)):
for j in range(len(A1[i])):
A[i][j].insert(1, X[i][j][0])
return A
def main():
#A = np.array( [ [ [1,2,3], [4,5,6] ] , [ [7,8,9] , [10,11,12], [13,14,15] ] ] , ndmin = 3 )
A = np.asarray([ [ [1,2,3], [4,5,6] ] , [ [7,8,9] , [10,11,12], [13,14,15] ] ])
B, X = f(A)
print(g(B,X))
if __name__ == '__main__':
main()
Finally please note that this is one solution among many possible alternatives.

Add in-between steps into array of numbers (Python)

I am looking for some function that takes an input array of numbers and adds steps (range) between these numbers. I need to specify the length of the output's array.
Example:
input_array = [1, 2, 5, 4]
output_array = do_something(input_array, output_length=10)
Result:
output_array => [1, 1.3, 1.6, 2, 3, 4, 5, 4.6, 4.3, 4]
len(output_array) => 10
Is there something like that, in Numpy for example?
I have a prototype of this function that uses dividing input array into pairs ([0,2], [2,5], [5,8]) and filling "spaces" between with np.linspace() but it don't work well: https://onecompiler.com/python/3xwcy3y7d
def do_something(input_array, output_length):
import math
import numpy as np
output = []
in_between_steps = math.ceil(output_length/len(input_array))
prev_num = None
for num in input_array:
if prev_num is not None:
for in_num in np.linspace(start=prev_num, stop=num, num=in_between_steps, endpoint=False):
output.append(in_num)
prev_num = num
output.append(input_array[len(input_array)-1]) # manually add last item
return output
How it works:
input_array = [1, 2, 5, 4]
print(len(do_something(input_array, output_length=10))) # result: 10 OK
print(len(do_something(input_array, output_length=20))) # result: 16 NOT OK
print(len(do_something(input_array, output_length=200))) # result: 151 NOT OK
I have an array [1, 2, 5, 4] and I need to "expand" a number of items in it but preserve the "shape":

There is numpy.interp which might be what you are looking for.
import numpy as np
points = np.arange(4)
values = np.array([1,2,5,4])
x = np.linspace(0, 3, num=10)
np.interp(x, points, values)
output:
array([1. , 1.33333333, 1.66666667, 2. , 3. ,
4. , 5. , 4.66666667, 4.33333333, 4. ])