Develop Reference

Allocate elements in python list with specific probobility

Assume, I have a specific proportions of slots proportion = [30,30,20,10,10]and I want to feed it with 1 element and get it allocated one by one. For example, we start with [0,0,0,0,0] and add 1 we get [1,0,0,0,0]. What I have so far is that (based on this post answer):
def distribute_elements_in_slots(total, slots, pct):
distr = [total * pct[i] / 100 for i in range(slots)]
solid = [int(elem) for elem in distr]
short = [distr[i] - solid[i] for i in range(slots)]
leftover = int(round(sum(short)))
for i in range(leftover):
shortest = short.index(max(short))
solid[shortest] += 1
short[shortest] = 0
return solid
To feed 1 element at the time I've generated the list on 1's:
randomlist = []
for i in range(0,30):
n = random.randint(1,1)
randomlist.append(n)
print(randomlist)
And addition function to loop over that list:
x = 5
flexibility = [30, 30, 20, 10 ,10]
total = 0
cars = 0
for n in randomlist:
cars += 1
total += n
distributed = distribute_elements_in_slots(total, x, flexibility)
print(distributed)
But the broblem is this fucnction does not remeber the previous step.
1-[1, 0, 0, 0, 0]
2-[1, 1, 0, 0, 0]
3-[1, 1, 1, 0, 0]
4-[1, 1, 1, 1, 0] - on this step we have 4 elements in 4 slots.
5-[2, 2, 1, 0, 0] - on this step we took 1 from the fourth element and "gave" it to second.
But I want it it be like this:
1-[1, 0, 0, 0, 0]
2-[1, 1, 0, 0, 0]
3-[1, 1, 1, 0, 0]
4-[1, 1, 1, 1, 0]
5-[2, 1, 1, 1, 0]

This simple code gives a slot filling sequence without reallocations:
slots = 5
dist = np.array([0]*slots)
proportion = np.array([30,30,20,10,10])
for i in range(0,30):
total = max(dist.sum(),1)
prop = dist/total*100
error = proportion - prop
idx = np.argmax(error)
dist[idx] += 1
print(dist)
[1 0 0 0 0]
[1 1 0 0 0]
[1 1 1 0 0]
[1 1 1 1 0]
[1 1 1 1 1]
[2 1 1 1 1]
[2 2 1 1 1]
[2 2 2 1 1]
[3 2 2 1 1]
[3 3 2 1 1]
[4 3 2 1 1]
[4 4 2 1 1]
[4 4 3 1 1]
[4 4 3 2 1]
[4 4 3 2 2]
[5 4 3 2 2]
[5 5 3 2 2]
[5 5 4 2 2]
[6 5 4 2 2]
[6 6 4 2 2]
[7 6 4 2 2]
[7 7 4 2 2]
[7 7 5 2 2]
[7 7 5 3 2]
[7 7 5 3 3]
[8 7 5 3 3]
[8 8 5 3 3]
[8 8 6 3 3]
[9 8 6 3 3]
[9 9 6 3 3]

Problem with understanding of work of np.argpartition

I have problem with execution of np.argpartition
I have nd.array
example = np.array([[5,6,7,3,4],[1,2,3,7,5],[6,7,4,2,3],[1,2,3,5,9],[2,3,6,1,2,]])
out: [[5 6 7 3 4]
[1 2 3 7 5]
[6 7 4 2 3]
[1 2 3 5 9]
[2 3 6 1 2]]
I can get indices for sorted array by np.argsort
print(np.argsort(example))
out:
[[3 4 0 1 2]
[0 1 2 4 3]
[3 4 2 0 1]
[0 1 2 3 4]
[3 0 4 1 2]]
I want to use np.argsort to economy some time for executing, because I need only 3 sorted element in each row of this array. I use this code to do it:
print(np.argpartition(example, 3, axis=1))
out: [[3 4 0 1 2]
[1 0 2 4 3]
[3 4 2 0 1]
[1 0 2 3 4]
[3 4 0 1 2]]
I expect that the first three indices of each row will match the indices in the sorted array, but this is not the caseю That doesn't work . I don't understand what I did wrong.

np.argpartition(example, k, axis=1) does not return sorted array for first k elements. It only returns indices such that only (k+1)th element is sorted. If you see in your output, only the 4th element matches with argsort()
If you want first three sorted elements, you have to give a list for k parameter
index_array = np.argpartition(example, [0,1,2], axis=1)
print(np.take_along_axis(example,index_array, axis=1)) ##this will give you first 3 sorted elements

How to overwrite 2-D numpy multi times symmetrically with given index?

I'm trying to change values in matrix a with given index matrix d and matrix e.
And the matrix should always be symmetrical.
What I come up with is to overwrite the primal matrix with given index, and try to make it symmetrical, then go for another overwrite, until all the given index matrix have been gone through. It's not efficient.
But I'm stuck with how make it symmetrical.
For example:
a = np.ones([4,4],dtype=np.object) #the primal matrix
d = np.array([[1],
[2],
[0],
[0]]) #the first index matrix
a[np.arange(a.shape[0])[:,None],d] =2 #the element change to 2 with the indexes shown in d matrix
Now the result is:
a = np.array([[1 2 1 1]
[1 1 2 1]
[2 1 1 1]
[2 1 1 1]])
After making it symmetrical (if a[ i ][ j ] was selected in d matrix, a[ j ][ i ] should also be changed to 2, how to do this part).
The expected output should be :
a = np.array([[1 2 2 2]
[2 1 2 1]
[2 2 1 1]
[2 1 1 1]])
Then, for another overwrite again:
e = np.array([[0],[2],[1],[1]])
a[np.arange(a.shape[0])[:,None],e] =3
Now the result is:
a = np.array([[3 2 2 2]
[2 1 3 1]
[2 3 1 1]
[2 3 1 1]])
Make it symmetrical, (I don't know how to do this part) the final output should be : (overwrite the values if they were given 2 or 1 before)
a = np.array([[3 2 2 2]
[2 1 3 3]
[2 3 1 1]
[2 3 1 1]])
What should I do to get symmetrical matrix?
And, is there anyway to change the primal matrix a directly to get the final result? In a more efficient way?
Thanks in advance !!

You can simply switch the first and second indices and apply the change, the result would be symmetrical:
a[np.arange(a.shape[0])[:,None], d] = 2
a[d, np.arange(a.shape[0])[:,None]] = 2
output:
[[1 2 2 2]
[2 1 2 1]
[2 2 1 1]
[2 1 1 1]]
Same with any number of other changes:
a[np.arange(a.shape[0])[:,None], e] = 3
a[e, np.arange(a.shape[0])[:,None]] = 3
output:
[[3 2 2 2]
[2 1 3 3]
[2 3 1 1]
[2 3 1 1]]

how can i swap list in a matrix in python?

I want to shuffle 3D matrix's rows but it doesn't work in a matrix
here is some example code
def shuffle(data,data_size):
for step in range(int(1*data_size)):
selected = int(np.random.uniform(0,data_size))
target = int(np.random.uniform(0,data_size))
print(data)
if selected!=target:
data[selected], data[target] = data[target], data[selected]
print(selected," and ",target, " are changed")
return data
data = [[[1,2,3,4],[1,2,3,5],[1,2,3,6]],
[[2,2,3,4],[2,2,3,5],[2,2,3,6]],
[[3,2,3,4],[3,2,3,5],[3,2,3,6]] ]
data = np.array(data)
data = shuffle(data,3)
in this code I want to shuffle data from some row list to another row list
but it's result doesn't work swaping but overwriting
here is result
[[[1 2 3 4]
[1 2 3 5]
[1 2 3 6]]
[[2 2 3 4]
[2 2 3 5]
[2 2 3 6]]
[[3 2 3 4]
[3 2 3 5]
[3 2 3 6]]]
2 and 1 are changed
[[[1 2 3 4]
[1 2 3 5]
[1 2 3 6]]
[[2 2 3 4]
[2 2 3 5]
[2 2 3 6]]
[[2 2 3 4]
[2 2 3 5]
[2 2 3 6]]]
1 and 0 are changed
[[[1 2 3 4]
[1 2 3 5]
[1 2 3 6]]
[[1 2 3 4]
[1 2 3 5]
[1 2 3 6]]
[[2 2 3 4]
[2 2 3 5]
[2 2 3 6]]]
0 and 2 are changed
[[[2 2 3 4]
[2 2 3 5]
[2 2 3 6]]
[[1 2 3 4]
[1 2 3 5]
[1 2 3 6]]
[[2 2 3 4]
[2 2 3 5]
[2 2 3 6]]]
2 and 1 are changed
how can i swap list in matrix?
thanks

import numpy as np
def shuffle(data,data_size):
for step in range(int(1*data_size)):
selected = int(np.random.uniform(0,data_size))
target = int(np.random.uniform(0,data_size))
print(data)
if selected!=target:
data[[selected, target]] = data[[target, selected]]
print(selected," and ",target, " are changed")
return data
data = [[[1,2,3,4],[1,2,3,5],[1,2,3,6]],
[[2,2,3,4],[2,2,3,5],[2,2,3,6]],
[[3,2,3,4],[3,2,3,5],[3,2,3,6]] ]
data = np.array(data)
data = shuffle(data,3)

If you want to shuffle along the first axis, just use np.random.shuffle:
data = np.array([
[[1,2,3,4],[1,2,3,5],[1,2,3,6]],
[[2,2,3,4],[2,2,3,5],[2,2,3,6]],
[[3,2,3,4],[3,2,3,5],[3,2,3,6]]
])
np.random.shuffle(data)
print(data)
Output:
[[[3 2 3 4]
[3 2 3 5]
[3 2 3 6]]
[[1 2 3 4]
[1 2 3 5]
[1 2 3 6]]
[[2 2 3 4]
[2 2 3 5]
[2 2 3 6]]]
If you want to shuffle along any other axis in data, you can shuffle the array view returned by np.swapaxes. For example, to shuffle the rows of the inner 2D matrices, do:
swap = np.swapaxes(data, 1, 0)
np.random.shuffle(swap)
print(data)
Output:
[[[1 2 3 6]
[1 2 3 4]
[1 2 3 5]]
[[2 2 3 6]
[2 2 3 4]
[2 2 3 5]]
[[3 2 3 6]
[3 2 3 4]
[3 2 3 5]]]

numpy: extract multiple subarrays of a position array in an efficient way

I have a 2D coefficient array COEFF with size row x col and a position array POS with size n x 2.
The goal is to create a batched array BAT with size n x (2*l) x (2*l) where l is the half length of subarray.
It looks like this
BAT[i, :, :] = COEFF[POS[i, 1] - l:POS[i, 1] + l, POS[i, 0] - l:POS[i, 0] + l]
It is possible to generate BAT based on above sequential code. However, I'm wondering is there an efficient way to construct the BAT array in parallel.
Thanks!

I'm not aware of a perfectly satisfactory solution to mixing advanced indexing and slicing in that way. But the following may be acceptable (assuming that by "parallel" you mean "vectorised"):
import numpy as np
nrow, ncol = 7, 7
n, l = 3, 2
coeff = np.random.randint(0,10, (nrow,ncol))
pos = np.c_[np.random.randint(l, nrow-l+1, (n,)),np.random.randint(l, ncol-l+1, (n,))]
i = (pos[:, :1] + np.arange(-l, l))[:, :, None]
j = (pos[:, 1:] + np.arange(-l, l))[:, None, :]
print(coeff, '\n')
print(pos, '\n')
print(coeff[i, j])
Prints:
# [[7 6 7 6 3 9 9]
# [3 6 8 3 4 8 6]
# [3 7 4 7 4 6 8]
# [0 7 2 3 7 0 4]
# [8 5 2 0 0 1 7]
# [4 6 1 9 4 5 4]
# [1 6 8 3 4 5 0]]
# [[2 2]
# [3 2]
# [2 4]]
# [[[7 6 7 6]
# [3 6 8 3]
# [3 7 4 7]
# [0 7 2 3]]
# [[3 6 8 3]
# [3 7 4 7]
# [0 7 2 3]
# [8 5 2 0]]
# [[7 6 3 9]
# [8 3 4 8]
# [4 7 4 6]
# [2 3 7 0]]]