Develop Reference

Python: How to split a list into ordered chunks

If I have the following list
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
Then
np.array_split([0, 1, 2, 3, 4, 5, 6, 7, 8, 9], 3)
Returns
[array([0, 1, 2, 3]), array([4, 5, 6]), array([7, 8, 9])]
Is there a way to get the sub-arrays in the following order?
[array([0, 3, 6, 9]), array([1, 4, 7]), array([2, 5, 8])]

As the lists are of differing lengths, a numpy.ndarray isn't possible without a bit of fiddling, as all sub-arrays must be the same length.
However, if a simple list meets your requirement, you can use:
l = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
l2 = []
for i in range(3):
l2.append(l[i::3])
Output:
[[0, 3, 6, 9], [1, 4, 7], [2, 5, 8]]
Or more concisely, giving the same output:
[l[i::3] for i in range(3)]

Let's look into source code refactor of np.array_split:
def array_split(arr, Nsections):
Neach_section, extras = divmod(len(arr), Nsections)
section_sizes = ([0] + extras * [Neach_section + 1] + (Nsections - extras) * [Neach_section])
div_points = np.array(section_sizes).cumsum()
sub_arrs = []
for i in range(Nsections):
st = div_points[i]
end = div_points[i + 1]
sub_arrs.append(arr[st:end])
return sub_arrs
Taking into account your example arr = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] and Nsections = 3 it will construct section sizes [0, 4, 3, 3] and dividing points [0, 4, 7, 10]. Then do something like this:
[arr[div_points[i]:div_points[i + 1]] for i in range(3)]
Trying to mimic behaviour of numpy, indeed,
def array_split_withswap(arr, N):
sub_arrs = []
for i in range(N):
sub_arrs.append(arr[i::N])
Is the best option to go with (like in #S3DEV solution).

How to shuffle a python list such that the occurrence of some elements are preserved?

I start with a list of integers:
A = [ 1, 2, 5, 3, 4, 6, 7, 8 ]
After a shuffle, I would like some elements (say 3, 4 and 5) to preserve their order of occurrence in A while the rest of the elements are free to be randomly shuffled. Something like:
Outcome #1:
A = [ 5, 2, 3, 1, 8, 4, 6, 7 ]
-or-
Outcome #2:
A = [ 7, 5, 6, 1, 3, 4, 8, 2 ]
-but not-
Outcome #3 (invalid outcome)
A = [7, 4, 6, 1, 3, 5, 8, 2]
Appreciate all suggestions!

Extract the elements you want to maintain relative ordering among, shuffle as normal, then glue the lists back together by randomly picking indexes for the "kept" elements to be inserted. All operations are linear if you use sets for speeding up in operations, which I didn't bother with. Ordering of the keep list matters.
>>> import random
>>> L = [1, 2, 5, 3, 4, 6, 7, 8]
>>> keep = [2, 3, 4]
>>> kept = [L[i] for i in keep][::-1]
>>> unkept = [x for i, x in enumerate(L) if i not in keep]
>>> random.shuffle(unkept)
>>> idxes = random.sample(list(range(len(L))), k=len(keep))
>>> result = [kept.pop() if i in idxes else unkept.pop() for i in range(len(L))]
>>> result
[6, 5, 3, 8, 4, 1, 7, 2]
Random tests:
import random
def shuffle_with_fixed_order(L, keep):
kept = [L[i] for i in keep][::-1]
unkept = [x for i, x in enumerate(L) if i not in keep]
random.shuffle(unkept)
idxes = random.sample(list(range(len(L))), k=len(keep))
return [kept.pop() if i in idxes else unkept.pop() for i in range(len(L))]
if __name__ == "__main__":
for _ in range(5000):
L = list(range(50))
random.shuffle(L)
keep = sorted(random.sample(L, k=20))
shuffled = shuffle_with_fixed_order(L, keep)
new_locs = [shuffled.index(L[i]) for i in keep]
assert all(x < y for x, y in zip(new_locs, new_locs[1:]))

This is one approach using random module
Ex:
import random
A = [ 1, 2, 5, 3, 4, 6, 7, 8 ]
result = A[2:5]
del A[2:5]
while A:
l = len(A)
result.insert(random.randint(0, l), A.pop(random.randrange(l)))
print(result)
Demo
def rand_shfl(lst):
result = lst[2:5]
del lst[2:5]
while A:
l = len(A)
result.insert(random.randint(0, l), A.pop(random.randrange(l)))
return result
for _ in range(10):
A = [ 1, 2, 5, 3, 4, 6, 7, 8 ]
print((rand_shfl(A)))
Output:
[1, 7, 6, 5, 8, 3, 4, 2]
[8, 1, 2, 7, 5, 3, 6, 4]
[8, 6, 7, 2, 1, 5, 3, 4]
[1, 7, 2, 8, 5, 3, 4, 6]
[6, 8, 7, 5, 1, 3, 4, 2]
[7, 2, 5, 6, 1, 3, 4, 8]
[1, 8, 5, 2, 3, 4, 6, 7]
[5, 7, 1, 6, 3, 8, 4, 2]
[1, 5, 7, 3, 2, 4, 8, 6]
[8, 2, 7, 6, 5, 3, 4, 1]

Probably below will work for you
import random
lst=[1, 2, 5, 3, 4, 6, 7, 8]
s_lst = {5:0,3:0,4:0}
idx_lst = [random.randint(0, len(lst)) for i in range(len(s_lst))]
idx_lst.sort()
for i, s in enumerate(s_lst.keys()):
s_lst[s] = idx_lst[i]
lst.remove(s)
random.shuffle(lst)
for k, v in s_lst.items():
lst.insert(v, k)
print(lst)

Python consecutive operations on list getting executed in wrong order

I have a function written in python that performs some consecutive operations on two lists. The problem is that at random times during the execution of these functions, they give wrong answer. The code inside the function is
def temp(c, p):
random.seed(0)
x = random.randint(0 , len(c)-1)
y = random.randint(0 , len(c)-1)
s_1 = c[x][0]
s_2 = c[y][0]
p[x] += [s_1]
p[y] += [s_2]
p[x].remove(s_2)
p[y].remove(s_1)
c[x], c[y] = c[y], c[x]
return c, p
def anotherFunction():
iter = 1000
for i in iter:
c_main, p_main = temp(c, p)
I have a list of list with numbers ranging from 0 to n. For example c contains the following
c = [[7], [6], [1], [2], [5], [4], [0], [3]]
And p is also a list of list that contains all the numbers from 0 to n except that at index which are there in c.
p = [[0, 2, 4, 6, 5, 1, 3]
[0, 1, 2, 3, 4, 5, 7]
[0, 2, 3, 4, 6, 7, 5]
[0, 1, 3, 5, 6, 7, 4]
[0, 2, 4, 6, 7, 3, 1]
[0, 1, 2, 3, 6, 7, 5]
[1, 2, 3, 4, 5, 6, 7]
[0, 1, 2, 4, 5, 6, 7]]
This is how the values should be at any random point in the function. That is the values at idx in c should not be present in the list at idx in p.
But sometimes during the execution of the function, the values selected by x and y are swapped but one other value also gets affected. This is how the two list looks like sometimes
c = [[3], [1], [4], [5], [7], [0], [2], [6]]
p = [[0, 1, 2, 4, 5, 6, 7]
[0, 2, 3, 4, 5, 6, 7]
[0, 1, 2, 3, 6, 7, 4]
[0, 1, 2, 3, 6, 5, 5]
[0, 1, 2, 3, 4, 6, 7]
[1, 2, 3, 4, 6, 7, 5]
[0, 1, 3, 4, 5, 6, 7]
[0, 1, 2, 3, 4, 5, 7]]
I'm unable to understand how these consecutive operations are getting affected by one another. This function gets called inside a loop of another function.
UPDATE:
I debugged my code more carefully and realized that at some iterations of the for loop two more values get swapped in c in addition to x and y. And because these values get swapped but they are not updated in p in some executions I get a faulty output. Any ideas why two more values are getting swapped.

Your code is not complete.
You seam to call your function like this:
c = [[3], [1], [4], [5], [7], [0], [2], [6]]
p = [[0, 1, 2, 4, 5, 6, 7],
[0, 2, 3, 4, 5, 6, 7],
[0, 1, 2, 3, 6, 7, 4],
[0, 1, 2, 3, 6, 5, 5],
[0, 1, 2, 3, 4, 6, 7],
[1, 2, 3, 4, 6, 7, 5],
[0, 1, 3, 4, 5, 6, 7],
[0, 1, 2, 3, 4, 5, 7]]
for i in range(1000):
c_main, p_main = temp(c, p)
note: if fixed your code: add range and add comma for each lines in p.
But inside your temp() function, your are modifying the content of p.
So, you may not have what you expect. Because you reuse the same p at each iteration. So sometimes it becomes inconsistent.
What you want, is certainly something like that:
import random
def temp(c):
# -- raw matrix
p = [[col for col in range(8)] for row in range(len(c))]
# -- drop a number
for p_row, c_row in zip(p, c):
p_row.pop(c_row[0])
# -- shuffle
for row in p:
random.shuffle(row)
return p
You can use it like this:
cols = [[3], [1], [4], [5], [7], [0], [2], [6]]
print(temp(cols))
You get:
[[6, 1, 4, 5, 7, 2, 0],
[3, 0, 5, 4, 2, 7, 6],
[2, 7, 0, 3, 6, 5, 1],
[1, 2, 7, 4, 6, 3, 0],
[3, 1, 4, 6, 2, 0, 5],
[4, 5, 6, 3, 7, 1, 2],
[0, 6, 1, 5, 7, 3, 4],
[4, 3, 7, 0, 1, 5, 2]]

Duplicating specific elements in lists or Numpy arrays

I work with large data sets in my research.
I need to duplicate an element in a Numpy array. The code below achieves this, but is there a function in Numpy that performs the operation in a more efficient manner?
"""
Example output
>>> (executing file "example.py")
Choose a number between 1 and 10:
2
Choose number of repetitions:
9
Your output array is:
[1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 4, 5, 6, 7, 8, 9, 10]
>>>
"""
x = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
y = int(input('Choose the number you want to repeat (1-10):\n'))
repetitions = int(input('Choose number of repetitions:\n'))
output = []
for i in range(len(x)):
if x[i] != y:
output.append(x[i])
else:
for j in range(repetitions):
output.append(x[i])
print('Your output array is:\n', output)

One approach would be to find the index of the element to be repeated with np.searchsorted. Use that index to slice the left and right sides of the array and insert the repeated array in between.
Thus, one solution would be -
idx = np.searchsorted(x,y)
out = np.concatenate(( x[:idx], np.repeat(y, repetitions), x[idx+1:] ))
Let's consider a bit more generic sample case with x as -
x = [2, 4, 5, 6, 7, 8, 9, 10]
Let the number to be repeated is y = 5 and repetitions = 7.
Now, use the proposed codes -
In [57]: idx = np.searchsorted(x,y)
In [58]: idx
Out[58]: 2
In [59]: np.concatenate(( x[:idx], np.repeat(y, repetitions), x[idx+1:] ))
Out[59]: array([ 2, 4, 5, 5, 5, 5, 5, 5, 5, 6, 7, 8, 9, 10])
For the specific case of x always being [1, 2, 3, 4, 5, 6, 7, 8, 9, 10], we would have a more compact/elegant solution, like so -
np.r_[x[:y-1], [y]*repetitions, x[y:]]

There is the numpy.repeat function:
>>> np.repeat(3, 4)
array([3, 3, 3, 3])
>>> x = np.array([[1,2],[3,4]])
>>> np.repeat(x, 2)
array([1, 1, 2, 2, 3, 3, 4, 4])
>>> np.repeat(x, 3, axis=1)
array([[1, 1, 1, 2, 2, 2],
[3, 3, 3, 4, 4, 4]])
>>> np.repeat(x, [1, 2], axis=0)
array([[1, 2],
[3, 4],
[3, 4]])

Python - Read nth line in a matrix

What is the simplest way I can read the nth letter of a matrix?
I thought this would be possible with a simple for loop but so far I haven't had any luck.
The best I can do so far is using a count which is not exactly elegant:
matrix = [[1, 3, 5, 2, 6, 2, 4, 1], [2, 6, 1, 6, 2, 5, 7], [1, 6, 2, 6, 8, 2, 6]]
count = 0
for n in matrix:
print matrix[count][nth]
count += 1
For example:
Read the 0th number of every row: 1, 2, 1.
Read the 4th number of every row: 6, 2, 8.

If your need to do this operation a lot you could transpose your matrix using zip(*matrix)
>>> matrix = [[1, 3, 5, 2, 6, 2, 4, 1], [2, 6, 1, 6, 2, 5, 7], [1, 6, 2, 6, 8, 2, 6]]
>>> matrix_t = zip(*matrix)
>>> matrix_t
[(1, 2, 1), (3, 6, 6), (5, 1, 2), (2, 6, 6), (6, 2, 8), (2, 5, 2), (4, 7, 6)]
>>> matrix_t[0]
(1, 2, 1)
>>> matrix_t[3]
(2, 6, 6)

Here's something that will handle rows of different lengths (as in your example), as well as supporting Python's special interpretation of negative indexes as relative to the end of the sequence (by changing them intolen(s) + n):
NULL = type('NULL', (object,), {'__repr__': lambda self: '<NULL>'})()
def nth_elems(n):
abs_n = abs(n)
return [row[n] if abs_n < len(row) else NULL for row in matrix]
matrix = [[1, 3, 5, 2, 6, 2, 4, 1], [2, 6, 1, 6, 2, 5, 7], [1, 6, 2, 6, 8, 2, 6]]
print nth_elems(0) # [1, 2, 1]
print nth_elems(6) # [4, 7, 6]
print nth_elems(7) # [1, <NULL>, <NULL>]
print nth_elems(-1) # [1, 7, 6]

Maybe this way?
column = [row[0] for row in matrix]
(for the 0th element)

In [1]: matrix = [[1, 3, 5, 2, 6, 2, 4, 1], [2, 6, 1, 6, 2, 5, 7], [1, 6, 2, 6, 8, 2, 6]]
In [2]: nth=0
In [3]: [row[nth] for row in matrix]
Out[3]: [1, 2, 1]
In [4]: nth=4
In [5]: [row[nth] for row in matrix]
Out[5]: [6, 2, 8]

Here is a solution using list comprehension:
[x[0] for x in matrix]
Which is basically, equal to:
for x in matrix:
print x[0]
You can also make it a function:
def getColumn(lst, col):
return [i[col] for i in lst]
Demo:
>>> matrix = [[1, 3, 5, 2, 6, 2, 4, 1], [2, 6, 1, 6, 2, 5, 7], [1, 6, 2, 6, 8, 2, 6]]
>>> def getColumn(lst, col):
return [i[col] for i in lst]
>>> getColumn(matrix, 0)
[1, 2, 1]
>>> getColumn(matrix, 5)
[2, 5, 2]
Hope this helps!

List comprehensions will work well here:
>>> matrix = [[1, 3, 5, 2, 6, 2, 4, 1], [2, 6, 1, 6, 2, 5, 7], [1, 6, 2, 6, 8, 2, 6]]
>>> # Get all the 0th indexes
>>> a = [item[0] for item in matrix]
>>> a
[1, 2, 1]
>>> # Get all the 4th indexes
>>> b = [item[4] for item in matrix]
>>> b
[6, 2, 8]
>>>

Your for loop is likely not doing what you expect. n is not an integer. It is the current row.
I think what you wanted to do was:
for row in matrix:
print row[0], row[4]
This prints,
1 6
2 2
1 8
Also, strictly speaking, matrix is a list of lists. To really have a matrix you might need to use numpy.

Lists in Python are not intended to be used like this. Using list comprehension may cause both memory and CPU issues if the data is sufficiently big. Consider using numpy if this is an issue.

Use zip:
>>> matrix = [[1, 3, 5, 2, 6, 2, 4, 1], [2, 6, 1, 6, 2, 5, 7], [1, 6, 2, 6, 8, 2, 6]]
>>> zip(*matrix)[0]
(1, 2, 1)
>>> zip(*matrix)[4]
(6, 2, 8)