Develop Reference

Change the format of a numpy array in python

I have a (2x2) numpy array which contains an (1x3) list, like this below:
[[ 1, 2, 3], [ 4, 5, 6]
[ 7, 8, 9], [10,11,12]]
and I want to break it to single elements like this (2x6) array (or matrix, I can't tell the difference):
[ 1, 2, 3, 4, 5, 6]
[ 7, 8, 9, 10, 11, 12]
I can create the second array by coping every single element of the first array and put it to the second one. Yet I wonder if there is a more easy way provided by the numpy library.

you can simply use resize function to do this
import numpy as np
a=[[ 1, 2, 3], [ 4, 5, 6],
[ 7, 8, 9], [10,11,12]]
a = numpy.array(a)
a.resize(2,6)
print(a)
OUTPUT
array([[ 1, 2, 3, 4, 5, 6],
[ 7, 8, 9, 10, 11, 12]])

Numpy operation to expand array into sequential slices of given length?

my_function must expand a 1D numpy array to a 2D numpy array, with the 2nd axis containing the slices of length starting from the first index until the end. Example:
import numpy as np
a = np.arange(10)
print (my_function(a, length=3))
Expected output
array([[0, 1, 2],
[1, 2, 3],
[2, 3, 4],
[3, 4, 5],
[4, 5, 6],
[5, 6, 7],
[6, 7, 8],
[7, 8, 9]])
I can achieve this using a for loop, but I was wondering if there is a numpy vectorization technique for this.
def my_function(a, length):
b = np.zeros((len(a)-(length-1), length))
for i in range(len(b)):
b[i] = a[i:i+length]
return b

If you're careful with the math and heed the warningin the docs, you can use np.lib.stride_tricks.as_strided(). You need to calculate the correct dimensions for your array so you don't overflow. Also note that as_strided() shares memory, so you will multiple references to the same memory in the final output. (You can of course, copy this to a new array).
>> import numpy as np
>> def my_function(a, length):
stride = a.strides[0]
l = len(a) - length + 1
return np.lib.stride_tricks.as_strided(a, (l, length), (stride,stride) )
>> np.array(my_function(np.arange(10), 3))
array([[0, 1, 2],
[1, 2, 3],
[2, 3, 4],
[3, 4, 5],
[4, 5, 6],
[5, 6, 7],
[6, 7, 8],
[7, 8, 9]])
>> np.array(my_function(np.arange(15), 7))
array([[ 0, 1, 2, 3, 4, 5, 6],
[ 1, 2, 3, 4, 5, 6, 7],
[ 2, 3, 4, 5, 6, 7, 8],
[ 3, 4, 5, 6, 7, 8, 9],
[ 4, 5, 6, 7, 8, 9, 10],
[ 5, 6, 7, 8, 9, 10, 11],
[ 6, 7, 8, 9, 10, 11, 12],
[ 7, 8, 9, 10, 11, 12, 13],
[ 8, 9, 10, 11, 12, 13, 14]])

How about this function?
import numpy as np
def my_function(a, length):
result = []
for i in range(length):
result.append(a + i)
return np.vstack(result).T[:len(a) - length + 1]
a = np.arange(10)
length = 3
my_function(a, length)

Weird "too many indices for array" error in python

Let's create a large np array 'a' with 10,000 entries
import numpy as np
a = np.arange(0, 10000)
Let's slice the array with 'n' indices 0->9, 1->10, 2->11, etc.
n = 32
b = list(map(lambda x:np.arange(x, x+10), np.arange(0, n)))
c = a[b]
The weird thing that I am getting, is that if n is smaller than 32, I get an error "IndexError: too many indices for array". If n is bigger or equal than 32, then the code works perfectly. The error occurs regardless of the size of the initial array, or the size of the individual slices, but always with number 32. Note that if n == 1, the code works.
Any idea on what is causing this? Thank you.

Your b is a list of arrays:
In [84]: b = list(map(lambda x:np.arange(x, x+10), np.arange(0, 5)))
In [85]: b
Out[85]:
[array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]),
array([ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10]),
array([ 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]),
array([ 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]),
array([ 4, 5, 6, 7, 8, 9, 10, 11, 12, 13])]
When used as an index:
In [86]: np.arange(1000)[b]
/usr/local/bin/ipython3:1: FutureWarning: Using a non-tuple sequence for multidimensional
indexing is deprecated; use `arr[tuple(seq)]` instead of `arr[seq]`.
In the future this will be interpreted as an array index, `arr[np.array(seq)]`,
which will result either in an error or a different result.
#!/usr/bin/python3
---------------------------------------------------------------
IndexError: too many indices for array
A[1,2,3] is the same as A[(1,2,3)] - that is, the comma separated indices are a tuple, which is then passed on to the indexing function. Or to put it another way, a multidimensional index should be a tuple (that includes ones with slices).
Up to now numpy has been a bit sloppy, and allowed us to use a list of indices in the same way. The warning tells us that the developers are in the process of tightening up those restrictions.
The error means it is trying to interpret each array in your list as the index for a separate dimension. An array can have at most 32 dimensions. Evidently for the longer list it doesn't try to treat it as a tuple, and instead creates a 2d array for indexing.
There are various ways we can use your b to index a 1d array:
In [87]: np.arange(1000)[np.hstack(b)]
Out[87]:
array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 1, 2, 3, 4, 5, 6, 7,
8, 9, 10, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 3, 4, 5, 6,
7, 8, 9, 10, 11, 12, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13])
In [89]: np.arange(1000)[np.array(b)] # or np.vstack(b)
Out[89]:
array([[ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
[ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
[ 2, 3, 4, 5, 6, 7, 8, 9, 10, 11],
[ 3, 4, 5, 6, 7, 8, 9, 10, 11, 12],
[ 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]])
In [90]: np.arange(1000)[b,] # 1d tuple containing b
Out[90]:
array([[ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
[ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10],
[ 2, 3, 4, 5, 6, 7, 8, 9, 10, 11],
[ 3, 4, 5, 6, 7, 8, 9, 10, 11, 12],
[ 4, 5, 6, 7, 8, 9, 10, 11, 12, 13]])
Note that if b is a ragged list - one or more of the arrays is shorter, only the hstack version works.

First of all, you're not slicing 0->9, 10->19, 20->29; your slices advance by 1 only: 0->9, 1->10, 11->20. Instead, try this:
n = 32
size = 10
b = list(map(lambda x:np.arange(x, x+size), np.arange(0, n*size, size)))
Next, you've misused the indexing notation. b is a list of arrays, and you've used this entire list to index a. When you have indexed more elements than exist in a, numpy assumes that you want the complex list taken as a sequence of references, and uses them as individual index arrays, one a element per leaf element in b.
However, once you drop below the limit of len(a), then numpy assume that you're trying to give a multi-dimensional slice into a: each element of b is taken as a slice into the corresponding dimension of a. Since a is only 1-dimensional, you get the error message. Your code will run in this mode with n=1, but fails with n=2 and above.
Although your question isn't a duplicate, also please see this one.

Stack slices of numpy array from given indices

I'm struggling to perform the below operation on a numpy vector.
I want to take previous_n samples from vector finishing at indices.
It's like I want to perform a np.take with slicing of the previous_n samples.
Example:
import numpy as np
vector = np.array([ 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14])
# number of previous samples
previous_n = 3
indices = np.array([ 5, 7, 12])
result
array([[ 3, 4, 5],
[ 5, 6, 7],
[10, 11, 12]])

Ok, this seems to do what I want. Found here
def stack_slices(arr, previous_n, indices):
all_idx = indices[:, None] + np.arange(previous_n) - (previous_n - 1)
return arr[all_idx]
>>> stack_slices(vector, 3, indices)
array([[ 3, 4, 5],
[ 5, 6, 7],
[10, 11, 12]])

Simplify merging of two numpy arrays at even and uneven points in Python

I wrote a program that does the job, however it is not very pythonic, not pythonic and definitly not beautiful.
The program must concatenate two numpy arrays in the following manner:
As an example list0 and list1 are the input
list0 = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
list1 = [ 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]
The output should look like the following:
[0, 2, 1, 3, 2, 4, 3, 5, 4, 6, 5, 7, 6, 8, 7, 9, 8, 10, 9, 11]
So basically put in the number of list0 at every even point of the output, and put in the number of list1 at every uneven point.
I am fairly new to python so I wrote it in a C-Style:
import numpy as np
list0 = np.arange(10)
list1 = np.arange(2,12)
new = []
cnt0 = 0
cnt1 = 0
for i in range(0,2*len(list0)):
if i % 2 == 0:
new.append(list0[cnt0])
cnt0 = cnt0 +1;
else:
new.append(list1[cnt1])
cnt1 = cnt1 +1;
Now I want to know if there is a more fancy, pythonic, faster way to achieve the same goal?

Being NumPy tagged, here's one with it -
np.vstack((list0, list1)).ravel('F').tolist()
ravel() here flattens in fortran order with the F specifier.
A shorter version with np.c_ that basically stacks the elements in columns -
np.c_[list0,list1].ravel().tolist()
ravel() here flattens in the default C order, so skipped here.
If the final output is to be kept as an array, skip the .tolist() from the approaches.

Nice one liner with itertools
from itertools import chain
chain(*zip(list0, list1))
[0, 2, 1, 3, 2, 4, 3, 5, 4, 6, 5, 7, 6, 8, 7, 9, 8, 10, 9, 11]

You can use zip
>>> output = [ data for elem in zip(list0,list1) for data in elem ]
[0, 2, 1, 3, 2, 4, 3, 5, 4, 6, 5, 7, 6, 8, 7, 9, 8, 10, 9, 11]