Develop Reference

flatten list of numpy arrays while keeping inner arrays

l = [np.array([[1,2],[3,4]]), np.array([5,6]), np.array([[7,8],[9,10],[11,12]])]
I'm trying to flatten this list of arrays but keeping the inner arrays:
[[1, 2], [3, 4], [5, 6], [7, 8], [9, 10], [11, 12]]
I tried itertools.chain, np.concatenate, np.flatten but none of these options give the output above

Your arrays have different numbers of dimensions, you need to ensure they all are 2D:
out = np.concatenate([x[None,:] if x.ndim == 1 else x for x in l])
output:
array([[ 1, 2],
[ 3, 4],
[ 5, 6],
[ 7, 8],
[ 9, 10],
[11, 12]])
or with itertools.chain with a list output:
from itertools import chain
list(chain.from_iterable([x.tolist()] if x.ndim == 1 else x.tolist()
for x in l))
output: [[1, 2], [3, 4], [5, 6], [7, 8], [9, 10], [11, 12]]

You haven't provided your desired output. But, as I understood, you are looking for this:
x = [[1, 2], [3, 4], [5, 6], [7, 8], [9, 10], [11, 12]]
print([n for m in x for n in m])
with output:
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12]
Please let me know if this is not your desired output.

You basically need to flatten each subarray, concatenate them and finally reshape it.
np.concatenate(list(map(np.ravel, l))).reshape(-1, 2)
output:
array([[ 1, 2],
[ 3, 4],
[ 5, 6],
[ 7, 8],
[ 9, 10],
[11, 12]])

How do I divide a Numpy array along axis-0 into a list of equal sized numpy arrays where the goal number of sub-arrays can be any value?

For example array x, np.array([[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]])
Divided into 3 "folds" along axis-0 would be like [array([[1, 2, 3, 4]]), array([[5, 6, 7, 8]]), array([[ 9, 10, 11, 12]] in a list.

In [1]: import numpy as np
In [2]: a = np.array([[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]])
In [3]: np.array_split(a, 3)
Out[3]: [array([[1, 2, 3, 4]]), array([[5, 6, 7, 8]]), array([[ 9, 10, 11, 12]])]
axis=0 is the default, you can specify another axis as well
In [4]: np.array_split(a, 4, axis=1)
Out[4]:
[array([[1],
[5],
[9]]), array([[ 2],
[ 6],
[10]]), array([[ 3],
[ 7],
[11]]), array([[ 4],
[ 8],
[12]])]

Numpy append vectors of same shape to different axis

I have a numpy array
import numpy as np
initial_array = np.array([[
[0, 1],
[1, 2],
[2, 3],
[3, 4]],
[[4, 5],
[5, 6],
[6, 7],
[7, 8]]])
I have an array I want to add in:
to_add = np.array([
[ 8, 9],
[ 9, 10],
[10, 11],
[11, 12]])
Here, initial_array has a shape of (2, 4, 2) and to_add has a shape of (4, 2). I'm looking for the final result with a shape (3, 4, 2):
result = np.array([[
[ 0, 1],
[ 1, 2],
[ 2, 3],
[ 3, 4]],
[[ 4, 5],
[ 5, 6],
[ 6, 7],
[ 7, 8]],
[[ 8, 9],
[ 9, 10],
[10, 11],
[11, 12]]])
How can this be done without converting the numpy array back to a python list, is it possible to do this using numpy alone?

A lot of ways actually, I'm showing a couple:
>>> result = np.insert(initial_array, initial_array.shape[0], to_add, axis=0)
# or
>>> result = np.vstack((initial_array,to_add[None,...]))
# or
>>> result = np.array([*initial_array, to_add])

In addition to the other answers, you can also do that with np.newaxis():
np.concatenate([initial_array, to_add[np.newaxis, :]])
The result:
Out[75]:
array([[[ 0, 1],
[ 1, 2],
[ 2, 3],
[ 3, 4]],
[[ 4, 5],
[ 5, 6],
[ 6, 7],
[ 7, 8]],
[[ 8, 9],
[ 9, 10],
[10, 11],
[11, 12]]])

Without reshape:
np.concatenate((initial_array, [to_add]))

You could just add an additional axis to to_add so they can be directly concatenated:
np.concatenate([initial_array, to_add[None,:]])
array([[[ 0, 1],
[ 1, 2],
[ 2, 3],
[ 3, 4]],
[[ 4, 5],
[ 5, 6],
[ 6, 7],
[ 7, 8]],
[[ 8, 9],
[ 9, 10],
[10, 11],
[11, 12]]])

You can use numpy.append with to_add inside a list and appending only on the axis 0.
initial_array = np.array([[
[0, 1],
[1, 2],
[2, 3],
[3, 4]],
[[4, 5],
[5, 6],
[6, 7],
[7, 8]]])
to_add = np.array([
[ 8, 9],
[ 9, 10],
[10, 11],
[11, 12]])
final = np.append(initial_array, [to_add], axis=0)

Trouble with dimensional order when cloning a 1-dimensional numpy array to 3 dimensions

I'm having trouble finding a way to clone a 1-D numpy array to create a 3-D array. Say for example, I have
z = np.array([0, 2, 3, 5, 7, 9, 10])
This represents a vertical column in 3-D space (let's say it's in meters, for example). I want to create a horizontal dimension (x, y), so that the final array has dimensions (len(z), len(x), len(y)) , where each column at each x,y point is the same. I'm doing this so that I can match the spatial dimensions to other 3-D data that I have.
So with the array z as input, and given horizontal dimensions of, say, ndimx=3, ndimy=2, I want to find a function that outputs the array
np.array([ [[0, 0], [0, 0], [0, 0]],
[[2, 2], [2, 2], [2, 2]],
[[3, 3], [3, 3], [3, 3]],
[[5, 5], [5, 5], [5, 5]],
[[7, 7], [7, 7], [7, 7]],
[[9, 9], [9, 9], [9, 9]],
[[10, 10], [10, 10], [10, 10]] ])
which has a shape of (7, 3, 2). This seemed trivial to me at first, but after ages of experimenting with np.dstack(), np.astype(), np.repeat(), and transposes with ().T, I can't get the dimensional order right.
That's the critical thing here, that the vertical column is the first dimension. I'm willing to bet the answer is indeed trivial, I just can't find the magic numpy function that will do it.
Any suggestions?

We can simply get a view into input z with np.broadcast_to with no additional memory overhead and as such virtually free on performance -
np.broadcast_to(z[:,None,None],(len(z),ndimx,ndimy))
Sample run -
In [23]: z
Out[23]: array([ 0, 2, 3, 5, 7, 9, 10])
In [24]: ndimx=3; ndimy=2
In [25]: np.broadcast_to(z[:,None,None],(len(z),ndimx,ndimy))
Out[25]:
array([[[ 0, 0],
[ 0, 0],
[ 0, 0]],
[[ 2, 2],
[ 2, 2],
[ 2, 2]],
[[ 3, 3],
[ 3, 3],
[ 3, 3]],
[[ 5, 5],
[ 5, 5],
[ 5, 5]],
[[ 7, 7],
[ 7, 7],
[ 7, 7]],
[[ 9, 9],
[ 9, 9],
[ 9, 9]],
[[10, 10],
[10, 10],
[10, 10]]])

In [360]: z = np.array([0, 2, 3, 5, 7, 9, 10])
In [361]: z1 = np.stack([z,z], axis=1)
In [362]: z2 = np.stack([z1,z1,z1],axis=1)
In [363]: z2
Out[363]:
array([[[ 0, 0],
[ 0, 0],
[ 0, 0]],
[[ 2, 2],
[ 2, 2],
[ 2, 2]],
[[ 3, 3],
[ 3, 3],
[ 3, 3]],
...
[[10, 10],
[10, 10],
[10, 10]]])

Extracting patches of a certain size from the image in python efficiently

I have an image and I want to extract square patches of different sizes from it.
I need dense patches, that is, I need a patch at every pixel in the image.
For example if the image is 100x100 and the patch size is 64.
The result will be 10000 patches of size 64x64
These are the same patches which we use for filtering operations for example.
In case there is a boundary I would like to mirror the image.
What is the most efficient way of extracting patches using python?
Thanks

I think you are looking for something like this:
http://scikit-image.org/docs/0.9.x/api/skimage.util.html#view-as-windows

sklearn
You might want to have a look at sklearn.feature_extraction.image.extract_patches_2d and skimage.util.pad:
>>> from sklearn.feature_extraction.image import extract_patches_2d
>>> import numpy as np
>>> A = np.arange(4*4).reshape(4,4)
>>> window_shape = (2, 2)
>>> B = extract_patches_2d(A, window_shape)
>>> B[0]
array([[0, 1],
[4, 5]])
>>> B
array([[[ 0, 1],
[ 4, 5]],
[[ 1, 2],
[ 5, 6]],
[[ 2, 3],
[ 6, 7]],
[[ 4, 5],
[ 8, 9]],
[[ 5, 6],
[ 9, 10]],
[[ 6, 7],
[10, 11]],
[[ 8, 9],
[12, 13]],
[[ 9, 10],
[13, 14]],
[[10, 11],
[14, 15]]])
skimage
Expanding the answer of Stefan van der Walt a bit:
Install skimage
On Ubuntu
$ sudo apt-get install python-skimage
or
$ pip install scikit-image
Example from the docs
>>> from skimage.util import view_as_windows
>>> import numpy as np
>>> A = np.arange(4*4).reshape(4,4)
>>> A
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11],
[12, 13, 14, 15]])
>>> window_shape = (2, 2)
>>> B = view_as_windows(A, window_shape)
>>> B[0]
array([[[0, 1],
[4, 5]],
[[1, 2],
[5, 6]],
[[2, 3],
[6, 7]]])
>>> B
array([[[[ 0, 1],
[ 4, 5]],
[[ 1, 2],
[ 5, 6]],
[[ 2, 3],
[ 6, 7]]],
[[[ 4, 5],
[ 8, 9]],
[[ 5, 6],
[ 9, 10]],
[[ 6, 7],
[10, 11]]],
[[[ 8, 9],
[12, 13]],
[[ 9, 10],
[13, 14]],
[[10, 11],
[14, 15]]]])