Related
While trying to flip an numpy array I discovered the following problem:
When the arry consists of different shapes for each subarray np.flipud() works fine.
As soon as 2 or more subarrays have the same shape, but the shape of a third subarray differs, it throws a ValueError. The order of the subarrays doesn't matter in this case. While looking into the error I noticed that the same error already occurs when trying to convert the original python list into a numpy array.
import numpy as np
liste = []
liste.append(np.asarray([[1,2],[5,1],[4,5],[3,4],[3,4]]))
liste.append(np.asarray([[1,3,3,1],[1,3,3,1],[1,3,3,1],[1,3,3,1],[1,2,3,1]]))
liste.append(np.asarray([[1,3,3,1],[1,3,3,1],[1,3,3,1],[1,3,3,1],[1,2,3,1]]))
liste = np.asarray(liste)
liste = np.flipud(liste)
print(liste)
ValueError: could not broadcast input array from shape (5,2) into shape (5,)
How can I get around this issue?
In [15]: liste = []
...: liste.append(np.asarray([[1,2],[5,1],[4,5],[3,4],[3,4]]))
...: liste.append(np.asarray([[1,3,3,1],[1,3,3,1],[1,3,3,1],[1,3,3,1],[1,2,3,1]]))
...: liste.append(np.asarray([[1,3,3,1],[1,3,3,1],[1,3,3,1],[1,3,3,1],[1,2,3,1]]))
In [16]: liste
Out[16]:
[array([[1, 2],
[5, 1],
[4, 5],
[3, 4],
[3, 4]]),
array([[1, 3, 3, 1],
[1, 3, 3, 1],
[1, 3, 3, 1],
[1, 3, 3, 1],
[1, 2, 3, 1]]),
array([[1, 3, 3, 1],
[1, 3, 3, 1],
[1, 3, 3, 1],
[1, 3, 3, 1],
[1, 2, 3, 1]])]
np.array can make an object dtype array from arrays that differ in shape, but if the arrays have a common initial shape, it runs into problems. Making array from arrays that can differ in shape is inherently a fudge, an iffy task.
In [18]: np.array(liste,object)
Traceback (most recent call last):
File "<ipython-input-18-6790af79ec47>", line 1, in <module>
np.array(liste,object)
ValueError: could not broadcast input array from shape (5,2) into shape (5,)
A way around that is to crate an "blank" array of the right size, and then assign the list to it. This also works if the subarrays are all the same size and you still want 1d object dtype array.
In [19]: arr = np.zeros(3, object)
In [20]: arr[:] = liste
In [21]: arr
Out[21]:
array([array([[1, 2],
[5, 1],
[4, 5],
[3, 4],
[3, 4]]), array([[1, 3, 3, 1],
[1, 3, 3, 1],
[1, 3, 3, 1],
[1, 3, 3, 1],
[1, 2, 3, 1]]), array([[1, 3, 3, 1],
[1, 3, 3, 1],
[1, 3, 3, 1],
[1, 3, 3, 1],
[1, 2, 3, 1]])],
dtype=object)
Then the flip does work
In [22]: np.flipud(arr)
Out[22]:
array([array([[1, 3, 3, 1],
[1, 3, 3, 1],
[1, 3, 3, 1],
[1, 3, 3, 1],
[1, 2, 3, 1]]), array([[1, 3, 3, 1],
[1, 3, 3, 1],
[1, 3, 3, 1],
[1, 3, 3, 1],
[1, 2, 3, 1]]), array([[1, 2],
[5, 1],
[4, 5],
[3, 4],
[3, 4]])],
dtype=object)
But it's just as easy to flip/reverse the list:
In [23]: liste[::-1]
Out[23]:
[array([[1, 3, 3, 1],
[1, 3, 3, 1],
[1, 3, 3, 1],
[1, 3, 3, 1],
[1, 2, 3, 1]]),
array([[1, 3, 3, 1],
[1, 3, 3, 1],
[1, 3, 3, 1],
[1, 3, 3, 1],
[1, 2, 3, 1]]),
array([[1, 2],
[5, 1],
[4, 5],
[3, 4],
[3, 4]])]
I have a large matrix where I want to permute (or shift) one row of it.
For example:
np.array([[1, 2, 3, 4],
[1, 2, 3, 4],
[1, 2, 3, 4],
[1, 2, 3, 4]])
And the desired shifting output is: (for the second row by 1, for that example)
np.array([[1, 2, 3, 4],
[2, 3, 4, 1],
[1, 2, 3, 4],
[1, 2, 3, 4]])
This can be done naively by extracting the row of interest, permute and stick it back in the matrix.
I want a better solution that is in-place and efficient.
How to shift desired row or column by n places?
How to permute (change the order as desired)?
Can this be done efficiently for more than 1 row? for example shift the i'th row i places forward:
np.array([[1, 2, 3, 4],
[2, 3, 4, 1],
[3, 4, 1, 2],
[4, 1, 2, 3]])
You can do it indexing by slicing the rows and rolling them:
import numpy as np
a = np.array([[1, 2, 3, 4],
[1, 2, 3, 4],
[1, 2, 3, 4],
[1, 2, 3, 4]])
shift = 2
rows = [1, 3]
a[rows] = np.roll(a[rows], shift, axis=1)
array([[1, 2, 3, 4],
[3, 4, 1, 2],
[1, 2, 3, 4],
[3, 4, 1, 2]])
Suppose I have the following array:
a = [[1, 4, 2, 3]
[3, 1, 5, 4]
[4, 3, 1, 2]]
What I'd like to do is impose a maximum value on the array, but have that maximum vary by row. For instance if I wanted to limit the 1st and 3rd row to a maximum value of 3, and the 2nd row to a value of 4, I could create something like:
[[1, 3, 2, 3]
[3, 1, 4, 4]
[3, 3, 1, 2]
Is there any better way than just looping over each row individually and setting it with 'nonzero'?
With numpy.clip (using the method version here):
a.clip(max=np.array([3, 4, 3])[:, None]) # np.clip(a, ...)
# array([[1, 3, 2, 3],
# [3, 1, 4, 4],
# [3, 3, 1, 2]])
Generalized:
def clip_2d_rows(a, maxs):
maxs = np.asanyarray(maxs)
if maxs.ndim == 1:
maxs = maxs[:, np.newaxis]
return np.clip(a, a_min=None, a_max=maxs)
You might be safer using the module-level function (np.clip) rather than the class method (np.ndarray.clip). The former uses a_max as a parameter, while the latter uses the builtin max as a parameter which is never a great idea.
With masking -
In [50]: row_lims = np.array([3,4,3])
In [51]: np.where(a > row_lims[:,None], row_lims[:,None], a)
Out[51]:
array([[1, 3, 2, 3],
[3, 1, 4, 4],
[3, 3, 1, 2]])
With
>>> a
array([[1, 4, 2, 3],
[3, 1, 5, 4],
[4, 3, 1, 2]])
Say you have
>>> maxs = np.array([[3],[4],[3]])
>>> maxs
array([[3],
[4],
[3]])
What about doing
>>> a.clip(max=maxs)
array([[1, 3, 2, 3],
[3, 1, 4, 4],
[3, 3, 1, 2]])
I'm trying to make an array of 2 by n using numpy, elements inside come from specific columns selected by an array of column numbers.
For example if I have something like this
[[1, 2, 3],
[2, 3, 4]]
as my input array, and i want to have columns
[2,3,1,2,3],
i will get
[[2, 3, 1, 2, 3],
[3, 4, 2, 3, 4]]
as my output array
You want to slice along the second dimension. However, keep in mind that numpy uses zero based indexing. You'll need [1, 2, 0, 1, 2] instead of [2, 3, 1, 2, 3]
a = np.array([
[1, 2, 3],
[2, 3, 4]])
a[:, [1, 2, 0, 1, 2]]
array([[2, 3, 1, 2, 3],
[3, 4, 2, 3, 4]])
I have a large 2d array of vectors. I want to split this array into several arrays according to one of the vectors' elements or dimensions. I would like to receive one such small array if the values along this column are consecutively identical. For example considering the third dimension or column:
orig = np.array([[1, 2, 3],
[3, 4, 3],
[5, 6, 4],
[7, 8, 4],
[9, 0, 4],
[8, 7, 3],
[6, 5, 3]])
I want to turn into three arrays consisting of rows 1,2 and 3,4,5 and 6,7:
>>> a
array([[1, 2, 3],
[3, 4, 3]])
>>> b
array([[5, 6, 4],
[7, 8, 4],
[9, 0, 4]])
>>> c
array([[8, 7, 3],
[6, 5, 3]])
I'm new to python and numpy. Any help would be greatly appreciated.
Regards
Mat
Edit: I reformatted the arrays to clarify the problem
Using np.split:
>>> a, b, c = np.split(orig, np.where(orig[:-1, 2] != orig[1:, 2])[0]+1)
>>> a
array([[1, 2, 3],
[1, 2, 3]])
>>> b
array([[1, 2, 4],
[1, 2, 4],
[1, 2, 4]])
>>> c
array([[1, 2, 3],
[1, 2, 3]])
Nothing fancy here, but this good old-fashioned loop should do the trick
import numpy as np
a = np.array([[1, 2, 3],
[1, 2, 3],
[1, 2, 4],
[1, 2, 4],
[1, 2, 4],
[1, 2, 3],
[1, 2, 3]])
groups = []
rows = a[0]
prev = a[0][-1] # here i assume that the grouping is based on the last column, change the index accordingly if that is not the case.
for row in a[1:]:
if row[-1] == prev:
rows = np.vstack((rows, row))
else:
groups.append(rows)
rows = [row]
prev = row[-1]
groups.append(rows)
print groups
## [array([[1, 2, 3],
## [1, 2, 3]]),
## array([[1, 2, 4],
## [1, 2, 4],
## [1, 2, 4]]),
## array([[1, 2, 3],
## [1, 2, 3]])]
if a looks like this:
array([[1, 1, 2, 3],
[2, 1, 2, 3],
[3, 1, 2, 4],
[4, 1, 2, 4],
[5, 1, 2, 4],
[6, 1, 2, 3],
[7, 1, 2, 3]])
than this
col = a[:, -1]
indices = np.where(col[:-1] != col[1:])[0] + 1
indices = np.concatenate(([0], indices, [len(a)]))
res = [a[start:end] for start, end in zip(indices[:-1], indices[1:])]
print(res)
results in:
[array([[1, 2, 3],
[1, 2, 3]]), array([[1, 2, 4],
[1, 2, 4],
[1, 2, 4]]), array([[1, 2, 3],
[1, 2, 3]])]
Update: np.split() is much nicer. No need to add first and last index:
col = a[:, -1]
indices = np.where(col[:-1] != col[1:])[0] + 1
res = np.split(a, indices)