The output from the method detect_keypoints(imagename, threshold) returns an ndarray, which looks like this:
[[131 326 1 0]
[135 281 1 0]
[159 405 1 0]]
Basically there's more data, but I'll keep it simple for this question. Just by this array, I'm not able to get certain columns, what is my actual goal. It keeps tellung me, that the above ndarray a tuple is and hence I can't access any value of it. So, I'm trying to convert the ndarray into a 2D array to read all integers of each array.
My implementation so far:
After research I implemented this one for converting:
def keypoints2D(imagename, threshold):
keypoints = detect_keypoints(imagename, threshold)
keypoints = np.array(keypoints)
lst = [list(i) for i in keypoints]
print(lst)
keypoints_lst = lst[:, 0]
print(keypoints_lst)
return lst
Doing so, I'll get
[[array([131, 326, 1, 0]), array([135, 281, 1, 0]), array([159, 405, 1, 0])]]
Even though, I'd like to get by converting something like this:
[[131, 326, 1, 0]
[135, 281, 1, 0]
[159, 405, 1, 0]]
If I try to get a certain column, it keeps telling me there's a exception in keypoints_lst = lst[:, 0]:
TypeError: list indices must be integers or slices, not tuple
In terms of the exception the return value is still a tuple. How can I change/convert it into a proper 2D array? Nevertheless the output of the ndarray should stay like this.
From the looks of it, the output
[[array([131, 326, 1, 0]), array([135, 281, 1, 0]), array([159, 405, 1, 0])]]
looks like a list of list of 1D-numpy arrays.
So a working but a bad way would be to convert lst to numpy array or maybe dont convert it into a list in the first place.
If you do plan to convert lst to numpy,here it goes.
lst = np.array(lst)
print(lst[:,:,0])#shall get you the first column
[[131 326 1 0]
[135 281 1 0]
[159 405 1 0]]
looks like a 2d array, which can simply be indexed with arr[:,0] for the first column, arr[0,:] the first row. arr[0,1] is actually arr[(0,1)]. Arrays take a tuple of indices.
Arrays display with the commas, lists with.
Your conversion looks like a nested list with array elements. Lists don't accept the multidimensional indexing (tuples).
alist[0] # first element
alist[0][1] # element of the first sublist
keypoints = np.array(keypoints)
lst = [list(i) for i in keypoints]
I don't know what the original keypoints (from detect_keypoints). But the lst line is creating a list of lists.
If I start with the nested list:
In [183]: target=[[131, 326, 1, 0],
...: [135, 281, 1, 0],
...: [159, 405, 1, 0]]
...:
In [184]: target
Out[184]: [[131, 326, 1, 0], [135, 281, 1, 0], [159, 405, 1, 0]]
I can make a 2d array, which displays as your first item:
In [185]: arr = np.array(target)
In [186]: arr
Out[186]:
array([[131, 326, 1, 0],
[135, 281, 1, 0],
[159, 405, 1, 0]])
In [187]: print(arr)
[[131 326 1 0]
[135 281 1 0]
[159 405 1 0]]
That is easily indexed by column
Simply passing that through your list comprehension produces:
In [190]: [list(i) for i in arr]
Out[190]: [[131, 326, 1, 0], [135, 281, 1, 0], [159, 405, 1, 0]]
tolist does the same thing:
In [191]: arr.tolist()
Out[191]: [[131, 326, 1, 0], [135, 281, 1, 0], [159, 405, 1, 0]]
The fact that you get a nested list of lists of arrays indicates that there's a another level of nesting in the source.
Just take out the comma in keypoints_lst = lst[:, 0]. When you include the comma, Python thinks you're giving a tuple, which you can't. Now, that will get rid of the error, but keypoints_lst = lst[:0] will just return an empty list, which I don't think you want. Python list slicing follows the format of mylist[start:stop:step]. When the start is not provided, it defaults to 0. So with the stop also being 0, you're going to get an empty list. Change the stop index to being the index of the last element you want to look at, plus one.
Related
I'm trying to merge two 2D NumPy arrays with a specific condition. Let's say we have:
A=[[100.121,200.129,1,2,3],
[105.343,203.347,2,2,1],
[107.426,201.657,1,3,1],
[100.121,300.010,1,1,1]]
and
B=[[107.426,201,675,80],
[100.121,200.129,70],
[100.121,300.010,90]]
I want to obtain:
C=[[100.121,200.129,1,2,3,70],
[105.343,203.347,2,2,1,0],
[107.426,201.657,1,3,1,80],
[100.121,300.010,1,1,1,90]]
So, when the values of the first and the second column match, take the third column in B and append it to A.
How can I do this?
Thanks.
You can try this
A=[[100,200,1,2,3],
[105,203,2,2,1],
[107,201,1,3,1]]
B=[[107,201,80],
[100,200,70],
[105,203,50]]
temp_B = list(zip(*B))
d_B = dict(zip(temp_B[0], temp_B[2]))
A = [i + [d_B.get(i[0],0)] for i in A]
# [[100, 200, 1, 2, 3, 70], [105, 203, 2, 2, 1, 50], [107, 201, 1, 3, 1, 80]]
I have an array (dataframe) with shape 9800, 9800. I need to index it (without labels) like:
x = (9800,9800)
a = x[0:7000,0:7000] (plus) x[7201:9800, 0:7000] (plus) x[0:7000, 7201:9800] (plus) x[7201:9800, 7201:9800]
b = x[7000:7200, 7000:7200]
c = x[7000:7200, 0:7000] (plus) x[7000:7200, 7201:9800]
d = x[0:7000, 7000:7200] (plus) x[7201:9800, 7000:7200]
What I mean by plus, is not a proper addition but more like a concatenation. Like putting the resulting dataframes together one next to the other. See attached image.
Is there any "easy" way of doing this? I need to replicate this to 10,000 dataframes and add them up individually to save memory.
You have np.r_, which basically creates an index array for you, for example:
np.r_[:3,4:6]
gives
array([0, 1, 2, 4, 5])
So in your case:
a_idx = np.r_[0:7000,7200:9000]
a = x[a_idx, a_idx]
c = x[7000:7200, a_idx]
In [167]: x=np.zeros((9800,9800),'int8')
The first list of slices:
In [168]: a = [x[0:7000,0:7000], x[7201:9800, 0:7000],x[0:7000, 7201:9800], x[7201:9800, 7201:9800]]
and their shapes:
In [169]: [i.shape for i in a]
Out[169]: [(7000, 7000), (2599, 7000), (7000, 2599), (2599, 2599)]
Since the shapes vary, you can't simply concatenate them all:
In [170]: np.concatenate(a, axis=0)
Traceback (most recent call last):
File "<ipython-input-170-c111dc665509>", line 1, in <module>
np.concatenate(a, axis=0)
File "<__array_function__ internals>", line 5, in concatenate
ValueError: all the input array dimensions for the concatenation axis must match exactly, but along dimension 1, the array at index 0 has size 7000 and the array at index 2 has size 2599
In [171]: np.concatenate(a, axis=1)
Traceback (most recent call last):
File "<ipython-input-171-227af3749524>", line 1, in <module>
np.concatenate(a, axis=1)
File "<__array_function__ internals>", line 5, in concatenate
ValueError: all the input array dimensions for the concatenation axis must match exactly, but along dimension 0, the array at index 0 has size 7000 and the array at index 1 has size 2599
You can concatenate subsets:
In [172]: np.concatenate(a[:2], axis=0)
Out[172]:
array([[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
...,
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0],
[0, 0, 0, ..., 0, 0, 0]], dtype=int8)
In [173]: _.shape
Out[173]: (9599, 7000)
I won't take the time to construct the other lists, but it looks like you could construct the first column with
np.concatenate([a[0], c[0], a[1]], axis=0)
similarly for the other columns, and then concatenate columns. Or join them by rows first.
np.block([[a[0],d[0],a[2]],[....]]) with an appropriate mix of list elements should do the same (just a difference in notation, same concatenation work).
I have the array
[[ 430 780 1900 420][ 0 0 2272 1704]]
and needs to convert it into this result:
[[[ 430 780 1] [1900 420 1]] [[ 0 0 1] [2272 1704 1]]]
basically turn a 2d array into 3d, separate each array into 2 and append the number 1 to it. How can I achieve it?
As pointed out in the comments, the question leaves some ambiguity about what would happen with bigger arrays, but one way to obtain the result that you indicate is this:
import numpy as np
a = np.array([[430, 780, 1900, 420], [0, 0, 2272, 1704]])
b = a.reshape(a.shape[0], -1, 2)
b = np.concatenate([b, np.ones_like(b[..., -1:])], -1)
print(b)
# [[[ 430 780 1]
# [1900 420 1]]
#
# [[ 0 0 1]
# [2272 1704 1]]]
Try this, for small size arrays(for large arrays consider #jdehesa answer).
>>> arr = [[ 430, 780, 1900, 420],[ 0, 0, 2272, 1704]]
>>> [[[a[0],a[1],1],[a[2],a[3],1]] for a in arr]
[[[430, 780, 1], [1900, 420, 1]], [[0, 0, 1], [2272, 1704, 1]]]
The first element of the first row should start with 0, and increment by 1 across the row, continues incrementing by 1 for the next column, and so on.
This is an example of what I am looking for
array([[0, 1, 2, 3],
[4, 5, 6, 7],
[8, 9, 10, 11],
...,
[5231, 5232, 5233, 5234],
[5235, 5236, 5237, 5238]], dtype=int32)
The solution should be able to apply for any specified 2D dimension, for example
array([[0, 1, 2, ..., 78, 79, 80],
[81, 82, 83, ..., 158, 159, 160],
...,
[2253, 2254, 2255, ..., 2453, 2454, 2455]], dtype=int32)
The examples aren't numerically accurate, I just wanted to demonstrate that it starts at 0, increments by 1 across the rows , and continues into the next row.
I was thinking of using a for loop to fill each value individually, but I am not sure if that is the fastest solution, nor the most pythonic and programmatically elegant solution.
You can use
np.arange(nrows*ncols).reshape(nrows,ncols)
Incidentally, this is how 90% of example 2D arrays are created in SO numpy posts.
Create a 1D array, initialize the array with the desired values, then use bumpy reshape to convert to a 2D array.
I am trying to remove the loop from this matrix multiplication (and learn more about optimizing code in general), and I think I need some form of np.broadcasting or np.einsum, but after reading up on them, I'm still not sure how to use them for my problem.
A = np.array([[1, 2, 3, 4, 5],
[6, 7, 8, 9, 10],
[11,12,13,14,15]])
#A is a 3x5 matrix, such that the shape of A is (3, 5) (and A[0] is (5,))
B = np.array([[1,0,0],
[0,2,0],
[0,0,3]])
#B is a 3x3 (diagonal) matrix, with a shape of (3, 3)
C = np.zeros(5)
for i in range(5):
C[i] = np.linalg.multi_dot([A[:,i].T, B, A[:,i]])
#Each row of matrix math is [1x3]*[3x3]*[3x1] to become a scaler value in each row
#C becomes a [5x1] matrix with a shape of (5,)
I know I can't just do np.multidot by itself, because that results in a (5,5) array.
I also found this: Multiply matrix by each row of another matrix in Numpy, but I can't tell if it's actually the same problem as mine.
In [601]: C
Out[601]: array([436., 534., 644., 766., 900.])
This is a natural for einsum. I use i as you do, to denote the index that carries through to the result. j and k are indices that are used in the sum of products.
In [602]: np.einsum('ji,jk,ki->i',A,B,A)
Out[602]: array([436, 534, 644, 766, 900])
It probably can also be done with mutmul, though it may require adding a dimension and latter squeezing.
dot approaches that use diag do a lot more work than necessary. The diag throws out a lot of values.
To use matmul we have to make the i dimension the first of 3d arrays. That's the 'passive' one carries over to the result:
In [603]: A.T[:,None,:]#B#A.T[:,:,None]
Out[603]:
array([[[436]], # (5,1,1) result
[[534]],
[[644]],
[[766]],
[[900]]])
In [604]: (A.T[:,None,:]#B#A.T[:,:,None]).squeeze()
Out[604]: array([436, 534, 644, 766, 900])
Or index the extra dimensions away: (A.T[:,None,:]#B#A.T[:,:,None])[:,0,0]
You can chain to calls to dot together, then get the diagonal:
# your original output:
# >>> C
# array([436., 534., 644., 766., 900.])
>>> np.diag(np.dot(np.dot(A.T,B), A))
array([436, 534, 644, 766, 900])
Or equivalently, use your original multi_dot train of thought, but take the diagonal of the resulting 5x5 array. This may have some performance boosts (according to the docs)
>>> np.diag(np.linalg.multi_dot([A.T, B, A]))
array([436, 534, 644, 766, 900])
atTo add to the answers. If you want to make multiply the matrices you can make use of broadcasting. Edit: Note this is element wise multiplication, not dot products. For that you can use the dot methods.
B [...,None] * A
Gives:
array([[[ 1, 2, 3, 4, 5],
[ 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 0]],
[[ 0, 0, 0, 0, 0],
[12, 14, 16, 18, 20],
[ 0, 0, 0, 0, 0]],
[[ 0, 0, 0, 0, 0],
[ 0, 0, 0, 0, 0],
[33, 36, 39, 42, 45]]])