Develop Reference

How to add or remove a specific element from a numpy 2d array?

Given the following numpy array:
arr = np.array([
[1,2,3],
[4,5,6],
[7,8,9]
])
delete and return:
arr = np.array([
[1,2,3],
[4,6],
[7,8,9]
])
I want to delete 5 from this array. or delete arr[1][2] only. When I am using del arr[i][j] it throws the following err. ValueError: cannot delete array elements and numpy documentation is not clear on this case for me.
Similarly how to add an element to some rows in the same array?
To be specific, When I am reading an image with opencv I am getting this err.
rgb_image = cv2.imread("image.png")
del operation gives me the top error and I couldnt make it with np.delete(...)

A numpy array (ndarray) is quote:
An ndarray is a (usually fixed-size) multidimensional container of items of the same type and size.
So you cannot have rows of different lengths if you want to use the ndarray data structure (with all of its optimizations).
A possible workaround is to have an array of lists
>>> arr=np.array([
[1,2,3],
[4,5,6],
[7,8,9],
[]
])
(note the empty row to escape the ndarray datatype)
so you can delete an element from one of the lists
>>> arr
array([list([1, 2, 3]), list([4, 5, 6]), list([7, 8, 9]), list([])],
dtype=object)
>>> arr[1]=np.delete(arr[1], [1], axis=0)
>>> arr
array([list([1, 2, 3]), array([4, 6]), list([7, 8, 9]), list([])],
dtype=object)

I think the one way would be to cast np.array to list and repeat cast to np.array, like this:
arr = arr.tolist()
arr[1].pop(1)
arr = np.array(arr)
Edit:
It seems to be right, numpy way:
np.delete(arr, [4, 4])
np.split(arr, [3, 5, 9])
Edit2:
Doesn't seems to be less time consuming, but you could check this way:
arr = np.empty(3, dtype=np.object)
arr[:] = [1,2,3], [4,5,6], [7,8,9]
arr[1].remove(5)

First convert the Array into a list using
new_list = list(old_array) function.(This will create a list of arrays)
Now,you can perfome all the operations of list like pop,filter, etc to remove whatever elements you want.
Finally when you have your filtered list, convert it back to array using
new_array = np.array(new_list) .(This new array will retain the dimensions of old array)

Python - How to create an empty numpy array and append to it, like lists [duplicate]

This question already has answers here:
How to add a new row to an empty numpy array
(7 answers)
Closed 4 years ago.
This is something I didn't find so easily on the internet. It's easy to create np.arrays, but how do you create an empty one so you can append on the go?

1. A Hack
You can create an empty array using the np.empty() function and specify the dimensions to be (0, 0) and then use np.append() to append items later.
>>> a = np.empty((0, 0))
>>> a
array([], shape=(0, 0), dtype=float64)
>>> b = np.append(a, [1, 2])
>>> b
array([1., 2.])
2. However...
The hack above is not advisable, use it with caution. Appending to lists has O(N) complexity, while appending to arrays has O(N^2) (besides different memory use). The proper way should be, then, to append to lists. Note that using list() on numpy arrays to transform them into lists is not correct, as you will get a list of numpy arrays. Instead, use the .tolist() method.
>>> a = np.array([[1, 2], [3, 4]])
>>>
>>> list(a)
[array([1, 2]), array([3, 4])]
>>>
>>> a.tolist()
[[1, 2], [3, 4]]

It is advised to use normal python lists and then at the end convert the list to a numpy array.
x = np.asarray(x)

Concatenate 2d list in python

I'm trying to create a 2d list with shape of [n,784] (the same shape as the MNIST image batches) using multiple [1,784] lists.
mylist.append(element) doesn't give me what I'm looking for, where mylist is the 2d [n,784] list and element is the [1,784] lists. It would return a list with shape [n,1,784].
I've also tried mylist[index].append(element), and I got a [784] 1d list instead.
Any idea how to solve my problem?
Thanks a lot

import numpy as np
myarray = np.array(mylist)
newarray = np.concatenate((myarray, element))
And if you want to turn it back into a list:
newlist = newarray.tolist()

a = [[1,1],[2,2]]
b = np.concatenate([a, a], axis=1).tolist()
The output will be:
[[1, 1, 1, 1], [2, 2, 2, 2]]

Appending a new row to a numpy array

I am trying to append a new row to an existing numpy array in a loop. I have tried the methods involving append, concatenate and also vstack none of them end up giving me the result I want.
I have tried the following:
for _ in col_change:
if (item + 2 < len(col_change)):
arr=[col_change[item], col_change[item + 1], col_change[item + 2]]
array=np.concatenate((array,arr),axis=0)
item+=1
I have also tried it in the most basic format and it still gives me an empty array.
array=np.array([])
newrow = [1, 2, 3]
newrow1 = [4, 5, 6]
np.concatenate((array,newrow), axis=0)
np.concatenate((array,newrow1), axis=0)
print(array)
I want the output to be [[1,2,3][4,5,6]...]

The correct way to build an array incrementally is to not start with an array:
alist = []
alist.append([1, 2, 3])
alist.append([4, 5, 6])
arr = np.array(alist)
This is essentially the same as
arr = np.array([ [1,2,3], [4,5,6] ])
the most common way of making a small (or large) sample array.
Even if you have good reason to use some version of concatenate (hstack, vstack, etc), it is better to collect the components in a list, and perform the concatante once.

If you want [[1,2,3],[4,5,6]] I could present you an alternative without append: np.arange and then reshape it:
>>> import numpy as np
>>> np.arange(1,7).reshape(2, 3)
array([[1, 2, 3],
[4, 5, 6]])
Or create a big array and fill it manually (or in a loop):
>>> array = np.empty((2, 3), int)
>>> array[0] = [1,2,3]
>>> array[1] = [4,5,6]
>>> array
array([[1, 2, 3],
[4, 5, 6]])
A note on your examples:
In the second one you forgot to save the result, make it array = np.concatenate((array,newrow1), axis=0) and it works (not exactly like you want it but the array is not empty anymore). The first example seems badly indented and without know the variables and/or the problem there it's hard to debug.

How to sort a list based on the output of numpy's argsort function

I have a list like this:
myList = [10,30,40,20,50]
Now I use numpy's argsort function to get the indices for the sorted list:
import numpy as np
so = np.argsort(myList)
which gives me the output:
array([0, 3, 1, 2, 4])
When I want to sort an array using so it works fine:
myArray = np.array([1,2,3,4,5])
myArray[so]
array([1, 4, 2, 3, 5])
But when I apply it to another list, it does not work but throws an error
myList2 = [1,2,3,4,5]
myList2[so]
TypeError: only integer arrays with one element can be converted to an
index
How can I now use so to sort another list without using a for-loop and without converting this list to an array first?

myList2 is a normal python list, and it does not support that kind of indexing.
You would either need to convert that to a numpy.array , Example -
In [8]: np.array(myList2)[so]
Out[8]: array([1, 4, 2, 3, 5])
Or you can use list comprehension -
In [7]: [myList2[i] for i in so]
Out[7]: [1, 4, 2, 3, 5]

You can't. You have to convert it to an array then back.
myListSorted = list(np.array(myList)[so])
Edit: I ran some benchmarks comparing the NumPy way to the list comprehension. NumPy is ~27x faster
>>> from timeit import timeit
>>> import numpy as np
>>> myList = list(np.random.rand(100))
>>> so = np.argsort(myList) #converts list to NumPy internally
>>> timeit(lambda: [myList[i] for i in so])
12.29590070003178
>>> myArray = np.random.rand(100)
>>> so = np.argsort(myArray)
>>> timeit(lambda: myArray[so])
0.42915570305194706