I replaced a Python list with a Numba int32 list and its concatenation behaviour wasn't as expected.
For example in Python: if a = [1,2,3] and b = [4,5,6] then a += b gives [1, 2, 3, 4, 5, 6].
In Numba, on the other hand:
if a = numba.int32([1,2,3]) and b = numba.int32([4,5,6]) then a += b gives array([5, 7, 9], dtype=int32).
Is there a way I can easily concatenate Numba lists as we do in Python? or is the only solution to write a function that loops over the two arrays and creates another one?
Thanks.
use the concatenate function. https://numpy.org/doc/stable/reference/generated/numpy.concatenate.html
import numpy as np
a = numba.int32([1,2,3])
b = numba.int32([4,5,6])
c = np.concatenate((a,b))
Use list()
a = numba.int32(list(a)+list(b))
Related
I have 3 lists and I need them to be 1 np.array() with 3 rows. The append method has not been working because it is created 3 separate arrays.
A = array([[1, 4, 1],
[4, 1, 9],
[1, 9, 1]])
[0.01665703 0.06662812 0.01665703]
[0.00049017 0.00012254 0.00110289]
[0.00012333 0.00110994 0.00012333]
ideal output (dtype: numpy.ndarray):
[[0.01665703 0.06662812 0.01665703]
[0.00049017 0.00012254 0.00110289]
[0.00012333 0.00110994 0.00012333]]
Attempted Code:
em = []
for list in A:
result = list / np.exp(list).sum(axis=0)
em.append(result)
Attempted code's output:
[array([0.01665703, 0.06662812, 0.01665703]),
array([0.00049017, 0.00012254, 0.00110289]),
array([0.00012333, 0.00110994, 0.00012333])]
import numpy as np
list_1 = [0.01665703, 0.06662812, 0.01665703]
list_2 = [0.00049017, 0.00012254, 0.00110289]
list_3 = [0.00012333, 0.00110994, 0.00012333]
combined = np.array([
list_1,
list_2,
list_3
])
Is this what you want?
This gives you the ideal output you specified.
probably not the best way to do it, but it works
import numpy as np
myList_1 = [0.01665703, 0.06662812, 0.01665703]
myList_2 = [0.00049017, 0.00012254, 0.00110289]
myList_3 = [0.00012333, 0.00110994, 0.00012333]
print(np.array([myList_1, myList_2, myList_3]))
Looking for a pythonic way to sum values from multiple lists:
I have got the following list of lists:
a = [0,5,2]
b = [2,1,1]
c = [1,1,1]
d = [5,3,4]
my_list = [a,b,c,d]
I am looking for the output:
[8,10,8]
I`ve used:
print ([sum(x) for x in zip(*my_list )])
but zip only works when I have 2 elements in my_list.
Any idea?
zip works for an arbitrary number of iterables:
>>> list(map(sum, zip(*my_list)))
[8, 10, 8]
which is, of course, roughly equivalent to your comprehension which also works:
>>> [sum(x) for x in zip(*my_list)]
[8, 10, 8]
Numpy has a nice way of doing this, it is also able to handle very large arrays. First we create the my_list as a numpy array as such:
import numpy as np
a = [0,5,2]
b = [2,1,1]
c = [1,1,1]
d = [5,3,4]
my_list = np.array([a,b,c,d])
To get the sum over the columns, you can do the following
np.sum(my_list, axis=0)
Alternatively, the sum over the rows can be retrieved by
np.sum(my_list, axis=1)
I'd make it a numpy array and then sum along axis 0:
my_list = numpy.array([a,b,c,d])
my_list.sum(axis=0)
Output:
[ 8 10 8]
I am trying to append a new row to an existing numpy array in a loop. I have tried the methods involving append, concatenate and also vstack none of them end up giving me the result I want.
I have tried the following:
for _ in col_change:
if (item + 2 < len(col_change)):
arr=[col_change[item], col_change[item + 1], col_change[item + 2]]
array=np.concatenate((array,arr),axis=0)
item+=1
I have also tried it in the most basic format and it still gives me an empty array.
array=np.array([])
newrow = [1, 2, 3]
newrow1 = [4, 5, 6]
np.concatenate((array,newrow), axis=0)
np.concatenate((array,newrow1), axis=0)
print(array)
I want the output to be [[1,2,3][4,5,6]...]
The correct way to build an array incrementally is to not start with an array:
alist = []
alist.append([1, 2, 3])
alist.append([4, 5, 6])
arr = np.array(alist)
This is essentially the same as
arr = np.array([ [1,2,3], [4,5,6] ])
the most common way of making a small (or large) sample array.
Even if you have good reason to use some version of concatenate (hstack, vstack, etc), it is better to collect the components in a list, and perform the concatante once.
If you want [[1,2,3],[4,5,6]] I could present you an alternative without append: np.arange and then reshape it:
>>> import numpy as np
>>> np.arange(1,7).reshape(2, 3)
array([[1, 2, 3],
[4, 5, 6]])
Or create a big array and fill it manually (or in a loop):
>>> array = np.empty((2, 3), int)
>>> array[0] = [1,2,3]
>>> array[1] = [4,5,6]
>>> array
array([[1, 2, 3],
[4, 5, 6]])
A note on your examples:
In the second one you forgot to save the result, make it array = np.concatenate((array,newrow1), axis=0) and it works (not exactly like you want it but the array is not empty anymore). The first example seems badly indented and without know the variables and/or the problem there it's hard to debug.
I have a list like this:
myList = [10,30,40,20,50]
Now I use numpy's argsort function to get the indices for the sorted list:
import numpy as np
so = np.argsort(myList)
which gives me the output:
array([0, 3, 1, 2, 4])
When I want to sort an array using so it works fine:
myArray = np.array([1,2,3,4,5])
myArray[so]
array([1, 4, 2, 3, 5])
But when I apply it to another list, it does not work but throws an error
myList2 = [1,2,3,4,5]
myList2[so]
TypeError: only integer arrays with one element can be converted to an
index
How can I now use so to sort another list without using a for-loop and without converting this list to an array first?
myList2 is a normal python list, and it does not support that kind of indexing.
You would either need to convert that to a numpy.array , Example -
In [8]: np.array(myList2)[so]
Out[8]: array([1, 4, 2, 3, 5])
Or you can use list comprehension -
In [7]: [myList2[i] for i in so]
Out[7]: [1, 4, 2, 3, 5]
You can't. You have to convert it to an array then back.
myListSorted = list(np.array(myList)[so])
Edit: I ran some benchmarks comparing the NumPy way to the list comprehension. NumPy is ~27x faster
>>> from timeit import timeit
>>> import numpy as np
>>> myList = list(np.random.rand(100))
>>> so = np.argsort(myList) #converts list to NumPy internally
>>> timeit(lambda: [myList[i] for i in so])
12.29590070003178
>>> myArray = np.random.rand(100)
>>> so = np.argsort(myArray)
>>> timeit(lambda: myArray[so])
0.42915570305194706
How do I convert a NumPy array into a Python List?
Use tolist():
>>> import numpy as np
>>> np.array([[1,2,3],[4,5,6]]).tolist()
[[1, 2, 3], [4, 5, 6]]
Note that this converts the values from whatever numpy type they may have (e.g. np.int32 or np.float32) to the "nearest compatible Python type" (in a list). If you want to preserve the numpy data types, you could call list() on your array instead, and you'll end up with a list of numpy scalars. (Thanks to Mr_and_Mrs_D for pointing that out in a comment.)
c = np.array([[1,2,3],[4,5,6]])
list(c.flatten())
The numpy .tolist method produces nested lists if the numpy array shape is 2D.
if flat lists are desired, the method below works.
import numpy as np
from itertools import chain
a = [1,2,3,4,5,6,7,8,9]
print type(a), len(a), a
npa = np.asarray(a)
print type(npa), npa.shape, "\n", npa
npa = npa.reshape((3, 3))
print type(npa), npa.shape, "\n", npa
a = list(chain.from_iterable(npa))
print type(a), len(a), a`
tolist() works fine even if encountered a nested array, say a pandas DataFrame;
my_list = [0,1,2,3,4,5,4,3,2,1,0]
my_dt = pd.DataFrame(my_list)
new_list = [i[0] for i in my_dt.values.tolist()]
print(type(my_list),type(my_dt),type(new_list))
Another option
c = np.array([[1,2,3],[4,5,6]])
c.ravel()
#>> array([1, 2, 3, 4, 5, 6])
# or
c.ravel().tolist()
#>> [1, 2, 3, 4, 5, 6]
also works.
The easiest way to convert array to a list is using the numpy package:
import numpy as np
#2d array to list
2d_array = np.array([[1,2,3],[8,9,10]])
2d_list = 2d_array.tolist()
To check the data type, you can use the following:
type(object)