Related
Consider 2D Numpy array A and in-place function x like
A = np.arange(9).reshape(3,3)
def x(M):
M[:,2] = 0
Now, I have a list (or 1D numpy array) L pointing the rows, I want to select and apply the function f on them like
L = [0, 1]
x(A[L, :])
where the output will be written to A. Since I used index access to A, the matrix A is not affected at all:
A = array([[0, 1, 2],
[3, 4, 5],
[6, 7, 8]])
What I actually need is to slice the matrix such as
x(A[:2, :])
giving me the desired output
A = array([[0, 1, 0],
[3, 4, 0],
[6, 7, 8]])
The question is now, how to provide Numpy array slicing by the list L (either any automatic conversion of list to slice or if there is any build in function for that), because I am not able to convert the list L easily to slice like :2 in this case.
Note that I have both large matrix A and list L in my problem - that is the reason, why I would need the in-place operations to control the available memory.
Can you modify the function so as you can pass slice L inside it:
def func(M,L):
M[L,2] = 0
func(A,L)
print(A)
Out:
array([[0, 1, 0],
[3, 4, 0],
[6, 7, 8]])
I have a 2d numpy array and a 2d numpy subarray that I want to add to the original array based on a condition.
I know that you can add the 2d subarray to the array like this:
original_array[start_x:end_x, start_y:end_y] = sub_array
but I dont know how to efficiently add only values of the sub_array that are bigger than 0?
Example:
orginal_array = np.array([2,2],[2,2],[2,2],[2,2])
sub_array = np.array([0,0],[1,1],[0,1],[0,0])
expected_result = np.array([2,2], [1,1], [2,1], [2,2])
You can index based on the condition >,< 0 and add the arrays.
orginal_array * (sub_array <= 0) + sub_array * (sub_array > 0)
array([[2, 2],
[1, 1],
[2, 1],
[2, 2]])
Another approach is to use the np.where function as:
np.where(sub_array > 0, sub_array, original_array)
Output:
array([[2, 2],
[1, 1],
[2, 1],
[2, 2]])
Try,
sub_array2 = np.select([sub_array>0],[sub_array])
original_array[start_x:end_x, start_y:end_y] = sub_array2
Suppose that two integer arrays min and max are given and they have equal shape. How to generate all Numpy arrays such that min[indices] <= ar[indices] <= max[indices] for all indices in np.ndindex(shape)? I have looked at the Numpy array creation routines but none of them seem to do what I want. I considered also starting with the min array and looping over its indices, adding 1 until the corresponding entry in max was reached, but I want to know if Numpy provides methods to do this more cleanly. As an example, if
min = np.array([[0, 1],
[2, 3]])
max = np.array([[0, 1],
[3, 4]])
Then I would like to have as an output:
[np.array([[0, 1],
[2, 3]]),
np.array([[0, 1],
[3, 3]]),
np.array([[0, 1],
[2, 4]]),
np.array([[0, 1],
[3, 4]])]
The reason I want to do this is that I am coding some implementations of tabular RL methods and the most natural way to specify states and actions is by using arrays. However, I also need to implement Q-tables and I would ideally have these tables be represented by 2D matrices. I want to map the states and actions to indices and use those to acces the Q-table. I know I can write a multidimensional Q-table, but then the TF-Agents library I need to use runs some checks on the inputs that throw errors. In any case, by assumption, the tabular RL problems I would solve with this method are relatively small, so the bounds on the matrices will restrict the number of resultants.
This will work,
Also since range and itertools.product both returns a generator it's memory efficient (O(1) space).
import numpy as np
from itertools import product
a = np.array([[0, 1],
[2, 3]])
b = np.array([[0, 1],
[3, 4]])
l, m = np.shape(a)
ranges = [range(i, j+1) for i, j in zip(a.ravel(), b.ravel())]
ans = [np.array(value).reshape(l,m) for value in product(*ranges)]
print(ans)
Output
[array([[0, 1],
[2, 3]]), array([[0, 1],
[2, 4]]), array([[0, 1],
[3, 3]]), array([[0, 1],
[3, 4]])]
It is a bit hard to explain what I want to do, so the best way is to show an example I think.
I have a 2D numpy array which contains a list of integer pairs. Those integers go from 0 to N and each appears in 2 pairs of the list, except for two of them (which will be called the extrema). So the list contains N-1 pairs.
I would like to reorder the array to have it starting by one of the pair containing an extremum, and having the next pair starting by the previous pair end... very confusing so look at this example :
I start with this array :
array([[3, 0], [3, 2], [4, 0], [1, 2]])
and I would like to end up with this one :
array([[1, 2], [2, 3], [3, 0], [0, 4]])
here is a algorithm that works but which contains a loop over N-1... in this case it is not a problem since N=5 but I would like to do it with N=100000 or more so I would like to do it without an explicit python loop but can't figure out a way...
import numpy as np
co = np.array([[3, 0], [3, 2], [4, 0], [1, 2]])
extrema = np.nonzero(np.bincount(co.flat) == 1)[0]
nb_el = co.shape[0]
new_co = np.empty_like(co)
start = extrema[0]
for el_idx in xrange(nb_el):
where_idx = np.where(co == start)
if where_idx[1][0] == 1:
new_co[el_idx] = co[where_idx[0][0]][::-1]
else:
new_co[el_idx] = co[where_idx[0][0]]
co = np.delete(co, where_idx[0], 0)
start = new_co[el_idx][-1]
print new_co
# array([[1, 2], [2, 3], [3, 0], [0, 4]])
How do I convert a simple list of lists into a numpy array? The rows are individual sublists and each row contains the elements in the sublist.
If your list of lists contains lists with varying number of elements then the answer of Ignacio Vazquez-Abrams will not work. Instead there are at least 3 options:
1) Make an array of arrays:
x=[[1,2],[1,2,3],[1]]
y=numpy.array([numpy.array(xi) for xi in x])
type(y)
>>><type 'numpy.ndarray'>
type(y[0])
>>><type 'numpy.ndarray'>
2) Make an array of lists:
x=[[1,2],[1,2,3],[1]]
y=numpy.array(x)
type(y)
>>><type 'numpy.ndarray'>
type(y[0])
>>><type 'list'>
3) First make the lists equal in length:
x=[[1,2],[1,2,3],[1]]
length = max(map(len, x))
y=numpy.array([xi+[None]*(length-len(xi)) for xi in x])
y
>>>array([[1, 2, None],
>>> [1, 2, 3],
>>> [1, None, None]], dtype=object)
>>> numpy.array([[1, 2], [3, 4]])
array([[1, 2], [3, 4]])
As this is the top search on Google for converting a list of lists into a Numpy array, I'll offer the following despite the question being 4 years old:
>>> x = [[1, 2], [1, 2, 3], [1]]
>>> y = numpy.hstack(x)
>>> print(y)
[1 2 1 2 3 1]
When I first thought of doing it this way, I was quite pleased with myself because it's soooo simple. However, after timing it with a larger list of lists, it is actually faster to do this:
>>> y = numpy.concatenate([numpy.array(i) for i in x])
>>> print(y)
[1 2 1 2 3 1]
Note that #Bastiaan's answer #1 doesn't make a single continuous list, hence I added the concatenate.
Anyway...I prefer the hstack approach for it's elegant use of Numpy.
It's as simple as:
>>> lists = [[1, 2], [3, 4]]
>>> np.array(lists)
array([[1, 2],
[3, 4]])
Again, after searching for the problem of converting nested lists with N levels into an N-dimensional array I found nothing, so here's my way around it:
import numpy as np
new_array=np.array([[[coord for coord in xk] for xk in xj] for xj in xi], ndmin=3) #this case for N=3
The OP specified that "the rows are individual sublists and each row contains the elements in the sublist".
Assuming that the use of numpy is not prohibited (given that the flair numpy has been added in the OP), use vstack:
import numpy as np
list_of_lists= [[1, 2, 3], [4, 5, 6], [7 ,8, 9]]
array = np.vstack(list_of_lists)
# array([[1, 2, 3],
# [4, 5, 6],
# [7, 8, 9]])
or simpler (as mentioned in another answer),
array = np.array(list_of_lists)
As mentioned in the other answers, np.vstack() will let you convert your list-of-lists(nested list) into a 1-dimensional array of sublists. But if you are looking to convert the list of lists into a 2-dimensional numpy.ndarray. Then you can use the numpy.asarray() function.
For example, if you have a list of lists named y_true that looks like:
[[0, 1, 0], [1, 0, 0], [0, 0, 1], [1, 0, 0], [0, 1, 0], [0, 0, 1], [1, 0, 0]]
<class 'list'>
This line y_true = np.asarray(y_true) will convert the list of lists into a 2-dimensional numpy ndarray that looks like:
[[0 1 0]
[1 0 0]
[0 0 1]
[1 0 0]
[0 1 0]
[0 0 1]
[1 0 0]]
<class 'numpy.ndarray'>
Additionally, you can also specify the dtype parameter like np.asarray(y_true, dtype = float) to have your array values in your desired data type.
I had a list of lists of equal length. Even then Ignacio Vazquez-Abrams's answer didn't work out for me. I got a 1-D numpy array whose elements are lists. If you faced the same problem, you can use the below method
Use numpy.vstack
import numpy as np
np_array = np.empty((0,4), dtype='float')
for i in range(10)
row_data = ... # get row_data as list
np_array = np.vstack((np_array, np.array(row_data)))
Just use pandas
list(pd.DataFrame(listofstuff).melt().values)
this only works for a list of lists
if you have a list of list of lists you might want to try something along the lines of
lists(pd.DataFrame(listofstuff).melt().apply(pd.Series).melt().values)