I have the following matrix
B = [[1,2], [3,4]]
and would like to store the matrix as lines of the syntax i j b_ij where i and j are the matrix indices and b_ij is the value at that indexed position.
That is, the matrix above would look like:
0 0 1
0 1 2
1 0 3
1 1 4
Is there anyway to do this with a library in Python? Also, is this a common format for storing a matrix as raw bytes? I know it is easy enough to iterate over a matrix to store it in this fashion but that seems rather inefficient.
It's not a library, but you can just use a list comprehension:
>>> B = [[1, 2], [3, 4]]
>>> matrix = [ [i, j, B[i][j]] for i in range(len(B)) for j in range(len(B[i])) ]
>>> print(matrix)
[[0, 0, 1], [0, 1, 2], [1, 0, 3], [1, 1, 4]]
You could also expand the for loop to do something similar, or nest comprehensions inside each other if your matrix B goes deeper than this.
Related
I have a relatively simple problem that I cannot solve without using loops. It is difficult for me to figure out the correct title for this problem.
Lets say we have two numpy arrays:
array_1 = np.array([[0, 1, 2],
[3, 3, 3],
[3, 3, 4],
[3, 6, 2]])
array_2 = np.array([[0, 0, 0],
[1, 1, 1],
[2, 2, 2],
[3, 3, 3],
[4, 4, 4],
[5, 5, 5],
[6, 6, 6]])
array_1 represents indices of the rows in array_2 that we want to sum. So for example, 4th row in result array should contain summed all rows in array_2 that have same row indices as all 3s in array_1.
It is much easier to understand it in the code:
result = np.empty(array_2.shape)
for i in range(array_1.shape[0]):
for j in range(array_1.shape[1]):
index = array_1[i, j]
result[index] = result[index] + array_2[i]
Result should be:
[[ 0 0 0]
[ 0 0 0]
[ 3 3 3]
[10 10 10]
[ 2 2 2]
[ 0 0 0]
[ 3 3 3]]
I tried to use np.einsum but I need to use both elements in array as indices and also its rows as indices so I'm not sure if np.einsum is the best path here.
This is the problem I have in graphics. array_1 represent indices of vertices for triangles and array_2 represents normals where index of a row corresponds to the index of the vertex
Any time you're adding something from a repeated index, normal ufuncs like np.add don't work out of the box because they only process a repeated fancy index once. Instead, you have to use the unbuffered version, which is np.add.at.
Here, you have a pair of indices: the row in array_1 is the row index into array_2, and the element of array_1 is the row index into the output.
First, construct the indices explicitly as fancy indices. This will make it much simpler to use them:
output_row = array_1.ravel()
input_row = np.repeat(np.arange(array_1.shape[0]), array_1.shape[1]).ravel()
You can apply input_row directly to array_2, but you need add.at to use output_row:
output = np.zeros_like(array_2)
np.add.at(output, output_row, array_2[input_row])
You really only use the first four rows of array_2, so it could be truncated to
array_2 = array2[:array_1.shape[0]]
In that case, you would want to initialize the output as:
output = np.zeros_like(array_2, shape=(output_row.max() + 1, array2.shape[1]))
I know that you cant stack or concatenate arrays of different lenghths in NumPy as all matrices need to be rectangular, but is there any other way to achieve this?
For example:
a = [1, 2 ,3]
b = [9, 8]
Stacking them would give:
c = [[1, 2, 3]
[9, 8]]
alternatively if there is no way to create the above how could I write a function to get this: (0 in place of missing element to fill matrix)?
c = [[1, 2, 3]
[9, 8, 0]]
This code worked for me:
a = [1, 2 ,3]
b = [9,8]
while len(b) != len(a):
if len(b) > len(a):
a.append(0)
else:
b.append(0)
final = np.array([a,b])
print(final)
The code is self explanatory, but I will try my best to give a valid explanation:
We take two lists (say a and b) and we compare there lengths, if they are unequal we add element (in this case 0) to the one whose length is lower, this loops until their lengths are equal, then it simply converts them into a 2D array in numpy
Also you can replace 0 with np.NaN if you want NaN values
I think what you are looking for is:
In:
from itertools import zip_longest
a = [1, 2 ,3]
b = [9, 8]
c = np.array(list(zip_longest(*[a,b])),dtype=float).transpose()
print(c)
Out:
[[ 1. 2. 3.]
[ 9. 8. nan]]
Take a look at the following code in MATLAB:
a = [1,2; 5,6]
b = [-1,1; -1,1]
d = a(b(:)>0)
Now d will be the 2x1 array,[2;6]. This is because array b has positive entry only at the positions (1,2) and (2,2), and the third line of the code is extracting elements of a in those positions.
Is there an equivalent method in Python that does this? I searched numpy documentation but could not find any. In my actual code, I have multiple large, multidimensional arrays from which I would want to extract elements based on the elements of other arrays. Of course, this can be done with nested for loops but it would be much better if there is a nicer way like MATLAB does.
Assuming a and b are numpy arrays use:
d = a[b > 0]
In numpy, indexing is done with the [] operator.
Without using any libraries:
a = [[1, 2], [5, 6]]
b = [[-1, 1], [-1, 1]]
d = [
a_xy
for a_x, b_x in zip(a, b)
for a_xy, b_xy in zip(a_x, b_x)
if b_xy > 0
]
Using numpy:
import numpy as np
a = np.array([[1, 2], [5, 6]])
b = np.array([[-1, 1], [-1, 1]])
d = a[b > 0]
I am relatively new to python, and a piece of existing code has created an object akin to per below. This is part of a legacy piece of code. i can unfortunately not change it. The code creates many objects that look like the following format:
[[{'a': 2,'b': 3}],[{'a': 1,'c': 3}],[{'c': 2,'d': 4}]]
I am trying to create transform this object into a matrix or numpy arrays. In this specific example - it would have three rows (1,2,3) and 4 columns (a,b,c,d), with the dictionary values inserted in the cells. (I have inserted how this matrix would look as a dinky toy example. However - i am not looking to recreate the table from scratch, but i am looking for code that translate the object per above in a matrix format).
I am struggling to find a fast and easy way to do this. Any tips or advice much appreciated.
a b c d
1 2 3 0 0
2 1 0 3 0
3 0 2 0 4
I suspect you are focusing on the fast and easy, when you need to address the how first. This isn't the normal input format for np.array or `pandas. So let's focus on that.
It's a list of lists; suggesting a 2d array. But each sublist contains one dictionary, not a list of values.
In [633]: dd=[[{'a': 2,'b': 3}],[{'a': 1,'c': 3}],[{'c': 2,'d': 4}]]
In [634]: dd[0]
Out[634]: [{'b': 3, 'a': 2}]
So let's define a function that converts a dictionary into a list of numbers. We can address the question of where a,b,c,d labels come from, and whether you need to collect them from dd or not, later.
In [635]: dd[0][0]
Out[635]: {'b': 3, 'a': 2}
In [636]: def mk_row(adict):
return [adict.get(k,0) for k in ['a','b','c','d']]
.....:
In [637]: mk_row(dd[0][0])
Out[637]: [2, 3, 0, 0]
So now we just need to apply the function to each sublist
In [638]: [mk_row(d[0]) for d in dd]
Out[638]: [[2, 3, 0, 0], [1, 0, 3, 0], [0, 0, 2, 4]]
This is the kind of list that #Colin fed to pandas. It can also be given to np.array:
In [639]: np.array([mk_row(d[0]) for d in dd])
Out[639]:
array([[2, 3, 0, 0],
[1, 0, 3, 0],
[0, 0, 2, 4]])
Simpy use:
import pandas as pd
df = pd.DataFrame.from_items([('1', [2, 3, 0,0]), ('2', [1, 0, 3,0]),('3', [0, 2, 0,4])], orient='index', columns=['a', 'b', 'c','d'])
arr = df.values
You can then reference it like a normal numpy array:
print(arr[0,:])
How do I convert a simple list of lists into a numpy array? The rows are individual sublists and each row contains the elements in the sublist.
If your list of lists contains lists with varying number of elements then the answer of Ignacio Vazquez-Abrams will not work. Instead there are at least 3 options:
1) Make an array of arrays:
x=[[1,2],[1,2,3],[1]]
y=numpy.array([numpy.array(xi) for xi in x])
type(y)
>>><type 'numpy.ndarray'>
type(y[0])
>>><type 'numpy.ndarray'>
2) Make an array of lists:
x=[[1,2],[1,2,3],[1]]
y=numpy.array(x)
type(y)
>>><type 'numpy.ndarray'>
type(y[0])
>>><type 'list'>
3) First make the lists equal in length:
x=[[1,2],[1,2,3],[1]]
length = max(map(len, x))
y=numpy.array([xi+[None]*(length-len(xi)) for xi in x])
y
>>>array([[1, 2, None],
>>> [1, 2, 3],
>>> [1, None, None]], dtype=object)
>>> numpy.array([[1, 2], [3, 4]])
array([[1, 2], [3, 4]])
As this is the top search on Google for converting a list of lists into a Numpy array, I'll offer the following despite the question being 4 years old:
>>> x = [[1, 2], [1, 2, 3], [1]]
>>> y = numpy.hstack(x)
>>> print(y)
[1 2 1 2 3 1]
When I first thought of doing it this way, I was quite pleased with myself because it's soooo simple. However, after timing it with a larger list of lists, it is actually faster to do this:
>>> y = numpy.concatenate([numpy.array(i) for i in x])
>>> print(y)
[1 2 1 2 3 1]
Note that #Bastiaan's answer #1 doesn't make a single continuous list, hence I added the concatenate.
Anyway...I prefer the hstack approach for it's elegant use of Numpy.
It's as simple as:
>>> lists = [[1, 2], [3, 4]]
>>> np.array(lists)
array([[1, 2],
[3, 4]])
Again, after searching for the problem of converting nested lists with N levels into an N-dimensional array I found nothing, so here's my way around it:
import numpy as np
new_array=np.array([[[coord for coord in xk] for xk in xj] for xj in xi], ndmin=3) #this case for N=3
The OP specified that "the rows are individual sublists and each row contains the elements in the sublist".
Assuming that the use of numpy is not prohibited (given that the flair numpy has been added in the OP), use vstack:
import numpy as np
list_of_lists= [[1, 2, 3], [4, 5, 6], [7 ,8, 9]]
array = np.vstack(list_of_lists)
# array([[1, 2, 3],
# [4, 5, 6],
# [7, 8, 9]])
or simpler (as mentioned in another answer),
array = np.array(list_of_lists)
As mentioned in the other answers, np.vstack() will let you convert your list-of-lists(nested list) into a 1-dimensional array of sublists. But if you are looking to convert the list of lists into a 2-dimensional numpy.ndarray. Then you can use the numpy.asarray() function.
For example, if you have a list of lists named y_true that looks like:
[[0, 1, 0], [1, 0, 0], [0, 0, 1], [1, 0, 0], [0, 1, 0], [0, 0, 1], [1, 0, 0]]
<class 'list'>
This line y_true = np.asarray(y_true) will convert the list of lists into a 2-dimensional numpy ndarray that looks like:
[[0 1 0]
[1 0 0]
[0 0 1]
[1 0 0]
[0 1 0]
[0 0 1]
[1 0 0]]
<class 'numpy.ndarray'>
Additionally, you can also specify the dtype parameter like np.asarray(y_true, dtype = float) to have your array values in your desired data type.
I had a list of lists of equal length. Even then Ignacio Vazquez-Abrams's answer didn't work out for me. I got a 1-D numpy array whose elements are lists. If you faced the same problem, you can use the below method
Use numpy.vstack
import numpy as np
np_array = np.empty((0,4), dtype='float')
for i in range(10)
row_data = ... # get row_data as list
np_array = np.vstack((np_array, np.array(row_data)))
Just use pandas
list(pd.DataFrame(listofstuff).melt().values)
this only works for a list of lists
if you have a list of list of lists you might want to try something along the lines of
lists(pd.DataFrame(listofstuff).melt().apply(pd.Series).melt().values)