I have a list of tables, where each table is a list of lists. I'd like to extract the columns from each table to get a lists of columns. An example should clarify:
input=[[[1,2,3],[4,5,6]],[[7,8,9],[10,11,12]],[[13,14,15],[16,17,18]]]
output=[[[1,4],[7,10],[13,16]],[[2,5],[8,11],[14,17]],[[3,6],[9,12],[15,18]]]
I know I can probably use several for loops, but it feels like there should be a nice one liner to do this.
I intend to assign the results to variables:
a,b,_=output
With a single table, I can extract the columns with:
>>>input=[[1,2,3],[4,5,6]]
>>>list(zip(*input))
[(1, 4), (2, 5), (3, 6)]
or for assignment:
>>>a,b,_=zip(*input)
>>>a
(1, 4)
>>>b
(2, 5)
But for a list of tables, I haven't been able to do it. Here are some of the things I've tried unsuccessfully:
>>>list(zip(*zip(*input)))
[([1, 2, 3], [4, 5, 6]), ([7, 8, 9], [10, 11, 12]), ([13, 14, 15], [16, 17, 18])]
>>>[list(zip(*inp)) for inp in input]
[[(1, 4), (2, 5), (3, 6)], [(7, 10), (8, 11), (9, 12)], [(13, 16), (14, 17), (15, 18)]]
>>>[[x,y,z] for x,y,z in [zip(*inp) for inp in input]]
#same as above
>>> [[x,y] for inp in input for x,y in zip(*inp)]
[[1, 4], [2, 5], [3, 6], [7, 10], [8, 11], [9, 12], [13, 16], [14, 17], [15, 18]]
And nested/unpacked assignment didn't work for me either:
>>>[[a,b,_]]=[zip(*inp) for inp in input]
ValueError: too many values to unpack (expected 1)
>>>[*a,*b,*_]=[[x,y] for x,y in [zip(*inp) for inp in input]]
SyntaxError: two starred expressions in assignment
Is there a one-liner to do what I'm trying to do?
Edit: Note that while the example is specifically 3 tables with 2 rows and 3 columns each, my actual use case has unknown numbers of tables and rows.
I ended up using this line in my code:
list(zip(*[zip(*inp) for inp in input]))
You got close with that last attempt. You need one more, well-chosen nesting level. EDIT: I added the final zip step to get the desired ordering. I also used that "star" notation to help show how to extend the concept.
given = [[[1,2,3],[4,5,6]],
[[7,8,9],[10,11,12]],
[[13,14,15],[16,17,18]]]
f1 = [[[a, b] for a, b in zip(list1, list2)] for list1, list2 in given]
print(f1)
f2 = list(zip(*f1))
print(f2)
Output (edited for readability)
[[[1, 4], [2, 5], [3, 6]],
[[7, 10], [8, 11], [9, 12]],
[[13, 16], [14, 17], [15, 18]]]
[([1, 4], [7, 10], [13, 16]),
([2, 5], [8, 11], [14, 17]),
([3, 6], [9, 12], [15, 18])]
The second one has tuples instead of lists at the middle level; is that okay? If not, can you fix it? (left as an exercise for the student ... )
Related
I am trying to create a for loop to combine lists within a list based on criteria.
My dataset has 300 records of pets. There are 3 types of animals (ex. dog, cat, bird), an interval for the year each animal was born (ex. (1990, 2000], (2000, 2004], etc.) and an interval for a friendly meter rating for each animal (ex. (0,2], (2, 4], (4, 6], etc.)
An example image of what the dataset may look like can be found here:
https://i.stack.imgur.com/YwbGN.png
The current code I am using to combine the friendly meter ratings is below:
friendly_meter_combos = []
combo = []
for val in df['friendly_meter'].unique():
if len(combo) < 2:
combo.append(val)
else:
friendly_meter_combos.append(combo)
combo = []
combo.append(val)
This provides me with an example outcome for
friendly_meter_combos =
[[(0, 2], (2, 4]],
[(4, 6], (6, 8]],
[(8, 10], (2, 4]],
[(4, 6], (6, 8]],
...].
As you can see in the second to last line I printed, the numbers start over ([(8, 10], (2, 4]]). This is because I'm now on a different interval for the year born for the same animal type in the dataset. I would like to figure out how to start the code over to get the output below instead
friendly_meter_combos =
[[(0, 2], (2, 4]],
[(4, 6], (6, 8]],
[(8, 10]],
[(2, 4], (4, 6]],
...].
Any help would be greatly appreciated! Thanks!
As much as I understand, you are looking to have a combination of consecutive "friendly meters"
# let's suppose unique_values is a list containing ordered friendly meters
unique_values = [ [0, 2], [2, 4], [4, 6], [6, 8], [8, 10] ]
# n is the size of the combo
n = 2
# use statement below
friendly_meter_combos = list(zip(*(unique_values[i:] for i in range(n))))
Result is:
[([0, 2], [2, 4]), ([2, 4], [4, 6]), ([4, 6], [6, 8]), ([6, 8], [8, 10])]
I'm desperately trying to find the python built-in equivalent of the following numpy.einsum expression:
>>> a = np.array((((1, 2), (3, 4)), ((5, 6), (7, 8))))
>>> a
array([[[1, 2],
[3, 4]],
[[5, 6],
[7, 8]]])
>>> b = np.array((((9, 10), (11, 12)), ((13, 14), (15, 16))))
>>> b
array([[[ 9, 10],
[11, 12]],
[[13, 14],
[15, 16]]])
>>> np.einsum("abc,abd->dc", a, b)
array([[212, 260],
[228, 280]])
As #AlexRiley comments the direct translation is something like:
(a[...,None,:]*b[...,None]).sum((0,1))
Let's parse the spec string 'abc,abd->dc' and let's rename the terms to x and y so they do not clash with the indices:
This is read as resultdc = ∑ab xabc yabd
As you can see the indices are taken verbatim from the spec string. Indices that do not occur in result spec are summed over. And that's it.
Side note: We can do better than that: Merging the first two axes the expression can be read as a matrix product for which numpy uses a highly optimized code path:
b.reshape(-1,b.shape[-1]).T#a.reshape(-1,a.shape[-1])
This is more than twice as fast as the direct translation and also a bit faster than the original einsum.
I have one array pat=[1,2,3,4,5,6,7] and a second array count=[5,6,7,8,9,10,11]. Is there a way without using dictionaries to get the following array newarray=[[1,5],[2,6],[3,7],[4,8],[5,9],[6,10],[7,11]]?
You can just zip the lists
>>> pat=[1,2,3,4,5,6,7]
>>> count=[5,6,7,8,9,10,11]
>>> list(zip(pat,count))
[(1, 5), (2, 6), (3, 7), (4, 8), (5, 9), (6, 10), (7, 11)]
Or if you want lists instead of tuples
>>> [[i,j] for i,j in zip(pat,count)]
[[1, 5], [2, 6], [3, 7], [4, 8], [5, 9], [6, 10], [7, 11]]
If you want inner elements to be list, you can use -
>>> pat=[1,2,3,4,5,6,7]
>>> count=[5,6,7,8,9,10,11]
>>> newarray = list(map(list,zip(pat,count)))
>>> newarray
[[1, 5], [2, 6], [3, 7], [4, 8], [5, 9], [6, 10], [7, 11]]
This first zips the two lists, combining the ith element of each list, then converts them into lists using map function, and later converts the complete outer map object (that we get from map function) into list
Without using zip, you can do the following:
def map_lists(l1, l2):
merged_list = []
for i in range(len(l1)):
merged_list.append([l1[i], l2[i]])
return merged_list
Or, the equivalent, using a list comprehension instead:
def map_lists(l1, l2):
return [[l1[i], l2[i]] for i in range(len(l1))]
I have a list whose nested list's size may vary with the multiple of 2. Currently, in this example, the nested list's length is 4.
a_list = [[1,2,3,4],[5,6,7,8],[9,10,11,12]]
According to length, I am trying to break the list to get the following result in the best possible pythonic way:
a = [[1,2], [5,6], [9,10]]
b = [[3,4], [7,8], [11,12]]
and if nested list's length is 6, then
c = [[..], [..], [..]]
Its kind of a transpose of a nested list but with sets of 2 values in a single row not to be transposed.
Using list comprehension:
>>> a_list = [[1,2,3,4],[5,6,7,8],[9,10,11,12]]
>>> a = [x[:2] for x in a_list]
>>> b = [x[2:] for x in a_list]
>>> a
[[1, 2], [5, 6], [9, 10]]
>>> b
[[3, 4], [7, 8], [11, 12]]
More general solution:
>>> [[x[i:i+2] for x in a_list] for i in range(0, len(a_list[0]), 2)]
[[[1, 2], [5, 6], [9, 10]],
[[3, 4], [7, 8], [11, 12]]]
I'd hesitate to call this "pythonic", since it's pretty much illegible, but:
>>> a, b = zip(*(zip(*[iter(s)]*2) for s in a_list))
>>> a
((1, 2), (5, 6), (9, 10))
>>> b
((3, 4), (7, 8), (11, 12))
Also works for 6-item lists:
>>> a_list = [[1,2,3,4,100,102],[5,6,7,8,103,104],[9,10,11,12,105,106]]
>>> a, b, c = zip(*(zip(*[iter(s)]*2) for s in a_list))
>>> a
((1, 2), (5, 6), (9, 10))
>>> b
((3, 4), (7, 8), (11, 12))
>>> c
((100, 102), (103, 104), (105, 106))
Almost the same as falsetru's answer, but first the nested lists are split into chunks of size 2 and then all of them are zipped together.
>>> a_list = [[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12]]
>>> zip(*([j[i*2: i*2 + 2] for i in range(len(j) / 2)] for j in a_list))
[([1, 2], [5, 6], [9, 10]), ([3, 4], [7, 8], [11, 12])]
>>> a_list = [[1, 2, 3, 4, 5, 6], [7, 8, 9, 10, 11, 12]]
>>> zip(*([j[i*2: i*2 + 2] for i in range(len(j) / 2)] for j in a_list))
[([1, 2], [7, 8]), ([3, 4], [9, 10]), ([5, 6], [11, 12])]
>>> a_list = [[1,2,3,4,100,102],[5,6,7,8,103,104],[9,10,11,12,105,106]]
>>> zip(*([j[i*2: i*2 + 2] for i in range(len(j) / 2)] for j in a_list))
[([1, 2], [5, 6], [9, 10]), ([3, 4], [7, 8], [11, 12]), ([100, 102], [103, 104], [105, 106])]
A Fast way is using numpy.hsplit :
>>> import numpy
>>> numpy.hsplit(numpy.array(a_list),2)
[array([[ 1, 2],[ 5, 6],[ 9, 10]]),array([[ 3, 4],[ 7, 8],[11, 12]])]
Since readability is pythonic, here's a simpler iterator-based solution (without the neat tricks that #Zero used to turn it into a one-liner):
First, an iterator that turns a list [1,2,3,4,5,6] into [(1, 2), (3, 4), (5, 6)].
def pairs(lst):
it=iter(lst)
return list(zip(it, it)) # Return a list of pairs drawn from the same iterator
The list a_list can be transformed into a list of such pair lists as follows:
a_list = [[1,2,3,4],[5,6,7,8],[9,10,11,12]]
pair_list = [ pairs(row) for row in a_list ]
Finally, we need to effectively transpose this list, making a list of the first pair/element from each sublist, another list of the second one, etc. A nice idiom for transposing a list is zip(*some_list). Let's use it to make the transformation requested by the OP:
a, b = zip(*pair_list)
or to collect any number of generated lists in one list:
results = list( zip(*pair_list) )
Feel free to pack these into a one-liner (though I wouldn't):
results = list(zip( *(pairs(row) for row in a_list) ))
I have a python dictionary of type defaultdict(list)
This dictionary is something like this:
a = {1:[1,2,3,4],2:[5,6,7,8]....n:[some 4 elements]}
So basically it has n keys which has a list as values and all the list are of same lenght.
Now, i want to build a list which has something like this.
[[1,5,...first element of all the list], [2,6.. second element of all the list]... and so on]
Soo basically how do i get the kth value from all the keys.. Is there a pythonic way to do this.. ??
THanks
>>> a = {1:[1,2,3,4],2:[5,6,7,8], 3:[9, 10, 11, 12]}
>>>
>>> zip(*(a[k] for k in sorted(a)))
[(1, 5, 9), (2, 6, 10), (3, 7, 11), (4, 8, 12)]
(Okay, this produces tuples, not lists, but hopefully that's not a problem.)
Update: I like the above more than this, but the following is a few keystrokes shorter:
>>> zip(*map(a.get, sorted(a)))
[(1, 5, 9), (2, 6, 10), (3, 7, 11), (4, 8, 12)]
How about this solution:
zip(*a.values())
For e.g.
>>> a = {1:[1,2,3,4],2:[5,6,7,8], 3:[9, 10, 11, 12]}
>>> zip(*a.values())
[(1, 5, 9), (2, 6, 10), (3, 7, 11), (4, 8, 12)]
Update: to preserve order use DSM's answer.
An alternative way to do this using numpy:
>>> import numpy
>>> a = {1:[1,2,3,4],2:[5,6,7,8], 3:[9, 10, 11, 12]}
>>> x = numpy.zeros((len(a),4), dtype=int)
>>> x[[i-1 for i in a.keys()]] = a.values()
>>> x.T
array([[ 1, 5, 9],
[ 2, 6, 10],
[ 3, 7, 11],
[ 4, 8, 12]])
List comprehension makes this easy. To get a list of the kth items:
k = 1
[a[key][k] for key in sorted(a.keys())]
The to build a list of lists:
[ [a[key][k] for key in sorted(a.keys())] for k in range(len(a[1]))]
if you want list of lists (assuming each list is of length=4):
>>> a = {1:[1, 2, 3, 4], 2:[5, 6, 7, 8], 3:[9, 10, 11, 12]}
>>> [[a[key][x] for key in sorted(a.keys())] for x in xrange(4)]
[[1, 5, 9], [2, 6, 10], [3, 7, 11], [4, 8, 12]]