Assume you have a list such as
x = [('Edgar',), ('Robert',)]
What would be the most efficient way to get to just the strings 'Edgar' and 'Robert'?
Don't really want x[0][0], for example.
Easy solution, and the fastest in most cases.
[item[0] for item in x]
#or
[item for (item,) in x]
Alternatively if you need a functional interface to index access (but slightly slower):
from operator import itemgetter
zero_index = itemgetter(0)
print map(zero_index, x)
Finally, if your sequence is too small to fit in memory, you can do this iteratively. This is much slower on collections but uses only one item's worth of memory.
from itertools import chain
x = [('Edgar',), ('Robert',)]
# list is to materialize the entire sequence.
# Normally you would use this in a for loop with no `list()` call.
print list(chain.from_iterable(x))
But if all you are going to do is iterate anyway, you can also just use tuple unpacking:
for (item,) in x:
myfunc(item)
This is pretty straightforward with a list comprehension:
x = [('Edgar',), ('Robert',)]
y = [s for t in x for s in t]
This does the same thing as list(itertools.chain.from_iterable(x)) and is equivalent in behavior to the following code:
y = []
for t in x:
for s in t:
y.append(s)
I need to send this string to another function.
If your intention is just to call a function for each string in the list, then there's no need to build a new list, just do...
def my_function(s):
# do the thing with 's'
x = [('Edgar',), ('Robert',)]
for (item,) in x:
my_function(item)
...or if you're prepared to sacrifice readability for performance, I suspect it's quickest to do...
def my_function(t):
s = t[0]
# do the thing with 's'
return None
x = [('Edgar',), ('Robert',)]
filter(my_function, x)
Both map() and filter() will do the iteration in C, rather than Python bytecode, but map() will need to build a list of values the same length of the input list, whereas filter() will only build an empty list, as long as my_function() returns a 'falsish' value.
Here is one way:
>>> [name for name, in x]
['Edgar', 'Robert']
Note the placement of the comma, which unpacks the tuple.
>>> from operator import itemgetter
>>> y = map(itemgetter(0), x)
>>> y
['Edgar', 'Robert']
>>> y[0]
'Edgar'
>>> y[1]
'Robert'
Related
When working with a function that returns multiple values with a tuple, I will often find myself using the following idiom to unpack the results from inside a list comprehension.
fiz, buz = zip(*[f(x) for x in input])
Most of the time this works fine, but it throws a ValueError: need more than 0 values to unpack if input is empty. The two ways I can think of to get around this are
fiz = []
buz = []
for x in input:
a, b = f(x)
fiz.append(a)
buz.append(b)
and
if input:
fiz, buz = zip(*[f(x) for x in input])
else:
fiz, buz = [], []
but neither of these feels especially Pythonic—the former is overly verbose and the latter doesn't work if input is a generator rather than a list (in addition to requiring an if/else where I feel like one really shouldn't be needed).
Is there a good simple way to do this? I've mostly been working in Python 2.7 recently, but would also be interested in knowing any Python 3 solutions if they are different.
If f = lambda x: (x,x**2) then this works
x,y = zip(*map(f,input)) if len(input) else ((),())
If input=[], x=() and y=().
If input=[2], x=(2,) and y=(4,)
If input=[2,3], x=(2,3) and y=(4,9)
They're tuples (not lists), but thats thats pretty easy to change.
I would consider using collections.namedtuple() for this sort of thing. I believe the named tuples are deemed more pythonic, and should avoid the need for complicated list comprehensions and zipping / unpacking.
From the documentation:
>>> p = Point(11, y=22) # instantiate with positional or keyword arguments
>>> p[0] + p[1] # indexable like the plain tuple (11, 22)
33
>>> x, y = p # unpack like a regular tuple
>>> x, y
(11, 22)
>>> p.x + p.y # fields also accessible by name
33
>>> p # readable __repr__ with a name=value style
Point(x=11, y=22)
You could use:
fiz = []
buz = []
results = [fiz, buz]
for x in input:
list(map(lambda res, val: res.append(val), results, f(x)))
print(results)
Note about list(map(...)): in Python3, map returns a generator, so we must use it if we want the lambda to be executed.list does it.
(adapted from my answer to Pythonic way to append output of function to several lists, where you could find other ideas.)
I have a list of dicts like so:
a = [ {'list':[1,2,3]}, {'list':[1,4,5]} ]
Am trying to get a flat set of the values in the list key like {1,2,3,4,5}. What's the quickest way?
You can write a loop like:
result = set()
for row in a:
result.update(row['list'])
which I think will work reasonably fast.
Or you can simply use set comprehension and that will result in the following one-liner:
result = {x for row in a for x in row['list']}
In case not all elements contain a 'list' key, you can use .get(..) with an empty tuple (this will reduce construction time):
result = {x for row in a for x in row.get('list',())}
It is not clear what your definition of "quickest" is, but whether it is speed or number of lines I would use a combination of itertools and a generator.
>>> import itertools
>>> a = [ {'list':[1,2,3]}, {'list':[1,4,5]} ]
>>> b = set(itertools.chain.from_iterable(x['list'] for x in a if 'list' in x))
Note that I have added a guard against any elements that may not contain a 'list' key; you can omit that if you know this will always be true.
flat list can be made through reduce easily.
All you need to use initializer - third argument in the reduce function.
reduce(
lambda _set, _dict, key='list': _set.update(
_dict.get(key) or set()) or _set,
a,
set())
Above code works for both python2 and python3, but you need to import reduce module as from functools import reduce. Refer below link for details.
for python2
for python3
I feel this can be done in one line, but I cannot find a way to do.
# final_list is what I want as an output
final_list = []
for x in some_list:
# y is a dictionary
y = x.get_some_dict()
# Want to add a new key/value pair to y, which comes from x
y.update({"new_key": x.property_in_x})
# append y to the output list
final_list.append(y)
return final_list
I wouldn't recommend collapsing this into a one line list comprehension. It can be done, but it's bad style. List comprehensions shouldn't have side effects (i.e. calling update).
You could get replace the explicit list appending with a generator. That wouldn't be a bad idea. And d[k] = v is simpler than d.update({k: v}).
def final_list(some_list):
for x in some_list:
y = x.get_some_dict()
y["new_key"] = x.property_in_x
yield y
I wouldn't recommend damping it into a one liner, which will (probably) be messy and unreadable. Also, the update makes a problem with the one liner solution.
But, In my opinion, this can be simplified, to be clearer:
(Shortened, but not to an unreadable one liner)
for x in some_list:
x.get_some_dict().update({"new_key": x.property_in_x})
final_list = [y.get_some_dict() for y in some_list]
Below is the equivalent list comprehension expression along with for loop as:
final_list = [x.get_some_dict() for x in some_list]
for dict_item, base_item in zip(final_list, some_list):
dict_item["new_key"] = base_item.property_in_x
I have a for loop in which I use the iterator always in the same manner, like this
for dict in mylist:
var = dict['prop']
var2 = f( dict['prop'] )
...
This is clumsy. The only alternatives I can think of:
local variable
wrapping the mylist in a list comprehension. But that seems overkill and is probably inefficient
Any better way?
One map call would work to give you a list of tuples of values:
listOfTuples = map(lambda x: (dict['prop'], f(dict['prop']), myList)
Or if you want two separate lists of values:
[varList, var2List] = zip(*zip(listOfTuples))
Hello and thanks for looking at my question! I was recently introduced to generators and I read the about on 'The Python yield keyword explained'. From my understanding, yield is generated on the fly, cannot be indexed and will remember where it left off after the yield command has been executed (correct me if I'm wrong I'm still new to this).
My question is then why does:
from itertools import combinations
x = [['4A'],['5A','5B','5C'],['7A','7B']]
y = list()
for combination in x:
for i in range(1,len(combination)+1):
y.append(list(combinations(combination,i)))
print y # [[('4A',)], [('5A',), ('5B',), ('5C',)],
# [('5A', '5B'), ('5A', '5C'), ('5B', '5C')],
# [('5A', '5B', '5C')], [('7A',), ('7B',)], [('7A', '7B')]]
But this doesn't work:
from itertools import combinations
x = [['4A'],['5A','5B','5C'],['7A','7B']]
y = list()
for combination in x:
for i in range(1,len(combination)+1):
y.append((combinations(combination,i)))
print y
Since I am appending the combinations to y straight after it is yielded why is it that when it is appended in a list form it works, but when I do it normally, it doesn't?
When you call list(generator_function()), generator_function is iterated to exhaustion, and each element is stored in a list. So, these three operations are doing the same thing:
l = list(generator_function())
...
l = [x for x in generator_function()]
...
l = []
for x in generator_function():
l.append(x)
In your example, without including list(..), you're just appending the combinations generator object to the y. With list(...), you're iterating over the combinations generator to build a list object, and then appending that to y.