When working with a function that returns multiple values with a tuple, I will often find myself using the following idiom to unpack the results from inside a list comprehension.
fiz, buz = zip(*[f(x) for x in input])
Most of the time this works fine, but it throws a ValueError: need more than 0 values to unpack if input is empty. The two ways I can think of to get around this are
fiz = []
buz = []
for x in input:
a, b = f(x)
fiz.append(a)
buz.append(b)
and
if input:
fiz, buz = zip(*[f(x) for x in input])
else:
fiz, buz = [], []
but neither of these feels especially Pythonic—the former is overly verbose and the latter doesn't work if input is a generator rather than a list (in addition to requiring an if/else where I feel like one really shouldn't be needed).
Is there a good simple way to do this? I've mostly been working in Python 2.7 recently, but would also be interested in knowing any Python 3 solutions if they are different.
If f = lambda x: (x,x**2) then this works
x,y = zip(*map(f,input)) if len(input) else ((),())
If input=[], x=() and y=().
If input=[2], x=(2,) and y=(4,)
If input=[2,3], x=(2,3) and y=(4,9)
They're tuples (not lists), but thats thats pretty easy to change.
I would consider using collections.namedtuple() for this sort of thing. I believe the named tuples are deemed more pythonic, and should avoid the need for complicated list comprehensions and zipping / unpacking.
From the documentation:
>>> p = Point(11, y=22) # instantiate with positional or keyword arguments
>>> p[0] + p[1] # indexable like the plain tuple (11, 22)
33
>>> x, y = p # unpack like a regular tuple
>>> x, y
(11, 22)
>>> p.x + p.y # fields also accessible by name
33
>>> p # readable __repr__ with a name=value style
Point(x=11, y=22)
You could use:
fiz = []
buz = []
results = [fiz, buz]
for x in input:
list(map(lambda res, val: res.append(val), results, f(x)))
print(results)
Note about list(map(...)): in Python3, map returns a generator, so we must use it if we want the lambda to be executed.list does it.
(adapted from my answer to Pythonic way to append output of function to several lists, where you could find other ideas.)
Related
I was recently surprised to find that the "splat" (unary *) operator always captures slices as a list during item unpacking, even when the sequence being unpacked has another type:
>>> x, *y, z = tuple(range(5))
>>> y
[1, 2, 3] # list, was expecting tuple
Compare to how this assignment would be written without unpacking:
>>> my_tuple = tuple(range(5))
>>> x = my_tuple[0]
>>> y = my_tuple[1:-1]
>>> z = my_tuple[-1]
>>> y
(1, 2, 3)
It is also inconsistent with how the splat operator behaves in function arguments:
>>> def f(*args):
... return args, type(args)
...
>>> f()
((), <class 'tuple'>)
In order to recover y as a tuple after unpacking, I now have to write:
>>> x, *y, z = tuple(range(5))
>>> y = tuple(y)
Which is still much better that the slice-based syntax, but nonetheless suffers from what I consider to be a very unnecessary and unexpected loss of elegance. Is there any way to recover y as a tuple instead of a list without post-assignment processing?
I tried to force python to interpret y as a tuple by writing x, *(*y,), z = ..., but it still ended up as a list. And of course silly things like x, *tuple(y), z don't work in python.
I am currently using Python 3.8.3 but solutions/suggestions/explanations involving higher versions (as they become available) are also welcome.
This is by design. Quoting the official docs about Assignment:
...The first items of the iterable are
assigned, from left to right, to the targets before the starred
target. The final items of the iterable are assigned to the
targets after the starred target. A list of the remaining items
in the iterable is then assigned to the starred target (the list
can be empty).
It is highly probable that the Python user wants to mutate your y afterwards, so the list type was chosen over the tuple.
Quoting the Acceptance section of PEP 3132 that I found through a link in this related question:
After a short discussion on the python-3000 list [1], the PEP was accepted by Guido in its current form. Possible changes discussed were:
Only allow a starred expression as the last item in the exprlist. This would simplify the unpacking code a bit and allow for the
starred expression to be assigned an iterator. This behavior was rejected because it would be too surprising.
Try to give the starred target the same type as the source iterable, for example, b in a, *b = "hello" would be assigned the
string "ello". This may seem nice, but is impossible to get right
consistently with all iterables.
Make the starred target a tuple instead of a list. This would be consistent with a function's *args, but make further processing of the
result harder.
So converting with y = tuple(y) afterwards is your only option.
I am reading a book, in which they write:
fp1, residuals, rank, sv, rcond = sp.polyfit(x, y, 1, full=True)
It seems sp.polyfit method assigns values to each of these variables in some sort of order.
For instance:
>>> print("Model parameters: %s" % fp1)
Model parameters: [ 2.59619213 989.02487106]
>>> print(res)
[ 3.17389767e+08]
(I don't know where res is being defined... but...) Is this Python's way of creating an object?
In other languages, you might do something like this:
Foo myFooObject = bar.GenerateFoo();
myFooObject.foo();
myFooObject.bar();
The general syntax of python in this way is confusing to me. Thanks for helping me to understand.
This has nothing to do with object creation -- it's an example of tuple (or more generally sequence) unpacking in python.
A tuple is a fixed sequence of items, and you can assign one set to another via a command like
a, b, c = 1, 'two', 3.0
which is the same as
a = 1
b = 'two'
c = 3.0
(Note that you can use this syntax to swap items: a, b = b,a.)
So what is happening in your example is that scipy.poylfit has a line like
return fp, resides, rank, eval, rcondnum
and you are assigning your variables to these.
It's tuple unpacking.
Say you have some tuple:
t = (1, 2, 3)
then you can use that to set three variables with:
x, y, z = t # x becomes 1, y 2, y 3
Your function sp.polyfit simply returns a tuple.
Actually it works with any iterable, not just tuples, but doing it with tuples is by far the most common way. Also, the number of elements in the iterables has to be exactly equal to the number of variables.
Assume you have a list such as
x = [('Edgar',), ('Robert',)]
What would be the most efficient way to get to just the strings 'Edgar' and 'Robert'?
Don't really want x[0][0], for example.
Easy solution, and the fastest in most cases.
[item[0] for item in x]
#or
[item for (item,) in x]
Alternatively if you need a functional interface to index access (but slightly slower):
from operator import itemgetter
zero_index = itemgetter(0)
print map(zero_index, x)
Finally, if your sequence is too small to fit in memory, you can do this iteratively. This is much slower on collections but uses only one item's worth of memory.
from itertools import chain
x = [('Edgar',), ('Robert',)]
# list is to materialize the entire sequence.
# Normally you would use this in a for loop with no `list()` call.
print list(chain.from_iterable(x))
But if all you are going to do is iterate anyway, you can also just use tuple unpacking:
for (item,) in x:
myfunc(item)
This is pretty straightforward with a list comprehension:
x = [('Edgar',), ('Robert',)]
y = [s for t in x for s in t]
This does the same thing as list(itertools.chain.from_iterable(x)) and is equivalent in behavior to the following code:
y = []
for t in x:
for s in t:
y.append(s)
I need to send this string to another function.
If your intention is just to call a function for each string in the list, then there's no need to build a new list, just do...
def my_function(s):
# do the thing with 's'
x = [('Edgar',), ('Robert',)]
for (item,) in x:
my_function(item)
...or if you're prepared to sacrifice readability for performance, I suspect it's quickest to do...
def my_function(t):
s = t[0]
# do the thing with 's'
return None
x = [('Edgar',), ('Robert',)]
filter(my_function, x)
Both map() and filter() will do the iteration in C, rather than Python bytecode, but map() will need to build a list of values the same length of the input list, whereas filter() will only build an empty list, as long as my_function() returns a 'falsish' value.
Here is one way:
>>> [name for name, in x]
['Edgar', 'Robert']
Note the placement of the comma, which unpacks the tuple.
>>> from operator import itemgetter
>>> y = map(itemgetter(0), x)
>>> y
['Edgar', 'Robert']
>>> y[0]
'Edgar'
>>> y[1]
'Robert'
I am working with Python list sort.
I have two lists: one is a list of integers, the other is a list of objects, and the second object list has the attribute id which is also an integer, I want to sort the object list based on the id attribute, in the order of the same id appears in the first list, well, this is an example:
I got a = [1,2,3,4,5]
and b = [o,p,q,r,s], where o.id = 2, p.id = 1, q.id = 3, r.id = 5, s.id = 4
and I want my list b to be sorted in the order of its id appears in list a, which is like this:
sorted_b = [p, o, q, s, r]
Of course, I can achieve this by using nested loops:
sorted_b = []
for i in a:
for j in b:
if j.id == i:
sorted_b.append(j)
break
but this is a classic ugly and non-Python way to solve a problem, I wonder if there is a way to solve this in a rather neat way, like using the sort method, but I don't know how.
>>> from collections import namedtuple
>>> Foo = namedtuple('Foo', 'name id') # this represents your class with id attribute
>>> a = [1,2,3,4,5]
>>> b = [Foo(name='o', id=2), Foo(name='p', id=1), Foo(name='q', id=3), Foo(name='r', id=5), Foo(name='s', id=4)]
>>> sorted(b, key=lambda x: a.index(x.id))
[Foo(name='p', id=1), Foo(name='o', id=2), Foo(name='q', id=3), Foo(name='s', id=4), Foo(name='r', id=5)]
This is a simple way to do it:
# Create a dictionary that maps from an ID to the corresponding object
object_by_id = dict((x.id, x) for x in b)
sorted_b = [object_by_id[i] for i in a]
If the list gets big, it's probably the fastest way, too.
You can do it with a list comprehension, but in general is it the same.
sorted_b = [ y for x in a for y in b if y.id == x ]
There is a sorted function in Python. It takes optional keyword argument cmp. You can pass there your customized function for sorting.
cmp definition from the docs:
custom comparison should return a negative, zero or positive number depending on whether the first argument is considered smaller than, equal to, or larger than the second argument
a = [1,2,3,4,5]
def compare(el1, el2):
if a.index(el1.id) < a.index(el2.id): return -1
if a.index(el1.id) > a.index(el2.id): return 1
return 0
sorted(b, cmp=compare)
This is more straightforward however I would encourage you to use the key argument as jamylak described in his answer, because it's more pythonic and in Python 3 the cmp is not longer supported.
Howdy, codeboys and codegirls!
I have came across a simple problem with seemingly easy solution. But being a Python neophyte I feel that there is a better approach somewhere.
Say you have a list of mixed strings. There are two basic types of strings in the sack - ones with "=" in them (a=potato) and ones without (Lady Jane). What you need is to sort them into two lists.
The obvious approach is to:
for arg in arguments:
if '=' in arg:
equal.append(arg)
else:
plain.append(arg)
Is there any other, more elegant way into it? Something like:
equal = [arg for arg in arguments if '=' in arg]
but to sort into multiple lists?
And what if you have more than one type of data?
Try
for arg in arguments:
lst = equal if '=' in arg else plain
lst.append(arg)
or (holy ugly)
for arg in arguments:
(equal if '=' in arg else plain).append(arg)
A third option: Create a class which offers append() and which sorts into several lists.
You can use itertools.groupby() for this:
import itertools
f = lambda x: '=' in x
groups = itertools.groupby(sorted(data, key=f), key=f)
for k, g in groups:
print k, list(g)
I would just go for two list comprehensions. While that does incur some overhead (two loops on the list), it is more Pythonic to use a list comprehension than to use a for. It's also (in my mind) much more readable than using all sorts of really cool tricks, but that less people know about.
def which_list(s):
if "=" in s:
return 1
return 0
lists = [[], []]
for arg in arguments:
lists[which_list(arg)].append(arg)
plain, equal = lists
If you have more types of data, add an if clause to which_list, and initialize lists to more empty lists.
I would go for Edan's approach, e.g.
equal = [arg for arg in arguments if '=' in arg]
plain = [arg for arg in arguments if '=' not in arg]
I read somewhere here that you might be interested in a solution that
will work for more than two identifiers (equals sign and space).
The following solution just requires you update the uniques set with
anything you would like to match, the results are placed in a dictionary of lists
with the identifier as the key.
uniques = set('= ')
matches = dict((key, []) for key in uniques)
for arg in args:
key = set(arg) & uniques
try:
matches[key.pop()].append(arg)
except KeyError:
# code to handle where arg does not contain = or ' '.
Now the above code assumes that you will only have a single match for your identifier
in your arg. I.e that you don't have an arg that looks like this 'John= equalspace'.
You will have to also think about how you would like to treat cases that don't match anything in the set (KeyError occurs.)
Another approach is to use the filter function, although it's not the most efficient solution.
Example:
>>> l = ['a=s','aa','bb','=', 'a+b']
>>> l2 = filter(lambda s: '=' in s, l)
>>> l3 = filter(lambda s: '+' in s, l)
>>> l2
['a=s', '=']
>>> l3
['a+b']
I put this together, and then see that Ned Batchelder was already on this same tack. I chose to package the splitting method instead of the list chooser, though, and to just use the implicit 0/1 values for False and True.
def split_on_condition(source, condition):
ret = [],[]
for s in source:
ret[condition(s)].append(s)
return ret
src = "z=1;q=2;lady jane;y=a;lucy in the sky".split(';')
plain,equal = split_on_condition(src, lambda s:'=' in s)
Your approach is the best one. For sorting just into two lists it can't get clearer than that. If you want it to be a one-liner, encapsulate it in a function:
def classify(arguments):
equal, plain = [], []
for arg in arguments:
if '=' in arg:
equal.append(arg)
else:
plain.append(arg)
return equal, plain
equal, plain = classify(lst)