Develop Reference

List comprehension with condition on the new element

I will be stealing the form of the question there: List comprehension with condition
I have a simple list.
>>> a = [0, 1, 2]
I want to make a new list from it using a list comprehension.
>>> b = [afunction(x) for x in a]
>>> b
[12458, None, 34745]
Pretty simple, but what if I want to operate only over non-None elements? I can do this:
>>> b = [y for y in [afunction(x) for x in a] if y != None]
I would rather like to be able to do it with single list comprehension, in a single run. This, I believe iterates over the list again to filter out the Nones, and nobody really likes that.

You don't need to build the list then iterate over it again, you can consume an iterator of the application of afunction to the elements in a, either using map:
[y for y in map(afunction, a) if y is not None]
or a generator expression (note change in the type of brackets):
[y for y in (afunction(x) for x in a) if y is not None]
# ^ here ^ and here
Note also that these test for None by identity, not equality, per PEP-8.

How does this input work with the Python 'any' function? [duplicate]

This question already has answers here:
How exactly does a generator comprehension work?
(8 answers)
Closed 7 months ago.
In the python docs page for any, the equivalent code for the any() function is given as:
def any(iterable):
for element in iterable:
if element:
return True
return False
How does this function know what element I wanna test if call it in this form?
any(x > 0 for x in list)
From the function definition, all I can see is that I'm passing an iterable object. How does the for loop know I am looking for something > 0?

If you use any(lst) you see that lst is the iterable, which is a list of some items. If it contained [0, False, '', 0.0, [], {}, None] (which all have boolean values of False) then any(lst) would be False. If lst also contained any of the following [-1, True, "X", 0.00001] (all of which evaluate to True) then any(lst) would be True.
In the code you posted, x > 0 for x in lst, this is a different kind of iterable, called a generator expression. Before generator expressions were added to Python, you would have created a list comprehension, which looks very similar, but with surrounding []'s: [x > 0 for x in lst]. From the lst containing [-1, -2, 10, -4, 20], you would get this comprehended list: [False, False, True, False, True]. This internal value would then get passed to the any function, which would return True, since there is at least one True value.
But with generator expressions, Python no longer has to create that internal list of True(s) and False(s), the values will be generated as the any function iterates through the values generated one at a time by the generator expression. And, since any short-circuits, it will stop iterating as soon as it sees the first True value. This would be especially handy if you created lst using something like lst = range(-1,int(1e9)) (or xrange if you are using Python2.x). Even though this expression will generate over a billion entries, any only has to go as far as the third entry when it gets to 1, which evaluates True for x>0, and so any can return True.
If you had created a list comprehension, Python would first have had to create the billion-element list in memory, and then pass that to any. But by using a generator expression, you can have Python's builtin functions like any and all break out early, as soon as a True or False value is seen.

>>> names = ['King', 'Queen', 'Joker']
>>> any(n in 'King and john' for n in names)
True
>>> all(n in 'King and Queen' for n in names)
False
It just reduce several line of code into one.
You don't have to write lengthy code like:
for n in names:
if n in 'King and john':
print True
else:
print False

(x > 0 for x in list) in that function call creates a generator expression eg.
>>> nums = [1, 2, -1, 9, -5]
>>> genexp = (x > 0 for x in nums)
>>> for x in genexp:
print x
True
True
False
True
False
Which any uses, and shortcircuits on encountering the first object that evaluates True

It's because the iterable is
(x > 0 for x in list)
Note that x > 0 returns either True or False and thus you have an iterable of booleans.

Simply saying, any() does this work : according to the condition even if it encounters one fulfilling value in the list, it returns true, else it returns false.
list = [2,-3,-4,5,6]
a = any(x>0 for x in lst)
print a:
True
list = [2,3,4,5,6,7]
a = any(x<0 for x in lst)
print a:
False

Comparing all elements of two tuples (with all() functionality)

So i know that comparisons on tuples work lexicographically:
Tuples and lists are compared lexicographically using comparison of corresponding elements. This means that to compare equal, each element must compare equal and the two sequences must be of the same type and have the same length.
If not equal, the sequences are ordered the same as their first differing elements. For example, cmp([1,2,x], [1,2,y]) returns the same as cmp(x,y). If the corresponding element does not exist, the shorter sequence is ordered first (for example, [1,2] < [1,2,3]).
So from this:
>>> a = (100, 0)
>>> b = (50, 50)
>>> a > b
True
But i want to compare all elements of 2 tuples in order, so functionally i want something akin to (using values from above):
>>> a > b
(True, False) #returned tuple containing each comparison
>>> all(a > b)
False
As an example in practice, for something like screen coordinates, if you wanted to check if something was 'inside' the screen at (0,0), but done a comparison like coord > (0,0), if the x coord was bigger than 0, but the y coord was smaller it would still return true, which isn't what is needed in this case.
As sort of a sub question/discussion:
I am not sure why comparing 2 tuples of different values is returned in such a way. You are not given any sort of index, so the only thing you get from comparing a tuple (that isn't testing equality) is that at some point in the tuple, one of the comparisons will throw a true or false value when they are not equal. How could you take advantage of that?

You can achieve this with a list comprehension and the zip built-in:
>>> a = (100, 0)
>>> b = (50, 50)
>>> [(a > b) for a, b in zip(a,b)]
[True, False]
You can use all() or any() on the returned list.

Replace a > b with tuple(i > j for i, j in zip(a,b)) in your second code sample.
>>> a = (100, 0)
>>> b = (50, 50)
>>> tuple(i > j for i, j in zip(a,b))
(True, False)
>>> all(i > j for i, j in zip(a,b))
False

You might consider using the following vectorized approach, which is usually more performant, and syntactically/semantically very clear:
>>> import numpy
>>>
>>> a = (100, 0)
>>> b = (50, 50)
>>> numpy.array(a) > b
array([ True, False], dtype=bool)
>>>
>>> (numpy.array(a) > b).any()
True
>>> (numpy.array(a) > b).all()
False
numpy is quite performant, and the resulting objects above also embed the any()/all() query methods you want. If you will be performing vector-like operations (as your screen coordinates example suggests), you may consider working with 'a' and 'b' as numpy arrays, instead of tuples. That results in the most efficient implementation of what you seek: no pre-conversion necessary, and Python-based loops are replaced with efficient numpy-based loops. This is worth highlighting because there are two and potentially three loops involved: (1) a preprocessing loop during conversion (which you can eliminate); (2) an item-by-item comparison loop; and (3) a query loop to answer the any/all question.
Note that I could've also created a numpy array from 'b', but not doing so eliminated one conversion step and pre-processing time. Since that approach results in one operand being a numpy array and the other a tuple, as the sequences grow, that may/may-not result in less speedy item-by-item comparisons (which strict numpy-to-numpy is good at). Try it. :)

I felt like the use of map and lambda functions was missing from the answers
>>> a = (100, 0)
>>> b = (50, 50)
>>> all(map(lambda x,y: x > y, a, b))
False

To get the described behavior, try:
[ai > bi for ai,bi in zip(a,b)]
The reason that comparisons of tuples are returned in that way is that you might want to write something like:
if a >= (0.,0.):
print "a has only positive values"
else:
print "a has at least one negative value"
If Python were to return the tuple that you describe, then the else would never happen. Try
if (False,False):
print "True!" # This is what is printed.
else:
print "False!"
I hope this helps.

How to get filter to work with a lambda taking multiple arguments?

Using Python, am finding it difficult to get filter() to work with lambda for cases where more than 1 argument needs to be passed as is the case in the following snippet:
max_validation = lambda x,y,z: x < y < z
sequence1 = [1,4,8]
filter(max_validation, sequence1)
It raises the following error:
TypeError: <lambda>() takes exactly 3 arguments (1 given)
Please suggest as to what am doing wrong here.

It's a little bit difficult to figure out exactly what you're trying to do. I'm going to interpret your question, then provide an answer. If this is not correct, please modify your question or comment on this answer.
Question
I have sequences that are exactly three elements long. Here's one:
sequence1 = [1, 4, 8]
I want to ensure that the first element is less than the second element, which should in turn be less than the third element. I've written the following function to do so:
max_validation = lambda x, y, z: x < y < z
How do I apply this using filter? Using filter(max_validation, sequence1) doesn't work.
Answer
Filter applies your function to each element of the provided iterable, picking it if the function returns True and discarding it if the function returns False.
In your case, filter first looks at the value 1. It tries to pass that into your function. Your function expects three arguments, and only one is provided, so this fails.
You need to make two changes. First, put your three-element sequence into a list or other sequence.
sequences = [[1, 4, 8], [2, 3, 9], [3, 2, 3]]
max_validation = lambda x: x[0] < x[1] < x[2] and len(x) == 3
I've added two other sequences to test. Because sequences is a list of a list, each list gets passed to your test function. Even if you're testing just one sequence, you should use [[1, 4, 8]] so that the entire sequence to test gets passed into your function.
I've also modified max_validation so that it accepts just one argument: the list to test. I've also added and len(x) == 3 to ensure that the sequences are only 3 elements in length

The function passed to filter() only gets a single argument passed to it, which is the current element in the iterable being iterated.. If you need something fancier than that then filter() won't do.

Straight from the docs of Python Filters
Note that filter(function, iterable)
is equivalent to [item for item in
iterable if function(item)] if
function is not None and [item for
item in iterable if item] if function
is None.
So, you can just process single arguments with Python filters. That effectively means you cannot use filters for the your example. You would have to write custom-code for the same.

It's possible to do this using a closure:
>>> def foo(a,b):
... def bar(c):
... return a+b+c
... return bar
...
>>> x = foo(1,2)
>>> x(3)
6
>>> y = foo(100,0)
>>> y(1)
101
>>> x(1)
4

I hope you are aware of?
>>> max([1, 4, 8])
8
filter() takes a single argument. In your case, it will take 1. Then 4. Then 8.

This would work for sequences of any length:
all(x < y for x, y in zip(seq, seq[1:]))
What does there happens?
For sequence 1, 2, 3... you take sequences 1, 2, 3... and 2, 3, 4... and zip them together to sequence (1, 2), (2, 3), ...
Then you check if statement 'x < y' holds for every pair.
And this will work for any associative rule you want to check.
Useful links:
slices in Python
zip in Python docs
all in Python docs

I think all others didn't get the point. the error message is for lambda function not for the filter. You should rather call it this way:
filter(max_validation, *sequence1)
add a star on the list transform it into three arguments, then it will work.

I'm in agreement with both #walkingpendulum and #Wesley, depending on the interpretation of the actual problem statement. So parsing through the ambiguity in the problem statement:
If you're sequentially comparing one item to its previous value in an iterable object, a lambda expression is overkill, just
use a list comprehension:
[1 if i < i+1 else 0 for i in sequence1]
If you're comparing objects, then just compare them -- a lambda expression wouldn't work firstly because you're only passing one argument where the lambda expression you defined you're passing three and lambda is generally applied across an iterable object. To compare objects, there are simple constructs for that:
sequence1 == some_other_sequence
and
x, y, z = 1, 2, 3
x < y < z
And lastly, if you want to apply a lambda expression to an iterable object, map can get you there: (arbitrary lambda function)
map(lambda x: x > 1, sequence1)
Otherwise #walkingpendulum and #Wesley cover the other interpretations

You could change
max_validation = lambda x,y,z: x < y < z
to
max_validation = lambda (x,y,z): x < y < z

filtering elements from list of lists in Python?

I want to filter elements from a list of lists, and iterate over the elements of each element using a lambda. For example, given the list:
a = [[1,2,3],[4,5,6]]
suppose that I want to keep only elements where the sum of the list is greater than N. I tried writing:
filter(lambda x, y, z: x + y + z >= N, a)
but I get the error:
<lambda>() takes exactly 3 arguments (1 given)
How can I iterate while assigning values of each element to x, y, and z? Something like zip, but for arbitrarily long lists.
thanks,
p.s. I know I can write this using: filter(lambda x: sum(x)..., a) but that's not the point, imagine that these were not numbers but arbitrary elements and I wanted to assign their values to variable names.

Using lambda with filter is sort of silly when we have other techniques available.
In this case I would probably solve the specific problem this way (or using the equivalent generator expression)
>>> a = [[1, 2, 3], [4, 5, 6]]
>>> [item for item in a if sum(item) > 10]
[[4, 5, 6]]
or, if I needed to unpack, like
>>> [(x, y, z) for x, y, z in a if (x + y) ** z > 30]
[(4, 5, 6)]
If I really needed a function, I could use argument tuple unpacking (which is removed in Python 3.x, by the way, since people don't use it much): lambda (x, y, z): x + y + z takes a tuple and unpacks its three items as x, y, and z. (Note that you can also use this in def, i.e.: def f((x, y, z)): return x + y + z.)
You can, of course, use assignment style unpacking (def f(item): x, y, z = item; return x + y + z) and indexing (lambda item: item[0] + item[1] + item[2]) in all versions of Python.

You can explicitly name the sublist elements (assuming there's always a fixed number of them), by giving lambda a tuple as its argument:
>>> a = [[1,2,3],[4,5,6]]
>>> N = 10
>>> filter(lambda (i, j, k): i + j + k > N, a)
[[4, 5, 6]]
If you specify "lambda i, j, k" as you tried to do, you're saying lambda will receive three arguments. But filter will give lambda each element of a, that is, one sublist at a time (thus the error you got). By enclosing the arguments to lambda in parenthesis, you're saying that lambda will receive one argument, but you're also naming each of its components.

You can do something like
>>> a=[[1,2,3],[4,5,6]]
>>> filter(lambda (x,y,z),: x+y+z>6, a)
[[4, 5, 6]]
Using the deconstruction syntax.

How about this?
filter(lambda b : reduce(lambda x, y : x + y, b) >= N, a)
This isn't answering the question asked, I know, but it works for arbitrarily-long lists, and arbitrarily-long sublists, and supports any operation that works under reduce().

Try something like this:
filter(lambda a: a[0] + a[1] + a[2] >= N, a)

Use a function instead of a lambda, then myVar = a[0], etc.

Ok, obviously you know that you can use sum. The goal of what you are trying to do seems a bit vague, but I think that the optional parameter syntax might help you out, or at least give you some inspiration. If you place a * before a parameter, it creates a tuple of all of itself and all of the remaining parameters. If you place a ** before it, you get a dictionary.
To see this:
def print_test(a,b,c,*d):
print a
print b
print c
print d
print_test(1,2,3,4,5,6)
prints
1
2
3
(4, 5, 6)
You can use this syntax with lambda too.
Like I said, I'm not sure exactly what you are trying to do, but it sounds like this might help. I don't think you can get local variable assignments in lambda without some hacking, but maybe you can use this to assign the values to variables somehow.
Edit: Ah, I understand what you are looking for moreso now. I think you want:
lambda (a, b, c): a+b+c > N