This question already has answers here:
How exactly does a generator comprehension work?
(8 answers)
Closed 7 months ago.
In the python docs page for any, the equivalent code for the any() function is given as:
def any(iterable):
for element in iterable:
if element:
return True
return False
How does this function know what element I wanna test if call it in this form?
any(x > 0 for x in list)
From the function definition, all I can see is that I'm passing an iterable object. How does the for loop know I am looking for something > 0?
If you use any(lst) you see that lst is the iterable, which is a list of some items. If it contained [0, False, '', 0.0, [], {}, None] (which all have boolean values of False) then any(lst) would be False. If lst also contained any of the following [-1, True, "X", 0.00001] (all of which evaluate to True) then any(lst) would be True.
In the code you posted, x > 0 for x in lst, this is a different kind of iterable, called a generator expression. Before generator expressions were added to Python, you would have created a list comprehension, which looks very similar, but with surrounding []'s: [x > 0 for x in lst]. From the lst containing [-1, -2, 10, -4, 20], you would get this comprehended list: [False, False, True, False, True]. This internal value would then get passed to the any function, which would return True, since there is at least one True value.
But with generator expressions, Python no longer has to create that internal list of True(s) and False(s), the values will be generated as the any function iterates through the values generated one at a time by the generator expression. And, since any short-circuits, it will stop iterating as soon as it sees the first True value. This would be especially handy if you created lst using something like lst = range(-1,int(1e9)) (or xrange if you are using Python2.x). Even though this expression will generate over a billion entries, any only has to go as far as the third entry when it gets to 1, which evaluates True for x>0, and so any can return True.
If you had created a list comprehension, Python would first have had to create the billion-element list in memory, and then pass that to any. But by using a generator expression, you can have Python's builtin functions like any and all break out early, as soon as a True or False value is seen.
>>> names = ['King', 'Queen', 'Joker']
>>> any(n in 'King and john' for n in names)
True
>>> all(n in 'King and Queen' for n in names)
False
It just reduce several line of code into one.
You don't have to write lengthy code like:
for n in names:
if n in 'King and john':
print True
else:
print False
(x > 0 for x in list) in that function call creates a generator expression eg.
>>> nums = [1, 2, -1, 9, -5]
>>> genexp = (x > 0 for x in nums)
>>> for x in genexp:
print x
True
True
False
True
False
Which any uses, and shortcircuits on encountering the first object that evaluates True
It's because the iterable is
(x > 0 for x in list)
Note that x > 0 returns either True or False and thus you have an iterable of booleans.
Simply saying, any() does this work : according to the condition even if it encounters one fulfilling value in the list, it returns true, else it returns false.
list = [2,-3,-4,5,6]
a = any(x>0 for x in lst)
print a:
True
list = [2,3,4,5,6,7]
a = any(x<0 for x in lst)
print a:
False
Related
x=[255, 255, 255, 250, 250, 250, 250, 250, 250, 255, 255, 255]
y=all([i for i in x if i>260])
y1 = all([i for i in x if i>220])
y2 = all([True for i in x if i>260])
y3 = all([True for i in x if i>220])
y1==y2==y3==y #gives True
How does all handle empty lists, and how can I use to filter items above 220.
the only thing working for me now is
y=len([i for i in x if i>220])== len(x)
what I want to know if all list is above certain number i.e 220
You want this:
all([i > 220 for i in x])
all for an empty list should be True, because all of its elements - all 0 of them - are truthy
Edit: if it's desired to rule out the trivial case where x is empty, then as suggested in the comments just add an additional check for that, such as len(X) > 0
There are some small tweaks required to make your list comprehensions work. For instance, you don't handle the case in which the value in the list is not greater than x. For example, let's assume our list is:
>>> x = [1, 2, 3, 4, 5, 6]
If we use the code snipped from your question:
>>> [True for i in x if i > 3]
[True, True, True]
The result is probably not exactly what you expected. Therefore, calling all() on that list will always evaluate to True. One small tweak, however (note the addition of the else statement):
>>> [True if i > 3 else False for i in x]
[False, False, False, True, True, True]
This is the desired output. Now you can also apply all() to the output list.
>>> y1 = [True if i > 3 else False for i in x]
>>> all(y1)
False
And if we want to know whether all values are greater than 0, we expect all() to return True in the following case:
>>> y2 = [True if i > 0 else False for i in x]
>>> all(y2)
True
To answer your question about how all() handles empty lists:
>>> all([])
True
Depending on the structure of your code and the desired output you may want to have an edge case statement that handles empty lists. Hopefully this is enough input for you to adapt it to your problem statement.
Making the code more compact: As very kindly pointed out in the comments, the code above is a bit "wordy" to make a point and for the sake of the explanation. When writing list comprehensions and condition statements in Python (and perhaps other programming languages), you can actually shorten the list comprehension and condition statement:
>>> y1 = [i > 3 for i in x]
>>> y1
[False, False, False, True, True, True]
>>> all(y1)
False
i > 3 in the list comprehension will turn into True or False and therefore yield the same result as writing True if i > 3 else False.
Finally, if you want to know if all elements meet the threshold and there is at least one element in the list:
>>> y1 = [i > 3 for i in x]
>>> y1
[False, False, False, True, True, True]
>>> all(y1) and y1
[] # this will evaluate to False if you use it in a condition statement
Alternatively you can also convert it to a boolean:
>>> bool(all(y1) and y1)
False
Which does the right thing for empty lists as well:
>>> bool(all([]) and [])
False
instead of True for i in x if i>260, do i>260 for i in x
i>260 will result in a boolean: True, or False for each element.
I have seen somewhere this usage of a for loop:
def func(seq=None):
for i in seq or [1, 2, 3]:
print i
func([3, 4, 5]) # Will print 3, 4, 5
func() # Will print 1, 2, 3
It seems that when using or/and when using the "in" keyword it will select the first/last "usable" sequence in order to run the for loop.
I have not seen any official documentation and was wondering how it works and if it's commonly used.
In case of or (or and ) operator, when you do -
a or b
It returns a if a is not a false-like value, otherwise it returns b . None (and empty lists) are false like value.
From documentation -
The expression x and y first evaluates x; if x is false, its value is returned; otherwise, y is evaluated and the resulting value is returned.
The expression x or y first evaluates x; if x is true, its value is returned; otherwise, y is evaluated and the resulting value is returned.
(Note that neither and nor or restrict the value and type they return to False and True, but rather return the last evaluated argument. This is sometimes useful, e.g., if s is a string that should be replaced by a default value if it is empty, the expression s or 'foo' yields the desired value. Because not has to invent a value anyway, it does not bother to return a value of the same type as its argument, so e.g., not 'foo' yields False, not ''.)
Also, in case of for loop, in is actually part of the for loop construct, its not an operator. Documentation for for loop construct here.
Hence when you do -
for i in seq or [1, 2, 3]:
It would first evaluate -
seq or [1 , 2, 3]
And returns the seq list if its not none or empty, so that you can iterate over that. Otherwise, it would return [1, 2, 3] and you would iterate over that.
No! It's the operator priority! or before in…
Precedence, §6.15.
So seq or [1, 2, 3] is evaluated before entering the loop. And seq is None.
I was asked if it was possible to compare two (say) lists without invoking operators, to determine if they were the same (or rather, contained the same elements).
I first entertained using
x in y
before I realised that it would not care for order, but for mere presence. Of course, if the lists were to contain purely numbers it would be trivial to do a modulus test or so, but lists can contain strings. (is didn't work either, but I didn't really expect it to, considering it tests identity...)
So I was wondering if it's (even) possible to pull off equality tests without using operators (==, !=)?
It was a mere rhetorical question, but it's been gnawing at me for some time and I've rather given up trying to solve it myself with my not-very extensive python knowledge.
Sure it is, just bypass the operators and go straight for the __eq__ special method:
>>> x = [1, 2, 3]
>>> y = [1, 2, 3]
>>> x.__eq__(y)
True
>>> z = [42]
>>> x.__eq__(z)
False
You can also use the operator module:
>>> import operator
>>> operator.eq(x, y)
True
>>> operator.eq(x, z)
False
In Python 2, you could use looping with any() and cmp(), with itertools.izip_longest() to make sure we don't ignore uneven lengths:
>>> from itertools import izip_longest
>>> not any(cmp(i, j) for i, j in izip_longest(x, y, fillvalue=object()))
True
>>> not any(cmp(i, j) for i, j in izip_longest(x, z, fillvalue=object()))
False
This works because cmp() returns 0 for values that are equal. any() returns False only if all results are false (e.g. 0).
Hell, go straight for cmp() without looping:
>>> not cmp(x, y)
True
>>> not cmp(x, z)
False
For Python 3 you'd have to create your own cmp() function, perhaps using .__lt__ and .__gt__ if you want to avoid the < and > operators too.
For lists with only integers, you can forgo the cmp() function and go straight to subtraction; let's use map() here and include the list lengths:
>>> not (len(x) - len(y)) and not any(map(lambda i, j: i - j, x, y))
True
>>> not (len(x) - len(z)) and not any(map(lambda i, j: i - j, x, z))
False
This works because map() zips up the values in the lists and passes these pairs to the first argument, a callable. That subtracts the values and only if the integers are equal do we get all 0 values and any() returns False.
Apart from Martijn Pieters's answer, i could think of following options:
using XOR:
x = [1, 2, 3]
y = [1, 2, 3]
result = "list equal"
if len(x)-len(y):
result = "list not equal"
else:
for i,j in zip(x,y):
if i ^ j:
result = "list is not equal"
break
print result
Using set:
if set(x).difference(set(y)):
print "list not equal"
else:
print "list equal"
How can I use list comprehension in python to compare if two arrays has same elements or not?
I did the following:
>>> aa=[12,3,13];
>>> bb=[3,13,12];
>>> pp=[True for x in aa for y in bb if y==x]
>>> pp
[True, True, True]
>>> bb=[3,13,123];
>>> pp=[True for x in aa for y in bb if y==x]
[True, True]
I also want to output the False if not true rather than outputting just two trues like in the latter case but don't know how to do it.
Finally, I want to get one True/False value (if all are true then true, if one of them is false, then false) rather than the list of true and/or false. I know the simple loop to iterate over pp(list of true and false) is enough for that but I am sure there is more pythonic way to that.
You are testing every element of each list against every element of the other list, finding all combinations that are True. Apart from inefficient, this is also the incorrect approach.
Use membership testing instead, and see all these tests are True with the all() function:
all(el in bb for el in aa)
all() returns True if each element of the iterable you give it is True, False otherwise.
This won't quite test if the lists are equivalent; you need to test for the length as well:
len(aa) == len(bb) and all(el in bb for el in aa)
To make this a little more efficient for longer bb lists; create a set() from that list first:
def equivalent(aa, bb):
if len(aa) != len(bb):
return False
bb_set = set(bb)
return all(el in bb_set for el in aa)
This still doesn't deal with duplicate numbers very well; [1, 1, 2] is equivalent to [1, 2, 2] with this approach. You underspecified what should happen in such cornercases; the only strict equivalent test would be to sort both inputs:
len(aa) == len(bb) and sorted(aa) == sorted(bb)
where we first test for length to avoid having to sort in case the lengths differ.
If duplicates are allowed, whatever the length of the input, you can forgo loops altogether and just use sets:
not set(aa).symmetric_difference(bb)
to test if they have the same unique elements.
set(aa) == set(bb)
This has the same effect but may be slightly faster
not set(aa).symmetric_difference(bb)
If you need [1,1] to be not equivalent to [1] do
sorted(aa) == sorted(bb)
"You are testing every element of each list against every element of
the other list, finding all combinations that are True. Apart from
inefficient, this is also the incorrect approach."
I agree with the above statement, the below code lists the False values as well but I don't think you really need this.
>>> bp = [y==x for x in aa for y in bb]
[False, False, True, True, False, False, False, True, False]
>>> False in bp
True
Using Python, am finding it difficult to get filter() to work with lambda for cases where more than 1 argument needs to be passed as is the case in the following snippet:
max_validation = lambda x,y,z: x < y < z
sequence1 = [1,4,8]
filter(max_validation, sequence1)
It raises the following error:
TypeError: <lambda>() takes exactly 3 arguments (1 given)
Please suggest as to what am doing wrong here.
It's a little bit difficult to figure out exactly what you're trying to do. I'm going to interpret your question, then provide an answer. If this is not correct, please modify your question or comment on this answer.
Question
I have sequences that are exactly three elements long. Here's one:
sequence1 = [1, 4, 8]
I want to ensure that the first element is less than the second element, which should in turn be less than the third element. I've written the following function to do so:
max_validation = lambda x, y, z: x < y < z
How do I apply this using filter? Using filter(max_validation, sequence1) doesn't work.
Answer
Filter applies your function to each element of the provided iterable, picking it if the function returns True and discarding it if the function returns False.
In your case, filter first looks at the value 1. It tries to pass that into your function. Your function expects three arguments, and only one is provided, so this fails.
You need to make two changes. First, put your three-element sequence into a list or other sequence.
sequences = [[1, 4, 8], [2, 3, 9], [3, 2, 3]]
max_validation = lambda x: x[0] < x[1] < x[2] and len(x) == 3
I've added two other sequences to test. Because sequences is a list of a list, each list gets passed to your test function. Even if you're testing just one sequence, you should use [[1, 4, 8]] so that the entire sequence to test gets passed into your function.
I've also modified max_validation so that it accepts just one argument: the list to test. I've also added and len(x) == 3 to ensure that the sequences are only 3 elements in length
The function passed to filter() only gets a single argument passed to it, which is the current element in the iterable being iterated.. If you need something fancier than that then filter() won't do.
Straight from the docs of Python Filters
Note that filter(function, iterable)
is equivalent to [item for item in
iterable if function(item)] if
function is not None and [item for
item in iterable if item] if function
is None.
So, you can just process single arguments with Python filters. That effectively means you cannot use filters for the your example. You would have to write custom-code for the same.
It's possible to do this using a closure:
>>> def foo(a,b):
... def bar(c):
... return a+b+c
... return bar
...
>>> x = foo(1,2)
>>> x(3)
6
>>> y = foo(100,0)
>>> y(1)
101
>>> x(1)
4
I hope you are aware of?
>>> max([1, 4, 8])
8
filter() takes a single argument. In your case, it will take 1. Then 4. Then 8.
This would work for sequences of any length:
all(x < y for x, y in zip(seq, seq[1:]))
What does there happens?
For sequence 1, 2, 3... you take sequences 1, 2, 3... and 2, 3, 4... and zip them together to sequence (1, 2), (2, 3), ...
Then you check if statement 'x < y' holds for every pair.
And this will work for any associative rule you want to check.
Useful links:
slices in Python
zip in Python docs
all in Python docs
I think all others didn't get the point. the error message is for lambda function not for the filter. You should rather call it this way:
filter(max_validation, *sequence1)
add a star on the list transform it into three arguments, then it will work.
I'm in agreement with both #walkingpendulum and #Wesley, depending on the interpretation of the actual problem statement. So parsing through the ambiguity in the problem statement:
If you're sequentially comparing one item to its previous value in an iterable object, a lambda expression is overkill, just
use a list comprehension:
[1 if i < i+1 else 0 for i in sequence1]
If you're comparing objects, then just compare them -- a lambda expression wouldn't work firstly because you're only passing one argument where the lambda expression you defined you're passing three and lambda is generally applied across an iterable object. To compare objects, there are simple constructs for that:
sequence1 == some_other_sequence
and
x, y, z = 1, 2, 3
x < y < z
And lastly, if you want to apply a lambda expression to an iterable object, map can get you there: (arbitrary lambda function)
map(lambda x: x > 1, sequence1)
Otherwise #walkingpendulum and #Wesley cover the other interpretations
You could change
max_validation = lambda x,y,z: x < y < z
to
max_validation = lambda (x,y,z): x < y < z