Adding or multiplying a large list of numbers in Python can elegantly be done by folding the list with the addition or multiplication operator:
import functools, operator
lst = range(1,100)
sum = functools.reduce(operator.add, lst)
prod = functools.reduce(operator.mul, lst)
This needs the function equivalents of the operators + and * which
are provided by the operator module as operator.add and
operator.mul, respectively.
If I want to use the same idiom with the operator or:
ingredients = ['onion', 'celery', 'cyanide', 'chicken stock']
soup_is_poisonous = functools.reduce(operator.or, map(is_poisonous, ingredients))
... then I discover that operator doesn't have a function equivalent of the logical and and or operators (though it has one for logical not)
Of course, I can trivially write one that works:
def operator_or(x,y):
return x or y
But I wonder: why are there no operator.or and operator.and in operator? Bitwise and and or are there, but not the logical ones.
Of course this is just a minor annoyance, and the answer may well be
the same as with the missing identity function:
that it is easy to write one. But this holds for * and + as well, so why the difference?
To wrap up all your helpful answers
and comments, in order of somewhat decreasing (to me) convincingness:
the addition of operator.or would break an important promise made by the module
For all operators <op> that have function equivalents
operator.op in the operator module, it is the case that a <op> b is equivalent to (i.e. can always, without changing program
behaviour, replace or be replaced by) operator.op(a, b). This
equivalence is actually mentioned in the module docstring. This is
impossible to do for the operators and and or as their
evaluation is short-circuiting while Python function calls are always evaluated after all of their arguments are.
On the values True and False, | and &, hence also the existing (bitwise) operator.and_ and operator.or_
already return the same results (if they return at all, that is) as or and and.
If is_poisonous() returns either True of False (not an unreasonable requirement), I could use
soup_is_poisonous = reduce(operator.or_, map(is_poisonous, ingredients), False)
in the example from the original question. However, many Python
programs conveniently use any "truthy" value as True in idioms like
your_model_T_color = "black" or any_color_you_like
using | or operator.or_ instead of or here will result in a
TypeError or, even worse, some unexpected value (if the operands
are ints)
The functions any and all can be used instead of
functools.reduce(operator.or, ....)
I'm not convinced by this
argument: operator functions are used in many more contexts than
as a first argument to reduce. Moreover, any always returns
either True or False, not the first truthy value:
any([0,0,0,5,6,7]) # returns True
reduce(lambda x, y: x or y, [0,0,0,5,6,7]) # returns 5
so any and reduce(operator.or would not really be equivalent
any([x,y]) does the same (and more, as it accepts iterables) as operator.or(x,y) would.
That is not quite true (see above), any([0,5]) returns True while operator.or(0,5) would return 5. Moreover, the number of arguments matters greatly if we use a function as an argument to another function like reduce()
all is short-circuiting logical-and.
any is short-circuiting logical-or.
No need to put versions that take exactly two arguments (instead of an iterable) into the operator module, I guess.
Related
We know that,
a = 1
b = 2
print(not a > b)
is the correct way of using the "not" keyword and the below throws an error
a = 1
b = 2
print(a not > b)
since "not" inverts the output Boolean.
Thus, by this logic the correct way for checking the presence of a member in a list should be
a = 1
b = [2,3,4,5]
print(not a in b)
But I find the most common way is
a = 1
b = [2,3,4,5]
print(a not in b)
which from the logic given in previous example should throw an error.
So what is the correct way of using the "not in" operator in Python3.x?
not in is a special case that simplifies to exactly what you tried first. Namely,
a not in b
literally simplifies to not (a in b). It also works (slightly differently, but same idea) for is.
a is not b`
is equivalent to not (a is b). Python added these because they flow naturally like English prose. On the other hand, a not < b doesn't look or feel natural, so it's not allowed. The not in and is not are special cases in the grammar, not small parts of a general rule about where not can go. The only general rule in play is that not can always be used as a prefix operator (like in not (a < b))
what is the correct way of using the "not in" operator
There is only one way to use the not in operator. Your not a in b instead uses the not operator and the in operator.
PEP 8 doesn't seem to have an opinion about which to use, but about the similar is not operator (thanks Silvio) it says:
Use is not operator rather than not ... is. While both expressions are functionally identical, the former is more readable and preferred:
# Correct:
if foo is not None:
# Wrong:
if not foo is None:
So I'd say not in should also be preferred, for the same reason.
not, not in and in are all valid operators. Transitively, not (in_expression) is also valid
Correct way? Refer Zen of Python.
First of all not in, is not a two separate operator, is constituently a single operator ,and also known as membership operator. There is another membership operator that is in. Membership operator has high precedence than logical NOT, AND and OR.
print(not a in b) -> This is actually first evaluating a in b then result is inverted by the logical 'not' and then result is printed.
So as per your example it should print True as a in b gives False then it is inverted to True via logical NOT operator.
print(a not in b) -> Here python checks if a is not a part of the b, if it is return 'False' else 'True` .
So as per your example it should return True as a is not a part of b.
I think a not in b is more clear than not a in b.I would suggest to use membership operator for testing the membership.
However the result will remain same for both kind of expression but the process of evaluating is completely different.
I'm learning Python and I just started learning conditionals with booleans
I am very confused though as to the specific topic of "If Not". Could someone please explain to me the difference between :
x = False
if not x:
print("hello")
if x == False:
print("hello")
When testing this code on a Python compiler, I receive "hello" twice. I can assume this means that they both mean the same thing to the computer.
Could someone please explain to me why one would use one method over the other method?
It depends™. Python doesn't know what any of its operators should do. It calls magic methods on objects and lets them decide. We can see this with a simple test
class Foo:
"""Demonstrates the difference between a boolean and equality test
by overriding the operations that implement them."""
def __bool__(self):
print("bool")
return True
def __eq__(self, other):
print("eq", repr(other))
return True
x = Foo()
print("This is a boolean operation without an additional parameter")
if not x:
print("one")
print("This is an equality operation with a parameter")
if x == False:
print("two")
Produces
This is a boolean operation without an additional parameter
bool
This is an equality operation with a parameter
eq False
two
In the first case, python did a boolean test by calling __bool__, and in the second, an equality test by calling __eq__. What this means depends on the class. Its usually obvious but things like pandas may decide to get tricky.
Usually not x is faster than x == False because the __eq__ operator will typically do a second boolean comparison before it knows for sure. In your case, when x = False you are dealing with a builtin class written in C and its two operations will be similar. But still, the x == False comparison needs to do a type check against the other side, so it will be a bit slower.
There are already several good answers here, but none discuss the general concept of "truthy" and "falsy" expressions in Python.
In Python, truthy expressions are expression that return True when converted to bool, and falsy expressions are expressions that return False when converted to bool. (Ref: Trey Hunner's regular expression tutorial; I'm not affiliated with Hunner, I just love his tutorials.)
Truthy stuff:
What's important here is that 0, 0.0, [], None and False are all falsy.
When used in an if statement, they will fail the test, and they will pass the test in an if not statement.
Falsy stuff:
Non-zero numbers, non-empty lists, many objects (but read #tdelaney's answer for more details here), and True are all truthy, so they pass if and fail if not tests.
Equality tests
When you use equality tests, you're not asking about the truthiness of an expression, you're asking whether it is equal to the other thing you provide, which is much more restrictive than general truthiness or falsiness.
EDIT: Additional references
Here are more references on "Truthy" and "Falsy" values in Python:
Truth value testing in the Python manual
The exhaustive list of Falsy values
Truthy and Falsy tutorial from freeCodeCamp
In one case you are checking for equality with the value "False", on the other you are performing a boolean test on a variable. In Python, several variables pass the "if not x" test but only x = False passes "if x == False".
See example code:
x = [] # an empty list
if not x: print("Here!")
# x passes this test
if x == False: print("There!")
# x doesn't pass this test
Try it with x = None: not x would be True then and x == False would be False. Unlike with x = False when both of these are True. not statement also accounts for an empty value.
Python has a great syntax for null coalescing:
c = a or b
This sets c to a if a is not False, None, empty, or 0, otherwise c is set to b.
(Yes, technically this is not null coalescing, it's more like bool coalescing, but it's close enough for the purpose of this question.)
There is not an obvious way to do this for a collection of objects, so I wrote a function to do this:
from functools import reduce
def or_func(x, y):
return x or y
def null_coalesce(*a):
return reduce(or_func, a)
This works, but writing my own or_func seems suboptimal - surely there is a built-in like __or__? I've attempted to use object.__or__ and operator.__or__, but the first gives an AttributeError and the second refers to the bitwise | (or) operator.
As a result I have two questions:
Is there a built-in function which acts like a or b?
Is there a built-in implementation of such a null coalesce function?
The answer to both seems to be no, but that would be somewhat surprising to me.
It's not exactly a single built-in, but what you want to achieve can be easily done with:
def null_coalesce(*a):
return next(x for x in a if x)
It's lazy, so it does short-circuit like a or b or c, but unlike reduce.
You can also make it null-specific with:
def null_coalesce(*a):
return next(x for x in a if x is not None)
Is there a built-in function which I can use which acts like a or b?
No. Quoting from this answer on why:
The or and and operators can't be expressed as functions because of their short-circuiting behavior:
False and some_function()
True or some_function()
in these cases, some_function() is never called.
A hypothetical or_(True, some_function()), on the other hand, would have to call some_function(), because function arguments are always evaluated before the function is called.
Is there a built-in implementation of such a null coalesce function?
No, there isn't. However, the Python documentation page for itertools suggests the following:
def first_true(iterable, default=False, pred=None):
"""Returns the first true value in the iterable.
If no true value is found, returns *default*
If *pred* is not None, returns the first item
for which pred(item) is true.
"""
# first_true([a,b,c], x) --> a or b or c or x
# first_true([a,b], x, f) --> a if f(a) else b if f(b) else x
return next(filter(pred, iterable), default)
Marco has it right, there's no built-in, and itertools has a recipe. You can also pip install boltons to use the boltons.iterutils.first() utility, which is perfect if you want short-circuiting.
from boltons.iterutils import first
c = first([a, b])
There are a few other related and handy reduction tools in iterutils, too, like one().
I've done enough of the above that I actually ended up wanting a higher-level tool that could capture the entire interaction (including the a and b references) in a Python data structure, yielding glom and its Coalesce functionality.
from glom import glom, Coalesce
target = {'b': 1}
spec = Coalesce('a', 'b')
c = glom(target, spec)
# c = 1
(Full disclosure, as hinted above, I maintain glom and boltons, which is good news, because you can bug me if you find bugs.)
operator module makes it easy to avoid unnecessary functions and lambdas
in situations like this:
import operator
def mytest(op, list1, list2):
ok = [op(i1, i2) for i1, i2 in zip(list1, list2)]
return all(ok)
mytest(operator.eq, [1, 2, 3], [1, 2, 3]) # True
mytest(operator.add, [-1, 2, -3], [1, -2, 33]) # False
Well, now I need to do i1 and i2, but to my surprise, I can't find and in the operator module! And the same applies to or! I know, and is not exactly operator, it's a keyword, but not, along with is and even del, are all keywords and all are included.
So what's the story? Why are they missing?
Because you cannot convert boolean operators into python functions. Functions always evaluate their arguments, and boolean operators do not. Adding and and or to the operators module would also require adding a special kind of functions (like lisp "macros") that evaluate their arguments on demand. Obviously, this is not something python designers ever wanted. Consider:
if obj is not None and obj.is_valid():
....
you cannot write this in a functional form. An attempt like
if operator.xyz(obj is not None, obj.is_valid())
will fail if obj is actually None.
You can write these yourself, but you'll need to pass a function (e.g. lambda) for the second argument to prevent it from being evaluated at call time, assuming that the usual short-circuiting behavior is important to you.
def func_or(val1, fval2):
return val1 or fval2()
def func_and(val1, fval2):
return val1 and fval2()
Usage:
func_or(False, lambda: True)
func_and(True, lambda: False)
The reason there's no operator.and is that and is a keyword, so that would be a SyntaxError.
As tgh435 explained, the reason there's no renamed and function in operator is that it would be misleading: a function call always evaluates its operands, but the and operator doesn't. (It would also be an exception to an otherwise consistent and simple rule.)
In your case, it looks like you don't actually care about short-circuiting at all, so can build your own version trivially:
def and_(a, b):
return a and b
Or, if you're just using it once, even inline:
mytest(lambda a, b: a and b, [-1, 2, -3], [1, -2, 33])
In some cases, it's worth looking at all (and, for or, any). It is effectively short-circuited and expanded to arbitrary operands. Of course it has a different API than the operator functions, taking a single iterable of operands instead of two separate operands. And the way it short-circuits is different; it just stops iterating the iterable, which only helps if you've set things up so the iterable is only evaluating things as needed. So, it's usually not usable as a drop-in replacement—but it's sometimes usable if you refactor your code a bit.
Python's and and or syntaxes cannot directly be mapped to functions. These syntaxes are lazy evaluated: If the result of the left part of the expression allows to know the value of the whole expression, the right part is skipped. Since they introduce flow control, their behavior cannot be reproduced using an operator.
To reduce confusion, python have chosen to simply not provide these methods.
georg gives a good example of a situation where and laziness matters:
if obj is not None and obj.is_valid():
...
Now, if you don't need lazy evaluation, you can use abarnert's answer implementation:
def and_(a, b):
return a and b
def or_(a, b):
return a or b
Usage:
>>> or_(False, True)
>>> and_(True, False)
If you need lazy evaluation, you can use kindall's answer implementation:
def func_or(val1, fval2):
return val1 or fval2()
def func_and(val1, fval2):
return val1 and fval2()
Usage:
>>> func_or(False, lambda: True)
>>> func_and(True, lambda: False)
Note:
As mentioned in the comments, the functions operator.and_ and operator.or_ correspond to the bitwise operators & and |. See: https://docs.python.org/3/library/operator.html#mapping-operators-to-functions
Note that the names operators.and and operators.or aren't used: and and or are Python keywords so it would be a syntax error.
This question already has answers here:
Why does comparing strings using either '==' or 'is' sometimes produce a different result?
(15 answers)
Closed 9 years ago.
I noticed a Python script I was writing was acting squirrelly, and traced it to an infinite loop, where the loop condition was while line is not ''. Running through it in the debugger, it turned out that line was in fact ''. When I changed it to !='' rather than is not '', it worked fine.
Also, is it generally considered better to just use '==' by default, even when comparing int or Boolean values? I've always liked to use 'is' because I find it more aesthetically pleasing and pythonic (which is how I fell into this trap...), but I wonder if it's intended to just be reserved for when you care about finding two objects with the same id.
For all built-in Python objects (like
strings, lists, dicts, functions,
etc.), if x is y, then x==y is also
True.
Not always. NaN is a counterexample. But usually, identity (is) implies equality (==). The converse is not true: Two distinct objects can have the same value.
Also, is it generally considered better to just use '==' by default, even
when comparing int or Boolean values?
You use == when comparing values and is when comparing identities.
When comparing ints (or immutable types in general), you pretty much always want the former. There's an optimization that allows small integers to be compared with is, but don't rely on it.
For boolean values, you shouldn't be doing comparisons at all. Instead of:
if x == True:
# do something
write:
if x:
# do something
For comparing against None, is None is preferred over == None.
I've always liked to use 'is' because
I find it more aesthetically pleasing
and pythonic (which is how I fell into
this trap...), but I wonder if it's
intended to just be reserved for when
you care about finding two objects
with the same id.
Yes, that's exactly what it's for.
I would like to show a little example on how is and == are involved in immutable types. Try that:
a = 19998989890
b = 19998989889 +1
>>> a is b
False
>>> a == b
True
is compares two objects in memory, == compares their values. For example, you can see that small integers are cached by Python:
c = 1
b = 1
>>> b is c
True
You should use == when comparing values and is when comparing identities. (Also, from an English point of view, "equals" is different from "is".)
The logic is not flawed. The statement
if x is y then x==y is also True
should never be read to mean
if x==y then x is y
It is a logical error on the part of the reader to assume that the converse of a logic statement is true. See http://en.wikipedia.org/wiki/Converse_(logic)
See This question
Your logic in reading
For all built-in Python objects (like
strings, lists, dicts, functions,
etc.), if x is y, then x==y is also
True.
is slightly flawed.
If is applies then == will be True, but it does NOT apply in reverse. == may yield True while is yields False.