Why do we see Python assignments with or?
For example:
def my_function(arg_1=None, arg_2=0):
determination = arg_1 or arg_2 or 'no arguments given!'
print(determination)
return determination
When called with no arguments, the above function would print and return 'no arguments given!'
Why does Python do this, and how can one best make best use of this functionality?
What the "or" expression does on assignment:
We sometimes see examples of this in Python as a substitute for conditional expression with ternary assignments, (in fact, it helped inspire the language to add conditional statements).
x = a or b
If bool(a) returns False, then x is assigned the value of b
Identical Use-case of Conditional Expressions (i.e. Ternary Assignments)
Here's an example of such a conditional expression that accomplishes the same thing, but with perhaps a bit less mystery.
def my_function(arg_1=None, arg_2=0):
determination = arg_1 if arg_1 else arg_2 if arg_2 else 'no arguments given!'
print(determination)
return determination
Repeating this syntax too much is considered to be bad style, otherwise it's OK for one-liners. The downside is that it is a bit repetitive.
or Expressions
The base case, x or y returns x if bool(x) evaluates True, else it evaluates y, (see the docs for reference). Therefore, a series of or expressions has the effect of returning the first item that evaluates True, or the last item.
For example
'' or [] or 'apple' or () or set(['banana'])
returns 'apple', the first item that evaluates as True, and
'' or [] or ()
returns (), even though it evaluates as False.
Extended and usage
For contrast, x and y returns x if bool(x) evaluates as False, else it returns y.
It makes sense that and would work this way when you consider that all of the conditions in a conditional and series needs to evaluate as True for the control flow to proceed down that path, and that it makes no sense to continue evaluating those items when you come across one that is False.
The utility of using and for assignment is not immediately as apparent as using or, but it was historically used for ternary assignment. That is, before this more clear and straightforward construction was available:
a = x if condition else y
the equivalent formed with boolean operators was:
a = condition and x or z # don't do this!
which while the meaning is derivable based on a full understanding of Python and and or evaluation, is not nearly as readable as the ternary conditional, and is best avoided altogether.
Conclusion
Using Boolean expressions for assignment must be done carefully. Definitely never use and for assignment, which is confusing enough to be quite error-prone. Style mavens will find use of or for assignments less preferable (than the more verbose ternary, if condition else), but I have found that it is so common in the professional Python community that it could be considered idiomatic.
If you choose to use it, do so cautiously with the understanding that the final element, if reached, will always be returned regardless of its evaluation, so that final element should probably be a literal, so that you know you have a good default fallback for your variable.
Related
I have a simple question regarding the use of parentheses in Python's conditional statements.
The following two snippets work just the same but I wonder if this is only true because of its simplicity:
>>> import os, socket
>>> if ((socket.gethostname() == "bristle") or (socket.gethostname() == "rete")):
... DEBUG = False
... else:
... DEBUG = True
...
>>> DEBUG
and now without parentheses
>>> import os, socket
>>> if socket.gethostname() == "bristle" or socket.gethostname() == "rete":
... DEBUG = False
... else:
... DEBUG = True
...
>>> DEBUG
Could anyone help shed some light on this? Are there any cases where I should definitely use them?
The other answers that Comparison takes place before Boolean are 100% correct. As an alternative (for situations like what you've demonstrated) you can also use this as a way to combine the conditions:
if socket.gethostname() in ('bristle', 'rete'):
# Something here that operates under the conditions.
That saves you the separate calls to socket.gethostname and makes it easier to add additional possible valid values as your project grows or you have to authorize additional hosts.
The parentheses just force an order of operations. If you had an additional part in your conditional, such as an and, it would be advisable to use parentheses to indicate which or that and paired with.
if (socket.gethostname() == "bristle" or socket.gethostname() == "rete") and var == condition:
...
To differentiate from
if socket.gethostname() == "bristle" or (socket.gethostname() == "rete" and var == condition):
...
The parentheses are redundant in this case. Comparison has a higher precedence than Boolean operators, so the comparisons will always be performed first regardless of the parentheses.
That said, a guideline I once saw (perhaps in Practical C Programming) said something like this:
Multiplication and division first
Addition and subtraction next
Parentheses around everything else
(Yes, IIRC they left out exponentiation!)
The idea being that the precedence rules are arcane enough that nobody should be expected to remember them all, neither the original programmer nor the maintenance programmer reading the code, so it is better to make it explicit. Essentially the parentheses serve both to communicate the intent to the compiler and as documentation for the next schmoe who has to work on it.
I believe in Python those two statements will generate the same bytecode so you're not even losing any efficiency.
I was always thinking that this is part of PEP8, but apparently it's not. However in all examples you meet in PEPs, code samples and documentation you never see redundant parentheses (there is even such an inspection in PyCharm, for example).
General recommendation is to use parentheses only if it improves readability or you actually want to change the order of expression calculation (such as (a or b) and c).
Do:
if (first_expr or second_expr) and third_expr:
if first_expr or second_expr:
Don't:
if ((first_expr or second_expr) and third_expr):
if (first_expr):
if (first_expr or (second_expr and third_expr)):
In your code sample, parentheses are completely redundant, just use if socket.gethostname() == "bristle" or socket.gethostname() == "rete": (in production code, of course, in will be much more readable, but that's rather off-topic now)
In Python and many other programming languages, parentheses are not required for every expression with multiple operators. This is because operators have a defined precedence. See the table here (Section 5.15) for information on operator precedence in Python.
You can draw an analogy to arithmetic. These expressions are equivalent:
5 * 5 + 3
(5 * 5) + 3
If you mean to add three first, then you need to use the parentheses like this:
5 * (5 + 3)
Have a look at the manual. The higher you are up in the list, the operator will be applied later. "or" is above "==" , and therefore, in this particular case the answers are the same. However, for readability, and just to be sure, I would recommend parenthesis.
I just met a similar question. My conditional statement is
if count1==0 & count2==0 & count3==0:
The first result is True, the second is False, and the third is True. Intuitively, the result for this conditional statement is true. However, it's false. But the result for the following sentence is correct；
if (count1==0) & (count2==0) & (count3==0):
I still have not figured out why this happening.
Clearly, it may not make that much sense to make an assignment to a function return, yet I've just encountered a strange situation and could not find a proper rule for that.
The code below does not run:
def foo():
a = [1,2,3,4,5]
return a
foo()= 10
however,
def foo():
a = [1,2,3,4,5]
return a
foo()[2] = 10
works perfectly. It is not clear to me what is going on here. It looks to me that there is an inconsistency here...
Thanks a lot.
It is not clear to me what is going on here.
It's fairly simple, really; the left-hand-side of an assignment statement is not allowed to just be any kind of expression, but some "kinds" of expression are valid assignment targets, and for those kinds of expression it doesn't matter what kinds of sub-expressions they may have. In your example, the left-hand-side is a subscription, which is a valid assignment target, so it's a syntactically valid assignment statement.
In particular, a subscription has two sub-expressions but it doesn't matter what lexical form those sub-expressions have. For example, the below may look perfectly normal to you, where the first sub-expression foo.bar is an attributeref and the second sub-expression index + 5 is an a_expr:
foo.bar[index + 5] = value
This example probably also looks perfectly normal: the first sub-expression baz[get_x()] is another subscription, and the second sub-expression get_y() is a call.
baz[get_x()][get_y()] = value
As for why it's allowed, the simple reason is that it would be a bit silly and arbitrary for the Python developers to decide on a list of restrictions on the sub-expressions. The meaning of the assignment statement is well-defined so long as the left-hand-side is one of the required lexical forms, so the only reason to impose further restrictions on the sub-expressions of those lexical forms would be to try to prevent mistakes by the programmer.
On the other hand, code like func_call()[index] = value is not necessarily a mistake anyway, because the list that func_call() returns might well be referenced elsewhere, so the mutation will be visible through other references to that list.
In python, not every value can be used of the left side of the assignment (so called lvalue). Basically, only 3 things can be lvalues:
a name, like abc
an attribute any_expression.name
a subscript any_expression[slice]
Other values, like literals, arithmetic expressions, function calls etc cannot be used on the left side:
123 = 123 # no
a + b = 123 # no
func() = 123 # no
See assignment in the grammar spec: https://docs.python.org/3/reference/grammar.html
First, the code:
>>> False or 'hello'
'hello'
This surprising behavior lets you check if x is not None and check the value of x in one line:
>>> x = 10 if randint(0,2) == 1 else None
>>> (x or 0) > 0
# depend on x value...
Explanation: or functions like this:
if x is false, then y, else x
No language that I know lets you do this. So, why does Python?
It sounds like you're combining two issues into one.
First, there's the issue of short-circuiting. Marcin's answer addresses this issue perfectly, so I won't try to do any better.
Second, there's or and and returning the last-evaluated value, rather than converting it to bool. There are arguments to be made both ways, and you can find many languages on either side of the divide.
Returning the last-evaluated value allows the functionCall(x) or defaultValue shortcut, avoids a possibly wasteful conversion (why convert an int 2 into a bool 1 if the only thing you're going to do with it is check whether it's non-zero?), and is generally easier to explain. So, for various combinations of these reasons, languages like C, Lisp, Javascript, Lua, Perl, Ruby, and VB all do things this way, and so does Python.
Always returning a boolean value from an operator helps to catch some errors (especially in languages where the logical operators and the bitwise operators are easy to confuse), and it allows you to design a language where boolean checks are strictly-typed checks for true instead of just checks for nonzero, it makes the type of the operator easier to write out, and it avoids having to deal with conversion for cases where the two operands are different types (see the ?: operator in C-family languages). So, for various combinations of these reasons, languages like C++, Fortran, Smalltalk, and Haskell all do things this way.
In your question (if I understand it correctly), you're using this feature to be able to write something like:
if (x or 0) < 1:
When x could easily be None. This particular use case isn't very useful, both because the more-explicit x if x else 0 (in Python 2.5 and later) is just as easy to write and probably easier to understand (at least Guido thinks so), but also because None < 1 is the same as 0 < 1 anyway (at least in Python 2.x, so you've always got at least one of the two options)… But there are similar examples where it is useful. Compare these two:
return launchMissiles() or -1
return launchMissiles() if launchMissiles() else -1
The second one will waste a lot of missiles blowing up your enemies in Antarctica twice instead of once.
If you're curious why Python does it this way:
Back in the 1.x days, there was no bool type. You've got falsy values like None, 0, [], (), "", etc., and everything else is true, so who needs explicit False and True? Returning 1 from or would have been silly, because 1 is no more true than [1, 2, 3] or "dsfsdf". By the time bool was added (gradually over two 2.x versions, IIRC), the current logic was already solidly embedded in the language, and changing would have broken a lot of code.
So, why didn't they change it in 3.0? Many Python users, including BDFL Guido, would suggest that you shouldn't use or in this case (at the very least because it's a violation of "TOOWTDI"); you should instead store the result of the expression in a variable, e.g.:
missiles = launchMissiles()
return missiles if missiles else -1
And in fact, Guido has stated that he'd like to ban launchMissiles() or -1, and that's part of the reason he eventually accepted the ternary if-else expression that he'd rejected many times before. But many others disagree, and Guido is a benevolent DFL. Also, making or work the way you'd expect everywhere else, while refusing to do what you want (but Guido doesn't want you to want) here, would actually be pretty complicated.
So, Python will probably always be on the same side as C, Perl, and Lisp here, instead of the same side as Java, Smalltalk, and Haskell.
No language that i know lets you do this. So, why Python do?
Then you don't know many languages. I can't think of one language that I do know that does not exhibit this "shortcircuiting" behaviour.
It does it because it is useful to say:
a = b or K
such that a either becomes b, if b is not None (or otherwise falsy), and if not it gets the default value K.
Actually a number of languages do. See Wikipedia about Short-Circuit Evaluation
For the reason why short-circuit evaluation exists, wikipedia writes:
If both expressions used as conditions are simple boolean variables,
it can be actually faster to evaluate both conditions used in boolean
operation at once, as it always requires a single calculation cycle,
as opposed to one or two cycles used in short-circuit evaluation
(depending on the value of the first).
This behavior is not surprising, and it's quite straightforward if you consider Python has the following features regarding or, and and not logical operators:
Short-circuit evaluation: it only evaluates operands up to where it needs to.
Non-coercing result: the result is one of the operands, not coerced to bool.
And, additionally:
The Truth Value of an object is False only for None, False, 0, "", [], {}. Everything else has a truth value of True (this is a simplification; the correct definition is in the official docs)
Combine those features, and it leads to:
or : if the first operand evaluates as True, short-circuit there and return it. Or return the 2nd operand.
and: if the first operand evaluates as False, short-circuit there and return it. Or return the 2nd operand.
It's easier to understand if you generalize to a chain of operations:
>>> a or b or c or d
>>> a and b and c and d
Here is the "rule of thumb" I've memorized to help me easily predict the result:
or : returns the first "truthy" operand it finds, or the last one.
and: returns the first "falsy" operand it finds, or the last one.
As for your question, on why python behaves like that, well... I think because it has some very neat uses, and it's quite intuitive to understand. A common use is a series of fallback choices, the first "found" (ie, non-falsy) is used. Think about this silly example:
drink = getColdBeer() or pickNiceWine() or random.anySoda or "meh, water :/"
Or this real-world scenario:
username = cmdlineargs.username or configFile['username'] or DEFAULT_USERNAME
Which is much more concise and elegant than the alternative.
As many other answers have pointed out, Python is not alone and many other languages have the same behavior, for both short-circuit (I believe most current languanges are) and non-coercion.
"No language that i know lets you do this. So, why Python do?" You seem to assume that all languages should be the same. Wouldn't you expect innovation in programming languages to produce unique features that people value?
You've just pointed out why it's useful, so why wouldn't Python do it? Perhaps you should ask why other languages don't.
You can take advantage of the special features of the Python or operator out of Boolean contexts. The rule of thumb is still that the result of your Boolean expressions is the first true operand or the last in the line.
Notice that the logical operators (or included) are evaluated before the assignment operator =, so you can assign the result of a Boolean expression to a variable in the same way you do with a common expression:
>>> a = 1
>>> b = 2
>>> var1 = a or b
>>> var1
1
>>> a = None
>>> b = 2
>>> var2 = a or b
>>> var2
2
>>> a = []
>>> b = {}
>>> var3 = a or b
>>> var3
{}
Here, the or operator works as expected, returning the first true operand or the last operand if both are evaluated to false.
For one of my sites, I need to check if several class attributes are defined and not empty. So far, I've happily used if self.attr:, which in my mind is the shorthand for if self.attr is not None and self.attr is not '':, or whatever the undefined value of the attribute is.
This works fine, but yields to surprising behavior when checking multiple string attributes. '' and '' is not False (as I expected), but ''.
This begs the question: are there other types for which the and operator does not force a typecast to bool?
I can't come up with an example where this difference in behavior would cause an actual different outcome for the if-clause (after all, '' still evaluates to False), but am left with the gut feeling that there's edge cases that could be a trap.
Lastly, I'd be keen to know if anybody knows why it was implemented that way? I thought the Zen of Python encourages one way and one way only, and the + operator already seems to bethe intuitive way for string concatenation.
and never typecasts to bool. Rather, if calls bool() on the result of expressions.
An expression using and (and or, for that matter), short-circuits when it can determine that the expression will not evaluate to True or False based on the first operand, and returns the last evaluated value:
>>> 0 and 'string'
0
>>> 1 and 'string'
'string'
>>> 'string' or 10
'string'
>>> '' or 10
10
This 'side-effect' is often used in python code. Note that not does return a boolean value. See the python documentation on boolean operators for the details.
The documentation also explains what constitutes a True or False equivalent for various types, such as None, 0, '' and empty containers being False, while most everything else is equivalent to True.
For custom classes, you need to define the .__nonzero__() method (returns True or False) or a .__len__() method (returns an int, where 0 is False and everything else is True), to influence their boolean equivalent, otherwise they'll always default to True.
What I found out:
In Dive in to Python I read about the peculiar nature of and and or operators
and how shortcircuit evaluation of boolean operators may be used to express conditionals more succinctly via the and-or trick that works very much like the ternary operator in C.
C:
result = condition ? a : b
Python:
result = condition and a or b
This seems to come in handy since lambda functions are restricted to one-liners in Python, but it uses logical syntax to express control flow.
Since Python 2.5 the inline-if seems to have come to the rescue as a more readable syntax for the and-or trick:
result = a if condition else b
So I guess that this is the pythonic replacement for the less readable and-or-construct.
Even if I want to nest multiple conditions, it still looks quite comprehensive:
result = a if condition1 else b if condition2 else c
But in a world of uncertainity I frequently find myself writing some code like this to access a.b.c :
result = a and hasattr(a, 'b') and hasattr(a.b, 'c') and a.b.c or None
So with the help of inline-if I could probably get rid of some ands and ors, resulting in a quite readable piece of code:
result = a.b.c if hasattr(a, 'b') and hasattr(a.b, 'c') else None
I also discovered a somewhat arcane approach for conditonals in this recipe
result = (a, b)[condition]
but this doesn't short-circuit and will result in all kinds of errors if the result of the condition does not return a boolean, 0 or 1.
What I'd like to know:
Now I wonder if it is considered preferable / more pythonic to use the inline-if as much as possible if downwards compatibility is not a concern, or is all just a matter of taste and how much one feels at home in the world of short-circuit evaluation?
Update
I just realized that inline-if is more than syntactic sugar for the and-or-trick, since it will not fail when a is false in a boolean context. So It's probably more fail-proof.
The pythonic thing to do is recognise when you've stretched your code past what is sensible to cram into a single function and just use an ordinary function. Remember, there is nothing you can do with a lambda that couldn't also be done with a named function.
The breaking point will be different for everyone of course, but if you find yourself writing:
return a.b.c if hasattr(a, 'b') and hasattr(a.b, 'c') else None
too much, consider just doing this instead:
try:
return a.b.c
except AttributeError:
return None
Since there is is a special language construct with the inline if-else that does what you want and which was introduced to replace ugly workarounds like those you mention, it is a good idea to use it. Especially since hacks like the and-or trick usually have unexpected corner cases/bugs.
The and-or trick for example fails in this case:
a = 0
b = 1
c = True and a or b
c will be 1, which is not what you expect if you are looking for if-else semantics.
So why use buggy workarounds when there is a language construct that does exactly what you want?
I would rather be explicit about what my code is doing. Inline if is very explicitly doing conditional assignment and readability does count. Inline if would not have made it into the language if and/or side affects were considered preferable.
The pep for conditional expressions goes in to more detail about why that specific syntax was selected and it specifically discusses the and/or hack:
http://www.python.org/dev/peps/pep-0308/
This is for the particular case you mentioned, but I think that mostly every case in which you need chained short-circuit logic can be handled with a more elegant solution. Which is obviously a matter of taste, let me add, so scratch that if you think the above is better than this:
try:
foo = a.b.c
except AttributeError:
print "woops"
In other, less straightforward cases, encapsulating all the testing in a function might improve readibility a lot.
EDIT: by the way, duck typing.