I often see code similar to this:
return [(var, val) for val in self.domains[var]
if self.nconflicts(var, val, assignment) == 0]
and I'm like DAMN that's sexy. But then I try to drop it sometimes and I get syntax errors. Are there any particular rules for this nice form of code writing that reverses the typical placement of for and if statements?
They're called list comprehensions. The basic syntax is (I'm using parens to group my words, not as part of the syntax):
[(an expression involving x) for x in someList if (some condition)]
If the condition evaluates to true, the resulting list includes the (expression involving x). So, for example, the following list comprehension uses this to only include strings in the resulting list.
>>> myList = [1,"hello",5.4,"world"]
>>> [elem for elem in myList if type(elem)==str]
['hello', 'world']
Note that the if part is optional, and the expression involving x can be as simple as just x (often used when you are just filtering out elements from another list).
In fact, the expression involving x doesn't really have to have x in it at all. For example, if for some reason you wanted a list of 0's as long as your name you could do this:
>>> [0 for letter in "Matthew"]
[0, 0, 0, 0, 0, 0, 0]
For when you don't need the list to stick around after you make it, use generator expressions instead. (Generator expressions and list comprehensions have the same syntax.)
See list comprehensions in the Python tutorial documentation. There are quite a number of things you can do with this syntax, including creating lists, sets, and dictionaries.
Although your example code is indeed a list comprehension, you do also occasionally see the reverse if syntax for inline conditionals:
a if b else c
The concept is called list comprehension,
http://www.secnetix.de/olli/Python/list_comprehensions.hawk
Take a look at lambda functions too,
http://www.secnetix.de/olli/Python/lambda_functions.hawk
Related
This question already has an answer here:
Why a generator object is obtained instead of a list
(1 answer)
Closed 3 years ago.
Not sure if this has been asked before but I couldn't find a proper, clear explanation.
I had a concern about something related to python syntax.
While practicing some python, I Intuitively assumed this would print all the elements of the list; list1.
But it doesn't seem to do so, why would that be?
I could obviously print it in many other ways; but I fail to understand the inherent python logic at play here.
list1 = [1,2,3,4]
print(list1[i] for i in range(len(list1)))
I expected the output to be '[1, 2, 3, 4]', but it instead prints a generator object.
You need to surround list1[i] for i in range(len(list)) with [] to indicate that it's a list. Although list1 is a list, you are trying to use a generator expression to print it out, which will return a generator object (type of iterable similar to a list.) Without specifying you want to convert the generator to a list, it won't print a list. (A generator expression converted to a list is called list comprehension.)
Even if you did do this, it would still print it [1, 2, 3, 4] rather than 1 2 3 4. You need to do [print(list1[i], end=" ") for i in range(len(list1)))] for that to work. There are far better ways of doing this: see donkopotamus's answer.
The expression (list1[i] for i in range(len(list))) defines a generator object. So that is what is printed.
If you wish to print a list, then make it a list comprehension rather than a generator, and print that:
print( [list1[i] for i in range(len(list1))] )
Alternatively, you could force evaluation of the generator into a tuple (or list or set), by passing the generator to the appropriate type using eg
print(tuple(list1[i] for i in range(len(list1))))
In order to get the specific output you intended (space separated) of 1 2 3 4 you could use str.join in the following way:
>>> list1 = [1, 2, 3, 4]
>>> print(" ".join(list1[i] for i in range(len(list1))))
1 2 3 4
or unpack the list into print (this will not work in python 2, as in python 2 print is not a function)
>>> print(*(list1[i] for i in range(len(list1))))
1 2 3 4
(list1[i] for i in range(len(list1)))
is indeed a generator object, equivalent to simply
(x for x in list1)
You're passing that generator to print as a single argument, so print simply prints it: it does not extract the elements from it.
Alternatively, you can unpack it as you pass it to print:
print(*(list1[i] for i in range(len(list1))))
This will pass each element of the generated sequence to print as a separate argument, so they should each get printed.
If you simply meant to print your list, any of the following would have worked:
print(list1)
print([list1[i] for i in range(len(list1))])
print([x for x in list1])
The use of square brackets makes a list comprehension rather than a generator expression.
There is something called list comprehension and generator expression in python. They are awesome tools, you can find more info by googling. Here is a link.
Basically, you can make a list on the fly in python.
list1 = [1,2,3,4]
squared_list = [i*i for i in list1]
would return a list with all the items squared. However, is we say
squared_gen_list = (i*i for i in list1)
this returns what is known as a generator object. That is what is happening in your case, as you can see from the syntax and so you are just printing that out. Hope that clears up the confusion.
Is there any way to rewrite the below python code in one line
for i in range(len(main_list)):
if main_list[i] != []:
for j in range(len(main_list[i])):
main_list[i][j][6]=main_list[i][j][6].strftime('%Y-%m-%d')
something like below,
[main_list[i][j][6]=main_list[i][j][6].strftime('%Y-%m-%d') for i in range(len(main_list)) if main_list[i] != [] for j in range(len(main_list[i]))]
I got SyntaxError for this.
Actually, i'm trying to storing all the values fetched from table into one list. Since the table contains date method/datatype, my requirement needs to convert it to string as i faced with malformed string error.
So my approach is to convert that element of list from datetime.date() to str. And i got it working. Just wanted it to work with one line
Use the explicit for loop. There's no better option.
A list comprehension is used to create a new list, not to modify certain elements of an existing list.
You may be able to update values via a list comprehension, e.g. [L.__setitem__(i, 'some_value') for i in range(len(L))], but this is not recommended as you are using a side-effect and in the process creating a list of None values which you then discard.
You could also write a convoluted list comprehension with a ternary statement indicating when you meet the 6th element in a 3rd nested sublist. But this will make your code difficult to maintain.
In short, use the for loop.
You're getting a syntax error because you're not allowed to perform assignments within a list comprehension. Python forbids assignments because it is discouraging over complex list comprehensions in favour of for loops.
Obviously you shouldn't do this on one line, but this is how to do it:
import datetime
# Example from your comment:
type1 = "some type"
main_list = [[], [],
[[1, 2, 3, datetime.date(2016, 8, 18), type1],
[3, 4, 5, datetime.date(2016, 8, 18), type1]], [], []]
def fmt_times(lst):
"""Format the fourth value of each element of each non-empty sublist"""
for i in range(len(lst)):
if lst[i] != []:
for j in range(len(lst[i])):
lst[i][j][3] = lst[i][j][3].strftime('%Y-%m-%d')
return lst
def fmt_times_one_line(main_list):
"""Format the fourth value of each element of each non-empty sublist"""
return [[] if main_list[i] == [] else [[main_list[i][j][k] if k != 3 else main_list[i][j][k].strftime('%Y-%m-%d') for k in range(len(main_list[i][j]))] for j in range(len(main_list[i])) ] for i in range(len(main_list))]
import copy
# Deep copy needed because fmt_times modifies the sublists.
assert fmt_times(copy.deepcopy(main_list)) == fmt_times_one_line(main_list)
The list comprehension is a functional thing. If you know how map() works in python or javascript then it's the same thing. In a map() or comprehension we generally don't mutate the data we're mapping over (and python discourages attempting it) so instead we recreate the entire object, substituting only the values we wanted to modify.
One line?
main_list = convert_list(main_list)
You will have to put a few more lines somewhere else though:
def convert_list(main_list):
for i, ml in enumerate(main_list):
if isinstance(ml, list) and len(ml) > 0:
main_list[i] = convert_list(ml)
elif isinstance(ml, datetime.date):
main_list[i] = ml.strftime('%Y-%m-%d')
return main_list
You might be able to whack this together with a list comprehension but it's a terrible idea (for reasons better explained in the other answer).
Can one-line loops in Python only be used to build lists? (i.e. list comprehensions), or can they use for more general computing?
For example, I am aware that list comprehensions (~single-line loop) in Python, e.g.
my_list = [ 2*i for i in range(10)]
can also be built with a multi-line loop:
my_list = []
for i in range(10):
my_list.append(2*i)
But can we always transform general multi-line loops into one-line loops?
For example, say we have the following multi-line for loop:
my_array = np.ones(10*10)
for x in range(10):
my_array[x,:] = 0
can we convert it into a single-line loop? More generally:
Q1. Are the two forms functionally equivalent? (i.e. they support the same set of manipulations/operations)
Q2. I think I have read before that one-line loops in Python are vectorized. Is this true? And does this mean that they can iterate faster than multi-line loops?
But can we we always transform general multi-line loops in Python into one-line loops?
The short answer is no.
List comprehensions are good for projections (mapping) and/or filtering.
For example, if you have code like this:
result = []
for x in seq:
if bar(x):
result.append(foo(x))
Then, as you point out, it can benefit from being rewritten as a list comprehension:
result = [foo(x) for f in seq if bar(x)]
However list comprehensions are generally not so good for operations that don't fit into this projection or filtering pattern.
For example if you need to mutate the elements but don't need the result then a list comprehension wouldn't be suitable. The following code would be inconvenient to write as a list comprehension (assuming that both methods return None):
for x in seq:
x.foo() # Modify x
x.bar() # Modify x again
Some operations are not allowed inside a comprehension. An example is breaking out of the loop early if a condition is met:
for x in seq:
if x.foo():
break
else:
x.bar()
One thing I'll point out is that it's not just lists, you can use comprehension to create sets and even dictionaries.
>>> {i**2 for i in range(5)}
set([0, 1, 4, 16, 9])
>>> {i : str(i) for i in range(5)}
{0: '0', 1: '1', 2: '2', 3: '3', 4: '4'}
Also, list comprehension is generally faster than using append numerous times (like your example) because the comprehension is done by underlying C code, as opposed to append, which has the extra Python-layer.
Of course, comprehension has limitations like anything else. If you wanted to perform a larger set of operations on each element of a list/set, then a normal loop might be more appropriate.
Let L = [1,2,3,4] be our list.
Then 1 in L is True. 2 in L is also True.
Is there a clean way to write (1,2) in L and have it come out true?
That is, given a list L and a test list T and the relation multi-in, if all members of T are in L, then T multi-in L is True, otherwise T multi-in L is False.
Of course I can write a multi-in function, but that seems ugly.
You want to treat (1,2) and L as sets:
set((1,2)).issubset(L)
or, nicer if you understand the notation:
set((1,2)) <= set(L)
all(x in L for x in [1, 2])
Unlike the set-based solutions, this (1) short-curcuits as soon as an element isn't found, (2) works for unhashable types and (3) reads out nice ;)
We can go improve complexity (O(n*m) currently) by going back to sets... although in a different way: Convert L to a set beforehand and you get O(1) membership test back (without needing a second set for the items to check for).
How about:
set((1,2)).issubset(L)
Use sets:
s = set([1,2])
l = set([1,2,3,4])
s.issubset(l)
The .issubset() method will tell you if all the elements in one set exist in another.
Good answers above. Another possibility is:
all(x in L for x in [1,2,3,4])
I'm not dutch, but that's the "single obvious way to do it" to me.
I was looking at the following code in python:
for ob in [ob for ob in context.scene.objects if ob.is_visible()]:
pass
Obviously, it's a for each loop, saying for each object in foo array. However, I'm having a bit of trouble reading the array. If it just had:
[for ob in context.scene.objects if ob.is_visible()]
that would make some sense, granted the use of different objects with the same name ob seems odd to me, but it looks readable enough, saying that the array consists of every element in that loop. However, the use of 'ob', before the for loop makes little sense, is there some syntax that isn't in what I've seen in the documentation?
Thank you
That is the syntax for list comprehension.
The first ob is an expression, take this simple example:
>>>> [ x**2 for x in range(10) if x%2 == 0 ]
[0, 4, 16, 36, 64]
Which reads, create a list from range(10), if x is even then square it. Discard any odd value of x.
It's a list comprehension. You can read it as:
create a list with [some_new_value for the_current_value in source_list_of_values if matches_criteria]
Think of:
[n*2 for n in [1,2,3,4]]
This means, for every item, n, create an entry in a new list with a value of n*2. So, this will yield:
[2,4,6,8]
In your case since the value ob is not modified, it's just filtering the list based on ob.is_visible(). So you're getting a new list of all ob's that are visible.
It might help you see what's going on to rewrite your example code into this equivalent code:
temp_list = [ob for ob in context.scene.objects if ob.is_visible()]
for ob in temp_list:
pass