Can one-line loops in Python only be used to build lists? (i.e. list comprehensions), or can they use for more general computing?
For example, I am aware that list comprehensions (~single-line loop) in Python, e.g.
my_list = [ 2*i for i in range(10)]
can also be built with a multi-line loop:
my_list = []
for i in range(10):
my_list.append(2*i)
But can we always transform general multi-line loops into one-line loops?
For example, say we have the following multi-line for loop:
my_array = np.ones(10*10)
for x in range(10):
my_array[x,:] = 0
can we convert it into a single-line loop? More generally:
Q1. Are the two forms functionally equivalent? (i.e. they support the same set of manipulations/operations)
Q2. I think I have read before that one-line loops in Python are vectorized. Is this true? And does this mean that they can iterate faster than multi-line loops?
But can we we always transform general multi-line loops in Python into one-line loops?
The short answer is no.
List comprehensions are good for projections (mapping) and/or filtering.
For example, if you have code like this:
result = []
for x in seq:
if bar(x):
result.append(foo(x))
Then, as you point out, it can benefit from being rewritten as a list comprehension:
result = [foo(x) for f in seq if bar(x)]
However list comprehensions are generally not so good for operations that don't fit into this projection or filtering pattern.
For example if you need to mutate the elements but don't need the result then a list comprehension wouldn't be suitable. The following code would be inconvenient to write as a list comprehension (assuming that both methods return None):
for x in seq:
x.foo() # Modify x
x.bar() # Modify x again
Some operations are not allowed inside a comprehension. An example is breaking out of the loop early if a condition is met:
for x in seq:
if x.foo():
break
else:
x.bar()
One thing I'll point out is that it's not just lists, you can use comprehension to create sets and even dictionaries.
>>> {i**2 for i in range(5)}
set([0, 1, 4, 16, 9])
>>> {i : str(i) for i in range(5)}
{0: '0', 1: '1', 2: '2', 3: '3', 4: '4'}
Also, list comprehension is generally faster than using append numerous times (like your example) because the comprehension is done by underlying C code, as opposed to append, which has the extra Python-layer.
Of course, comprehension has limitations like anything else. If you wanted to perform a larger set of operations on each element of a list/set, then a normal loop might be more appropriate.
Related
Suppose I have a list of lists say
A = [[1,1,1,1],[2,2,2,2]]
and I want to create two strings from that to be
'1111'
'2222'
How would we do this in python?
Maybe list comprehension:
>>> A = [[1,1,1,1],[2,2,2,2]]
>>> l=[''.join(map(str,i)) for i in A]
>>> l
['1111', '2222']
>>>
Now you've got it.
This is pretty easily done using join and a list comprehension.
A = [[1,1,1,1],[2,2,2,2]]
a_strings = [''.join(map(str, sub_list)) for sublist in A]
See, join() takes a list of strings and makes a string concatenating all the substrings and the list comprehension I used just loops through them all. Above I combined the 2 together.
On a second thought
map() is actually deemed more efficient (when not using lambda.. etc) and for SOME more readable. I'll just add an approach using map instead of a comprehension.
a_strings = map(''.join(), map(str, A))
This first takes the inner map and makes all the ints > strs then joins all the strs together for every sub-list.
Hopefully this makes things a bit more chewable for ya, each method is close to equivalent such that for this case you could consider them style choices.
I have this:
self.lines = [...]
cnt = defaultdict(int)
for line in self.lines:
cnt[line] += 1
Now this works. But I wonder if it (incrementing count of particular lines in defaultdict) could be done using list comprehension?
This is a syntax error:
[cnt[line] += 1 for line in self.lines]
Incidentally, why can't one use expressions like this within list comprehension? It's straightforward and would greatly improved both terseness and performance of such code.
Your list comprehension doesn't work because assignments are not expressions.
You shouldn't use list comprehension to replace a loop. Write a loop. List comprehensions are used for constructing lists.
Why do you think list comprehension would improve performance? If anything, it potentially hurts performance, because it needs to allocate and assign to the temporary list it builds, which is then never used. Imagine you have 1,000,000,000 lines in your original list.
You could use collections.Counter here:
>>> from collections import Counter
>>> lis = [1,2,3,3,3,5,6,1,2,2]
>>> Counter(lis)
Counter({2: 3, 3: 3, 1: 2, 5: 1, 6: 1})
cnt[line] += 1 is an assignment LC don't support assignments, and even using LCs for side effect is also a bad practice.
lis = []
[lis.append(x) for x in xrange(5)] #bad
Read: Is it Pythonic to use list comprehensions for just side effects?
I often see code similar to this:
return [(var, val) for val in self.domains[var]
if self.nconflicts(var, val, assignment) == 0]
and I'm like DAMN that's sexy. But then I try to drop it sometimes and I get syntax errors. Are there any particular rules for this nice form of code writing that reverses the typical placement of for and if statements?
They're called list comprehensions. The basic syntax is (I'm using parens to group my words, not as part of the syntax):
[(an expression involving x) for x in someList if (some condition)]
If the condition evaluates to true, the resulting list includes the (expression involving x). So, for example, the following list comprehension uses this to only include strings in the resulting list.
>>> myList = [1,"hello",5.4,"world"]
>>> [elem for elem in myList if type(elem)==str]
['hello', 'world']
Note that the if part is optional, and the expression involving x can be as simple as just x (often used when you are just filtering out elements from another list).
In fact, the expression involving x doesn't really have to have x in it at all. For example, if for some reason you wanted a list of 0's as long as your name you could do this:
>>> [0 for letter in "Matthew"]
[0, 0, 0, 0, 0, 0, 0]
For when you don't need the list to stick around after you make it, use generator expressions instead. (Generator expressions and list comprehensions have the same syntax.)
See list comprehensions in the Python tutorial documentation. There are quite a number of things you can do with this syntax, including creating lists, sets, and dictionaries.
Although your example code is indeed a list comprehension, you do also occasionally see the reverse if syntax for inline conditionals:
a if b else c
The concept is called list comprehension,
http://www.secnetix.de/olli/Python/list_comprehensions.hawk
Take a look at lambda functions too,
http://www.secnetix.de/olli/Python/lambda_functions.hawk
I would like to extend a list while looping over it:
for idx in xrange(len(a_list)):
item = a_list[idx]
a_list.extend(fun(item))
(fun is a function that returns a list.)
Question:
Is this already the best way to do it, or is something nicer and more compact possible?
Remarks:
from matplotlib.cbook import flatten
a_list.extend(flatten(fun(item) for item in a_list))
should work but I do not want my code to depend on matplotlib.
for item in a_list:
a_list.extend(fun(item))
would be nice enough for my taste but seems to cause an infinite loop.
Context:
I have have a large number of nodes (in a dict) and some of them are special because they are on the boundary.
'a_list' contains the keys of these special/boundary nodes. Sometimes nodes are added and then every new node that is on the boundary needs to be added to 'a_list'. The new boundary nodes can be determined by the old boundary nodes (expresses here by 'fun') and every boundary node can add several new nodes.
Have you tried list comprehensions? This would work by creating a separate list in memory, then assigning it to your original list once the comprehension is complete. Basically its the same as your second example, but instead of importing a flattening function, it flattens it through stacked list comprehensions. [edit Matthias: changed + to +=]
a_list += [x for lst in [fun(item) for item in a_list] for x in lst]
EDIT: To explain what going on.
So the first thing that will happen is this part in the middle of the above code:
[fun(item) for item in a_list]
This will apply fun to every item in a_list and add it to a new list. Problem is, because fun(item) returns a list, now we have a list of lists. So we run a second (stacked) list comprehension to loop through all the lists in our new list that we just created in the original comprehension:
for lst in [fun(item) for item in a_list]
This will allow us to loop through all the lists in order. So then:
[x for lst in [fun(item) for item in a_list] for x in lst]
This means take every x (that is, every item) in every lst (all the lists we created in our original comprehension) and add it to a new list.
Hope this is clearer. If not, I'm always willing to elaborate further.
Using itertools, it can be written as:
import itertools
a_list += itertools.chain(* itertools.imap(fun, a_list))
or, if you're aiming for code golf:
a_list += sum(map(fun, a_list), [])
Alternatively, just write it out:
new_elements = map(fun, a_list) # itertools.imap in Python 2.x
for ne in new_elements:
a_list.extend(ne)
As you want to extend the list, but loop only over the original list, you can loop over a copy instead of the original:
for item in a_list[:]:
a_list.extend(fun(item))
Using generator
original_list = [1, 2]
original_list.extend((x for x in original_list[:]))
# [1, 2, 1, 2]
This question already has answers here:
How can I use list comprehensions to process a nested list?
(13 answers)
Closed last month.
I have the following code which I use to map a nested list in Python to produce a list with the same structure.
>>> nested_list = [['Hello', 'World'], ['Goodbye', 'World']]
>>> [map(str.upper, x) for x in nested_list]
[['HELLO', 'WORLD'], ['GOODBYE', 'WORLD']]
Can this be done with list comprehension alone (without using the map function)?
For nested lists you can use nested list comprehensions:
nested_list = [[s.upper() for s in xs] for xs in nested_list]
Personally I find map to be cleaner in this situation, even though I almost always prefer list comprehensions. So it's really your call, since either will work.
Remember the Zen of Python:
There is generally more than one -- and probably several -- obvious ways to do it.**
** Note: Edited for accuracy.
Anyway, I prefer map.
from functools import partial
nested_list = map( partial(map, str.upper), nested_list )
Map is certainly a much cleaner way of doing what you want. You can nest the list comprehensions though, maybe that's what you're after?
[[ix.upper() for ix in x] for x in nested_list]
Other posters have given the answer, but whenever I'm having trouble wrapping my head around a functional construct, I swallow my pride and spell it out longhand with explicitly non-optimal methods and/or objects. You said you wanted to end up with a generator, so:
for xs in n_l:
def doUpper(l):
for x in l:
yield x.upper()
yield doUpper(xs)
for xs in n_l:
yield (x.upper() for x in xs)
((x.upper() for x in xs) for xs in n_l)
Sometimes it's cleaner to keep one of the longhand versions. For me, map and reduce sometimes make it more obvious, but Python idioms might be more obvious for others.
Here is solution for nested list that has arbitrary depth:
def map_nlist(nlist=nlist,fun=lambda x: x*2):
new_list=[]
for i in range(len(nlist)):
if isinstance(nlist[i],list):
new_list += [map_nlist(nlist[i],fun)]
else:
new_list += [fun(nlist[i])]
return new_list
you want to upper case all you list element, just type
In [26]: nested_list = [['Hello', 'World'], ['Goodbye', [['World']]]]
In [27]: map_nlist(nested_list,fun=str.upper)
Out[27]: [['HELLO', 'WORLD'], ['GOODBYE', [['WORLD']]]]
And more important, this recursive function can do more than this!
I am new to python, feel free to discuss!