The goal of this post is to put multiple print functions throughout a list comprehension to visually understand what's happening within.
Important notes:
This should not be used for anything other than educational purposes and trying to understand code.
If you are using Python 2.x, you need to add a future import (it's in the code I pasted) or else print won't work. Only functions work in list comprehension. Print in 2.x does not operate as a function. Or...just switch to Python 3.x.
This was the original question:
## Using future to switch Print to a function
from __future__ import print_function
reg = []
for x in [1,2,3]:
for y in [3,1,4]:
print('looping through',x,'then',y)
if x == y:
print('success',x,y)
reg.append((x,y))
print(reg)
Here's the equivalent list comprehension with no print statements.
from __future__ import print_function
comp = [(x,y) for x in [1,2,3] for y in [3,1,4] if x == y]
print(comp)
So is there any way to put in a bunch of print statements so both code print the same things?
Edit with solution to original question:
Using the methods in the comments - I've figured it out!
So say you want to convert this.
from __future__ import print_function
x = 1
y = 2
z = 1
n = 2
[[a,b,c] for a in range(x+1) for b in range(y+1) for c in range(z+1) if a + b + c != n]
Adding print statements to print each loop, showing if it failed or not.
from __future__ import print_function
x = 1
y = 2
z = 1
n = 2
[
[a,b,c] for a in range(x+1) for b in range(y+1) for c in range(z+1) if
(print('current loop is',a,b,c) or a + b + c != n)
and
(print('condition true at',a,b,c) or True)
]
So really the only thing that was changed was the conditional at the end.
(a + b + c != n)
to
(print('current loop is',a,b,c) or a + b + c != n)
and
(print('condition true at',a,b,c) or True)
Additional Information:
So there's good stuff in the comment section that I think would help others as well. I'm a visual learner so this website was great.
http://treyhunner.com/2015/12/python-list-comprehensions-now-in-color/#colored-comprehension
(credits to Tadhg McDonald-Jensen)
I think you shouldn't running debug code inside list comprehensions, that said, if you wanted to do so, you could wrap your code inside a function like this:
from __future__ import print_function
def foo(x, y):
print('looping through', x, 'then', y)
if x == y:
print('success', x, y)
return (x, y)
comp = [foo(x, y) for x in [1, 2, 3] for y in [3, 1, 4] if x == y]
print(comp)
You need to evaluate your print function, but the return value isn't useful since it's always None. You can use and/or to combine it with another expression.
comp = [(x,y) for x in [1,2,3] for y in [3,1,4] if (print('looping through',x,'then',y) or x == y) and (print('success', x, y) or True)]
I really hope you're only doing this for educational purposes, because it's ugly as heck. Just because you can do something doesn't mean you should.
List comprehension was introduced with PEP 202 which states:
It is proposed to allow conditional construction of list literals
using for and if clauses. They would nest in the same way for loops
and if statements nest now.
List comprehension was designed to replace constructs that formed a list using only for loops, if conditionals and .append method once per iteration. Any additional structure is not possible in list comprehensions so unless you stuck your prints into one of the allowed components you cannot add them.
That being said, putting a print statement in the conditional - while technically possible - is highly not recommended.
[a for a in x if print("this is a bad way to test",a)]
Related
In the following code, I use abs(v - i) three times on the same line. Is this expression computed three times when the code is run? Is there a way to avoid this without having to complicate the code?
x = sum(abs(v-i) if s == 1 else int((abs(v-i)*(abs(v-i)+1))/2) for v in list)
Is this expression computed three times when the code is run?
No, once or twice for every list value.
Is there a way to avoid this without having to complicate the code?
Depends on what you consider complicating the code.
You could use the idiom that even got optimized in Python 3.9:
x = sum(a if s == 1 else int((a*(a+1))/2)
for v in list_
for a in [abs(v-i)])
Or if your list values are ints, you could use math.comb:
x = sum(abs(v-i) if s == 1 else comb(abs(v-i)+1, 2) for v in list_)
While https://stackoverflow.com/a/70268402/1126841 regarding the assignment operator is correct, this is a case where I really dislike the assignment expression, as you have to hunt for where a is actually defined. I would probably ditch sum and accumulate the value in a for loop instead.
x = 0
for v in list_:
a = abs(v-i)
if s == 1:
x += a
else:
x += int(a*(a+1)/2)
However, since s never changes in the loop, I would refactor this into two separate loops chosen by the value of s, one of which can use sum without difficulty.
if s == 1:
x = sum(abs(v-i) for v in list_)
else:
x = 0
for v in list_:
a = abs(v-i)
x += int(a*(a+1)/2)
My answer only shows that it's possible to do what you want with a one-liner, but I would still advise to use a longer approach with an explicit if/else + caching the value, or using numpy arrays and masks.
You can use the walrus operator := to store the value as a variable. This line is equivalent to your original code and will only compute a = abs(v-i) once per loop instead of 1-2 times:
x = sum(a if ((a := abs(v-i)) is not None) and s == 1 else int(a*(a+1)/2) for v in list_)
The problem is that the walrus operator can only be used in a if check, so we need to add a check that's always true... It really doesn't help reading comprehension.
"Long" approach:
v = np.array(list_)
a = np.abs(v - i)
x = np.sum(a if s == 1 else np.int(a*(a+1)/2))```
Can't you just save abs(v-i) to a variable and then substitute in that variable?
I would create a variable called my_calc = abs(v-i) then use that name. This will clean up your code
I come across a list comprehension that is not quite the same as usual. So I am confused about the list-compresion execution order.
import re
folders = ['train_frames001', 'train_masks002',
'val_frames003','val_masks004', 'test_frames005', 'test_masks006']
folders.sort(key=lambda var:[int(x) if x.isdigit() else x
for x in re.findall(r'[^0-9]|[0-9]+', var)])
print(folders)
#Whether the list compresion part means
#for x in re.findall(r'[^0-9]|[0-9]+', var):
# if x.isdigit():
# int(x)
# else:
# x
I did't find related samples and docs.
I think you are confused between order of if-else.
a = [1,2,3,4,5,6,7,8]
If you want simply square of each number
b = [i**2 for i in a]
# [1,4,9,16,25,36,49,64]
If you want even numbers (if statement in list-comprehension)
c = [i for i in a if i%2==0]
# [2,4,6,8]
If you want to square only even numbers(if-else statement ternary operator)
c = [i**2 if i%2==0 else i for i in a]
# [1,4,3,16,5,36,7,64]
I run the code, and get ['test_frames005', 'test_masks006', 'train_frames001', 'train_masks002', 'val_frames003', 'val_masks004'], I think the result is right.
If you want get result like ['train_frames001', 'train_masks002', 'val_frames003', 'val_masks004', 'test_frames005', 'test_masks006'], which sorted by the end number. Maybe you should change your code like below.
import re
folders = ['train_frames001', 'train_masks002',
'val_frames003', 'val_masks004', 'test_frames005', 'test_masks006']
folders.sort(key=lambda var: [int(x)
for x in re.findall(r'[^0-9]|[0-9]+', var) if x.isdigit()])
print(folders)
For example, how could I condense
In [1]: for x in xrange(1,11):
...: if x%2==0:
...: print x
into one line?
Edit: Thanks guys! That was exactly what I was looking for.
To make this a little more challenging though, is there a way to add elif & else and still have it be on one line?
To use the previous example,
for x in xrange(1,11):
if x%2==0:
print x
else
print "odd"
For your specific example:
for x in xrange(2, 11, 2): print x
More generally, in terms of whether you can nest blocks on one line, the answer is no. Paraphrasing the documentation on compound statements, a "suite" may not contain nested compound statements if it is in the one-line form. A "suite" is the group of statements controlled by a clause (like a conditional block or the body of a loop).
Maybe something like this:
from __future__ import print_function
map(print, [x for x in xrange(1,11) if x % 2 == 0])
This isn't quite the same and isn't "one line", but consider removing the side-effect and using a list filter/comprehension.
evens = [x for x in xrange(1,11) if x % 2 == 0]
print "\n".join(evens)
# or (now a saner "one line", the evens-expr could even be moved in-place)
for x in evens: print x
for x in [y for y in xrange(1,11) if y%2==0]:
print x
I recently started learning Python, and the concept of for loops is still a little confusing for me. I understand that it generally follows the format for x in y, where y is just some list.
The for-each loop for (int n: someArray)
becomes for n in someArray,
And the for loop for (i = 0; i < 9; i-=2) can be represented by for i in range(0, 9, -2)
Suppose instead of a constant increment, I wanted i*=2, or even i*=i. Is this possible, or would I have to use a while loop instead?
As you say, a for loop iterates through the elements of a list. The list can contain anything you like, so you can construct a list beforehand that contains each step.
A for loop can also iterate over a "generator", which is a small piece of code instead of an actual list. In Python, range() is actually a generator (in Python 2.x though, range() returned a list while xrange() was the generator).
For example:
def doubler(x):
while True:
yield x
x *= 2
for i in doubler(1):
print i
The above for loop will print
1
2
4
8
and so on, until you press Ctrl+C.
You can use a generator expression to do this efficiently and with little excess code:
for i in (2**x for x in range(10)): #In Python 2.x, use `xrange()`.
...
Generator expressions work just like defining a manual generator (as in Greg Hewgill's answer), with a syntax similar to a list comprehension. They are evaluated lazily - meaning that they don't generate a list at the start of the operation, which can cause much better performance on large iterables.
So this generator works by waiting until it is asked for a value, then asking range(10) for a value, doubling that value, and passing it back to the for loop. It does this repeatedly until the range() generator yields no more values.
Bear in mind that the 'list' part of the Python can be any iterable sequence.
Examples:
A string:
for c in 'abcdefg':
# deal with the string on a character by character basis...
A file:
with open('somefile','r') as f:
for line in f:
# deal with the file line by line
A dictionary:
d={1:'one',2:'two',3:'three'}
for key, value in d.items():
# deal with the key:value pairs from a dict
A slice of a list:
l=range(100)
for e in l[10:20:2]:
# ever other element between 10 and 20 in l
etc etc etc etc
So it really is a lot deeper than 'just some list'
As others have stated, just set the iterable to be what you want it to be for your example questions:
for e in (i*i for i in range(10)):
# the squares of the sequence 0-9...
l=[1,5,10,15]
for i in (i*2 for i in l):
# the list l as a sequence * 2...
You will want to use list comprehensions for this
print [x**2 for x in xrange(10)] # X to the 2nd power.
and
print [x**x for x in xrange(10)] # X to the Xth power.
The list comprehension syntax is a follows:
[EXPRESSION for VARIABLE in ITERABLE if CONDITION]
Under the hood, it acts similar to the map and filter function:
def f(VARIABLE): return EXPRESSION
def c(VARIABLE): return CONDITION
filter(c, map(f, ITERABLE))
Example given:
def square(x): return x**2
print map(square, xrange(10))
and
def hypercube(x): return x**x
print map(hypercube, xrange(10))
Which can be used as alternative approach if you don't like list comprehensions.
You could as well use a for loop, but that would step away from being Python idiomatic...
Just for an alternative, how about generalizing the iterate/increment operation to a lambda function so you can do something like this:
for i in seq(1, 9, lambda x: x*2):
print i
...
1
2
4
8
Where seq is defined below:
#!/bin/python
from timeit import timeit
def seq(a, b, f):
x = a;
while x < b:
yield x
x = f(x)
def testSeq():
l = tuple(seq(1, 100000000, lambda x: x*2))
#print l
def testGen():
l = tuple((2**x for x in range(27)))
#print l
testSeq();
testGen();
print "seq", timeit('testSeq()', 'from __main__ import testSeq', number = 1000000)
print "gen", timeit('testGen()', 'from __main__ import testGen', number = 1000000)
The difference in performance isn't that much:
seq 7.98655080795
gen 6.19856786728
[EDIT]
To support reverse iteration and with a default argument...
def seq(a, b, f = None):
x = a;
if b > a:
if f == None:
f = lambda x: x+1
while x < b:
yield x
x = f(x)
else:
if f == None:
f = lambda x: x-1
while x > b:
yield x
x = f(x)
for i in seq(8, 0, lambda x: x/2):
print i
Note: This behaves differently to range/xrange in which the direction </> test is chosen by the iterator sign, rather than the difference between start and end values.
Like the title says, is there any way to name variables (i.e., lists) used within a nested list comprehension in Python?
I could come up with a fitting example, but I think the question is clear enough.
Here is an example of pseudo code:
[... [r for r in some_list if r.some_attribute == something_from_within_this_list comprehension] ... [r for r in some_list if r.some_attribute == something_from_within_this_list comprehension] ...]
Is there any way to avoid the repetition here and simply add a variable for this temporary list only for use within the list comprehension?
CLARIFICATION:
The list comprehension is already working fine, so it's not a question of 'can it be done with a list comprehension'. And it is quicker than it's original form of a for statement too, so it's not one of those 'for statements vs list comprehensions' questions either. It is simply a question of making the list comprehension more readable by making variable names for variables internal to the list comprehension alone. Just googling around I haven't really found any answer. I found this and this, but that's not really what I am after.
Based on my understanding of what you want to do, No you cannot do it.
You cannot carry out assignments in list comprehensions because a list comprehension is essentially of the form
[expression(x, y) for x in expression_that_creates_a_container
for y in some_other_expression_that_creates_a_container(x)
if predicate(y, x)]
Granted there are a few other cases but they're all about like that. Note that nowhere does there exist room for a statement which is what a name assignment is. So you cannot assign to a name in the context of a list comprehension except by using the for my_variable in syntax.
If you have the list comprehension working, you could post it and see if it can be simplified. Solutions based on itertools are often a good alternative to burly list comprehensions.
I think I understand exactly what you meant, and I came up with a "partial solution" to this problem. The solution works fine, but is not efficent.
Let me explain with an example:
I was just trying to solve a Pythagorean triplet which sum was 1000. The python code to solve it is just:
def pythagoreanTriplet(sum):
for a in xrange(1, sum/2):
for b in xrange(1, sum/3):
c = sum - a - b
if c > 0 and c**2 == a**2 + b**2:
return a, b, c
But I wanted to code it in a functional programming-like style:
def pythagoreanTriplet2(sum):
return next((a, b, sum-a-b) for a in xrange(1, sum/2) for b in xrange(1, sum/3) if (sum-a-b) > 0 and (sum-a-b)**2 == a**2 + b**2)
As can be seen in the code, I calc 3 times (sum-a-b), and I wanted to store the result in an internal varible to avoid redundant calculation. The only way I found to do that was by adding another loop with a single value to declare an internal variable:
def pythagoreanTriplet3(sum):
return next((a, b, c) for a in xrange(1, sum/2) for b in xrange(1, sum/3) for c in [sum-a-b] if c > 0 and c**2 == a**2 + b**2)
It works fine... but as I said at the begin of the post, is not an efficent method. Comparing the 3 methods with cProfile, the time required for each method is the next one:
First method: 0.077 seconds
Secnd method: 0.087 seconds
Third method: 0.109 seconds
Some people could classify the following as a "hack", but it is definitely useful in some cases.
f = lambda i,j: int(i==j) #A dummy function (here Kronecker's delta)
a = tuple(tuple(i + (2+f_ij)*j + (i + (1+f_ij)*j)**2
for j in range(4)
for f_ij in (f(i,j),) ) #"Assign" value f(i,j) to f_ij.
for i in range(4) )
print(a)
#Output: ((0, 3, 8, 15), (2, 13, 14, 23), (6, 13, 44, 33), (12, 21, 32, 93))
This approach is particularly convenient if the function f is costly to evaluate. Because it is somewhat unusual, it may be a good idea to document the "assignment" line, as I did above.
I'm just gonna go out on a limb here, because I have no idea what you really are trying to do. I'm just going to guess that you are trying to shoehorn more than you should be into a single expression. Don't do that, just assign subexpressions to variables:
sublist = [r for r in some_list if r.some_attribute == something_from_within_this_list comprehension]
composedlist = [... sublist ... sublist ...]
This feature was added in Python 3.8 (see PEP 572), it's called "assignment expressions" and the operator is := .
Examples from the documentation:
results = [(x, y, x/y) for x in input_data if (y := f(x)) > 0]
stuff = [[y := f(x), x/y] for x in range(5)]