So I have the following generator function:
def gen(n=5):
for i in range(n):
n = yield n
for i in gen(3):
print(i)
The result:
3
None
None
I understand the first result of yield is 3. Because I assigned 3 to function argument n. But where are the None in the second and third yield coming from? Is it because in the for-loop, yield n returns None and this None is assigned to n in this line: n = yield n?
This is explained in the documentation of yield expressions, especially this part:
The value of the yield expression after resuming depends on the method
which resumed the execution. If next() is used (typically via
either a for or the next() builtin) then the result is None.
Otherwise, if send() is used, then the result will be the value passed
in to that method.
As you use a for loop, n just gets None as a value when resuming after the first yield. So, from now on, n is None, and this is what will be yielded the last two times.
It seems to me that you answered your own question:
because in the for-loop, yield n returns None and None is assigned to n in this line: n = yield n
You can read more at this answer.
Related
I recently studied a python recursion function and found that the recursion stops when it uses element in []. So I made a simple test function, found that there is even no print out. So how can I understand the element in []? Why does the function stop when referring to element in []?
b=1
def simple():
for i in []:
print('i am here')
return i+b
a = simple()
Python's in keyword has two purposes.
One use in as part of a for loop, which is written for element in iterable. This assigns each value from iterable to element on each pass through the loop body. This is how your example function is using in (though since the list you're looping over is empty, the loop never does anything).
The other way you can use in is as an operator. An expression like x in y tests if element x is present in container y. (There's also a negated version of the in operator, not in. The expression x not in y is exactly equivalent to not (x in y).) I suspect this is what your recursive code is doing. This would also not be useful to do with an empty list literal (since an empty list by definition doesn't contain anything), but I'm guessing the real recursive function is a bit more complicated.
As an example of both uses of in, here's a generator function that uses a set to filter out duplicate items from some other iterable. It has a for loop that has in, and it also uses in (well, technically not in) as an operator to test if the next value from the input iterator is contained in the seen set:
def unique(iterable):
seen = set()
for item in iterable: # "in" used by for loop
if item not in seen: # "in" used here as an operator
yield item
seen.add(item)
A recursive function calls itself n-number of times, then returns a terminating value on the last recursion that backs out of the recursive stacks.
Example:
compute the factorial of a number:
def fact(n):
# ex: 5 * 4 * 3 * 2 * 1
# n == 0 is your terminating recursion
if n == 0:
return 1
# else is your recursion call to fact(n-1)
else:
return n * fact(n-1)
In your example, there is no recursive call to simple() within the function, nor are there any element inside the empty list [] to step through, therefore your for loop never executed
Its concerned about mechanism of 'for loop'.
Superficially, the iterator you want to travese (which is "[]" in you example) has a length of 0, so the body of the loop (which include "print" an so on) will not be executed.
Hope it helps.
I am trying to understand the behaviour of the yield statement by building a generator which behaves similarly to the 'enumerate' built-in function but I am witnessing inconsistencies depending on how I iterate through it.
def enumerate(sequence, start=0):
n = start
for elem in sequence:
print("Before the 'yield' statement in the generator, n = {}".format(n))
yield n, elem
n += 1
print("After the 'yield' statement in the generator, n = {}".format(n))
My understanding of generators is that the execution of the code will stop once a yield statement has been reached, upon which it returns a value. This matches what I get with the script below.
a = 'foo'
b = enumerate(a)
n1,v1 = next(b)
print('n1 = {}, v1 = {}\n'.format(n1,v1))
n2,v2 = next(b)
print('n2 = {}, v2 = {}'.format(n2,v2))
In this case, the generator seems to stop exactly at the yield statement and resumes in the n+=1 one with the second 'next' statement:
Before the 'yield' statement in the generator, n = 0
n1 = 0, v1 = f
After the 'yield' statement in the generator, n = 1
Before the 'yield' statement in the generator, n = 1
n2 = 1, v2 = o
However, if I use the for loop below, the generator does not seem to stop at the yield statement.
for n,v in enumerate(a[0:1]):
print('n = {}, v = {}'.format(n,v))
This is what I get:
Before the 'yield' statement in the generator, n = 0
n = 0, v = f
After the 'yield' statement in the generator, n = 1
Edit taking comments into account
I realise I'm iterating over just one element, but I was not expecting to see the very last "After the 'yield' statement in the generator" sentence (which appears even if I iterate over ALL the elements.
print('\n\n')
for n,v in enumerate(a):
print('n = {}, v = {}'.format(n,v))
Before the 'yield' statement in the generator, n = 0
n = 0, v = f
After the 'yield' statement in the generator, n = 1
Before the 'yield' statement in the generator, n = 1
n = 1, v = o
After the 'yield' statement in the generator, n = 2
Before the 'yield' statement in the generator, n = 2
n = 2, v = o
After the 'yield' statement in the generator, n = 3
Why does this happen?
The fundamental issue here is that you are confusing the fact that you know when the generator will be exhausted just by looking at it, with the fact that Python can only know by running the code. When Python reaches the yield that you consider to be the last one, it does not actually know that it is the last one. What if your generator looked like this:
def enumeratex(x, start=0):
for elem in x:
yield start, x
start += 1
yield start, None
Here, for reasons no one will ever know, a final None element is returned after the main generator loop. Python would have no way of knowing that the generator is done until you either
Return from the generator.
Raise an error, in which case everything will grind to a halt.
In versions before Python 3.7, generators could raise StopIteration to indicate termination. In fact, a return statement would be equivalent to either raise StopIteration (if returning None) or raise StopIteration(return_value).
So while the exact manner in which you tell Python to end the generator is up to you, you do have to be explicit about it. A yield does not by itself end the generator.
TL;DR
All of the code in a loop in a generator will always run, even after the last value has been yielded because Python can only know it was the last value by actually executing all the code.
the answer lies in understanding what for loop in python does:
It get the iterator (i.e. iter()) of an object and continues until a StopIteration exception is raised.
StopIteration exception is thrown when the code of the generator is done, meaning getting the return statement which exists the function (could be implicit also).
This is why it doesn't stops at yield, it keeps asking for the next yield until the generator is done.
after reading documentation, questions, and making my own test code, I believe I have understood how a yield expression works.
Nevertheless, I am surprised of the behavior of the following example code:
def gen(n=0):
while True:
n = (yield n) or n+1
g=gen()
print( next(g) )
print( next(g) )
print( g.send(5) )
print( next(g) )
print( next(g) )
I would have expected that it returned 0, 1, 2, 5, 6, while instead it produces: 0, 1, 5, 6, 7.
I.e: I would have expected that the yield expression produce these effects:
calculate the value of the yield expression , and return it to the caller
get the value(s) from the caller's send() and use them in the as the value of the yield expression which the generator function code receives
suspend execution before anything else is executed; it will be resumed at the same point at the same next(g) or g.send() call
... and/or that Python would care to avoid any interference between the two
flows of information in (1) and (2), i.e. that they were guaranteed independent such as in a tuple assignment a, b = f(a,b), g(a,b)
(I would even wonder if it were better to make the suspension happen in between (1) and (2), but maybe it would be quite complicated because it would imply that only part of the statement is executed and the rest is held for the next resume)
Anyway, the order of the operations is rather (2), then (1), then (3), so that the assignment in (2) occurs before, and can influence the assignment in (1). I.e. the value injected by the g.send() call is used before calculating the yield expression itself, which is directly exposed to the caller as the value of the same g.send() expression.
I am astonished because from the point of view of the code in the generator expression, the value received in its lhs can influence the value taken by the rhs!
To me, this is kind of misleading because one expects that in a statement like lhs expr = rhs expr, all calculations in the rhs expr are finished before doing the assignment, and frozen during the assignment. It looks really weird that the lhs of an assignment can influence it's own rhs!
The question: which are the reasons why it was made this way? Any clue?
(I know that "We prefer questions that can be answered, not just discussed", but this is something in which I stumbled and made me consume a lot of time. I believe a bit of discussion won't to any bad and maybe will save someone else's time)
PS. of course I understand that I can separate the assignment into two steps, so that any value received from send() will be used only after resuming the operation. Like this:
def gen(n=0):
while True:
received = (yield n)
n = received or (n+1)
Your confusion lies with generator.send(). Sending is just the same thing as using next(), with the difference being that the yield expression produces a different value. Put differently, next(g) is the same thing as g.send(None), both operations resume the generator there and then.
Remember that a generator starts paused, at the top. The first next() call advances to the first yield expression, stops the generator and then pauses. When a yield expression is paused and you call either next(g) or g.send(..), the generator is resumed where it is right now, and then runs until the next yield expression is reached, at which point it pauses again.
For your code, this happens:
g is created, nothing happens in gen()
next(g) actually enters the function body, n = 0 is executed, yield n pauses g and yields 0. This is printed.
next(g) resumes the generator; None is returned for yield n (nothing was sent after all), so None or n + 1 is executed an n = 1 is set. The loop continues on and yield n is reached again, the generator pauses and 1 is yielded. This is printed.
g.send(5) resumes the generator. 5 or n + 1 means n = 5 is executed. The loop continues until yield n is reached, the generator is paused, 5 is yielded and you print 5.
next(g) resumes the generator; None is returned (nothing was sent again), so None or n + 1 is executed an n = 6 is set. The loop continues on and yield n is reached again, the generator pauses and 6 is yielded and printed.
next(g) resumes the generator; None is returned (nothing was sent again), so None or n + 1 is executed an n = 7 is set. The loop continues on and yield n is reached again, the generator pauses and 7 is yielded and printed.
Given your steps 1., 2. and 3., the actual order is 3., 2., 1. then, with the addition that next() also goes through step 2. producing None, and 1. being the next invocation of yield encountered after un-pausing.
I have the following code which is not working for me:
def bitmask_generator(n):
if n > 0:
for x in bitmask_generator(n-1):
yield 1 + (x << 1)
for x in bitmask_generator(n-1):
yield x << 1
...
for x in bitmask_generator(5):
print x
The code is supposed to generate all possible bitmasks with length n. However, it prints nothing. What am I doing wrong here?
Update: adding a print(n) on the first line of bitmask_generator does actually print a bunch of n values.
Your innermost generator will not generate anything; when n > 0 is false, the function just returns. As such, your outer generator functions have nothing to loop over.
Say you called bitmask_generator(1); it would call bitmask_generator(0) twice. Each such generator would produce an empty sequence, so both for loops in the bitmask_generator(1) stack frame have nothing to loop over, so no yield is ever reached.
You need to add a yield statement before the first for statement. It is a requirement of recursive generators.
def exampleGen(item):
yield item
for x in exampleGen(getNextValueFuntion(x)):
yield x
When i was googling for python yield i found something interesting and I never knew before i.e we can pass value to yield to change the next() value. I hope some of the new pythonist might now aware of this and I am not sure how it works too. So, If any body could explain how it works and how the behavior changes when i send a new index to yield using send().
Here is my snippets :
def print_me (count):
i = 0
while i < count:
val = (yield i)
if val is not None:
i = val
else:
i += 1
gen = print_me(10)
for i in gen:
if i == 5:
gen.send(8)
else:
print i
Here is my output:
0
1
2
3
4
9
Your test code is pretty instructive actually. What happens in a generator function like this is that whenever you call next on the generator, code gets executed until it hits a yield statement. The value is yielded and then execution in the generator "pauses". If you .send some data to the generator, that value is returned from yield. If you don't .send anything, yield returns None.
So in your code, when you .send(8), then it sets val = 8 inside your generator (val = yield i). since val is not None, i is set to 8. .send actually executes until the next yield statement (returning that value -- 8). Then things resume as normal (the next number to be yielded is 9).