Python: I don't understand what's happening with this generator

Python: I don't understand what's happening with this generator - python

I'm curious as to what's happening here. Can someone who knows generators and coroutines well explain this code.
def b():
for i in range(5):
yield i
x = (yield)
print(x)
def a():
g = b()
next(g)
for i in range(4):
g.send(5)
print(next(g))
a()
output
None
1
None
2
None
3
None
4
but when I switch around lines 3 and 4: the lines yield i and x = (yield), I get the following.
5
None
5
None
5
None
5
None
I suspect the problem might me from trying to use the yield statement to both receive and send values in the same function. Is this not possible in Python?
I have successfully written a couple of programs that use coroutines, so I am familiar with the way they work, but I am confused as to the way this snippet of code is behaving. Any insights into this would be appreciated.
Thanks
Edit: Thanks BrenBarn and unutbu for your answers. What's happening here makes more sense when you expand the problem out as such.
def b():
for i in range(5):
yield i
x = yield None
def a():
g = b()
print('* got', g.send(None) )
for i in range(4):
print('+ got', g.send(5) )
print('- got', g.send(None))
a()

I don't quite get what you're asking, but basically: when you use send, it causes the most-recently-reached yield expression in the generator to evaluate to the value you send. Note also that send advances the generator to the next yield. One thing that may be confusing you is that you are printing the value of x inside the generator, and you are printing the value of next(g) inside b, but the generator is also yielding values at g.send(5), and you aren't printing those.
In your first case, your first send causes the yield i statement to evaluate to 5 inside b, but you don't use this value inside b (you don't assign yield i to anything), so it does nothing. Also, when you do send(5), the generator is yielding None (from the x = (yield) line), but you don't print it so you don't know this. Then you advance the generator again with next(g). The most-recently-reached yield is the x = yield, but next(g) passes no value, so x gets set to None.
In the second case, the parity of the calls is reversed. Now your first send does send to the x = yield line, so x gets set to 5. This send also yields the loop value in b, but you ignore this value in a and don't print it. You then print next(g), which is None. On each subsequent send, b prints the value of x, which is always 5 because that's what you always send, and then a prints the next yielded value, which is always None (because that's what x = yield yields).
I don't quite get what you mean by "using the yield statement to both receive and send values in the same function". You can certainly do this, but you have to realize that: a) a value (None) is still sent even when you call next(g); and b) a value is still yielded when you call g.send(5).

Using traceit to step through the program line-by-line:
import sys
import linecache
class SetTrace(object):
'''
with SetTrace(monitor):
...
'''
def __init__(self, func):
self.func = func
def __enter__(self):
sys.settrace(self.func)
return self
def passit(self, frame, event, arg):
return self.passit
def __exit__(self, ext_type, exc_value, traceback):
sys.settrace(self.passit)
def traceit(frame, event, arg):
'''
http://www.dalkescientific.com/writings/diary/archive/2005/04/20/tracing_python_code.html
'''
if event == "line":
lineno = frame.f_lineno
filename = frame.f_globals["__file__"]
if (filename.endswith(".pyc") or
filename.endswith(".pyo")):
filename = filename[:-1]
name = frame.f_globals["__name__"]
line = linecache.getline(filename, lineno)
print("%s # %s:%s" % (line.rstrip(), name, lineno, ))
return traceit
def b():
for i in range(5):
yield i
x = (yield)
print(x)
def a():
g = b()
next(g)
for i in range(4):
g.send(5)
print(next(g))
with SetTrace(traceit):
a()
we obtain
g = b() # __main__:44
next(g) # __main__:45 # runs b until you get to a yield
for i in range(5): # __main__:38
yield i # __main__:39 # stop before the yield; resume a
^
for i in range(4): # __main__:46
g.send(5) # __main__:47 # resume b; (yield i) expression evals to 5 then thrown away
x = (yield) # __main__:40 # stop before yield; resume a
^
print(next(g)) # __main__:48 # next(g) called; resume b; print not called yet
print(x) # __main__:41 # next(g) causes (yield) to evaluate to None
None
for i in range(5): # __main__:38
yield i # __main__:39 # yield 1; resume a; `print(next(g))` prints 1
1
for i in range(4): # __main__:46
g.send(5) # __main__:47 # resume b; (yield i) expression evals to 5 then thrown away
The comments on the right-hand side (above) explain why Python prints None then 1. If you get that far, I think it is clear why you get None, 2, etc. -- it's the same story all over again with different values for i.
The other scenario, where x = (yield) and yield i are reverse can be analyzed similarly.

Related

In a generator, equality to yield [duplicate]

I understand yield. But what does a generator's send function do? The documentation says:
generator.send(value)
Resumes the execution and “sends” a value into the generator function. The value argument becomes the result of the current yield expression. The send() method returns the next value yielded by the generator, or raises StopIteration if the generator exits without yielding another value.
What does that mean? I thought value was the input to the generator function? The phrase "The send() method returns the next value yielded by the generator" seems to be also the exact purpose of yield, which also returns the next value yielded by the generator.
Is there an example of a generator utilizing send that accomplishes something yield cannot?

It's used to send values into a generator that just yielded. Here is an artificial (non-useful) explanatory example:
>>> def double_inputs():
... while True:
... x = yield
... yield x * 2
...
>>> gen = double_inputs()
>>> next(gen) # run up to the first yield
>>> gen.send(10) # goes into 'x' variable
20
>>> next(gen) # run up to the next yield
>>> gen.send(6) # goes into 'x' again
12
>>> next(gen) # run up to the next yield
>>> gen.send(94.3) # goes into 'x' again
188.5999999999999
You can't do this just with yield.
As to why it's useful, one of the best use cases I've seen is Twisted's #defer.inlineCallbacks. Essentially it allows you to write a function like this:
#defer.inlineCallbacks
def doStuff():
result = yield takesTwoSeconds()
nextResult = yield takesTenSeconds(result * 10)
defer.returnValue(nextResult / 10)
What happens is that takesTwoSeconds() returns a Deferred, which is a value promising a value will be computed later. Twisted can run the computation in another thread. When the computation is done, it passes it into the deferred, and the value then gets sent back to the doStuff() function. Thus the doStuff() can end up looking more or less like a normal procedural function, except it can be doing all sorts of computations & callbacks etc. The alternative before this functionality would be to do something like:
def doStuff():
returnDeferred = defer.Deferred()
def gotNextResult(nextResult):
returnDeferred.callback(nextResult / 10)
def gotResult(result):
takesTenSeconds(result * 10).addCallback(gotNextResult)
takesTwoSeconds().addCallback(gotResult)
return returnDeferred
It's a lot more convoluted and unwieldy.

This function is to write coroutines
def coroutine():
for i in range(1, 10):
print("From generator {}".format((yield i)))
c = coroutine()
c.send(None)
try:
while True:
print("From user {}".format(c.send(1)))
except StopIteration: pass
prints
From generator 1
From user 2
From generator 1
From user 3
From generator 1
From user 4
...
See how the control is being passed back and forth? Those are coroutines. They can be used for all kinds of cool things like asynch IO and similar.
Think of it like this, with a generator and no send, it's a one way street
========== yield ========
Generator | ------------> | User |
========== ========
But with send, it becomes a two way street
========== yield ========
Generator | ------------> | User |
========== <------------ ========
send
Which opens up the door to the user customizing the generators behavior on the fly and the generator responding to the user.

This may help someone. Here is a generator that is unaffected by send function. It takes in the number parameter on instantiation and is unaffected by send:
>>> def double_number(number):
... while True:
... number *=2
... yield number
...
>>> c = double_number(4)
>>> c.send(None)
8
>>> c.next()
16
>>> c.next()
32
>>> c.send(8)
64
>>> c.send(8)
128
>>> c.send(8)
256
Now here is how you would do the same type of function using send, so on each iteration you can change the value of number:
def double_number(number):
while True:
number *= 2
number = yield number
Here is what that looks like, as you can see sending a new value for number changes the outcome:
>>> def double_number(number):
... while True:
... number *= 2
... number = yield number
...
>>> c = double_number(4)
>>>
>>> c.send(None)
8
>>> c.send(5) #10
10
>>> c.send(1500) #3000
3000
>>> c.send(3) #6
6
You can also put this in a for loop as such:
for x in range(10):
n = c.send(n)
print n
For more help check out this great tutorial.

The send() method controls what the value to the left of the yield expression will be.
To understand how yield differs and what value it holds, lets first quickly refresh on the order python code is evaluated.
Section 6.15 Evaluation order
Python evaluates expressions from left to right. Notice that while evaluating an assignment, the right-hand side is evaluated before the left-hand side.
So an expression a = b the right hand side is evaluated first.
As the following demonstrates that a[p('left')] = p('right') the right hand side is evaluated first.
>>> def p(side):
... print(side)
... return 0
...
>>> a[p('left')] = p('right')
right
left
>>>
>>>
>>> [p('left'), p('right')]
left
right
[0, 0]
What does yield do?, yield, suspends execution of the function and returns to the caller, and resumes execution at the same place it left off prior to suspending.
Where exactly is execution suspended? You might have guessed it already...
the execution is suspended between the right and left side of the yield expression. So new_val = yield old_val the execution is halted at the = sign, and the value on the right (which is before suspending, and is also the value returned to the caller) may be something different then the value on the left (which is the value being assigned after resuming execution).
yield yields 2 values, one to the right and another to the left.
How do you control the value to the left hand side of the yield expression? via the .send() method.
6.2.9. Yield expressions
The value of the yield expression after resuming depends on the method which resumed the execution. If __next__() is used (typically via either a for or the next() builtin) then the result is None. Otherwise, if send() is used, then the result will be the value passed in to that method.

Some use cases for using generator and send()
Generators with send() allow:
remembering internal state of the execution
what step we are at
what is current status of our data
returning sequence of values
receiving sequence of inputs
Here are some use cases:
Watched attempt to follow a recipe
Let us have a recipe, which expects predefined set of inputs in some order.
We may:
create a watched_attempt instance from the recipe
let it get some inputs
with each input return information about what is currently in the pot
with each input check, that the input is the expected one (and fail if it is not)
def recipe():
pot = []
action = yield pot
assert action == ("add", "water")
pot.append(action[1])
action = yield pot
assert action == ("add", "salt")
pot.append(action[1])
action = yield pot
assert action == ("boil", "water")
action = yield pot
assert action == ("add", "pasta")
pot.append(action[1])
action = yield pot
assert action == ("decant", "water")
pot.remove("water")
action = yield pot
assert action == ("serve")
pot = []
yield pot
To use it, first create the watched_attempt instance:
>>> watched_attempt = recipe()
>>> watched_attempt.next()
[]
The call to .next() is necessary to start execution of the generator.
Returned value shows, our pot is currently empty.
Now do few actions following what the recipe expects:
>>> watched_attempt.send(("add", "water"))
['water']
>>> watched_attempt.send(("add", "salt"))
['water', 'salt']
>>> watched_attempt.send(("boil", "water"))
['water', 'salt']
>>> watched_attempt.send(("add", "pasta"))
['water', 'salt', 'pasta']
>>> watched_attempt.send(("decant", "water"))
['salt', 'pasta']
>>> watched_attempt.send(("serve"))
[]
As we see, the pot is finally empty.
In case, one would not follow the recipe, it would fail (what could be desired outcome of watched
attempt to cook something - just learning we did not pay enough attention when given instructions.
>>> watched_attempt = running.recipe()
>>> watched_attempt.next()
[]
>>> watched_attempt.send(("add", "water"))
['water']
>>> watched_attempt.send(("add", "pasta"))
---------------------------------------------------------------------------
AssertionError Traceback (most recent call last)
<ipython-input-21-facdf014fe8e> in <module>()
----> 1 watched_attempt.send(("add", "pasta"))
/home/javl/sandbox/stack/send/running.py in recipe()
29
30 action = yield pot
---> 31 assert action == ("add", "salt")
32 pot.append(action[1])
33
AssertionError:
Notice, that:
there is linear sequence of expected steps
the steps may differ (some are removing, some are adding to the pot)
we manage to do all that by a functions/generator - no need to use complex class or similar
strutures.
Running totals
We may use the generator to keep track of running total of values sent to it.
Any time we add a number, count of inputs and total sum is returned (valid for
the moment previous input was send into it).
from collections import namedtuple
RunningTotal = namedtuple("RunningTotal", ["n", "total"])
def runningtotals(n=0, total=0):
while True:
delta = yield RunningTotal(n, total)
if delta:
n += 1
total += delta
if __name__ == "__main__":
nums = [9, 8, None, 3, 4, 2, 1]
bookeeper = runningtotals()
print bookeeper.next()
for num in nums:
print num, bookeeper.send(num)
The output would look like:
RunningTotal(n=0, total=0)
9 RunningTotal(n=1, total=9)
8 RunningTotal(n=2, total=17)
None RunningTotal(n=2, total=17)
3 RunningTotal(n=3, total=20)
4 RunningTotal(n=4, total=24)
2 RunningTotal(n=5, total=26)
1 RunningTotal(n=6, total=27)

The send method implements coroutines.
If you haven't encountered Coroutines they are tricky to wrap your head around because they change the way a program flows. You can read a good tutorial for more details.

The word "yield" has two meanings: to produce something (e.g., to yield corn), and to halt to let someone/thing else continue (e.g., cars yielding to pedestrians). Both definitions apply to Python's yield keyword; what makes generator functions special is that unlike in regular functions, values can be "returned" to the caller while merely pausing, not terminating, a generator function.
It is easiest to imagine a generator as one end of a bidirectional pipe with a "left" end and a "right" end; this pipe is the medium over which values are sent between the generator itself and the generator function's body. Each end of the pipe has two operations: push, which sends a value and blocks until the other end of the pipe pulls the value, and returns nothing; and pull, which blocks until the other end of the pipe pushes a value, and returns the pushed value. At runtime, execution bounces back and forth between the contexts on either side of the pipe -- each side runs until it sends a value to the other side, at which point it halts, lets the other side run, and waits for a value in return, at which point the other side halts and it resumes. In other words, each end of the pipe runs from the moment it receives a value to the moment it sends a value.
The pipe is functionally symmetric, but -- by convention I'm defining in this answer -- the left end is only available inside the generator function's body and is accessible via the yield keyword, while the right end is the generator and is accessible via the generator's send function. As singular interfaces to their respective ends of the pipe, yield and send do double duty: they each both push and pull values to/from their ends of the pipe, yield pushing rightward and pulling leftward while send does the opposite. This double duty is the crux of the confusion surrounding the semantics of statements like x = yield y. Breaking yield and send down into two explicit push/pull steps will make their semantics much more clear:
Suppose g is the generator. g.send pushes a value leftward through the right end of the pipe.
Execution within the context of g pauses, allowing the generator function's body to run.
The value pushed by g.send is pulled leftward by yield and received on the left end of the pipe. In x = yield y, x is assigned to the pulled value.
Execution continues within the generator function's body until the next line containing yield is reached.
yield pushes a value rightward through the left end of the pipe, back up to g.send. In x = yield y, y is pushed rightward through the pipe.
Execution within the generator function's body pauses, allowing the outer scope to continue where it left off.
g.send resumes and pulls the value and returns it to the user.
When g.send is next called, go back to Step 1.
While cyclical, this procedure does have a beginning: when g.send(None) -- which is what next(g) is short for -- is first called (it is illegal to pass something other than None to the first send call). And it may have an end: when there are no more yield statements to be reached in the generator function's body.
Do you see what makes the yield statement (or more accurately, generators) so special? Unlike the measly return keyword, yield is able to pass values to its caller and receive values from its caller all without terminating the function it lives in! (Of course, if you do wish to terminate a function -- or a generator -- it's handy to have the return keyword as well.) When a yield statement is encountered, the generator function merely pauses, and then picks back up right where it left off upon being sent another value. And send is just the interface for communicating with the inside of a generator function from outside it.
If we really want to break this push/pull/pipe analogy down as far as we can, we end up with the following pseudocode that really drives home that, aside from steps 1-5, yield and send are two sides of the same coin pipe:
right_end.push(None) # the first half of g.send; sending None is what starts a generator
right_end.pause()
left_end.start()
initial_value = left_end.pull()
if initial_value is not None: raise TypeError("can't send non-None value to a just-started generator")
left_end.do_stuff()
left_end.push(y) # the first half of yield
left_end.pause()
right_end.resume()
value1 = right_end.pull() # the second half of g.send
right_end.do_stuff()
right_end.push(value2) # the first half of g.send (again, but with a different value)
right_end.pause()
left_end.resume()
x = left_end.pull() # the second half of yield
goto 6
The key transformation is that we have split x = yield y and value1 = g.send(value2) each into two statements: left_end.push(y) and x = left_end.pull(); and value1 = right_end.pull() and right_end.push(value2). There are two special cases of the yield keyword: x = yield and yield y. These are syntactic sugar, respectively, for x = yield None and _ = yield y # discarding value.
For specific details regarding the precise order in which values are sent through the pipe, see below.
What follows is a rather long concrete model of the above. First, it should first be noted that for any generator g, next(g) is exactly equivalent to g.send(None). With this in mind we can focus only on how send works and talk only about advancing the generator with send.
Suppose we have
def f(y): # This is the "generator function" referenced above
while True:
x = yield y
y = x
g = f(1)
g.send(None) # yields 1
g.send(2) # yields 2
Now, the definition of f roughly desugars to the following ordinary (non-generator) function:
def f(y):
bidirectional_pipe = BidirectionalPipe()
left_end = bidirectional_pipe.left_end
right_end = bidirectional_pipe.right_end
def impl():
initial_value = left_end.pull()
if initial_value is not None:
raise TypeError(
"can't send non-None value to a just-started generator"
)
while True:
left_end.push(y)
x = left_end.pull()
y = x
def send(value):
right_end.push(value)
return right_end.pull()
right_end.send = send
# This isn't real Python; normally, returning exits the function. But
# pretend that it's possible to return a value from a function and then
# continue execution -- this is exactly the problem that generators were
# designed to solve!
return right_end
impl()
The following has happened in this transformation of f:
We've moved the implementation into a nested function.
We've created a bidirectional pipe whose left_end will be accessed by the nested function and whose right_end will be returned and accessed by the outer scope -- right_end is what we know as the generator object.
Within the nested function, the very first thing we do is check that left_end.pull() is None, consuming a pushed value in the process.
Within the nested function, the statement x = yield y has been replaced by two lines: left_end.push(y) and x = left_end.pull().
We've defined the send function for right_end, which is the counterpart to the two lines we replaced the x = yield y statement with in the previous step.
In this fantasy world where functions can continue after returning, g is assigned right_end and then impl() is called. So in our example above, were we to follow execution line by line, what would happen is roughly the following:
left_end = bidirectional_pipe.left_end
right_end = bidirectional_pipe.right_end
y = 1 # from g = f(1)
# None pushed by first half of g.send(None)
right_end.push(None)
# The above push blocks, so the outer scope halts and lets `f` run until
# *it* blocks
# Receive the pushed value, None
initial_value = left_end.pull()
if initial_value is not None: # ok, `g` sent None
raise TypeError(
"can't send non-None value to a just-started generator"
)
left_end.push(y)
# The above line blocks, so `f` pauses and g.send picks up where it left off
# y, aka 1, is pulled by right_end and returned by `g.send(None)`
right_end.pull()
# Rinse and repeat
# 2 pushed by first half of g.send(2)
right_end.push(2)
# Once again the above blocks, so g.send (the outer scope) halts and `f` resumes
# Receive the pushed value, 2
x = left_end.pull()
y = x # y == x == 2
left_end.push(y)
# The above line blocks, so `f` pauses and g.send(2) picks up where it left off
# y, aka 2, is pulled by right_end and returned to the outer scope
right_end.pull()
x = left_end.pull()
# blocks until the next call to g.send
This maps exactly to the 16-step pseudocode above.
There are some other details, like how errors are propagated and what happens when you reach the end of the generator (the pipe is closed), but this should make clear how the basic control flow works when send is used.
Using these same desugaring rules, let's look at two special cases:
def f1(x):
while True:
x = yield x
def f2(): # No parameter
while True:
x = yield x
For the most part they desugar the same way as f, the only differences are how the yield statements are transformed:
def f1(x):
# ... set up pipe
def impl():
# ... check that initial sent value is None
while True:
left_end.push(x)
x = left_end.pull()
# ... set up right_end
def f2():
# ... set up pipe
def impl():
# ... check that initial sent value is None
while True:
left_end.push(x)
x = left_end.pull()
# ... set up right_end
In the first, the value passed to f1 is pushed (yielded) initially, and then all values pulled (sent) are pushed (yielded) right back. In the second, x has no value (yet) when it first come times to push, so an UnboundLocalError is raised.

These confused me too. Here is an example I made when trying to set up a generator which yields and accepts signals in alternating order (yield, accept, yield, accept)...
def echo_sound():
thing_to_say = '<Sound of wind on cliffs>'
while True:
thing_to_say = (yield thing_to_say)
thing_to_say = '...'.join([thing_to_say]+[thing_to_say[-6:]]*2)
yield None # This is the return value of send.
gen = echo_sound()
print 'You are lost in the wilderness, calling for help.'
print '------'
in_message = gen.next()
print 'You hear: "{}"'.format(in_message)
out_message = 'Hello!'
print 'You yell "{}"'.format(out_message)
gen.send(out_message)
print '------'
in_message = gen.next()
print 'You hear: "{}"'.format(in_message)
out_message = 'Is anybody out there?'
print 'You yell "{}"'.format(out_message)
gen.send(out_message)
print '------'
in_message = gen.next()
print 'You hear: "{}"'.format(in_message)
out_message = 'Help!'
print 'You yell "{}"'.format(out_message)
gen.send(out_message)
The output is:
You are lost in the wilderness, calling for help.
------
You hear: "<Sound of wind on cliffs>"
You yell "Hello!"
------
You hear: "Hello!...Hello!...Hello!"
You yell "Is anybody out there?"
------
You hear: "Is anybody out there?...there?...there?"
You yell "Help!"

itr.send(None) is the same thing as next(itr) and what you're doing is giving the value given by yield in the generator.
Here's an example that clearly shows this, and how it can be used more practically.
def iterator_towards(dest=100):
value = 0
while True:
n = yield value
if n is not None:
dest = n
if dest > value:
value += 1
elif dest < value:
value -= 1
else:
return
num = iterator_towards()
for i in num:
print(i)
if i == 5:
num.send(0)
This will print:
0
1
2
3
4
5
3
2
1
0
The code at i == 5 tells it to send 0. This is not None in the iterator_towards and so it changes the value of dest. We then iterate towards 0.
However note, there is no value 4 after the value 5. This is because the nature of .send(0) is that it was yielded the 4 value and that was not printed.
If we add a continue we can re-yield the same value.
def iterator_towards(dest=100):
value = 0
while True:
n = yield value
if n is not None:
dest = n
continue
if dest > value:
value += 1
elif dest < value:
value -= 1
else:
return
Which will allow you to iterate a list but also dynamically send it new destination values onthefly.

What is the python syntax when a variable is after 'yield'? [duplicate]

I understand yield. But what does a generator's send function do? The documentation says:
generator.send(value)
Resumes the execution and “sends” a value into the generator function. The value argument becomes the result of the current yield expression. The send() method returns the next value yielded by the generator, or raises StopIteration if the generator exits without yielding another value.
What does that mean? I thought value was the input to the generator function? The phrase "The send() method returns the next value yielded by the generator" seems to be also the exact purpose of yield, which also returns the next value yielded by the generator.
Is there an example of a generator utilizing send that accomplishes something yield cannot?

It's used to send values into a generator that just yielded. Here is an artificial (non-useful) explanatory example:
>>> def double_inputs():
... while True:
... x = yield
... yield x * 2
...
>>> gen = double_inputs()
>>> next(gen) # run up to the first yield
>>> gen.send(10) # goes into 'x' variable
20
>>> next(gen) # run up to the next yield
>>> gen.send(6) # goes into 'x' again
12
>>> next(gen) # run up to the next yield
>>> gen.send(94.3) # goes into 'x' again
188.5999999999999
You can't do this just with yield.
As to why it's useful, one of the best use cases I've seen is Twisted's #defer.inlineCallbacks. Essentially it allows you to write a function like this:
#defer.inlineCallbacks
def doStuff():
result = yield takesTwoSeconds()
nextResult = yield takesTenSeconds(result * 10)
defer.returnValue(nextResult / 10)
What happens is that takesTwoSeconds() returns a Deferred, which is a value promising a value will be computed later. Twisted can run the computation in another thread. When the computation is done, it passes it into the deferred, and the value then gets sent back to the doStuff() function. Thus the doStuff() can end up looking more or less like a normal procedural function, except it can be doing all sorts of computations & callbacks etc. The alternative before this functionality would be to do something like:
def doStuff():
returnDeferred = defer.Deferred()
def gotNextResult(nextResult):
returnDeferred.callback(nextResult / 10)
def gotResult(result):
takesTenSeconds(result * 10).addCallback(gotNextResult)
takesTwoSeconds().addCallback(gotResult)
return returnDeferred
It's a lot more convoluted and unwieldy.

This function is to write coroutines
def coroutine():
for i in range(1, 10):
print("From generator {}".format((yield i)))
c = coroutine()
c.send(None)
try:
while True:
print("From user {}".format(c.send(1)))
except StopIteration: pass
prints
From generator 1
From user 2
From generator 1
From user 3
From generator 1
From user 4
...
See how the control is being passed back and forth? Those are coroutines. They can be used for all kinds of cool things like asynch IO and similar.
Think of it like this, with a generator and no send, it's a one way street
========== yield ========
Generator | ------------> | User |
========== ========
But with send, it becomes a two way street
========== yield ========
Generator | ------------> | User |
========== <------------ ========
send
Which opens up the door to the user customizing the generators behavior on the fly and the generator responding to the user.

This may help someone. Here is a generator that is unaffected by send function. It takes in the number parameter on instantiation and is unaffected by send:
>>> def double_number(number):
... while True:
... number *=2
... yield number
...
>>> c = double_number(4)
>>> c.send(None)
8
>>> c.next()
16
>>> c.next()
32
>>> c.send(8)
64
>>> c.send(8)
128
>>> c.send(8)
256
Now here is how you would do the same type of function using send, so on each iteration you can change the value of number:
def double_number(number):
while True:
number *= 2
number = yield number
Here is what that looks like, as you can see sending a new value for number changes the outcome:
>>> def double_number(number):
... while True:
... number *= 2
... number = yield number
...
>>> c = double_number(4)
>>>
>>> c.send(None)
8
>>> c.send(5) #10
10
>>> c.send(1500) #3000
3000
>>> c.send(3) #6
6
You can also put this in a for loop as such:
for x in range(10):
n = c.send(n)
print n
For more help check out this great tutorial.

The send() method controls what the value to the left of the yield expression will be.
To understand how yield differs and what value it holds, lets first quickly refresh on the order python code is evaluated.
Section 6.15 Evaluation order
Python evaluates expressions from left to right. Notice that while evaluating an assignment, the right-hand side is evaluated before the left-hand side.
So an expression a = b the right hand side is evaluated first.
As the following demonstrates that a[p('left')] = p('right') the right hand side is evaluated first.
>>> def p(side):
... print(side)
... return 0
...
>>> a[p('left')] = p('right')
right
left
>>>
>>>
>>> [p('left'), p('right')]
left
right
[0, 0]
What does yield do?, yield, suspends execution of the function and returns to the caller, and resumes execution at the same place it left off prior to suspending.
Where exactly is execution suspended? You might have guessed it already...
the execution is suspended between the right and left side of the yield expression. So new_val = yield old_val the execution is halted at the = sign, and the value on the right (which is before suspending, and is also the value returned to the caller) may be something different then the value on the left (which is the value being assigned after resuming execution).
yield yields 2 values, one to the right and another to the left.
How do you control the value to the left hand side of the yield expression? via the .send() method.
6.2.9. Yield expressions
The value of the yield expression after resuming depends on the method which resumed the execution. If __next__() is used (typically via either a for or the next() builtin) then the result is None. Otherwise, if send() is used, then the result will be the value passed in to that method.

Some use cases for using generator and send()
Generators with send() allow:
remembering internal state of the execution
what step we are at
what is current status of our data
returning sequence of values
receiving sequence of inputs
Here are some use cases:
Watched attempt to follow a recipe
Let us have a recipe, which expects predefined set of inputs in some order.
We may:
create a watched_attempt instance from the recipe
let it get some inputs
with each input return information about what is currently in the pot
with each input check, that the input is the expected one (and fail if it is not)
def recipe():
pot = []
action = yield pot
assert action == ("add", "water")
pot.append(action[1])
action = yield pot
assert action == ("add", "salt")
pot.append(action[1])
action = yield pot
assert action == ("boil", "water")
action = yield pot
assert action == ("add", "pasta")
pot.append(action[1])
action = yield pot
assert action == ("decant", "water")
pot.remove("water")
action = yield pot
assert action == ("serve")
pot = []
yield pot
To use it, first create the watched_attempt instance:
>>> watched_attempt = recipe()
>>> watched_attempt.next()
[]
The call to .next() is necessary to start execution of the generator.
Returned value shows, our pot is currently empty.
Now do few actions following what the recipe expects:
>>> watched_attempt.send(("add", "water"))
['water']
>>> watched_attempt.send(("add", "salt"))
['water', 'salt']
>>> watched_attempt.send(("boil", "water"))
['water', 'salt']
>>> watched_attempt.send(("add", "pasta"))
['water', 'salt', 'pasta']
>>> watched_attempt.send(("decant", "water"))
['salt', 'pasta']
>>> watched_attempt.send(("serve"))
[]
As we see, the pot is finally empty.
In case, one would not follow the recipe, it would fail (what could be desired outcome of watched
attempt to cook something - just learning we did not pay enough attention when given instructions.
>>> watched_attempt = running.recipe()
>>> watched_attempt.next()
[]
>>> watched_attempt.send(("add", "water"))
['water']
>>> watched_attempt.send(("add", "pasta"))
---------------------------------------------------------------------------
AssertionError Traceback (most recent call last)
<ipython-input-21-facdf014fe8e> in <module>()
----> 1 watched_attempt.send(("add", "pasta"))
/home/javl/sandbox/stack/send/running.py in recipe()
29
30 action = yield pot
---> 31 assert action == ("add", "salt")
32 pot.append(action[1])
33
AssertionError:
Notice, that:
there is linear sequence of expected steps
the steps may differ (some are removing, some are adding to the pot)
we manage to do all that by a functions/generator - no need to use complex class or similar
strutures.
Running totals
We may use the generator to keep track of running total of values sent to it.
Any time we add a number, count of inputs and total sum is returned (valid for
the moment previous input was send into it).
from collections import namedtuple
RunningTotal = namedtuple("RunningTotal", ["n", "total"])
def runningtotals(n=0, total=0):
while True:
delta = yield RunningTotal(n, total)
if delta:
n += 1
total += delta
if __name__ == "__main__":
nums = [9, 8, None, 3, 4, 2, 1]
bookeeper = runningtotals()
print bookeeper.next()
for num in nums:
print num, bookeeper.send(num)
The output would look like:
RunningTotal(n=0, total=0)
9 RunningTotal(n=1, total=9)
8 RunningTotal(n=2, total=17)
None RunningTotal(n=2, total=17)
3 RunningTotal(n=3, total=20)
4 RunningTotal(n=4, total=24)
2 RunningTotal(n=5, total=26)
1 RunningTotal(n=6, total=27)

The send method implements coroutines.
If you haven't encountered Coroutines they are tricky to wrap your head around because they change the way a program flows. You can read a good tutorial for more details.

The word "yield" has two meanings: to produce something (e.g., to yield corn), and to halt to let someone/thing else continue (e.g., cars yielding to pedestrians). Both definitions apply to Python's yield keyword; what makes generator functions special is that unlike in regular functions, values can be "returned" to the caller while merely pausing, not terminating, a generator function.
It is easiest to imagine a generator as one end of a bidirectional pipe with a "left" end and a "right" end; this pipe is the medium over which values are sent between the generator itself and the generator function's body. Each end of the pipe has two operations: push, which sends a value and blocks until the other end of the pipe pulls the value, and returns nothing; and pull, which blocks until the other end of the pipe pushes a value, and returns the pushed value. At runtime, execution bounces back and forth between the contexts on either side of the pipe -- each side runs until it sends a value to the other side, at which point it halts, lets the other side run, and waits for a value in return, at which point the other side halts and it resumes. In other words, each end of the pipe runs from the moment it receives a value to the moment it sends a value.
The pipe is functionally symmetric, but -- by convention I'm defining in this answer -- the left end is only available inside the generator function's body and is accessible via the yield keyword, while the right end is the generator and is accessible via the generator's send function. As singular interfaces to their respective ends of the pipe, yield and send do double duty: they each both push and pull values to/from their ends of the pipe, yield pushing rightward and pulling leftward while send does the opposite. This double duty is the crux of the confusion surrounding the semantics of statements like x = yield y. Breaking yield and send down into two explicit push/pull steps will make their semantics much more clear:
Suppose g is the generator. g.send pushes a value leftward through the right end of the pipe.
Execution within the context of g pauses, allowing the generator function's body to run.
The value pushed by g.send is pulled leftward by yield and received on the left end of the pipe. In x = yield y, x is assigned to the pulled value.
Execution continues within the generator function's body until the next line containing yield is reached.
yield pushes a value rightward through the left end of the pipe, back up to g.send. In x = yield y, y is pushed rightward through the pipe.
Execution within the generator function's body pauses, allowing the outer scope to continue where it left off.
g.send resumes and pulls the value and returns it to the user.
When g.send is next called, go back to Step 1.
While cyclical, this procedure does have a beginning: when g.send(None) -- which is what next(g) is short for -- is first called (it is illegal to pass something other than None to the first send call). And it may have an end: when there are no more yield statements to be reached in the generator function's body.
Do you see what makes the yield statement (or more accurately, generators) so special? Unlike the measly return keyword, yield is able to pass values to its caller and receive values from its caller all without terminating the function it lives in! (Of course, if you do wish to terminate a function -- or a generator -- it's handy to have the return keyword as well.) When a yield statement is encountered, the generator function merely pauses, and then picks back up right where it left off upon being sent another value. And send is just the interface for communicating with the inside of a generator function from outside it.
If we really want to break this push/pull/pipe analogy down as far as we can, we end up with the following pseudocode that really drives home that, aside from steps 1-5, yield and send are two sides of the same coin pipe:
right_end.push(None) # the first half of g.send; sending None is what starts a generator
right_end.pause()
left_end.start()
initial_value = left_end.pull()
if initial_value is not None: raise TypeError("can't send non-None value to a just-started generator")
left_end.do_stuff()
left_end.push(y) # the first half of yield
left_end.pause()
right_end.resume()
value1 = right_end.pull() # the second half of g.send
right_end.do_stuff()
right_end.push(value2) # the first half of g.send (again, but with a different value)
right_end.pause()
left_end.resume()
x = left_end.pull() # the second half of yield
goto 6
The key transformation is that we have split x = yield y and value1 = g.send(value2) each into two statements: left_end.push(y) and x = left_end.pull(); and value1 = right_end.pull() and right_end.push(value2). There are two special cases of the yield keyword: x = yield and yield y. These are syntactic sugar, respectively, for x = yield None and _ = yield y # discarding value.
For specific details regarding the precise order in which values are sent through the pipe, see below.
What follows is a rather long concrete model of the above. First, it should first be noted that for any generator g, next(g) is exactly equivalent to g.send(None). With this in mind we can focus only on how send works and talk only about advancing the generator with send.
Suppose we have
def f(y): # This is the "generator function" referenced above
while True:
x = yield y
y = x
g = f(1)
g.send(None) # yields 1
g.send(2) # yields 2
Now, the definition of f roughly desugars to the following ordinary (non-generator) function:
def f(y):
bidirectional_pipe = BidirectionalPipe()
left_end = bidirectional_pipe.left_end
right_end = bidirectional_pipe.right_end
def impl():
initial_value = left_end.pull()
if initial_value is not None:
raise TypeError(
"can't send non-None value to a just-started generator"
)
while True:
left_end.push(y)
x = left_end.pull()
y = x
def send(value):
right_end.push(value)
return right_end.pull()
right_end.send = send
# This isn't real Python; normally, returning exits the function. But
# pretend that it's possible to return a value from a function and then
# continue execution -- this is exactly the problem that generators were
# designed to solve!
return right_end
impl()
The following has happened in this transformation of f:
We've moved the implementation into a nested function.
We've created a bidirectional pipe whose left_end will be accessed by the nested function and whose right_end will be returned and accessed by the outer scope -- right_end is what we know as the generator object.
Within the nested function, the very first thing we do is check that left_end.pull() is None, consuming a pushed value in the process.
Within the nested function, the statement x = yield y has been replaced by two lines: left_end.push(y) and x = left_end.pull().
We've defined the send function for right_end, which is the counterpart to the two lines we replaced the x = yield y statement with in the previous step.
In this fantasy world where functions can continue after returning, g is assigned right_end and then impl() is called. So in our example above, were we to follow execution line by line, what would happen is roughly the following:
left_end = bidirectional_pipe.left_end
right_end = bidirectional_pipe.right_end
y = 1 # from g = f(1)
# None pushed by first half of g.send(None)
right_end.push(None)
# The above push blocks, so the outer scope halts and lets `f` run until
# *it* blocks
# Receive the pushed value, None
initial_value = left_end.pull()
if initial_value is not None: # ok, `g` sent None
raise TypeError(
"can't send non-None value to a just-started generator"
)
left_end.push(y)
# The above line blocks, so `f` pauses and g.send picks up where it left off
# y, aka 1, is pulled by right_end and returned by `g.send(None)`
right_end.pull()
# Rinse and repeat
# 2 pushed by first half of g.send(2)
right_end.push(2)
# Once again the above blocks, so g.send (the outer scope) halts and `f` resumes
# Receive the pushed value, 2
x = left_end.pull()
y = x # y == x == 2
left_end.push(y)
# The above line blocks, so `f` pauses and g.send(2) picks up where it left off
# y, aka 2, is pulled by right_end and returned to the outer scope
right_end.pull()
x = left_end.pull()
# blocks until the next call to g.send
This maps exactly to the 16-step pseudocode above.
There are some other details, like how errors are propagated and what happens when you reach the end of the generator (the pipe is closed), but this should make clear how the basic control flow works when send is used.
Using these same desugaring rules, let's look at two special cases:
def f1(x):
while True:
x = yield x
def f2(): # No parameter
while True:
x = yield x
For the most part they desugar the same way as f, the only differences are how the yield statements are transformed:
def f1(x):
# ... set up pipe
def impl():
# ... check that initial sent value is None
while True:
left_end.push(x)
x = left_end.pull()
# ... set up right_end
def f2():
# ... set up pipe
def impl():
# ... check that initial sent value is None
while True:
left_end.push(x)
x = left_end.pull()
# ... set up right_end
In the first, the value passed to f1 is pushed (yielded) initially, and then all values pulled (sent) are pushed (yielded) right back. In the second, x has no value (yet) when it first come times to push, so an UnboundLocalError is raised.

These confused me too. Here is an example I made when trying to set up a generator which yields and accepts signals in alternating order (yield, accept, yield, accept)...
def echo_sound():
thing_to_say = '<Sound of wind on cliffs>'
while True:
thing_to_say = (yield thing_to_say)
thing_to_say = '...'.join([thing_to_say]+[thing_to_say[-6:]]*2)
yield None # This is the return value of send.
gen = echo_sound()
print 'You are lost in the wilderness, calling for help.'
print '------'
in_message = gen.next()
print 'You hear: "{}"'.format(in_message)
out_message = 'Hello!'
print 'You yell "{}"'.format(out_message)
gen.send(out_message)
print '------'
in_message = gen.next()
print 'You hear: "{}"'.format(in_message)
out_message = 'Is anybody out there?'
print 'You yell "{}"'.format(out_message)
gen.send(out_message)
print '------'
in_message = gen.next()
print 'You hear: "{}"'.format(in_message)
out_message = 'Help!'
print 'You yell "{}"'.format(out_message)
gen.send(out_message)
The output is:
You are lost in the wilderness, calling for help.
------
You hear: "<Sound of wind on cliffs>"
You yell "Hello!"
------
You hear: "Hello!...Hello!...Hello!"
You yell "Is anybody out there?"
------
You hear: "Is anybody out there?...there?...there?"
You yell "Help!"

itr.send(None) is the same thing as next(itr) and what you're doing is giving the value given by yield in the generator.
Here's an example that clearly shows this, and how it can be used more practically.
def iterator_towards(dest=100):
value = 0
while True:
n = yield value
if n is not None:
dest = n
if dest > value:
value += 1
elif dest < value:
value -= 1
else:
return
num = iterator_towards()
for i in num:
print(i)
if i == 5:
num.send(0)
This will print:
0
1
2
3
4
5
3
2
1
0
The code at i == 5 tells it to send 0. This is not None in the iterator_towards and so it changes the value of dest. We then iterate towards 0.
However note, there is no value 4 after the value 5. This is because the nature of .send(0) is that it was yielded the 4 value and that was not printed.
If we add a continue we can re-yield the same value.
def iterator_towards(dest=100):
value = 0
while True:
n = yield value
if n is not None:
dest = n
continue
if dest > value:
value += 1
elif dest < value:
value -= 1
else:
return
Which will allow you to iterate a list but also dynamically send it new destination values onthefly.

python3 send() function in generators

According to docs, the send() function:
"Resumes the execution and “sends” a value into the generator function. The value argument becomes the result of the current yield expression. The send() method returns the next value yielded by the generator, or raises StopIteration if the generator exits without yielding another value. When send() is called to start the generator, it must be called with None as the argument, because there is no yield expression that could receive the value."
But i can't understand, why "The value argument becomes the result of the current yield expression" is not happened in the following example:
def gen():
yield 1
x = (yield 42)
print(x)
yield 2
>>>c=gen() #create generator
>>>next(c) #prints '1' and stop execution, which is caused by yield 1
>>>c.send(100) #prints '42', because 'The send() method returns the next value yielded by the generator'
>>>next(c) #prints 'None' and '2'
So why x variable stays 'None' dispite i send 100 to it by c.send(100)? It seems, that yield expression in right hand side works in two steps: first it return the value to generator's caller and the second it returns argument of send function inside generator. And if add extra next(c) before send(42) i'll get expected behavior and programm prints '100'. It's not clear for me from documentation, why these two steps should not happen simultaneously when i call send().

So why x variable stays 'None' dispite i send 100 to it by
c.send(100)?
Because the current yield is not the one you think it is, it's the one before.
When you send 100, the generator is still stopped at yield 1, not yet at yield 42, hence 100 will be the result of yield 1, not of yield 42.
To see this clearer, if you modify the generator in order to retrieve the content of yield 1 in a z variable, you'll see that z does contain 100:
>>> def gen():
... z = yield 1
... x = (yield 42)
... print(x, z)
... yield 2
...
>>> c=gen()
>>> next(c)
1
>>> c.send(100)
42
>>> next(c)
None 100
2
>>>
And that's the reason why an extra next() will print 100. Back to your code:
def gen():
yield 1 # first next(), stops here (before "getting out" of yield)
x = (yield 42) # second next(), stops here (before "getting out" of yield),
# so, when you send(100), then 100 is given to x and execution goes on, so:
print(x) # 100 is printed
yield 2 # ... and 2.

I think I've figured it out.
c = gen()
You create a genertor in the c variable, there none of the generator's code is executed.
next(c)
Then you use the next() function, the next function go to the next yield statement who is yield 1 here.
So this yield returns 1 and stops the generator's execution to try to catch a value.
The next line is
c.send(100), so the yield 1 was waiting for a value and you provided it, but this value isn't saved.
The send method also execute the rest of the generator until the next yield statement who is:
x = (yield 42)
So the yield here return 42 and stops the generator program to try to catch a value. But after that you call
next(c)
And you didn't provided a value so x is None now. Then the rest of the code is executed (until the next yield statement, don't forget)
print(x)
yield 2
So it prints x who is None and then the yield returns 2 and try to catch a value.
Try to write this
c = gen()
next(c)
next(c)
c.send(100)
And it will work (understand why !)

From documentation:
The value argument becomes the result of the current yield expression. The send() method returns the next value yielded by the generator.
assume we have following infinite generator:
def infgen():
a = 0
while True:
a = yield a
a += 1
generato = infgen()
generato.send(None)
output:0
.send(None) is executed almost the same as next()
now we have yeild first value of a = 0 and
code stopped here a = yield a
as value becomes result of the current yield expression
yielded expression is yield a. Result of yielded expresion is a
generato.send(5)
.send(5) we send 5 to a in generator and it continue:
Result of yielded expresion is a = 5
a = 5
a += 1
Go to while loop with a = 5 + 1 = 6
And stops and yield a where a is 6
output:6

How does "line = (yield)" work? [duplicate]

I understand yield. But what does a generator's send function do? The documentation says:
generator.send(value)
Resumes the execution and “sends” a value into the generator function. The value argument becomes the result of the current yield expression. The send() method returns the next value yielded by the generator, or raises StopIteration if the generator exits without yielding another value.
What does that mean? I thought value was the input to the generator function? The phrase "The send() method returns the next value yielded by the generator" seems to be also the exact purpose of yield, which also returns the next value yielded by the generator.
Is there an example of a generator utilizing send that accomplishes something yield cannot?

It's used to send values into a generator that just yielded. Here is an artificial (non-useful) explanatory example:
>>> def double_inputs():
... while True:
... x = yield
... yield x * 2
...
>>> gen = double_inputs()
>>> next(gen) # run up to the first yield
>>> gen.send(10) # goes into 'x' variable
20
>>> next(gen) # run up to the next yield
>>> gen.send(6) # goes into 'x' again
12
>>> next(gen) # run up to the next yield
>>> gen.send(94.3) # goes into 'x' again
188.5999999999999
You can't do this just with yield.
As to why it's useful, one of the best use cases I've seen is Twisted's #defer.inlineCallbacks. Essentially it allows you to write a function like this:
#defer.inlineCallbacks
def doStuff():
result = yield takesTwoSeconds()
nextResult = yield takesTenSeconds(result * 10)
defer.returnValue(nextResult / 10)
What happens is that takesTwoSeconds() returns a Deferred, which is a value promising a value will be computed later. Twisted can run the computation in another thread. When the computation is done, it passes it into the deferred, and the value then gets sent back to the doStuff() function. Thus the doStuff() can end up looking more or less like a normal procedural function, except it can be doing all sorts of computations & callbacks etc. The alternative before this functionality would be to do something like:
def doStuff():
returnDeferred = defer.Deferred()
def gotNextResult(nextResult):
returnDeferred.callback(nextResult / 10)
def gotResult(result):
takesTenSeconds(result * 10).addCallback(gotNextResult)
takesTwoSeconds().addCallback(gotResult)
return returnDeferred
It's a lot more convoluted and unwieldy.

This function is to write coroutines
def coroutine():
for i in range(1, 10):
print("From generator {}".format((yield i)))
c = coroutine()
c.send(None)
try:
while True:
print("From user {}".format(c.send(1)))
except StopIteration: pass
prints
From generator 1
From user 2
From generator 1
From user 3
From generator 1
From user 4
...
See how the control is being passed back and forth? Those are coroutines. They can be used for all kinds of cool things like asynch IO and similar.
Think of it like this, with a generator and no send, it's a one way street
========== yield ========
Generator | ------------> | User |
========== ========
But with send, it becomes a two way street
========== yield ========
Generator | ------------> | User |
========== <------------ ========
send
Which opens up the door to the user customizing the generators behavior on the fly and the generator responding to the user.

This may help someone. Here is a generator that is unaffected by send function. It takes in the number parameter on instantiation and is unaffected by send:
>>> def double_number(number):
... while True:
... number *=2
... yield number
...
>>> c = double_number(4)
>>> c.send(None)
8
>>> c.next()
16
>>> c.next()
32
>>> c.send(8)
64
>>> c.send(8)
128
>>> c.send(8)
256
Now here is how you would do the same type of function using send, so on each iteration you can change the value of number:
def double_number(number):
while True:
number *= 2
number = yield number
Here is what that looks like, as you can see sending a new value for number changes the outcome:
>>> def double_number(number):
... while True:
... number *= 2
... number = yield number
...
>>> c = double_number(4)
>>>
>>> c.send(None)
8
>>> c.send(5) #10
10
>>> c.send(1500) #3000
3000
>>> c.send(3) #6
6
You can also put this in a for loop as such:
for x in range(10):
n = c.send(n)
print n
For more help check out this great tutorial.

The send() method controls what the value to the left of the yield expression will be.
To understand how yield differs and what value it holds, lets first quickly refresh on the order python code is evaluated.
Section 6.15 Evaluation order
Python evaluates expressions from left to right. Notice that while evaluating an assignment, the right-hand side is evaluated before the left-hand side.
So an expression a = b the right hand side is evaluated first.
As the following demonstrates that a[p('left')] = p('right') the right hand side is evaluated first.
>>> def p(side):
... print(side)
... return 0
...
>>> a[p('left')] = p('right')
right
left
>>>
>>>
>>> [p('left'), p('right')]
left
right
[0, 0]
What does yield do?, yield, suspends execution of the function and returns to the caller, and resumes execution at the same place it left off prior to suspending.
Where exactly is execution suspended? You might have guessed it already...
the execution is suspended between the right and left side of the yield expression. So new_val = yield old_val the execution is halted at the = sign, and the value on the right (which is before suspending, and is also the value returned to the caller) may be something different then the value on the left (which is the value being assigned after resuming execution).
yield yields 2 values, one to the right and another to the left.
How do you control the value to the left hand side of the yield expression? via the .send() method.
6.2.9. Yield expressions
The value of the yield expression after resuming depends on the method which resumed the execution. If __next__() is used (typically via either a for or the next() builtin) then the result is None. Otherwise, if send() is used, then the result will be the value passed in to that method.

Some use cases for using generator and send()
Generators with send() allow:
remembering internal state of the execution
what step we are at
what is current status of our data
returning sequence of values
receiving sequence of inputs
Here are some use cases:
Watched attempt to follow a recipe
Let us have a recipe, which expects predefined set of inputs in some order.
We may:
create a watched_attempt instance from the recipe
let it get some inputs
with each input return information about what is currently in the pot
with each input check, that the input is the expected one (and fail if it is not)
def recipe():
pot = []
action = yield pot
assert action == ("add", "water")
pot.append(action[1])
action = yield pot
assert action == ("add", "salt")
pot.append(action[1])
action = yield pot
assert action == ("boil", "water")
action = yield pot
assert action == ("add", "pasta")
pot.append(action[1])
action = yield pot
assert action == ("decant", "water")
pot.remove("water")
action = yield pot
assert action == ("serve")
pot = []
yield pot
To use it, first create the watched_attempt instance:
>>> watched_attempt = recipe()
>>> watched_attempt.next()
[]
The call to .next() is necessary to start execution of the generator.
Returned value shows, our pot is currently empty.
Now do few actions following what the recipe expects:
>>> watched_attempt.send(("add", "water"))
['water']
>>> watched_attempt.send(("add", "salt"))
['water', 'salt']
>>> watched_attempt.send(("boil", "water"))
['water', 'salt']
>>> watched_attempt.send(("add", "pasta"))
['water', 'salt', 'pasta']
>>> watched_attempt.send(("decant", "water"))
['salt', 'pasta']
>>> watched_attempt.send(("serve"))
[]
As we see, the pot is finally empty.
In case, one would not follow the recipe, it would fail (what could be desired outcome of watched
attempt to cook something - just learning we did not pay enough attention when given instructions.
>>> watched_attempt = running.recipe()
>>> watched_attempt.next()
[]
>>> watched_attempt.send(("add", "water"))
['water']
>>> watched_attempt.send(("add", "pasta"))
---------------------------------------------------------------------------
AssertionError Traceback (most recent call last)
<ipython-input-21-facdf014fe8e> in <module>()
----> 1 watched_attempt.send(("add", "pasta"))
/home/javl/sandbox/stack/send/running.py in recipe()
29
30 action = yield pot
---> 31 assert action == ("add", "salt")
32 pot.append(action[1])
33
AssertionError:
Notice, that:
there is linear sequence of expected steps
the steps may differ (some are removing, some are adding to the pot)
we manage to do all that by a functions/generator - no need to use complex class or similar
strutures.
Running totals
We may use the generator to keep track of running total of values sent to it.
Any time we add a number, count of inputs and total sum is returned (valid for
the moment previous input was send into it).
from collections import namedtuple
RunningTotal = namedtuple("RunningTotal", ["n", "total"])
def runningtotals(n=0, total=0):
while True:
delta = yield RunningTotal(n, total)
if delta:
n += 1
total += delta
if __name__ == "__main__":
nums = [9, 8, None, 3, 4, 2, 1]
bookeeper = runningtotals()
print bookeeper.next()
for num in nums:
print num, bookeeper.send(num)
The output would look like:
RunningTotal(n=0, total=0)
9 RunningTotal(n=1, total=9)
8 RunningTotal(n=2, total=17)
None RunningTotal(n=2, total=17)
3 RunningTotal(n=3, total=20)
4 RunningTotal(n=4, total=24)
2 RunningTotal(n=5, total=26)
1 RunningTotal(n=6, total=27)

The send method implements coroutines.
If you haven't encountered Coroutines they are tricky to wrap your head around because they change the way a program flows. You can read a good tutorial for more details.

The word "yield" has two meanings: to produce something (e.g., to yield corn), and to halt to let someone/thing else continue (e.g., cars yielding to pedestrians). Both definitions apply to Python's yield keyword; what makes generator functions special is that unlike in regular functions, values can be "returned" to the caller while merely pausing, not terminating, a generator function.
It is easiest to imagine a generator as one end of a bidirectional pipe with a "left" end and a "right" end; this pipe is the medium over which values are sent between the generator itself and the generator function's body. Each end of the pipe has two operations: push, which sends a value and blocks until the other end of the pipe pulls the value, and returns nothing; and pull, which blocks until the other end of the pipe pushes a value, and returns the pushed value. At runtime, execution bounces back and forth between the contexts on either side of the pipe -- each side runs until it sends a value to the other side, at which point it halts, lets the other side run, and waits for a value in return, at which point the other side halts and it resumes. In other words, each end of the pipe runs from the moment it receives a value to the moment it sends a value.
The pipe is functionally symmetric, but -- by convention I'm defining in this answer -- the left end is only available inside the generator function's body and is accessible via the yield keyword, while the right end is the generator and is accessible via the generator's send function. As singular interfaces to their respective ends of the pipe, yield and send do double duty: they each both push and pull values to/from their ends of the pipe, yield pushing rightward and pulling leftward while send does the opposite. This double duty is the crux of the confusion surrounding the semantics of statements like x = yield y. Breaking yield and send down into two explicit push/pull steps will make their semantics much more clear:
Suppose g is the generator. g.send pushes a value leftward through the right end of the pipe.
Execution within the context of g pauses, allowing the generator function's body to run.
The value pushed by g.send is pulled leftward by yield and received on the left end of the pipe. In x = yield y, x is assigned to the pulled value.
Execution continues within the generator function's body until the next line containing yield is reached.
yield pushes a value rightward through the left end of the pipe, back up to g.send. In x = yield y, y is pushed rightward through the pipe.
Execution within the generator function's body pauses, allowing the outer scope to continue where it left off.
g.send resumes and pulls the value and returns it to the user.
When g.send is next called, go back to Step 1.
While cyclical, this procedure does have a beginning: when g.send(None) -- which is what next(g) is short for -- is first called (it is illegal to pass something other than None to the first send call). And it may have an end: when there are no more yield statements to be reached in the generator function's body.
Do you see what makes the yield statement (or more accurately, generators) so special? Unlike the measly return keyword, yield is able to pass values to its caller and receive values from its caller all without terminating the function it lives in! (Of course, if you do wish to terminate a function -- or a generator -- it's handy to have the return keyword as well.) When a yield statement is encountered, the generator function merely pauses, and then picks back up right where it left off upon being sent another value. And send is just the interface for communicating with the inside of a generator function from outside it.
If we really want to break this push/pull/pipe analogy down as far as we can, we end up with the following pseudocode that really drives home that, aside from steps 1-5, yield and send are two sides of the same coin pipe:
right_end.push(None) # the first half of g.send; sending None is what starts a generator
right_end.pause()
left_end.start()
initial_value = left_end.pull()
if initial_value is not None: raise TypeError("can't send non-None value to a just-started generator")
left_end.do_stuff()
left_end.push(y) # the first half of yield
left_end.pause()
right_end.resume()
value1 = right_end.pull() # the second half of g.send
right_end.do_stuff()
right_end.push(value2) # the first half of g.send (again, but with a different value)
right_end.pause()
left_end.resume()
x = left_end.pull() # the second half of yield
goto 6
The key transformation is that we have split x = yield y and value1 = g.send(value2) each into two statements: left_end.push(y) and x = left_end.pull(); and value1 = right_end.pull() and right_end.push(value2). There are two special cases of the yield keyword: x = yield and yield y. These are syntactic sugar, respectively, for x = yield None and _ = yield y # discarding value.
For specific details regarding the precise order in which values are sent through the pipe, see below.
What follows is a rather long concrete model of the above. First, it should first be noted that for any generator g, next(g) is exactly equivalent to g.send(None). With this in mind we can focus only on how send works and talk only about advancing the generator with send.
Suppose we have
def f(y): # This is the "generator function" referenced above
while True:
x = yield y
y = x
g = f(1)
g.send(None) # yields 1
g.send(2) # yields 2
Now, the definition of f roughly desugars to the following ordinary (non-generator) function:
def f(y):
bidirectional_pipe = BidirectionalPipe()
left_end = bidirectional_pipe.left_end
right_end = bidirectional_pipe.right_end
def impl():
initial_value = left_end.pull()
if initial_value is not None:
raise TypeError(
"can't send non-None value to a just-started generator"
)
while True:
left_end.push(y)
x = left_end.pull()
y = x
def send(value):
right_end.push(value)
return right_end.pull()
right_end.send = send
# This isn't real Python; normally, returning exits the function. But
# pretend that it's possible to return a value from a function and then
# continue execution -- this is exactly the problem that generators were
# designed to solve!
return right_end
impl()
The following has happened in this transformation of f:
We've moved the implementation into a nested function.
We've created a bidirectional pipe whose left_end will be accessed by the nested function and whose right_end will be returned and accessed by the outer scope -- right_end is what we know as the generator object.
Within the nested function, the very first thing we do is check that left_end.pull() is None, consuming a pushed value in the process.
Within the nested function, the statement x = yield y has been replaced by two lines: left_end.push(y) and x = left_end.pull().
We've defined the send function for right_end, which is the counterpart to the two lines we replaced the x = yield y statement with in the previous step.
In this fantasy world where functions can continue after returning, g is assigned right_end and then impl() is called. So in our example above, were we to follow execution line by line, what would happen is roughly the following:
left_end = bidirectional_pipe.left_end
right_end = bidirectional_pipe.right_end
y = 1 # from g = f(1)
# None pushed by first half of g.send(None)
right_end.push(None)
# The above push blocks, so the outer scope halts and lets `f` run until
# *it* blocks
# Receive the pushed value, None
initial_value = left_end.pull()
if initial_value is not None: # ok, `g` sent None
raise TypeError(
"can't send non-None value to a just-started generator"
)
left_end.push(y)
# The above line blocks, so `f` pauses and g.send picks up where it left off
# y, aka 1, is pulled by right_end and returned by `g.send(None)`
right_end.pull()
# Rinse and repeat
# 2 pushed by first half of g.send(2)
right_end.push(2)
# Once again the above blocks, so g.send (the outer scope) halts and `f` resumes
# Receive the pushed value, 2
x = left_end.pull()
y = x # y == x == 2
left_end.push(y)
# The above line blocks, so `f` pauses and g.send(2) picks up where it left off
# y, aka 2, is pulled by right_end and returned to the outer scope
right_end.pull()
x = left_end.pull()
# blocks until the next call to g.send
This maps exactly to the 16-step pseudocode above.
There are some other details, like how errors are propagated and what happens when you reach the end of the generator (the pipe is closed), but this should make clear how the basic control flow works when send is used.
Using these same desugaring rules, let's look at two special cases:
def f1(x):
while True:
x = yield x
def f2(): # No parameter
while True:
x = yield x
For the most part they desugar the same way as f, the only differences are how the yield statements are transformed:
def f1(x):
# ... set up pipe
def impl():
# ... check that initial sent value is None
while True:
left_end.push(x)
x = left_end.pull()
# ... set up right_end
def f2():
# ... set up pipe
def impl():
# ... check that initial sent value is None
while True:
left_end.push(x)
x = left_end.pull()
# ... set up right_end
In the first, the value passed to f1 is pushed (yielded) initially, and then all values pulled (sent) are pushed (yielded) right back. In the second, x has no value (yet) when it first come times to push, so an UnboundLocalError is raised.

These confused me too. Here is an example I made when trying to set up a generator which yields and accepts signals in alternating order (yield, accept, yield, accept)...
def echo_sound():
thing_to_say = '<Sound of wind on cliffs>'
while True:
thing_to_say = (yield thing_to_say)
thing_to_say = '...'.join([thing_to_say]+[thing_to_say[-6:]]*2)
yield None # This is the return value of send.
gen = echo_sound()
print 'You are lost in the wilderness, calling for help.'
print '------'
in_message = gen.next()
print 'You hear: "{}"'.format(in_message)
out_message = 'Hello!'
print 'You yell "{}"'.format(out_message)
gen.send(out_message)
print '------'
in_message = gen.next()
print 'You hear: "{}"'.format(in_message)
out_message = 'Is anybody out there?'
print 'You yell "{}"'.format(out_message)
gen.send(out_message)
print '------'
in_message = gen.next()
print 'You hear: "{}"'.format(in_message)
out_message = 'Help!'
print 'You yell "{}"'.format(out_message)
gen.send(out_message)
The output is:
You are lost in the wilderness, calling for help.
------
You hear: "<Sound of wind on cliffs>"
You yell "Hello!"
------
You hear: "Hello!...Hello!...Hello!"
You yell "Is anybody out there?"
------
You hear: "Is anybody out there?...there?...there?"
You yell "Help!"

itr.send(None) is the same thing as next(itr) and what you're doing is giving the value given by yield in the generator.
Here's an example that clearly shows this, and how it can be used more practically.
def iterator_towards(dest=100):
value = 0
while True:
n = yield value
if n is not None:
dest = n
if dest > value:
value += 1
elif dest < value:
value -= 1
else:
return
num = iterator_towards()
for i in num:
print(i)
if i == 5:
num.send(0)
This will print:
0
1
2
3
4
5
3
2
1
0
The code at i == 5 tells it to send 0. This is not None in the iterator_towards and so it changes the value of dest. We then iterate towards 0.
However note, there is no value 4 after the value 5. This is because the nature of .send(0) is that it was yielded the 4 value and that was not printed.
If we add a continue we can re-yield the same value.
def iterator_towards(dest=100):
value = 0
while True:
n = yield value
if n is not None:
dest = n
continue
if dest > value:
value += 1
elif dest < value:
value -= 1
else:
return
Which will allow you to iterate a list but also dynamically send it new destination values onthefly.

What is the purpose of the "send" function on Python generators?

I understand yield. But what does a generator's send function do? The documentation says:
generator.send(value)
Resumes the execution and “sends” a value into the generator function. The value argument becomes the result of the current yield expression. The send() method returns the next value yielded by the generator, or raises StopIteration if the generator exits without yielding another value.
What does that mean? I thought value was the input to the generator function? The phrase "The send() method returns the next value yielded by the generator" seems to be also the exact purpose of yield, which also returns the next value yielded by the generator.
Is there an example of a generator utilizing send that accomplishes something yield cannot?

It's used to send values into a generator that just yielded. Here is an artificial (non-useful) explanatory example:
>>> def double_inputs():
... while True:
... x = yield
... yield x * 2
...
>>> gen = double_inputs()
>>> next(gen) # run up to the first yield
>>> gen.send(10) # goes into 'x' variable
20
>>> next(gen) # run up to the next yield
>>> gen.send(6) # goes into 'x' again
12
>>> next(gen) # run up to the next yield
>>> gen.send(94.3) # goes into 'x' again
188.5999999999999
You can't do this just with yield.
As to why it's useful, one of the best use cases I've seen is Twisted's #defer.inlineCallbacks. Essentially it allows you to write a function like this:
#defer.inlineCallbacks
def doStuff():
result = yield takesTwoSeconds()
nextResult = yield takesTenSeconds(result * 10)
defer.returnValue(nextResult / 10)
What happens is that takesTwoSeconds() returns a Deferred, which is a value promising a value will be computed later. Twisted can run the computation in another thread. When the computation is done, it passes it into the deferred, and the value then gets sent back to the doStuff() function. Thus the doStuff() can end up looking more or less like a normal procedural function, except it can be doing all sorts of computations & callbacks etc. The alternative before this functionality would be to do something like:
def doStuff():
returnDeferred = defer.Deferred()
def gotNextResult(nextResult):
returnDeferred.callback(nextResult / 10)
def gotResult(result):
takesTenSeconds(result * 10).addCallback(gotNextResult)
takesTwoSeconds().addCallback(gotResult)
return returnDeferred
It's a lot more convoluted and unwieldy.

This function is to write coroutines
def coroutine():
for i in range(1, 10):
print("From generator {}".format((yield i)))
c = coroutine()
c.send(None)
try:
while True:
print("From user {}".format(c.send(1)))
except StopIteration: pass
prints
From generator 1
From user 2
From generator 1
From user 3
From generator 1
From user 4
...
See how the control is being passed back and forth? Those are coroutines. They can be used for all kinds of cool things like asynch IO and similar.
Think of it like this, with a generator and no send, it's a one way street
========== yield ========
Generator | ------------> | User |
========== ========
But with send, it becomes a two way street
========== yield ========
Generator | ------------> | User |
========== <------------ ========
send
Which opens up the door to the user customizing the generators behavior on the fly and the generator responding to the user.

This may help someone. Here is a generator that is unaffected by send function. It takes in the number parameter on instantiation and is unaffected by send:
>>> def double_number(number):
... while True:
... number *=2
... yield number
...
>>> c = double_number(4)
>>> c.send(None)
8
>>> c.next()
16
>>> c.next()
32
>>> c.send(8)
64
>>> c.send(8)
128
>>> c.send(8)
256
Now here is how you would do the same type of function using send, so on each iteration you can change the value of number:
def double_number(number):
while True:
number *= 2
number = yield number
Here is what that looks like, as you can see sending a new value for number changes the outcome:
>>> def double_number(number):
... while True:
... number *= 2
... number = yield number
...
>>> c = double_number(4)
>>>
>>> c.send(None)
8
>>> c.send(5) #10
10
>>> c.send(1500) #3000
3000
>>> c.send(3) #6
6
You can also put this in a for loop as such:
for x in range(10):
n = c.send(n)
print n
For more help check out this great tutorial.

The send() method controls what the value to the left of the yield expression will be.
To understand how yield differs and what value it holds, lets first quickly refresh on the order python code is evaluated.
Section 6.15 Evaluation order
Python evaluates expressions from left to right. Notice that while evaluating an assignment, the right-hand side is evaluated before the left-hand side.
So an expression a = b the right hand side is evaluated first.
As the following demonstrates that a[p('left')] = p('right') the right hand side is evaluated first.
>>> def p(side):
... print(side)
... return 0
...
>>> a[p('left')] = p('right')
right
left
>>>
>>>
>>> [p('left'), p('right')]
left
right
[0, 0]
What does yield do?, yield, suspends execution of the function and returns to the caller, and resumes execution at the same place it left off prior to suspending.
Where exactly is execution suspended? You might have guessed it already...
the execution is suspended between the right and left side of the yield expression. So new_val = yield old_val the execution is halted at the = sign, and the value on the right (which is before suspending, and is also the value returned to the caller) may be something different then the value on the left (which is the value being assigned after resuming execution).
yield yields 2 values, one to the right and another to the left.
How do you control the value to the left hand side of the yield expression? via the .send() method.
6.2.9. Yield expressions
The value of the yield expression after resuming depends on the method which resumed the execution. If __next__() is used (typically via either a for or the next() builtin) then the result is None. Otherwise, if send() is used, then the result will be the value passed in to that method.

Some use cases for using generator and send()
Generators with send() allow:
remembering internal state of the execution
what step we are at
what is current status of our data
returning sequence of values
receiving sequence of inputs
Here are some use cases:
Watched attempt to follow a recipe
Let us have a recipe, which expects predefined set of inputs in some order.
We may:
create a watched_attempt instance from the recipe
let it get some inputs
with each input return information about what is currently in the pot
with each input check, that the input is the expected one (and fail if it is not)
def recipe():
pot = []
action = yield pot
assert action == ("add", "water")
pot.append(action[1])
action = yield pot
assert action == ("add", "salt")
pot.append(action[1])
action = yield pot
assert action == ("boil", "water")
action = yield pot
assert action == ("add", "pasta")
pot.append(action[1])
action = yield pot
assert action == ("decant", "water")
pot.remove("water")
action = yield pot
assert action == ("serve")
pot = []
yield pot
To use it, first create the watched_attempt instance:
>>> watched_attempt = recipe()
>>> watched_attempt.next()
[]
The call to .next() is necessary to start execution of the generator.
Returned value shows, our pot is currently empty.
Now do few actions following what the recipe expects:
>>> watched_attempt.send(("add", "water"))
['water']
>>> watched_attempt.send(("add", "salt"))
['water', 'salt']
>>> watched_attempt.send(("boil", "water"))
['water', 'salt']
>>> watched_attempt.send(("add", "pasta"))
['water', 'salt', 'pasta']
>>> watched_attempt.send(("decant", "water"))
['salt', 'pasta']
>>> watched_attempt.send(("serve"))
[]
As we see, the pot is finally empty.
In case, one would not follow the recipe, it would fail (what could be desired outcome of watched
attempt to cook something - just learning we did not pay enough attention when given instructions.
>>> watched_attempt = running.recipe()
>>> watched_attempt.next()
[]
>>> watched_attempt.send(("add", "water"))
['water']
>>> watched_attempt.send(("add", "pasta"))
---------------------------------------------------------------------------
AssertionError Traceback (most recent call last)
<ipython-input-21-facdf014fe8e> in <module>()
----> 1 watched_attempt.send(("add", "pasta"))
/home/javl/sandbox/stack/send/running.py in recipe()
29
30 action = yield pot
---> 31 assert action == ("add", "salt")
32 pot.append(action[1])
33
AssertionError:
Notice, that:
there is linear sequence of expected steps
the steps may differ (some are removing, some are adding to the pot)
we manage to do all that by a functions/generator - no need to use complex class or similar
strutures.
Running totals
We may use the generator to keep track of running total of values sent to it.
Any time we add a number, count of inputs and total sum is returned (valid for
the moment previous input was send into it).
from collections import namedtuple
RunningTotal = namedtuple("RunningTotal", ["n", "total"])
def runningtotals(n=0, total=0):
while True:
delta = yield RunningTotal(n, total)
if delta:
n += 1
total += delta
if __name__ == "__main__":
nums = [9, 8, None, 3, 4, 2, 1]
bookeeper = runningtotals()
print bookeeper.next()
for num in nums:
print num, bookeeper.send(num)
The output would look like:
RunningTotal(n=0, total=0)
9 RunningTotal(n=1, total=9)
8 RunningTotal(n=2, total=17)
None RunningTotal(n=2, total=17)
3 RunningTotal(n=3, total=20)
4 RunningTotal(n=4, total=24)
2 RunningTotal(n=5, total=26)
1 RunningTotal(n=6, total=27)

The send method implements coroutines.
If you haven't encountered Coroutines they are tricky to wrap your head around because they change the way a program flows. You can read a good tutorial for more details.

The word "yield" has two meanings: to produce something (e.g., to yield corn), and to halt to let someone/thing else continue (e.g., cars yielding to pedestrians). Both definitions apply to Python's yield keyword; what makes generator functions special is that unlike in regular functions, values can be "returned" to the caller while merely pausing, not terminating, a generator function.
It is easiest to imagine a generator as one end of a bidirectional pipe with a "left" end and a "right" end; this pipe is the medium over which values are sent between the generator itself and the generator function's body. Each end of the pipe has two operations: push, which sends a value and blocks until the other end of the pipe pulls the value, and returns nothing; and pull, which blocks until the other end of the pipe pushes a value, and returns the pushed value. At runtime, execution bounces back and forth between the contexts on either side of the pipe -- each side runs until it sends a value to the other side, at which point it halts, lets the other side run, and waits for a value in return, at which point the other side halts and it resumes. In other words, each end of the pipe runs from the moment it receives a value to the moment it sends a value.
The pipe is functionally symmetric, but -- by convention I'm defining in this answer -- the left end is only available inside the generator function's body and is accessible via the yield keyword, while the right end is the generator and is accessible via the generator's send function. As singular interfaces to their respective ends of the pipe, yield and send do double duty: they each both push and pull values to/from their ends of the pipe, yield pushing rightward and pulling leftward while send does the opposite. This double duty is the crux of the confusion surrounding the semantics of statements like x = yield y. Breaking yield and send down into two explicit push/pull steps will make their semantics much more clear:
Suppose g is the generator. g.send pushes a value leftward through the right end of the pipe.
Execution within the context of g pauses, allowing the generator function's body to run.
The value pushed by g.send is pulled leftward by yield and received on the left end of the pipe. In x = yield y, x is assigned to the pulled value.
Execution continues within the generator function's body until the next line containing yield is reached.
yield pushes a value rightward through the left end of the pipe, back up to g.send. In x = yield y, y is pushed rightward through the pipe.
Execution within the generator function's body pauses, allowing the outer scope to continue where it left off.
g.send resumes and pulls the value and returns it to the user.
When g.send is next called, go back to Step 1.
While cyclical, this procedure does have a beginning: when g.send(None) -- which is what next(g) is short for -- is first called (it is illegal to pass something other than None to the first send call). And it may have an end: when there are no more yield statements to be reached in the generator function's body.
Do you see what makes the yield statement (or more accurately, generators) so special? Unlike the measly return keyword, yield is able to pass values to its caller and receive values from its caller all without terminating the function it lives in! (Of course, if you do wish to terminate a function -- or a generator -- it's handy to have the return keyword as well.) When a yield statement is encountered, the generator function merely pauses, and then picks back up right where it left off upon being sent another value. And send is just the interface for communicating with the inside of a generator function from outside it.
If we really want to break this push/pull/pipe analogy down as far as we can, we end up with the following pseudocode that really drives home that, aside from steps 1-5, yield and send are two sides of the same coin pipe:
right_end.push(None) # the first half of g.send; sending None is what starts a generator
right_end.pause()
left_end.start()
initial_value = left_end.pull()
if initial_value is not None: raise TypeError("can't send non-None value to a just-started generator")
left_end.do_stuff()
left_end.push(y) # the first half of yield
left_end.pause()
right_end.resume()
value1 = right_end.pull() # the second half of g.send
right_end.do_stuff()
right_end.push(value2) # the first half of g.send (again, but with a different value)
right_end.pause()
left_end.resume()
x = left_end.pull() # the second half of yield
goto 6
The key transformation is that we have split x = yield y and value1 = g.send(value2) each into two statements: left_end.push(y) and x = left_end.pull(); and value1 = right_end.pull() and right_end.push(value2). There are two special cases of the yield keyword: x = yield and yield y. These are syntactic sugar, respectively, for x = yield None and _ = yield y # discarding value.
For specific details regarding the precise order in which values are sent through the pipe, see below.
What follows is a rather long concrete model of the above. First, it should first be noted that for any generator g, next(g) is exactly equivalent to g.send(None). With this in mind we can focus only on how send works and talk only about advancing the generator with send.
Suppose we have
def f(y): # This is the "generator function" referenced above
while True:
x = yield y
y = x
g = f(1)
g.send(None) # yields 1
g.send(2) # yields 2
Now, the definition of f roughly desugars to the following ordinary (non-generator) function:
def f(y):
bidirectional_pipe = BidirectionalPipe()
left_end = bidirectional_pipe.left_end
right_end = bidirectional_pipe.right_end
def impl():
initial_value = left_end.pull()
if initial_value is not None:
raise TypeError(
"can't send non-None value to a just-started generator"
)
while True:
left_end.push(y)
x = left_end.pull()
y = x
def send(value):
right_end.push(value)
return right_end.pull()
right_end.send = send
# This isn't real Python; normally, returning exits the function. But
# pretend that it's possible to return a value from a function and then
# continue execution -- this is exactly the problem that generators were
# designed to solve!
return right_end
impl()
The following has happened in this transformation of f:
We've moved the implementation into a nested function.
We've created a bidirectional pipe whose left_end will be accessed by the nested function and whose right_end will be returned and accessed by the outer scope -- right_end is what we know as the generator object.
Within the nested function, the very first thing we do is check that left_end.pull() is None, consuming a pushed value in the process.
Within the nested function, the statement x = yield y has been replaced by two lines: left_end.push(y) and x = left_end.pull().
We've defined the send function for right_end, which is the counterpart to the two lines we replaced the x = yield y statement with in the previous step.
In this fantasy world where functions can continue after returning, g is assigned right_end and then impl() is called. So in our example above, were we to follow execution line by line, what would happen is roughly the following:
left_end = bidirectional_pipe.left_end
right_end = bidirectional_pipe.right_end
y = 1 # from g = f(1)
# None pushed by first half of g.send(None)
right_end.push(None)
# The above push blocks, so the outer scope halts and lets `f` run until
# *it* blocks
# Receive the pushed value, None
initial_value = left_end.pull()
if initial_value is not None: # ok, `g` sent None
raise TypeError(
"can't send non-None value to a just-started generator"
)
left_end.push(y)
# The above line blocks, so `f` pauses and g.send picks up where it left off
# y, aka 1, is pulled by right_end and returned by `g.send(None)`
right_end.pull()
# Rinse and repeat
# 2 pushed by first half of g.send(2)
right_end.push(2)
# Once again the above blocks, so g.send (the outer scope) halts and `f` resumes
# Receive the pushed value, 2
x = left_end.pull()
y = x # y == x == 2
left_end.push(y)
# The above line blocks, so `f` pauses and g.send(2) picks up where it left off
# y, aka 2, is pulled by right_end and returned to the outer scope
right_end.pull()
x = left_end.pull()
# blocks until the next call to g.send
This maps exactly to the 16-step pseudocode above.
There are some other details, like how errors are propagated and what happens when you reach the end of the generator (the pipe is closed), but this should make clear how the basic control flow works when send is used.
Using these same desugaring rules, let's look at two special cases:
def f1(x):
while True:
x = yield x
def f2(): # No parameter
while True:
x = yield x
For the most part they desugar the same way as f, the only differences are how the yield statements are transformed:
def f1(x):
# ... set up pipe
def impl():
# ... check that initial sent value is None
while True:
left_end.push(x)
x = left_end.pull()
# ... set up right_end
def f2():
# ... set up pipe
def impl():
# ... check that initial sent value is None
while True:
left_end.push(x)
x = left_end.pull()
# ... set up right_end
In the first, the value passed to f1 is pushed (yielded) initially, and then all values pulled (sent) are pushed (yielded) right back. In the second, x has no value (yet) when it first come times to push, so an UnboundLocalError is raised.

These confused me too. Here is an example I made when trying to set up a generator which yields and accepts signals in alternating order (yield, accept, yield, accept)...
def echo_sound():
thing_to_say = '<Sound of wind on cliffs>'
while True:
thing_to_say = (yield thing_to_say)
thing_to_say = '...'.join([thing_to_say]+[thing_to_say[-6:]]*2)
yield None # This is the return value of send.
gen = echo_sound()
print 'You are lost in the wilderness, calling for help.'
print '------'
in_message = gen.next()
print 'You hear: "{}"'.format(in_message)
out_message = 'Hello!'
print 'You yell "{}"'.format(out_message)
gen.send(out_message)
print '------'
in_message = gen.next()
print 'You hear: "{}"'.format(in_message)
out_message = 'Is anybody out there?'
print 'You yell "{}"'.format(out_message)
gen.send(out_message)
print '------'
in_message = gen.next()
print 'You hear: "{}"'.format(in_message)
out_message = 'Help!'
print 'You yell "{}"'.format(out_message)
gen.send(out_message)
The output is:
You are lost in the wilderness, calling for help.
------
You hear: "<Sound of wind on cliffs>"
You yell "Hello!"
------
You hear: "Hello!...Hello!...Hello!"
You yell "Is anybody out there?"
------
You hear: "Is anybody out there?...there?...there?"
You yell "Help!"

itr.send(None) is the same thing as next(itr) and what you're doing is giving the value given by yield in the generator.
Here's an example that clearly shows this, and how it can be used more practically.
def iterator_towards(dest=100):
value = 0
while True:
n = yield value
if n is not None:
dest = n
if dest > value:
value += 1
elif dest < value:
value -= 1
else:
return
num = iterator_towards()
for i in num:
print(i)
if i == 5:
num.send(0)
This will print:
0
1
2
3
4
5
3
2
1
0
The code at i == 5 tells it to send 0. This is not None in the iterator_towards and so it changes the value of dest. We then iterate towards 0.
However note, there is no value 4 after the value 5. This is because the nature of .send(0) is that it was yielded the 4 value and that was not printed.
If we add a continue we can re-yield the same value.
def iterator_towards(dest=100):
value = 0
while True:
n = yield value
if n is not None:
dest = n
continue
if dest > value:
value += 1
elif dest < value:
value -= 1
else:
return
Which will allow you to iterate a list but also dynamically send it new destination values onthefly.

Develop Reference

Python is a programming language that lets you work quickly and integrate systems more effectively.

Python: I don't understand what's happening with this generator - python

Related

In a generator, equality to yield [duplicate]

What is the python syntax when a variable is after 'yield'? [duplicate]

python3 send() function in generators

How does "line = (yield)" work? [duplicate]

What is the purpose of the "send" function on Python generators?

Categories

Resources