How does this code, involving assignment and the yield operator, work? The results are rather confounding.
def test1(x):
for i in x:
_ = yield i
yield _
def test2(x):
for i in x:
_ = yield i
r1 = test1([1,2,3])
r2 = test2([1,2,3])
print list(r1)
print list(r2)
Output:
[1, None, 2, None, 3, None]
[1, 2, 3]
The assignment syntax ("yield expression") allows you to treat the generator as a rudimentary coroutine.
First proposed in PEP 342 and documented here: https://docs.python.org/2/reference/expressions.html#yield-expressions
The client code that is working with the generator can communicate data back into the generator using its send() method. That data is accessible via the assignment syntax.
send() will also iterate - so it actually includes a next() call.
Using your example, this is what it would be like to use the couroutine functionality:
>>> def test1(x):
... for i in x:
... _ = yield i
... yield _
...
>>> l = [1,2,3]
>>> gen_instance = test1(l)
>>> #First send has to be a None
>>> print gen_instance.send(None)
1
>>> print gen_instance.send("A")
A
>>> print gen_instance.send("B")
2
>>> print gen_instance.send("C")
C
>>> print gen_instance.send("D")
3
>>> print gen_instance.send("E")
E
>>> print gen_instance.send("F")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
StopIteration
Note that some of the sends are lost because of the second yield in each loop iteration that doesn't capture the sent data.
EDIT:
Forgot to explain the Nones yielded in your example.
From https://docs.python.org/2/reference/expressions.html#generator.next:
When a generator function is resumed with a next() method, the current
yield expression always evaluates to None.
next() is used when using the iteration syntax.
_ = yield i
yield _
First it yields the value referenced by i, e.g. 1. Then it yields the value returned by the yield operation, which is None. It does this on each iteration of the loop.
for i in x:
_ = yield i
This simply yields the value referenced by i, e.g. 1, then proceeds to the next iteration of the loop, producing 2, then 3.
Unlike return, the yield keyword can be used in an expression:
x = return 0 # SyntaxError
x = yield 0 # perfectly fine
Now, when the interpreter sees a yield, it will generate the indicated value. However, when it does so, that operation returns the value None, just like mylist.append(0) or print('hello') will return the value None. When you assign that result to a reference like _, you're saving that None.
So, in the first snippet, you're yielding an object, then you save the "result" of that yield operation, which is None, and then you yield that None. In the second snippet, you yield an object, then you save the "result" of that yield operation, but you never yield that result, so None does not appear in the output.
Note that yield won't always return None - this is just what you sent to the generator with send(). Since that was nothing in this case, you get None. See this answer for more on send().
To expand on TigerhawkT3's answer, the reason that the yield operation is returning None in your code is because list(r1) isn't sending anything into the generator. Try this:
def test1(x):
for i in x:
_ = yield i
yield _
r1 = test1([1, 2, 3])
for x in r1:
print(' x', x)
print('send', r1.send('hello!'))
Output:
x 1
send hello!
x 2
send hello!
x 3
send hello!
Here's a somewhat manufactured example where sending values into a generator could be useful:
def changeable_count(start=0):
current = start
while True:
changed_current = yield current
if changed_current:
current = changed_current
else:
current += 1
counter = changeable_count(10)
for x in range(20):
print(next(counter), end=' ')
print()
print()
print('Sending 51, printing return value:', counter.send(51))
print()
for x in range(20):
print(next(counter), end=' ')
print()
print()
print('Sending 42, NOT printing return value')
print()
counter.send(42)
for x in range(20):
print(next(counter), end=' ')
print()
Output:
10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
Sending 51, printing return value: 51
52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71
Sending 42, NOT printing return value
43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62
Related
I'm trying to writing some codes that defines the outer function including inner function.
def outer():
t = 1
for i in range(3):
print(f'## {i}th loop##')
t = inner(t)
return t
def inner(t):
print(f't value is now {t}')
t += 1
if (t % 10) != 0: # imaginary if-condition to reproduce my original codes
inner(t)
return t
outer()
While doing this, I confronted the problem that implementing inner function few times does not change the variable value that was defined with the definition of outer function at the same time.
The result of codes above is:
The result
In first iter, t : 1,2,3,4,5,6,7,8,9
In second iter, t : 2,3,4,5,6,7,8,9 ..
I expected that the t value in the second iteration will start with the value of 10. Would u let me know what the problem is?
The problem is how you are calling:
outer -> inner -> inner -> last inner, etc.
Resulting in t += 1 at the end, from the last inner call.
If you want to get it functional, and still use inner and outer you can do this:
def outer():
t = 0
for i in range(3):
print(f'## {i}th loop##')
t = inner(t)
def inner(num):
for j in range(num, num + 10):
print(f't value is now {j}')
num += 1
return num
outer()
This issue is in line:
inner(t)
Changes in the recursive calls has no effect on the parent--see How do I pass a variable by reference?.
This can be fixed by changing to:
t = inner(t)
Codes becomes:
def outer():
t = 1
for i in range(3):
print(f'## {i}th loop##')
t = inner(t)
return t
def inner(t):
print(f't value is now {t}')
t += 1 # equivalent to t = t + 1
# which creates a new local value of t and assigns it to passed in argument t + 1
if (t % 10) != 0: # imaginary if-condition to reproduce my original codes
t = inner(t)
return t
outer()
Output
## 0th loop##
t value is now 1
t value is now 2
t value is now 3
t value is now 4
t value is now 5
t value is now 6
t value is now 7
t value is now 8
t value is now 9
## 1th loop##
t value is now 10
t value is now 11
t value is now 12
t value is now 13
t value is now 14
t value is now 15
t value is now 16
t value is now 17
t value is now 18
t value is now 19
## 2th loop##
t value is now 20
t value is now 21
t value is now 22
t value is now 23
t value is now 24
t value is now 25
t value is now 26
t value is now 27
t value is now 28
t value is now 29
I know that this error message 'TypeError: 'NoneType' object is not iterable' means that there is None is the data. However, I am looking through all of my list and there is no part which has a no value element.
This is the section of my code that correlates to my issue.
def printList(heights):
print(" ".join(str(h) for h in heights))
def nextYear(heights):
next_heights = []
for i in range(len(heights)):
next_heights.append(heights[i] + 5)
i += 1
print(" ".join(str(h) for h in next_heights))
#main routine
heights = [33, 45, 23, 43, 48, 32, 35, 46, 48, 39, 41]
printList(heights)
print("Predicted heights after a year")
printList(nextYear(heights))
This is my codes output:
33 45 23 43 48 32 35 46 48 39 41
Predicted heights after a year
38 50 28 48 53 37 40 51 53 44 46
Traceback (most recent call last):
File "/Users/ellarendell/Desktop/test.py", line 17, in <module>
printList(nextYear(heights))
File "/Users/ellarendell/Desktop/test.py", line 2, in printList
print(" ".join(str(h) for h in heights))
TypeError: 'NoneType' object is not iterable
I want my code to do the same output without the error message.
Do you know what part of the list could have 'None'?
Thank you :)
There are two things wrong with your code:
First you are not returning next_heights in your function
def nextYear(heights):
next_heights = []
for i in range(len(heights)):
next_heights.append(heights[i] + 5)
i += 1
return next_heights
Without the return line, it will return None and pass it to printList function, also you do not need to print inside nextYear as you already calling printList to print after the function returns the highlights
Second thing is increasing the iterator, you literally cannot do that, you could try this little example to understand what I mean
for i in range(10):
print(i)
i +=15555
So first thing to do is to remove this line from your loop
def nextYear(heights):
next_heights = []
for i in range(len(heights)):
next_heights.append(heights[i] + 5)
return next_heights
It will be incremented automatically every iteration, if you want it to be increased by two instead of one then you can specify that in range() function as a step size.
You're not returning anything in the nextYear function, which is why the argument heights in the printList function is None.
You are missing a return statement in nextYear.
Your function should be:
def nextYear(heights):
next_heights = []
for i in range(len(heights)):
next_heights.append(heights[i] + 5)
i += 1
print(" ".join(str(h) for h in next_heights))
return next_heights
... at least from my understanding of what you want nextYear to do. In any case, you need a return statement.
My output is:
33 45 23 43 48 32 35 46 48 39 41
Predicted heights after a year
38 50 28 48 53 37 40 51 53 44 46
38 50 28 48 53 37 40 51 53 44 46
For clarification, the None is coming from the return of your nextYear function.
In your line printList(nextYear(heights)), nextYear(heights) returns None, which is passed to printList.
I am trying to write a code for class that identifies the probability that two or more people given can have the same birthday out of a group by running this experiment 10^6 times and then calculating the fraction of times during which two or more people had the same birthday. I attached the code below, but when I try and run it absolutely nothing happens. It gives no error message, it just stops working. Does anyone have an idea why?
-
import random
def calc_birthday_probability (num_people):
random.seed (2020) # Don't change this value
num_trials = 1000000
probability = 0
list1 = []
num_people = int(input())
repeats = 0
for i in range(0,num_trials+1):
for i in range (0,num_people+1):
list1.append(random.randint (1,3655))
for i in list1:
if list1.count(i)>1:
repeats +=1
i = i+1
i = i+1
prob = repeats//num_trials
probability = probability + prob
return probability
a = calc_birthday_probability(10)
print(a)
num_people = int(input())
Classic, you are already given the parameter in the function but you are retaking input for it. Your program simply waits for you to enter something.
Usually, a good trick when a python program does nothing despite it being expected to perform some kind of action is to check whether or not it is waiting for some kind of input :)
Edit #1: as #rkatkam noted, you are using the same loop variable (specifically, i) for both of the for loops.
In all the for loops in the function, you have used variable i and so its scope is improper to define too.
Something like following should work, it worked for me:
def calc_birthday_probability (num_people):
random.seed (2020) # Don't change this value
num_trials = 1000000
probability = 0
list1 = []
repeats = 0
for i in range(0,num_trials+1):
for j in range (0,num_people+1):
list1.append(random.randint (1,3655))
for k in list1:
if list1.count(j)>1:
repeats +=1
k = k+1
prob = repeats//num_trials
probability = probability + prob
return probability
And some tips:
Try testing your code with smaller num_trials initially till you find the results are accurate.
You have argument that accepts the value of num_people and the function also has an input() for the same.
When you trying to print output, print some other string as well along, to identify if the function finished its execution.
Simpler version of your code
import random
def calc_birthday_probability(number_of_people = 30, num_trials=1000):
dups_found = 0
for _ in range(num_trials):
birthdays = [random.randint (1,365) for _ in range(number_of_people)]
# set of birthdays which are duplicates
duplicates = set(x for x in birthdays if birthdays.count(x) > 1)
if len(duplicates) >= 1:
dups_found += 1 # increment since at least one duplicate
return number_of_people, dups_found/num_trials * 100
num_people = int(input("Number of people: "))
print(f'{calc_birthday_probability(num_people, 1000):.2f}%')
Test
Testing with only 1,000 trials since sufficient for results to compare with reference
for num_people in range(1, 52, 2):
print(f'{num_people} --> {calc_birthday_probability(num_people, 1000):.2f}%')
Output
1 --> 0.00%
3 --> 0.40%
5 --> 3.10%
7 --> 6.10%
9 --> 9.90%
11 --> 13.70%
13 --> 19.00%
15 --> 25.30%
17 --> 34.60%
19 --> 37.70%
21 --> 46.50%
23 --> 53.30%
25 --> 57.30%
27 --> 59.60%
29 --> 70.40%
31 --> 72.40%
33 --> 77.90%
35 --> 81.60%
37 --> 84.30%
39 --> 87.90%
41 --> 89.30%
43 --> 93.40%
45 --> 93.70%
47 --> 95.00%
49 --> 96.10%
51 --> 96.60%
21 def power(values):
22 print "power", values
23 for value in values:
24 print 'powering %s' % value
25 yield value
26
27 def adder(values):
28 print "adder", values
29 for value in values:
30 print 'adding to %s' % value
31 if value % 2 ==0:
32 yield value + 3
33 else:
34 yield value + 2
35
36 elements = [1, 4, 7, 9, 12, 19]
37 res = adder(power(elements))
38 print res.next()
39 print res.next()
40 print res.next()
41 print res.next()
OUTPUT:
adder <generator object power at 0x7fb6b9ee7910> <--- is this the stdout flush matter?
power [1, 4, 7, 9, 12, 19]
powering 1
adding to 1
3
powering 4
adding to 4
7
powering 7
adding to 7
9
powering 9
adding to 9
11
I am trying to understand the above code.
1) why did adder got printed before power [1, 4, 7, 9, 12, 19]?
2) adder is not iterating through elements but iterating through power generator, correct?
3) confirm my thought on (1). so adder gets called first then in for value in values for adder, it's consulting power generator and hence, powering print line gets triggered and then adder print line gets triggered?
4) If so, why doesnt the print statement power [1, 4, 7, 9, 12, 19] gets called along with the powering <$> print statement every time?
power has a yield return, which makes it a generator. The code in the function itself is only execute when next() is called on it.
correct. adder relies on the generator, and has no information on the data being iterated. (such as the size)
correct again
yield is a particular instruction. It does not return from the generator function (power). Instead, it provides one value, and suspend its execution until the next call to next(). At this point, the execution is resumed at the same point, which is inside the loop.
Edit
Illustration of the yield stopping point:
def gene():
print 'first!'
yield 1
print 'second!'
yield 2
g = gene()
g.next()
# first!
# 1
g.next()
# second!
# 2
As you can see, the generator is interrupted exactly after the yield instruction, ready to execute the next one
Generator functions don't begin execution until the first call to __next__ on the generator. For example:
>>> def gen():
... print 'starting execution'
... for i in range(10): yield i
...
>>> itr = gen()
>>> next(itr)
starting execution
0
So to answer you first question, "why did adder got printed before power", when you do adder(power(elements)) the generator power(elements) is created first, but the execution of that generator will not begin until the first iteration of for value in values within adder().
Code inside a generator is not run until next is called:
def gen():
print "called"
yield 3.14
g = gen() # nothing is printed
g.next() # print "called" and return 3.14
The for loop is doing the job of calling next here, in your code this is happening after the adder is printed:
g = gen()
print 'adder' # prints adder
for i in g: # prints called (as inside generator, *before* yields)
print i # prints 3.14
I have a minor problem while checking for elements in a list:
I have two files with contents something like this
file 1: file2:
47 358 47
48 450 49
49 56 50
I parsed both files into two lists and used the following code to check
for i in file_1:
for j in file_2:
j = j.split()
if i == j[1]:
x=' '.join(j)
print >> write_in, x
I am now trying to get a "0" if the value of file_1 is not there in file_2 for example, value "48" is not there is file_2 so I need to get the output like (with only one space in between the two numbers) Also both the conditions should produce only one output file:
output_file:
358 47
0 48
450 49
56 50
I tried using the dictionary approach but I didn't quite get what I wanted (actually I don't know how to use dictionary in python correctly ;)). Any help will be great.
r1=open('file1').read().split()
r2=open('file2').read().split()
d=dict(zip(r2[1::2],r2[::2]))
output='\n'.join(x in d and d[x]+' '+x or '0 '+x for x in r1)
open('output_file','wb').write(output)
Test
>>> file1='47\n48\n49\n50'
>>> file2='358 47\n450 49\n56 50'
>>>
>>> r1=file1.split()
>>> r2=file2.split()
>>>
>>> d=dict(zip(r2[1::2],r2[::2])) #
>>> d
{'47': '358', '50': '56', '49': '450'}
>>>
>>> print '\n'.join(x in d and d[x]+' '+x or '0 '+x for x in r1)
358 47
0 48
450 49
56 50
>>>
You could modify your code quite easily:
for i in file_1:
x = None
for j in file_2:
j = j.split()
if i == j[1]:
x = ' '.join(j)
if x is None:
x = ' '.join(['0', i])
Depending on your inputs, the whole task might be of course simplified even further. At the moment, your code is 0(n**2) complexity.
Here's a readable solution using a dictionary:
d = {}
for k in file1:
d[k] = 0
for line in file2:
v, k = line.split()
d[k] = v
for k in sorted(d):
print d[k], k
You can try something like:
l1 = open('file1').read().split()
l2 = [line.split() for line in open('file2')]
for x, y in zip(l1, l2):
if x not in y:
print 0, x
print ' '.join(y)
but if you follow your logic, the output should be
358 47
0 48
450 49
0 49
56 50
and not
358 47
0 48
450 49
56 50
def file_process(filename1, filename2):
# read first file with zeroes as values
with open(filename1) as fp:
adict= dict( (line.rstrip(), 0) for line in fp)
# read second file as "value key"
with open(filename2) as fp:
adict.update(
line.rstrip().partition(" ")[2::-2] # tricky, read notes
for line in fp)
for key in sorted(adict):
yield adict[key], key
fp= open("output_file", "w")
fp.writelines("%s %s\n" % items for items in file_process("file1", "file2"))
fp.close()
str.partition(" ") returns a tuple of (pre-space, space, post-space). By slicing the tuple, starting at item 2 (post-space) and moving by a step of -2, we return a tuple of (post-space, pre-space), which are (key, value) for the dictionary that describes the solution.
PS Um :) I just noticed that my answer is essentially the same as Daniel Stutzbach's.