In this PyCon talk, Jack Diederich shows this "simple" implementation of Conway's Game of Life. I am not intimately familiar with either GoL or semi-advanced Python, but the code seems quite easy to grasp, if not for two things:
The use of yield. I have seen the use of yield to create generators before, but eight of them in a row is new... Does it return a list of eight generators, or how does this thing work?
set(itertools.chain(*map(neighbors, board))). The star unpacks the resulting list (?) from applying neighbours to board, and ... my mind just blew.
Could someone try to explain these two parts for a programmer that is used to hacking together some python code using map, filter and reduce, but that is not using Python on a daily basis? :-)
import itertools
def neighbors(point):
x, y = point
yield x + 1, y
yield x - 1, y
yield x, y + 1
yield x, y - 1
yield x + 1, y + 1
yield x + 1, y - 1
yield x - 1, y + 1
yield x - 1, y - 1
def advance(board):
newstate = set()
recalc = board | set(itertools.chain(*map(neighbors, board)))
for point in recalc:
count = sum((neigh in board) for neigh in neighbors(point))
if count == 3 or (count == 2 and point in board):
newstate.add(point)
return newstate
glider = set([(0,0), (1,0), (2, 0), (0,1), (1,2)])
for i in range(1000):
glider = advance(glider)
print glider
Generators operate on two principles: they produce a value each time a yield statement is encountered, and unless it is iterated over, their code is paused.
It doesn't matter how many yield statements are used in a generator, the code is still run in normal python ordering. In this case, there is no loop, just a series of yield statements, so each time the generator is advanced, python executes the next line, which is another yield statement.
What happens with the neighbors generator is this:
Generators always start paused, so calling neighbors(position) returns a generator that hasn't done anything yet.
When it is advanced (next() is called on it), the code is run until the first yield statement. First x, y = point is executed, then x + 1, y is calculated and yielded. The code pauses again.
When advanced again, the code runs until the next yield statement is encountered. It yields x - 1, y.
etc. until the function completes.
The set(itertools.chain(*map(neighbors, board))) line does:
map(neighbors, board) produces an iterator for each and every position in the board sequence. It simply loops over board, calls neighbors on each value, and returns a new sequence of the results. Each neighbors() function returns a generator.
The *parameter syntax expands the parameter sequence into a list of parameters, as if the function was called with each element in parameter as a separate positional parameter instead. param = [1, 2, 3]; foo(*param) would translate to foo(1, 2, 3).
itertools.chain(*map(..)) takes each and every generator produced by the map, and applies that as a series of positional parameters to itertools.chain(). Looping over the output of chain means that each and every generator for each and every board position is iterated over once, in order.
All the generated positions are added to a set, essentially removing duplicates
You could expand the code to:
positions = set()
for board_position in board:
for neighbor in neighbors(board):
positions.add(neighbor)
In python 3, that line could be expressed a little more efficiently still by using itertools.chain.from_iterable() instead, because map() in Python 3 is a generator too; .from_iterable() doesn't force the map() to be expanded and will instead loop over the map() results one by one as needed.
Wow, that's a neat implementation, thanks for posting it !
For the yield, there is nothing to add to Martijn's answer.
As for the star : the map returns a generator or a list (depending on python 2 or 3), and each item of this list is a generator (from neighbors), so we have a list of generators.
chain takes many arguments that are iterables and chains them, meaning it returns a single iterable while iterate over all of them in turn.
Because we have a list of generators, and chain takes many arguments, we use a star to convert the list of generator to arguments. We could have done the same with chain.from_iterable.
it just returns a tuple of all cell's neighbours. If you do understand what generators do, it is pretty clear that using generators is a good practice when working with big amount of data. you do not need to store all this in memory, you calculate it only when you need it.
Related
I recently studied a python recursion function and found that the recursion stops when it uses element in []. So I made a simple test function, found that there is even no print out. So how can I understand the element in []? Why does the function stop when referring to element in []?
b=1
def simple():
for i in []:
print('i am here')
return i+b
a = simple()
Python's in keyword has two purposes.
One use in as part of a for loop, which is written for element in iterable. This assigns each value from iterable to element on each pass through the loop body. This is how your example function is using in (though since the list you're looping over is empty, the loop never does anything).
The other way you can use in is as an operator. An expression like x in y tests if element x is present in container y. (There's also a negated version of the in operator, not in. The expression x not in y is exactly equivalent to not (x in y).) I suspect this is what your recursive code is doing. This would also not be useful to do with an empty list literal (since an empty list by definition doesn't contain anything), but I'm guessing the real recursive function is a bit more complicated.
As an example of both uses of in, here's a generator function that uses a set to filter out duplicate items from some other iterable. It has a for loop that has in, and it also uses in (well, technically not in) as an operator to test if the next value from the input iterator is contained in the seen set:
def unique(iterable):
seen = set()
for item in iterable: # "in" used by for loop
if item not in seen: # "in" used here as an operator
yield item
seen.add(item)
A recursive function calls itself n-number of times, then returns a terminating value on the last recursion that backs out of the recursive stacks.
Example:
compute the factorial of a number:
def fact(n):
# ex: 5 * 4 * 3 * 2 * 1
# n == 0 is your terminating recursion
if n == 0:
return 1
# else is your recursion call to fact(n-1)
else:
return n * fact(n-1)
In your example, there is no recursive call to simple() within the function, nor are there any element inside the empty list [] to step through, therefore your for loop never executed
Its concerned about mechanism of 'for loop'.
Superficially, the iterator you want to travese (which is "[]" in you example) has a length of 0, so the body of the loop (which include "print" an so on) will not be executed.
Hope it helps.
The following is a simplified example of my code.
>>> def action(num):
print "Number is", num
>>> items = [1, 3, 6]
>>> for i in [j for j in items if j > 4]:
action(i)
Number is 6
My question is the following: is it bad practice (for reasons such as code clarity) to simply replace the for loop with a comprehension which will still call the action function? That is:
>>> (action(j) for j in items if j > 2)
Number is 6
This shouldn't use a generator or comprehension at all.
def action(num):
print "Number is", num
items = [1, 3, 6]
for j in items:
if j > 4:
action(i)
Generators evaluate lazily. The expression (action(j) for j in items if j > 2) will merely return a generator expression to the caller. Nothing will happen in it unless you explicitly exhaust it. List comprehensions evaluate eagerly, but, in this particular case, you are left with a list with no purpose. Just use a regular loop.
This is bad practice. Firstly, your code fragment does not produce the desired output. You would instead get something like: <generator object <genexpr> at 0x03D826F0>.
Secondly, a list comprehension is for creating sequences, and generators a for creating streams of objects. Typically, they do not have side effects. Your action function is a prime example of a side effect -- it prints its input and returns nothing. Rather, a generator should for each item it generates, take an input and compute some output. eg.
doubled_odds = [x*2 for x in range(10) if x % 2 != 0]
By using a generator you are obfuscating the purpose of your code, which is to mutate global state (printing something), and not to create a stream of objects.
Whereas, just using a for loop makes the code slightly longer (basically just more whitespace), but immediately you can see that the purpose is to apply function to a selection of items (as opposed to creating a new stream/list of items).
for i in items:
if i < 4:
action(i)
Remember that generators are still looping constructs and that the underlying bytecode is more or less the same (if anything, generators are marginally less efficient), and you lose clarity. Generators and list comprehensions are great, but this is not the right situation for them.
While I personally favour Tigerhawk's solution, there might be a middle ground between his and willywonkadailyblah's solution (now deleted).
One of willywonkadailyblah's points was:
Why create a new list instead of just using the old one? You already have the condition to filter out the correct elements, so why put them away in memory and come back for them?
One way to avoid this problem is to use lazy evaluation of the filtering i.e. have the filtering done only when iterating using the for loop by making the filtering part of a generator expression rather than a list comprehension:
for i in (j for j in items if j > 4):
action(i)
Output
Number is 6
In all honesty, I think Tigerhawk's solution is the best for this, though. This is just one possible alternative.
The reason that I proposed this is that it reminds me a lot of LINQ queries in C#, where you define a lazy way to extract, filter and project elements from a sequence in one statement (the LINQ expression) and can then use a separate for each loop with that query to perform some action on each element.
I am trying to evaluate power series using python. series => e^x = 1+ x+ x^2/2! + x^3/3!...x^n/n!
I am getting this error ''int' object has no attribute 'extend'.
My code:
import math
print('give x')
x = float(input())
n =100
k = 0
list = (1)
while 0<x<1:
list.extend([math.pow(x,K+1))])
k = k+1
if x==n:
break
print(sum(list))
Please help!
There are multiple problems with your code.
Firstly, you are attempting to create a list with (1) - that just creates the integer object 1, the parentheses have no effect here. To create a list containing 1 you need [1]. And you shouldn't use the names of Python built-ins (like list) as variable names - not only is it confusing to other people who may read your code it makes the built-in inaccessible, which can lead to mysterious bugs.
K is not the same as k.
Your while 0<x<1: test does't make much sense; FWIW, the Taylor series for ex converges for all values of x.
Your if x==n: test should be if k==n:, although it'd be better to use a for loop with range (or maybe xrange in Python 2).
You don't need to save the terms in a list - just add them as you go.
You don't need math.pow - x**k calculates the same thing as math.pow(x, k), but even that's unnecessary here: you should just keep track of the previous term and multiply it by x on each loop.
You forgot the /n!. Once again, you don't really need to compute the factorial (or call the math.factorial function) since you can just divide the previous term by k.
Hopefully, that's given you enough clues to fix your code. I won't provide working code at this stage, since I suspect this is a homework problem. Note that the math module has an exp function which you can use to test the accuracy of your power series calculations.
A list literal is created with square brackets, []. You can use parentheses, (), for grouping or for creating a tuple. In the case of list = (1), they are being used for grouping, so this is the same as list = 1. (A tuple with one element is created with mytuple = (1,) or just mytuple = 1,.)
At this point, I'll mention that naming one of your variables list masks the built-in function list, so after you do that you can't access that function anymore without some effort. It's best to name your object something else, like lst.
A list's extend() method adds all the elements from the passed list onto the object you're accessing, so if mylist was [1, 2, 3], mylist.extend([4, 5]) would result in mylist becoming [1, 2, 3, 4, 5]. However, you only have one object to add, so it makes more sense to use append(), which adds the passed object to the given list.
x = float(input('Give x: ')) # the input function can be passed a prompt string
n = 100
k = 0
lst = [1] # changed name, created a list
while 0<x<1:
list.append(x**(k+1)) # you can just use the ** operator if you want
# also, k isn't K
k = k+1
if x==n: # x is never changed, so your loop either never runs
# or goes forever
break
print(sum(lst))
Note the while loop that will either never be entered or never finish. You'll have to take another look at your program's logic.
I'm using Python 3.4.* and I am trying to execute the following code:
def P(n):
if n == 0:
yield []
return
for p in P(n-1):
p.append(1)
yield p
p.pop()
if p and (len(p) < 2 or p[-2] > p[-1]):
p[-1] += 1
yield p
print(P(5)) # this line doesn't make sense
for i in P(5): # but this line does make sense thanks to furkle
print(i)
but I am getting <generator object P at 0x02DAC198> rather than the output.
Can someone explain where in my code needs to be fixed? I don't think py likes the function name P but I could be wrong.
Edit: furkle clarified <generator object P at 0x02DAC198>.
By the way, I'm currently trying to write my own modified partition function and I was trying to understand this one corresponding to the classical setting.
I think you're misunderstanding the concept of a generator. A generator object is like a list, but you can iterate through its results lazily, without having to wait for the whole list to be constructed. Calling an operation on a function that returns a generator will not perform that operation in sequence on every item yielded by the generator.
If you wanted to print all the output of P(5), you should write:
for i in P(5):
print(i)
If you just want to print a list of the content returned by the generator, that largely seems to defeat the purpose of the generator.
Many things are wrong with this code, and your understanding of how generators work and what they are used for.
First, with respect to your print statement, that is exactly what it should print. Generators are never implicitly expanded, because there is no guarantee that a generator would ever terminate. It's perfectly valid, and sometimes very desirable, to construct a generator that produces an endless sequence. To get what you'd want (which I assume is produce output similar to a list), you'd do:
print(list(P(5))
But that brings me to my second point; Generators yield values in a sequence (in 99% of uses for them, unless you're using it as a coroutine). You are trying to use your generator to construct a list; however, if n is not 0 this will never yield a value and will immediately return. If you goal is to construct a generator that makes a list of 1's of a given length, it should look like this:
def P(n):
while n >= 0:
yield n
n -=1
This will produce a sequence of 1's of length n. To get the list form, you'd do list(P(n)).
I suggest you have another read over the Generator Documentation and get a better feel for them and see if they're really the right tool for the job.
In reading the function, I try to find what the call will produce. Let's start with the full original code:
def P(n):
if n == 0:
yield []
return
for p in P(n-1):
p.append(1)
yield p
p.pop()
if p and (len(p) < 2 or p[-2] > p[-1]):
p[-1] += 1
yield p
print(P(5)) # this line doesn't make sense
Okay, so it calls P(5). Since that's not 0, P recurses, until we reach P(0) which yields an empty list. That's the first time p receives a value. Then P(1) appends 1 into that list, and yields it to P(2) which repeats the process.. and so on. All the same list, originally created by P(0), and eventually yielded out as [1,1,1,1,1] by P(5) - but then the magic happens. Let's call this first list l0.
When you ask the generator for the second item, control returns to P(5) which now removes a value from l0. Depending on a bunch of conditions, it may increment the last value and yield p, which is l0, again. So the first item we received has been changing while we asked for the second. This will eventually terminate, but means there's a difference between these two:
print list(P(5)) # Eventually prints a list of l0 which has been emptied!
for item in P(5):
print item # Prints l0 at each point it was yielded
In [225]: for i in P(5): print i
[1, 1, 1, 1, 1]
[2, 1, 1, 1]
[2, 2, 1]
[3, 1, 1]
[3, 2]
[4, 1]
[5]
In [226]: list(P(5))
Out[226]: [[], [], [], [], [], [], []]
This is why I called it post-modifying; the values it returns keep changing after they've been produced (since they are in fact the same object being manipulated).
print max(3 for i in range(4))
#output is 3
Using Python 2.6
The 3 is throwing me off, heres my attempt at explaining whats going on.
for i in range(4) makes a loop that loops 4 times, incrementing i from 0 to 3 at the start of each loop. [no idea what the 3 means in this context...] max() returns the biggest iterable passed to it and the result is printed to screen.
3 for i in range(4) is a generator that yields 3 four times in a row and max takes an iterable and returns the element with the highest value, which is, obviously, three here.
This evaluates to:
print max([3,3,3,3])
... which is an elaborate way to say print 3.
expr for x in xs is a generator expression. Typically, you would use x in expr. For example:
[2*i for i in range(4)] #=> [0, 2, 4, 6]
It can be rewritten as:
nums = []
for i in range(4):
nums.append(3)
print max(nums) # 3! Hurrah!
I hope that makes its pointlessness more obvious.
The expression:
print max(3 for i in range(4))
is printing the result of the max() function, applied to what is inside the parentheses. Inside the parentheses however, you have a generator expression creating something similar to an array, with all elements equal to 3, but in a more efficient way than the expression:
print max([3 for i in range(4)])
which will create an array of 3s and destroy it after it is no longer needed.
Basically: because inside the parentheses you will create only values that are equal, and the max() function returns the biggest one, you do not need to create more than one element. Because with the number of elements always equal to one, the max() function becomes not needed and your code can be effectively replaced (at least in the case you have given) by the following code:
print 3
That is simply all ;)
To read more about differences between comprehension and generator expression, you can visit this documentation page.