Suppose I have li = iter([1,2,3,4]).
Will the garbage collector drop the references to inaccessible element when I do next(li).
And what about deque, will elements in di = iter(deque([1,2,3,4])) be collectable once consumed.
If not, does a native data structure in Python implement such behaviour.
https://github.com/python/cpython/blob/bb86bf4c4eaa30b1f5192dab9f389ce0bb61114d/Objects/iterobject.c
A reference to the list is held until you iterate to the end of the sequence. You can see this in the iternext function.
The deque is here and has no special iterator.
https://github.com/python/cpython/blob/master/Modules/_collectionsmodule.c
You can create your own class and define __iter__ and __next__ to do what you want. Something like this
class CList(list):
def __init__(self, lst):
self.lst = lst
def __iter__(self):
return self
def __next__(self):
if len(self.lst) == 0:
raise StopIteration
item = self.lst[0]
del self.lst[0]
return item
def __len__(self):
return len(self.lst)
l = CList([1,2,3,4])
for item in l:
print( len(l) )
Related
I have the following code which I wrote for a course which runs fine:
def reverse_iter(iterable):
"""Return a generator that yields items from iterable in reverse order"""
last = len(iterable) - 1
while last >= 0:
yield iterable[last]
last -= 1
The next part of the assignment is to turn this function into a class. I know this not practical but it is what is being asked. I have very limited knowledge about classes but I came up with the following code:
class ReverseIter:
"""Class whose instances iterate the initial iterable in reverse order"""
def __init__(self, iterable):
self.iterable = iterable
def iterreverse(self):
last = len(self.iterable) - 1
while last >= 0:
yield self.iterable[last]
last -= 1
nums = [1, 2, 3, 4]
it = ReverseIter(nums)
print(iter(it) is it)
print(next(it) == 4)
print(next(it))
print(next(it))
print(next(it))
print(next(it))
The output always reads TypeError: 'ReverseIter' object is not iterable. I hav eno idea why this is coming up. Any explanations or suggestions would be helpful. The output of the print statements above by the way should be True, True, 3, 2, 1, Stop Iteration respectively.
An object is an iterator if it has a __next__ method, which is called to get the next item (and should raise StopIteration when it runs out), and __iter__ which returns an iterator (usually just self)
class ReverseIter:
def __init__(self, sequence):
self.sequence = sequence
self.index = len(sequence)
def __iter__(self):
return self
def __next__(self):
if self.index > 0:
self.index -= 1
return self.sequence[self.index]
else:
raise StopIteration
you must implement the __next__ method, as code below.
class ReverseIter:
"""Class whose instances iterate the initial iterable in reverse order"""
def __init__(self, iterable):
self.iterable = iterable
def __iter__(self):
last = len(self.iterable) - 1
while last >= 0:
yield self.iterable[last]
last -= 1
I have class Count which takes 3 parameters including self, mystart and myend. It should count from mystart until myend (also reversed) using magic methods __iter__, __next__ and __reversed__. I have implemented all three magic methods. But I am still not sure whether it is the right way to implemnet next and reversed methods. Is it possible that I can call built-in functions next and reversed inside my __next__ and __reversed__ methods or is there any pythonic way?
class Count:
def __init__(self,mystart,myend):
self.mystart=mystart
self.myend=myend
self.current=mystart
self.reverse=[]
def __iter__(self):
"Returns itself as an Iterator Object"
return self
def __next__(self):
if self.current > self.myend:
raise StopIteration
else:
self.current+=1
return self.current-1
def __reversed__(self):
for i in range(self.myend,self.mystart,-1):
self.reverse.append(i)
return self.reverse
obj1=Count(0,10)
print("FOR LOOP")
for i in obj1:
print (i,end=",")
print ("\nNEXT")
obj2=Count(1,4)
print(next(obj2))
print(next(obj2))
print ("Reversed")
print(reversed(obj1))
Now I have done it using yield statement. #jedwards thanks for your tipp.
class Count:
def __init__(self, mystart,myend):
self.mystart = mystart
self.myend = myend
self.current=None
def __iter__(self):
self.current = self.mystart
while self.current < self.myend:
yield self.current
self.current += 1
def __next__(self):
if self.current is None:
self.current=self.mystart
if self.current > self.myend:
raise StopIteration
else:
self.current+=1
return self.current-1
def __reversed__(self):
self.current = self.myend
while self.current >= self.mystart:
yield self.current
self.current -= 1
obj1=Count(0,10)
for i in obj1:
print (i)
obj2=reversed(obj1)
for i in obj2:
print (i)
obj3=Count(0,10)
print (next(obj3))
print (next(obj3))
print (next(obj3))
You are mixing up Iterators and Iterables:
Iterators:
Keep a state associated with their current iteration progress
Implement __next__ to get the next state
Implement __iter__ to return themselves.
Iterables:
Contain (or define with some rule) a collection of elements that can be traversed
Implement __iter__ to return an iterator that can traverse the elements
can implement __reversed__ to return an iterator that goes backwards.
The __reversed__ magic method is:
Called (if present) by the reversed() built-in to implement reverse
iteration. It should return a new iterator object that iterates over
all the objects in the container in reverse order.
So you probably don't want to implement an iterator that can be __reversed__ mid iteration, for example the implementation in your answer means that this code:
x = Count(1,10)
for i in x:
for j in x:
print(i,j)
Will cause an infinite loop, the output is just this pattern repeated:
1 4
1 3
1 2
1 1
the reason for this is because both for loops are changing self.current in opposite directions, the outer loop will increment it by 1 then the inner loop will set it to self.myend and lower it back to 0, then the process repeats.
The only way to properly implement all three magic methods is to use two classes, one for the iterator and one for the iterable:
class _Count_iter:
def __init__(self, start, stop, step=1):
self.current = start
self.step = step
self.myend = stop
def __iter__(self):return self
def __next__(self):
#if current is one step over the end
if self.current == self.myend+self.step:
raise StopIteration
else:
self.current+=self.step
return self.current-self.step
class Count:
def __init__(self, mystart,myend):
self.mystart = mystart
self.myend = myend
def __iter__(self):
return _Count_iter(self.mystart,self.myend,1)
def __reversed__(self):
return _Count_iter(self.myend, self.mystart, -1)
From two options below what is more appealing(readable code, more pythonic, efficiency , etc..) for running through iterable and if I want add more logic in the future(for example add 1 to every returned value)?
One of options returns generator, the other returns next item and None when done all iterations.
Is there a preferable method? If yes, why? Is there limitations some method has that another don't.
class Option1():
def __init__(self):
self.current = 0
def get_next_batch(self, my_list):
if self.current == len(my_list):
self.current = 0
return None
self.current += 1
return [my_list[self.current-1]]
class Option2():
def __init__(self):
self.current = 0
def get_next_batch(self, my_list):
while self.current < len(my_list):
yield [my_list[self.current]]
self.current += 1
self.current = 0
raise StopIteration()
Usage:
o1 = Option1()
o2 = Option2()
arr = [1,2,3,4,5]
a = o1.get_next_batch(arr)
while a is not None:
print a
a = o1.get_next_batch(arr)
for item in o2.get_next_batch(arr):
print item
Output in both cases:
[1]
[2]
[3]
[4]
[5]
You almost certainly want to go with the second. Less lines, less opportunity for things to go wrong.
However, seeing as you're not using current outside the iterator I would just optimise the whole thing down to:
def get_next_batch(my_list):
for item in my_list:
yield [item]
arr = [1,2,3,4,5]
for item in get_next_batch(arr):
print item
Side points. Always make your classes inherit from object in python 2.7, and don't raise StopIteration to halt a generator in python -- it's deprecated behaviour that can lead to bugs. Use return instead. eg.
def get_next_batch(my_list):
for item in my_list:
if item > 3:
return
yield [item]
batched = list(get_next_batch([1,2,3,4,5]))
expected = [[1], [2], [3]]
print batched == expected
You can tidy up Option1 to make it easy to use in for loops. To do this you can make it with the iterator protocol. That is an __iter__ method that returns self and a next method to get the next item. eg.
class UsingNext(object):
def __init__(self, mylist):
self.current = 0
self.mylist = mylist
def __iter__(self):
return self
def next(self): # __next__ in python 3
if self.current >= len(self.mylist):
raise StopIteration
if self.current == 2:
self.current += 1
return "special"
item = self.mylist[self.current]
self.current += 1
return [item]
class UsingYield(object):
def __init__(self, mylist):
self.mylist = mylist
def __iter__(self):
for current, item in enumerate(self.mylist):
if current == 2:
yield "special"
continue
yield [item]
arr = range(1, 6)
# both print
# [1]
# [2]
# special
# [4]
# [5]
for item in UsingNext(arr):
print item
for item in UsingYield(arr):
print item
In my mind the generator version is cleaner, and easier to understand.
I want to create a class that behaves like a list. The challenge is doing it without using lists nor dictionaries. So far I've created a node class that goes like this:
class Node:
def __init__(self, value=None):
self.next = None
self.last = None
self.value = valor
def __repr__(self):
return self.value
And MyList class that's basically a chain of nodes, with a head node and a tail node. Thing is, I want to make it iterable so I can run a for with it. I searched how iter and next works and came up with something like this:
class MyList:
def __init__(self):
self.head = None
self.tail = None
def __iter__(self):
return self
def __next__(self):
if self.head:
if self.head.next:
self.head = self.head.next
return self.head.last
aux = self.head
self.head = None
return aux
raise StopIteration
It works but it obviously delete the data inside MyList so I can't use it again. Any advices on how to get the same results without messing up with the info inside the object?
Note that the iterator protocol only requires that the container's __iter__ returns an iterator; you can also implement __iter__ as a generator, rather than returning the instance itself:
def __iter__(self):
node = self.head
while node is not None:
yield node
node = node.next
You need to add a "current" marker to your class to indicate the node currently pointed at by iteration. Something like this:
class MyList:
def __init__(self):
self.head = None
self.tail = None
self.current = self.head
def __iter__(self):
return self
def __next__(self):
if self.current is not None:
it = self.current
self.current = self.current.next
return it
raise StopIteration
Right now, your list doesn't distinguish between its head and its current iteration position, but these are two totally distinct concepts.
Of course, if you do it this way, all iterations over the same MyList will be "linked", so that if you do something like:
x = MyList(1, 2, 3, 4)
for item in x:
print(x)
if item == 2:
break
for item in x:
print(x)
Then the second iteration will pick up where the first left off. If you don't want this behavior, you'll have to create a separate iterator class, and have MyList.__iter__ return an instance of that rather than self. If you return self from __iter__, then the object can't have multiple independent iterations going on, because the iteration state is stored in the object as the data being iterated.
What would be the nice way to return something from an iterator one last time when it's exhausted. I'm using a flag, but this is rather ugly:
class Example():
def __iter__(self):
self.lst = [1,2,3]
self.stop = False # <-- ugly
return self
def next(self):
if self.stop: # <-- ugly
raise StopIteration
if len(self.lst) == 0:
self.stop = True
return "one last time"
return self.lst.pop()
Background: I'm fetching an unknown amount of strings from an external source and send them further down to the caller. When the process is over, I want to emit a string "x records processed". I have no control over calling code, so this must be done inside my iterator.
You could just yield from __iter__ which would turn it into a generator function (alternately you could just write a generator function as suggested by Dan). Just as a warning, this might be misleading to people that abuse the next method.
class Example():
def __iter__(self):
lst = [1,2,3]
for i in reversed(lst):
yield i
yield "one last time"
Maybe you can use a generator function instead:
def example():
lst = [1, 2, 3]
while lst:
yield lst.pop()
yield 'one last time'
Don't return one extra thing. That's a bad idea because it doesn't extend well. What if you want to do sum as well as count? Or hash as well as count? An iterator is a stateful object. Make use of that.
class Example( collections.Iterator ):
def __iter__(self):
self.lst = [1,2,3]
self.count = 0
return self
def next(self):
if self.lst:
self.count += 1
return self.lst.pop()
raise StopIteration()
Use it like this.
my_iter= iter(Example())
for item in my_iterprin:
print( item )
print( my_iter.count )
You could do something like this:
class Example():
def __iter__(self):
self.lst = [1, 2, 3]
return self
def next(self):
try:
return self.lst.pop()
except IndexError:
print "done iterating"
raise StopIteration
>>> for i in Example():
... print i
...
3
2
1
done iterating
In your actual code you will probably need to change the exception type that you are catching, but this format should still be applicable.