I've created two enumeration methods, one which returns a list and the other which returns a yield/generator:
def enum_list(sequence, start=0):
lst = []
num = start
for sequence_item in sequence:
lst.append((num, sequence_item))
num += 1
return lst
def enum_generator(sequence, start=0):
num = start
for sequence_item in sequence:
yield (num, sequence_item)
num += 1
A few questions on this:
(1) Is changing a list to a generator as simple as doing:
# build via list
l = list()
for item in items:
l.append(item)
# build via iterator
# l = list() (1) <== delete this line
for item in items:
yield item # (2) change l.append(...) to yield ...
(2) Is "lazy evaluation" the only reason to use a generator, or are there other reasons as well?
(1) generator are simply created as adding yield to your iteration.
(2) Yes, for lazy evaluation. But generators are also used to create stack and queue as they can be only iterate once. This property is also exploited in context manager, by yielding the context.
An additional difference in your case is that since list is created before use and generator is evaluated at each next call, the generator function can check the context and come to different result for each yield, depending on external conditions, which vary with time.
Consider pseudocode:
def alloted_time():
while True:
if len(global_queue)>10:
yield 5
else:
yield 10
If queue is large, allot 5 mins for next person, else 10.
Related
Recently i have been using the 'yield' in python. And I find generator functions very useful. My query is that, is there something which could decrement the imaginative cursor in the generator object. Just how next(genfun) moves and outputs +i'th item in the container, i would like to know if there exists any function that may call upon something like previous(genfun) and moves to -1th item in the conatiner.
Actual Working
def wordbyword():
words = ["a","b","c","d","e"]
for word in words:
yield word
getword = wordbyword()
next(getword)
next(getword)
Output's
a
b
What I would like to see and achieve is
def wordbyword():
words = ["a","b","c","d","e"]
for word in words:
yield word
getword = wordbyword()
next(getword)
next(getword)
previous(getword)
Expected Output
a
b
a
This may sound silly, but is there someway there is this previous in generator, if not why is it so?. Why not we could decrement the iterator, or am I ignorant of an existing method, pls shower some light. What can be the closest way to implement what I have here in hand.
No there is no such function to sort of go back in a generator function. The reason is that Python does not store up the previous value in a generator function natively, and as it does not store it, it also cannot perform a recalculation.
For example, if your generator is a time-sensitive function, such as
def time_sensitive_generator():
yield datetime.now()
You will have no way to recalculate the previous value in this generator function.
Of course, this is only one of the many possible cases that a previous value cannot be calculated, but that is the idea.
If you do not store the value yourself, it will be lost forever.
As already said, there is no such function since the entire point of a generator is to have a small memory footprint. You would need to store the result.
You could automate the storing of previous results. One use-case of generators is when you have a conceptually infinite list (e.g. that of prime numbers) for which you only need an initial segment. You could write a generator that builds up these initial segments as a side effect. Have an optional history parameter that the generator appends to while it is yielding. For example:
def wordbyword(history = None):
words = ["a","b","c","d","e"]
for word in words:
if isinstance(history,list): history.append(word)
yield word
If you use the generator without an argument, getword = wordbyword(), it will work like an ordinary generator, but if you pass it a list, that list will store the growing history:
hist = []
getword = wordbyword(hist)
print(next(getword)) #a
print(next(getword)) #b
print(hist) #['a','b']
Iterating over a generator object consumes its elements, so there is nothing to go back to after using next. You could convert the generator to a list and implement your own next and previous
index = 0
def next(lst):
global index
index += 1
if index > len(lst):
raise StopIteration
return lst[index - 1]
def previous(lst):
global index
index -= 1
if index == 0:
raise StopIteration
return lst[index - 1]
getword = list(wordbyword())
print(next(getword)) # a
print(next(getword)) # b
print(previous(getword)) # a
One option is to wrap wordbyword with a class that has a custom __next__ method. In this way, you can still use the built-in next function to consume the generator on-demand, but the class will store all the past results from the next calls and make them accessible via a previous attribute:
class save_last:
def __init__(self, f_gen):
self.f_gen = f_gen
self._previous = []
def __next__(self):
self._previous.append(n:=next(self.i_gen))
return n
def __call__(self, *args, **kwargs):
self.i_gen = self.f_gen(*args, **kwargs)
return self
#property
def previous(self):
if len(self._previous) < 2:
raise Exception
return self._previous[-2]
#save_last
def wordbyword():
words = ["a","b","c","d","e"]
for word in words:
yield word
getword = wordbyword()
print(next(getword))
print(next(getword))
print(getword.previous)
Output:
a
b
a
TL;DR is what I'm trying to do too complicated for a yield-based generator?
I have a python application where I need to repeat an expensive test on a list of objects, one at a time, and then mangle those that pass. I expect several objects to pass, but I do not want to create a list of all those that pass, as mangle will alter the state of some of the other objects. There is no requirement to test in any particular order. Then rinse and repeat until some stop condition.
My first simple implementation was this, which runs logically correctly
while not stop_condition:
for object in object_list:
if test(object):
mangle(object)
break
else:
handle_no_tests_passed()
unfortunately, for object in object_list: always restarts at the beginning of the list, where the objects probably haven't been changed, and there are objects at the end of the list ready to test. Picking them at random would be slightly better, but I would rather carry on where I left off from the previous for/in call. I still want the for/in call to terminate when it's traversed the entire list.
This sounded like a job for yield, but I tied my brain in knots failing to make it do what I wanted. I can use it in the simple cases, iterating over a range or returning filtered records from some source, but I couldn't find out how to make it save state and restart reading from its source.
I can often do things the long wordy way with classes, but fail to understand how to use the alleged simplifications like yield. Here is a solution that does exactly what I want.
class CyclicSource:
def __init__(self, source):
self.source = source
self.pointer = 0
def __iter__(self):
# reset how many we've done, but not where we are
self.done_this_call = 0
return self
def __next__(self):
ret_val = self.source[self.pointer]
if self.done_this_call >= len(self.source):
raise StopIteration
self.done_this_call += 1
self.pointer += 1
self.pointer %= len(self.source)
return ret_val
source = list(range(5))
q = CyclicSource(source)
print('calling once, aborted early')
count = 0
for i in q:
count += 1
print(i)
if count>=2:
break
else:
print('ran off first for/in')
print('calling again')
for i in q:
print(i)
else:
print('ran off second for/in')
which demonstrates the desired behaviour
calling once, aborted early
0
1
calling again
2
3
4
0
1
ran off second for/in
Finally, the question. Is it possible to do what I want with the simplified generator syntax using yield, or does maintaining state between successive for/in calls require the full class syntax?
Your use of the __iter__ method causes your iterator to be reset. This actually goes quite counter to regular behaviour of an iterator; the __iter__ method should just return self, nothing more. You rely on a side effect of for applying iter() to your iterator each time you create a for i in q: loop. This makes your iterator work, but the behaviour is surprising and will trip up future maintainers. I'd prefer that effect to be split out to a separate .reset() method, for example.
You can reset a generator too, using generator.send() to signal it to reset:
def cyclic_source(source):
pointer = 0
done_this_call = 0
while done_this_call < len(source):
ret_val = source[pointer]
done_this_call += 1
pointer = (pointer + 1) % len(source)
reset = yield ret_val
if reset is not None:
done_this_call = 0
yield # pause again for next iteration sequence
Now you can 'reset' your count back to zero:
q = cyclic_source(source)
for count, i in enumerate(q):
print(i)
if count == 1:
break
else:
print('ran off first for/in')
print('explicitly resetting the generator')
q.send(True)
for i in q:
print(i)
else:
print('ran off second for/in')
This is however, rather.. counter to readability. I'd instead use an infinite generator by using itertools.cycle() that is limited in the number of iterations with itertools.islice():
from itertools import cycle, islice
q = cycle(source)
for count, i in enumerate(islice(q, len(source))):
print(i)
if count == 1:
break
else:
print('ran off first for/in')
for i in islice(q, len(source)):
print(i)
else:
print('ran off second for/in')
q will produce values from source in an endless loop. islice() cuts off iteration after len(source) elements. But because q is reused, it is still maintaining the iteration state.
If you must have a dedicated iterator, stick to a class object and make an iterable, so have it return a new iterator each time __iter__ is called:
from itertools import cycle, islice
class CyclicSource:
def __init__(self, source):
self.length = len(source)
self.source = cycle(source)
def __iter__(self):
return islice(self.source, self.length)
This keeps state in the cycle() iterator still, but simply creates a new islice() object each time you create an iterator for this. It basically encapsulates the islice() approach above.
This question already has answers here:
Looping from 1 to infinity in Python
(8 answers)
Closed 5 months ago.
The community reviewed whether to reopen this question 4 months ago and left it closed:
Original close reason(s) were not resolved
Is it possible to get an infinite loop in for loop?
My guess is that there can be an infinite for loop in Python. I'd like to know this for future references.
You can use the second argument of iter(), to call a function repeatedly until its return value matches that argument. This would loop forever as 1 will never be equal to 0 (which is the return value of int()):
for _ in iter(int, 1):
pass
If you wanted an infinite loop using numbers that are incrementing you could use itertools.count:
from itertools import count
for i in count(0):
....
The quintessential example of an infinite loop in Python is:
while True:
pass
To apply this to a for loop, use a generator (simplest form):
def infinity():
while True:
yield
This can be used as follows:
for _ in infinity():
pass
Yes, use a generator that always yields another number:
Here is an example
def zero_to_infinity():
i = 0
while True:
yield i
i += 1
for x in zero_to_infinity():
print(x)
It is also possible to achieve this by mutating the list you're iterating on, for example:
l = [1]
for x in l:
l.append(x + 1)
print(x)
In Python 3, range() can go much higher, though not to infinity:
import sys
for i in range(sys.maxsize**10): # you could go even higher if you really want but not infinity
pass
Here's another solution using the itertools module:
import itertools
for _ in itertools.repeat([]): # return an infinite iterator
pass
It's also possible to combine built-in functions iter (see also this answer) and enumerate for an infinite for loop which has a counter:
for i, _ in enumerate(iter(bool, True)):
input(i)
Which prints:
0
1
2
3
4
...
This uses iter to create an infinite iterator and enumerate provides the counting loop variable. You can even set a start value other than 0 with enumerate's start argument:
for i, _ in enumerate(iter(bool, True), start=42):
input(i)
Which prints:
42
43
44
45
46
...
Python infinite for loop
Well, there are a few ways, but the easiest one I found is to Iterate over a list
To understand this you must be knowing about:
python basics
python loops
python lists ( and also its append() function
The syntax is something like this...
l = ['#'] # Creating a list
for i in l: # Iterating over the same list
print(l)
l.append(i) # Appending the same element
With every iteration, the an element gets appended to the list. This way the loop never stops iterating.
Happy coding : )
While there have been many answers with nice examples of how an infinite for loop can be done, none have answered why (it wasn't asked, though, but still...)
A for loop in Python is syntactic sugar for handling the iterator object of an iterable an its methods. For example, this is your typical for loop:
for element in iterable:
foo(element)
And this is what's sorta happening behind the scenes:
iterator = iterable.__iter__()
try:
while True:
element = iterator.next()
foo(element)
except StopIteration:
pass
An iterator object has to have, as it can be seen, anextmethod that returns an element and advances once (if it can, or else it raises a StopIteration exception).
So every iterable object of which iterator'snextmethod does never raise said exception has an infinite for loop. For example:
class InfLoopIter(object):
def __iter__(self):
return self # an iterator object must always have this
def next(self):
return None
class InfLoop(object):
def __iter__(self):
return InfLoopIter()
for i in InfLoop():
print "Hello World!" # infinite loop yay!
we can actually have a for infinite loop
list = []
for i in list:
list.append(i)
print("Your thing")
i found a way without using yield or a while loop.
my python version is python 3.10.1
x = [1]
for _ in x:
x.append(1)
print('Hello World!')
if you need loop count, you can use i+1:
x = [1]
for i in x:
x.append(i+1)
print(f'Hello {i}')
you should know that this is not really an "infinite" loop.
because as the loop runs, the list grows and eventually, you will run out of ram.
Best way in my opinion:
for i in range(int(1e18)):
...
The loop will run for thousands of years
You can configure it to use a list. And append an element to the list everytime you iterate, so that it never ends.
Example:
list=[0]
t=1
for i in list:
list.append(i)
#do your thing.
#Example code.
if t<=0:
break
print(t)
t=t/10
This exact loop given above, won't get to infinity. But you can edit the if statement to get infinite for loop.
I know this may create some memory issues, but this is the best that I could come up with.
n = 0
li = [0]
for i in li:
n += 1
li.append(n)
print(li)
In the above code, we iterate over the list (li).
So in the 1st iteration, i = 0 and the code in for block will run, that is li will have a new item (n+1) at index 1 in this iteration so our list becomes [ 0, 1 ]
Now in 2nd iteration, i = 1 and new item is appended to the li (n+1), so the li becomes [0, 1, 2]
In 3rd iteration, i = 2, n+1 will be appended again to the li, so the li becomes [ 0, 1, 2, 3 ]
This will keep on going as in each iteration the size of list is increasing.
The other solutions solutions have a few issues, such as:
consuming a lot of memory which may
cause memory overflow
consuming a lot of processor power.
creating deadlock.
using 3rd party library
Here is an answer, which will overcome these problems.
from asyncio import run, sleep
async def generator():
while True:
await sleep(2)
yield True
async def fun():
async for _ in generator():
print("Again")
if __name__ == '__main__':
run(fun())
In case you want to do something that will take time, replace sleep with your desired function.
i'm newbie in python but try this
for i in range(2):
# your code here
i = 0
can improve this code
In Python 2.x, you can do:
my_list = range(10)
for i in my_list:
print "hello python!!"
my_list.append(i)
I can't figure out how to make my doubly linked list's iterableness work correctly when using nested loops.
My code thus far: http://pastebin.com/PU9iFggr
I have attempted to make it iterable:
def __iter__(self):
self.index = 0
return (self)
def next(self):
try:
result = self._findNode(self.index).get()
except IndexError:
self.index = 0
raise StopIteration
self.index += 1
return result
def __getitem__(self, item):
return self._findNode(item).get()
It seems to work if inside one for loop, but not inside of two:
myList = DoublyLinkedList()
myList.append(0)
myList.append(1)
myList.append(2)
myList.append(3)
for i in myList:
print i #works as expected
for i in myList:
for j in myList:
print j #goes forever
I imagine that the issue is that there is only one self.index inside of the object that is being updated by both of the for loops, but I don't know how to fix this.
Containers should be Iterable, not Iterators. Don't implement next on the class itself. Either make __iter__ a generator function, or write a separate class for it to return that wraps the linked list and implements next.
The easiest approach is to define __iter__ as a generator function:
def __iter__(self):
cur = self.head
while cur is not None:
yield cur.value
cur = cur.nextNode
Remove the next function from DoubleLinkedList and that's it. When you try to iterate it with a for loop, the call to the generator function returns a new, independent generator object which then iterates independently of any other generators that may have been requested. And it's much faster than repeated indexing like you were doing (which has to start from the head and traverse every time; the generator saves state as it goes, so it's only traversing one link in the chain for each item yielded).
I think you know very well where the problem is:
1 for i in mylist:
2 for j in mylist:
3 print j
4 # when j loop ends index goes back 0, this is where the infinite
5 # loop is,next line in execution is 1, and the method called is
6 # "next()", it will read linkedlist[0] for the second time (and
7 # then repeat...forever)
in short every time you call next in i loop, it will just return doubleLinkedList[0], it make to progress towards the index exception.
There are a lot of solutions,
1. if all you do in the nested for loop is print j,you can simply just iterate through the length of your linkedlist:
for i in range(len(mylist)): # I see that you already have the __len__ method
for j in mylist:
print j
2.This is my favorite solution: Instead pf implementing an iterator interface,use python generator:
def traverseList(doubly_linked_list):
# select and delete all of your __iter__() and next(), use the following code
index = 0
while True:
try:
yield doubly_linked_list._findNode(index).get()
index += 1
except IndexError:
break
for i in traverseList(mylist):
for j in traverseList(mylist):
# do things, note that I did not create two linked list
# I sort of create two iterators...
you can look up coroutine if you are not too familiar with generators, but basically they have their own stack, so each iterator of your doubly linked list maintains its own index (what you try to achieve in your code)
3.hmmm I am still thinking, I will update if I got any new ideas
Is there a better way to write the following:
row_counter = 0
for item in iterable_sequence:
# do stuff with the item
counter += 1
if not row_counter:
# handle the empty-sequence-case
Please keep in mind that I can't use len(iterable_sequence) because 1) not all sequences have known lengths; 2) in some cases calling len() may trigger loading of the sequence's items into memory (as the case would be with sql query results).
The reason I ask is that I'm simply curious if there is a way to make above more concise and idiomatic. What I'm looking for is along the lines of:
for item in sequence:
#process item
*else*:
#handle the empty sequence case
(assuming "else" here worked only on empty sequences, which I know it doesn't)
for item in iterable:
break
else:
# handle the empty-sequence-case here
Or
item = next(iterator, sentinel)
if item is sentinel:
# handle the empty-sequence-case here
In each case one item is consumed if it is present.
An example of empty_adapter()'s implementation mentioned in the comments:
def empty_adaptor(iterable, sentinel=object()):
it = iter(iterable)
item = next(it, sentinel)
if item is sentinel:
return None # empty
else:
def gen():
yield item
for i in it:
yield i
return gen()
You could use it as follows:
it = empty_adaptor(some_iter)
if it is not None:
for i in it:
# handle items
else:
# handle empty case
Introducing special case for an empty sequence for a general case seems wrong. There should be a better solution for a domain specific problem.
It may be a job for itertools.tee
You "trigger" the sequence on the verification, but you are left with an untouched copy of the sequence afterwards:
from itertools import tee
check, sequence = tee(sequence, 2)
try:
check.next():
except StopIteration:
#empty sequence
for item in sequence:
#do stuff
(it's worth nting that tee does the "right" thing here: it will load just the first element of the sequence in the moment check.next() is executed - and this first elment will remain available in the sequence. The remaining items will only be retrieved as part of the for loop
Or just keeping it simple:
If you can't use len, you can't check if the sequence has a bool value of True, for the same reasons.
Therefore, your way seens simple enough -
another way would be to delete the name "item" before the "for" statement and
check if it exists after the loop:
del item
for item in sequence:
# do stuff
try:
item
except NameError:
# sequence is empty.
But your code should be used as its more clear than this.
The second example from J.F. Sebastian seems to be the ticket with a while loop.
NoItem = object()
myiter = (x for x in range(10))
item = next(myiter, NoItem)
if item is NoItem:
...
else:
while item is not NoItem:
print item
item = next(myiter, NoItem)
Not the most concise but objectively the clearest... Mud, no?
This shouldn't trigger len():
def handle_items(items):
index = -1
for index, item in enumerate(items):
print 'processing item #%d: %r' % (index, item)
# at this point, index will be the offset of the last item,
# i.e. length of items minus one
if index == -1:
print 'there were no items to process'
print 'done'
print
# test with an empty generator and two generators of different length:
handle_items(x for x in ())
handle_items(x for x in (1,))
handle_items(x for x in (1, 2, 3))
if not iterable_sequence.__length_hint__():
empty()
else:
for item in iterable_sequence:
dostuff()