I want to have a generator that may or may not have anything to yield, and if .next() or similar is used it will not have a StopIteration error if none of the conditions to yield are met.
An example:
def A(iterable):
for x in iterable:
if x == 1:
yield True
Which works like the following:
>>> list(A([1,2]))
[True]
>>> A([1]).next()
True
>>> A([2]).next()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
StopIteration
But what I would want is for A([2]).next() to return is None.
generally, in these circumstances you'd use the builtin next function:
my_iterator = A([2])
value = next(my_iterator, None)
In addition to being able to pass an optional "default" value when the iterator is empty, next has the advantage of working on python2.x and python3.x where the name of the method changes to __next__.
How about this:
def A(iterable):
for x in iterable:
yield True if x == 1 else None
Related
I've learned about iterators and such and discovered this quite interesting way of getting the first element in a list that a condition is applied (and also with default value in case we don't find it):
first_occurence = next((x for x in range(1,10) if x > 5), None)
For me, it seems a very useful, clear way of obtaining the result.
But since I've never seen that in production code, and since next is a little more "low-level" in the python structure I was wondering if that could be bad practice for some reason. Is that the case? and why?
It's fine. It's efficient, it's fairly readable, etc.
If you're expecting a result, or None is a possible result (so using None as a placeholder makes it hard to figure out if you got a result or got the default) it may be better to use the EAFP form rather than providing a default, catching the StopIteration it raises if no item is found, or just letting it bubble up if the problem is from the caller's input not meeting specs (so it's up to them to handle it). It looks even cleaner at point of use that way:
first_occurence = next(x for x in range(1,10) if x > 5)
Alternatively, when None is a valid result, you can use an explicit sentinel object that's guaranteed unique like so:
sentinel = object() # An anonymous object you construct can't possibly appear in the input
first_occurence = next((x for x in range(1,10) if x > 5), sentinel)
if first_occurence is not sentinel: # Compare with is for performance and to avoid broken __eq__ comparing equal to sentinel
A common use case for this one of these constructs to replace a call to any when you not only need to know if any item passed the test, but which item (any can only return True or False, so it's unsuited to finding which item passed).
We can wrap it up in a function to provide an even nicer interface:
_raise = object()
# can pass either an iterable or an iterator
def first(iterable, condition, *, default=_raise, exctype=None):
"""Get the first value from `iterable` which meets `condition`.
Will consume elements from the iterable.
default -> if no element meets the condition, return this instead.
exctype -> if no element meets the condition and there is no default,
raise this kind of exception rather than `StopIteration`.
(It will be chained from the original `StopIteration`.)
"""
try:
# `iter` is idempotent; this makes sure we have an iterator
return next(filter(condition, iter(iterable)))
except StopIteration as e:
if default is not _raise:
return default
if exctype:
raise exctype() from e
raise
Let's test it:
>>> first(range(10), lambda x: x > 5)
6
>>> first(range(10), lambda x: x > 11)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 4, in first
StopIteration
>>> first(range(10), lambda x: x > 11, exctype=ValueError)
Traceback (most recent call last):
File "<stdin>", line 4, in first
StopIteration
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 9, in first
ValueError
>>> first(range(10), lambda x: x > 11, default=None)
>>>
I am pretty much copying from the example given in online docs, but using filter() in windows-based Python 3 is perplexing me. What is wrong here:
a=[1,2,3,4]
b=[1,0,1,0]
f=filter(b,a)
for fs in f : print(fs)
...
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'list' object is not callable
f=list(filter(b,a))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'list' object is not callable
Online docs say to throw list() in, but that clearly helps not.
First argument of filter must be a function or lambda. You passed b which is a list.
Filter's documentation clearly states that it takes a function and an iterable as input. Perhaps if you want to check if a value is in both a and b, you could do something like:
f = filter(lambda x: x in b, a)
and then loop over f.
You are misunderstanding how filter works, filter needs a function that returns True for each item that you want to keep.
I'm assuming b describes the things in a you want to select, then you meant something like::
f = filter(lambda x: x[1], zip(a, b))
for fs, _ in f: ...
You can also replace filter with a list comprehension:
f = [x for x, y in zip(a, b) if y]
for fs in f: ...
But there is a function in itertools that does exactly this:
import itertools as it
f = it.compress(a, b)
for fs in f: ...
Basically filter takes a function and an iterable value to filter in
a = [1,2,3,4]
b = [1,0,1,0]
f = filter(lambda val: val not in b, a)
for fs in f: print(fs)
output:
2
3
4
Confusion occurs when we call filter with None
list(filter(None, [0,1,2]))
# This returns [1,2] Which is a special case
# and it is not equal to:
list(filter(1, [0,1,2])) #invalid
The reason being When we pass filter condition as None filter function checks each value in the list and sees if they qualify the if value condition.
Which roughly translates to this.
[value for value in [0,1,2] if value]
As indicated in the documentation, the default value is returned if the iterator is exhausted. However, in the following program, the g(x) function is not exhausted, and I hope that the error from f(x) would not be processed in the next function.
def f(x) :
if 0 : # to make sure that nothing is generated
yield 10
def g(x) :
yield next(f(x))
# list(g(3))
next(g(3), None)
What I expect:
Traceback (most recent call last):
File "a.py", line 9, in <module>
next(g(3), None)
File "a.py", line 6, in g
yield next(f(x))
StopIteration
What I encountered is that the program was running successfully.
Can I use an alternating approach to achieve the goal? Or can it be fixed in Python?
Edit: The program mentioned above may be modified like this in order to prevent ambiguation.
def f(x) :
if 0 : # to make sure that nothing is generated
yield 10
def g(x) :
f(x).__next__() # g(x) is not exhausted at this time
yield 'something meaningful'
# I hope that the next function will only catch this line
# list(g(3))
next(g(3), None)
next with a default parameter catches the StopIteration no matter the source.
The behavior you're seeing is expected, and maybe better understood using this code:
def justraise():
yield next(iter([])) # raises StopIteration
next(justraise(), None) # None
next(justraise()) # raises StopIteration
Moving to your code - even though the inner use is of next without a default argument, the StopIteration it raised is caught in the outer next with the default argument.
If you have a meaningful exception to raise, you should raise a meaningful exception and not StopIteration which indicates the iteration ended (and not erroneously) - which is what next relies on.
g(x) is an iterator that always yields f(x), which yields Nothing, and raises a StopIteration (in f)
You can check that next(f(some_value)) does throw an exception when called itself.
As will
def g(x):
return next(f(x))
But, you've added the default None, so that g(x) will run, but simply return back None since the iterator is exhausted.
If you remove the None, then you see
In [5]: next(g(3))
---------------------------------------------------------------------------
StopIteration Traceback (most recent call last)
<ipython-input-14-05eb86fce40b> in <module>()
----> 1 next(g(3))
<ipython-input-13-a4323284f776> in g(x)
1 def g(x) :
----> 2 yield next(f(x))
3
StopIteration:
I have a function that either returns a tuple or None. How is the Caller supposed to handle that condition?
def nontest():
return None
x,y = nontest()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'NoneType' object is not iterable
EAFP:
try:
x,y = nontest()
except TypeError:
# do the None-thing here or pass
or without try-except:
res = nontest()
if res is None:
....
else:
x, y = res
How about:
x,y = nontest() or (None,None)
If nontest returns a two-item tuple like it should, then x and y are assigned to the items in the tuple. Otherwise, x and y are each assigned to none. Downside to this is that you can't run special code if nontest comes back empty (the above answers can help you if that is your goal). Upside is that it is clean and easy to read/maintain.
If you can change the function itself, it's probably a better idea to make it raise a relevant exception instead of returning None to signal an error condition. The caller should then just try/except that.
If the None isn't signalling an error condition, you'll want to rethink your semantics altogether.
I have a program as follows:
a=reader.next()
if *some condition holds*:
#Do some processing and continue the iteration
else:
#Append the variable a back to the iterator
#That is nullify the operation *a=reader.next()*
How do I add an element to the start of the iterator?
(Or is there an easier way to do this?)
EDIT: OK let me put it this way. I need the next element in an iterator without removing it.
How do I do this>?
You're looking for itertools.chain:
import itertools
values = iter([1,2,3]) # the iterator
value = 0 # the value to prepend to the iterator
together = itertools.chain([value], values) # there it is
list(together)
# -> [0, 1, 2, 3]
Python iterators, as such, have very limited functionality -- no "appending" or anything like that. You'll need to wrap the generic iterator in a wrapper adding that functionality. E.g.:
class Wrapper(object):
def __init__(self, it):
self.it = it
self.pushedback = []
def __iter__(self):
return self
def next(self):
if self.pushedback:
return self.pushedback.pop()
else:
return self.it.next()
def pushback(self, val):
self.pushedback.append(val)
This is Python 2.5 (should work in 2.6 too) -- slight variants advised for 2.6 and mandatory for 3.any (use next(self.it) instead of self.it.next() and define __next__ instead of next).
Edit: the OP now says what they need is "peek ahead without consuming". Wrapping is still the best option, but an alternative is:
import itertools
...
o, peek = itertools.tee(o)
if isneat(peek.next()): ...
this doesn't advance o (remember to advance it if and when you decide you DO want to;-).
By design (in general development concepts) iterators are intended to be read-only, and any attempt to change them would break.
Alternatively, you could read the iterator backwards, and add it to the end of hte element (which is actually the start :) )?
This isn't too close what you asked for, but if you have control over the generator and you don't need to "peek" before the value is generated (and any side effects have occurred), you can use the generator.send method to tell the generator to repeat the last value it yielded:
>>> def a():
... for x in (1,2,3):
... rcvd = yield x
... if rcvd is not None:
... yield x
...
>>> gen = a()
>>> gen.send("just checking")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: can't send non-None value to a just-started generator
>>> gen.next()
1
>>> gen.send("just checking")
1
>>> gen.next()
2
>>> gen.next()
3
>>> gen.next()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
StopIteration