The itertools.count counter in Python (2.7.9) is very handy for thread-safe counting. How can I get the current value of the counter though?
The counter increments and returns the last value every time you call next():
import itertools
x = itertools.count()
print x.next() # 0
print x.next() # 1
print x.next() # 2
So far, so good.
I can't find a way to get the current value of the counter without calling next(), which would have the undesirable side-effect of increasing the counter, or using the repr() function.
Following on from the above:
print repr(x) # "count(3)"
So you could parse the output of repr(). Something like
current_value = int(repr(x)[6:-1])
would do the trick, but is really ugly.
Is there a way to get the current value of the counter more directly?
Another hack to get next value without advancing iterator is to abuse copy protocol:
>>> c = itertools.count()
>>> c.__reduce__()[1][0]
0
>>> next(c)
0
>>> c.__reduce__()[1][0]
1
Or just take it from object copy:
>>> from copy import copy
>>> next(copy(c))
1
Use the source, Luke!
According to module implementation, it's not possible.
typedef struct {
PyObject_HEAD
Py_ssize_t cnt;
PyObject *long_cnt;
PyObject *long_step;
} countobject;
Current state is stored in cnt and long_cnt members, and neither of them is exposed in object API. Only place where it may be retrieved is object __repr__, as you suggested.
Note that while parsing string you have to consider a non-singular increment case. repr(itertools.count(123, 4)) is equal to 'count(123, 4)' - logic suggested by you in question would fail in that case.
According to the documentation there is no way to access the current value of the function. itertools.count() is a generator method from the itertools module. As such, it is common practice to just simply assign the value of a generator's current value to a variable.
Simply store the the result of the next call:
current_value = x.next()
or ( Built-in python method for Python version ≥ 2.6 )
current_value = next(x)
You could make a wrapper function, or a utility decorator class if you would like some added syntactic sugar, but assignment is standard.
It is a generator, it wouldn't be easy to do what you want.
If you want to use it's value in several places, I'd recommend getting a value via .next() and storing it in a variable. If you are concerned that counter may be incremented between these 2 uses, you'd need to put them both in a critical section anyway.
If you don't want to pollute that counter with additional '+1's generated by those checks, you can use one more counter to count checks (put this in critical section too). Substracting latter from the former would give you what you need.
Also, are you really sure about thread-safety? Docs page has nothing about threads.
Ran into the same thing today. Here's what I ended up with:
class alt_count:
def __init__(self, start=0, step=1):
self.current = start - step
self.step = step
def __next__(self):
self.current = self.current + self.step
return self.current
Should give you almost all the itertools.count functionality plus the current property.
i = alt_count()
print(next(i)) # 0
print(next(i)) # 1
print(i.current) # 1
If the current value is not needed, I found using this simple closure also works. Note that nonlocal only works for Python version > 3.
def alt_next_maker(start=0, step=1):
res = start - step
def alt_next():
nonlocal res, step
res = res + step
return res
return alt_next
Can be used as a simple alternative if you don't want to use the itertools module.
alt_next = alt_next_maker()
print(alt_next()) # 0
print(alt_next()) # 1
The docs also mention the following as equivalent:
def count(start=0, step=1):
# count(10) --> 10 11 12 13 14 ...
# count(2.5, 0.5) -> 2.5 3.0 3.5 ...
n = start
while True:
yield n
n += step
Related
Is there anything like this in the Python built-in library:
class Counter:
def __init__(self, start=0):
self.val = start
def consume(self):
val = self.val
self.val += 1
return val
I see this as much safer way of implementing code where a counter needs to be used and then immediately incremented on the next line. Kind of like using i++ in other languages. But I'm trying to avoid clogging up my library with definitions like this if there's an inbuilt method.
You have basically reimplemented itertools.count, with consume standing in for __next__. (__next__ is not typically called directly, but by passing the instance of count to the next function.)
>>> from itertools import count
>>> c = count()
>>> next(c)
0
>>> next(c)
1
>>> next(c)
2
Elegant way to increment a global variable in Python:
This is what I have so far:
my_i = -1
def get_next_i():
global my_i
my_i += 1
return my_i
with generator:
my_iter = iter(range(100000000))
def get_next_i():
return next(my_iter)
with class:
class MyI:
MyI.my_i = -1
#staticmethod
def next():
MyI.my_i += 1
return MyI.my_i
The first one is long and I don't consider it as a better way to code .
The second one is a bit more elegant, but have an upper limit.
The third one is long, but at least have no global variable to work with.
What would be the best alternative to those?
The purpose of these functions is to assign a unique number to a specific event in my code. The code is not just a single loop, so using for i in range(...): is not suitable here. A later version might use multiple indices assigned to different events. The first code would require duplication to solve such an issue. (get_next_i(), get_next_j(), ...)
Thank You.
As others suggested, itertools.count() is the best option, e.g.
import itertools
global_counter1 = itertools.count()
global_counter2 = itertools.count()
# etc.
And then, when you need it, simply call next:
def some_func():
next_id = next(global_counter1)
EDIT: Changed global_counter1.next() (which worked only in Python 2) to next(global_counter1), which works also in Python 3.
You can create a generator that has an infinite loop. Each call of next(generator) will return a next value, without limit. See What does the "yield" keyword do in Python?
def create_generator()
i=0
while True:
i+=1
yield i
generator = create_generator()
print(next(generator))
print(next(generator))
Is there any non-explicit for way to call a member n times upon an object?
I was thinking about some map/reduce/lambda approach, but I couldn't figure out a way to do this -- if it's possible.
Just to add context, I'm using BeautifulSoup, and I'm extracting some elements from an html table; I extract some elements, and then, the last one.
Since I have:
# First value
print value.text
# Second value
value = value.nextSibling
print value.text
# Ninth value
for i in xrange(1, 7):
value = value.nextSibling
print value.text
I was wondering if there's any lambda approach -- or something else -- that would allow me to do this:
# Ninth value
((value = value.nextSibling) for i in xrange(1, 7))
print value.text
P.S.: No, there's no problem whatsoever with the for approach, except I really enjoy one-liner solutions, and this would fit really nice in my code.
I have a strong preference for the loop, but you could use reduce:
>>> class Foo(object):
... def __init__(self):
... self.count = 0
... def callme(self):
... self.count += 1
... return self
...
>>> a = Foo()
>>> reduce(lambda x,y:x.callme(),range(7),a)
<__main__.Foo object at 0xec390>
>>> a.count
7
You want a one-liner equivalent of this:
for i in xrange(1, 7):
value = value.nextSibling
This is one line:
for i in xrange(1, 7): value = value.nextSibling
If you're looking for something more functional, what you really want is a compose function, so you can compose callme() (or attrgetter('my_prop'), or whatever) 7 times.
In case of BS you can use nextSiblingGenerator() with itertools.islice to get the nth sibling. It would also handle situations where there is no nth element.
from itertools import islice
nth = 7
next(islice(elem.nextSiblingGenerator(), nth, None), None)
Disclaimer: eval is evil.
value = eval('value' + ('.nextSibling' * 7))
Ah! But reduce is not available in Python3, at least not as a built in.
So here is my try, portable to Python2/3 and based on the OP failed attempt:
[globals().update(value=value.nextSibling) for i in range(7)]
That assumes that value is a global variable. If value happens to be a member variable, then write instead:
[self.__dict__.update(value=value.nextSibling) for i in range(7)]
You cannot use locals() because the list comprehension creates a nested local scope, so the real locals() is not directly available. However, you can capture it with a bit of work:
(lambda loc : [loc.update(x=x.nextSibling) for i in range(7)])(locals())
Or easier if you don't mind duplicating the number of lines:
loc = locals()
[loc.update(value=value.nextSibling) for i in range(7)]
Or if you really fancy one-liners:
loc = locals() ; [loc.update(value=value.nextSibling) for i in range(7)]
Yes, Python can use ; too 8-)
UPDATE:
Another fancy variation, now with map instead of the list comprehension:
list(map(lambda d : d.update(value=value.nextSibling), 7 * [locals()]))
Note the clever use of list multiplication to capture the current locals() and create the initial iterable at the same time.
The most direct way to write it would be:
value = reduce(lambda x, _: x.nextSibling, xrange(1,7), value)
This question already has answers here:
Length of generator output [duplicate]
(9 answers)
What's the shortest way to count the number of items in a generator/iterator?
(7 answers)
Closed 9 years ago.
Python generators are very useful. They have advantages over functions that return lists. However, you could len(list_returning_function()). Is there a way to len(generator_function())?
UPDATE:
Of course len(list(generator_function())) would work.....
I'm trying to use a generator I've created inside a new generator I'm creating. As part of the calculation in the new generator it needs to know the length of the old one. However I would like to keep both of them together with the same properties as a generator, specifically - not maintain the entire list in memory as it may be very long.
UPDATE 2:
Assume the generator knows it's target length even from the first step. Also, there's no reason to maintain the len() syntax. Example - if functions in Python are objects, couldn't I assign the length to a variable of this object that would be accessible to the new generator?
The conversion to list that's been suggested in the other answers is the best way if you still want to process the generator elements afterwards, but has one flaw: It uses O(n) memory. You can count the elements in a generator without using that much memory with:
sum(1 for x in generator)
Of course, be aware that this might be slower than len(list(generator)) in common Python implementations, and if the generators are long enough for the memory complexity to matter, the operation would take quite some time. Still, I personally prefer this solution as it describes what I want to get, and it doesn't give me anything extra that's not required (such as a list of all the elements).
Also listen to delnan's advice: If you're discarding the output of the generator it is very likely that there is a way to calculate the number of elements without running it, or by counting them in another manner.
Generators have no length, they aren't collections after all.
Generators are functions with a internal state (and fancy syntax). You can repeatedly call them to get a sequence of values, so you can use them in loop. But they don't contain any elements, so asking for the length of a generator is like asking for the length of a function.
if functions in Python are objects, couldn't I assign the length to a
variable of this object that would be accessible to the new generator?
Functions are objects, but you cannot assign new attributes to them. The reason is probably to keep such a basic object as efficient as possible.
You can however simply return (generator, length) pairs from your functions or wrap the generator in a simple object like this:
class GeneratorLen(object):
def __init__(self, gen, length):
self.gen = gen
self.length = length
def __len__(self):
return self.length
def __iter__(self):
return self.gen
g = some_generator()
h = GeneratorLen(g, 1)
print len(h), list(h)
Suppose we have a generator:
def gen():
for i in range(10):
yield i
We can wrap the generator, along with the known length, in an object:
import itertools
class LenGen(object):
def __init__(self,gen,length):
self.gen=gen
self.length=length
def __call__(self):
return itertools.islice(self.gen(),self.length)
def __len__(self):
return self.length
lgen=LenGen(gen,10)
Instances of LenGen are generators themselves, since calling them returns an iterator.
Now we can use the lgen generator in place of gen, and access len(lgen) as well:
def new_gen():
for i in lgen():
yield float(i)/len(lgen)
for i in new_gen():
print(i)
You can use len(list(generator_function()). However, this consumes the generator, but that's the only way you can find out how many elements are generated. So you may want to save the list somewhere if you also want to use the items.
a = list(generator_function())
print(len(a))
print(a[0])
You can len(list(generator)) but you could probably make something more efficient if you really intend to discard the results.
You can use reduce.
For Python 3:
>>> import functools
>>> def gen():
... yield 1
... yield 2
... yield 3
...
>>> functools.reduce(lambda x,y: x + 1, gen(), 0)
In Python 2, reduce is in the global namespace so the import is unnecessary.
You can use send as a hack:
def counter():
length = 10
i = 0
while i < length:
val = (yield i)
if val == 'length':
yield length
i += 1
it = counter()
print(it.next())
#0
print(it.next())
#1
print(it.send('length'))
#10
print(it.next())
#2
print(it.next())
#3
You can combine the benefits of generators with the certainty of len(), by creating your own iterable object:
class MyIterable(object):
def __init__(self, n):
self.n = n
def __len__(self):
return self.n
def __iter__(self):
self._gen = self._generator()
return self
def _generator(self):
# Put your generator code here
i = 0
while i < self.n:
yield i
i += 1
def next(self):
return next(self._gen)
mi = MyIterable(100)
print len(mi)
for i in mi:
print i,
This is basically a simple implementation of xrange, which returns an object you can take the len of, but doesn't create an explicit list.
Is it possible to do following without the i?
for i in range(some_number):
# do something
If you just want to do something N amount of times and don't need the iterator.
Off the top of my head, no.
I think the best you could do is something like this:
def loop(f,n):
for i in xrange(n): f()
loop(lambda: <insert expression here>, 5)
But I think you can just live with the extra i variable.
Here is the option to use the _ variable, which in reality, is just another variable.
for _ in range(n):
do_something()
Note that _ is assigned the last result that returned in an interactive python session:
>>> 1+2
3
>>> _
3
For this reason, I would not use it in this manner. I am unaware of any idiom as mentioned by Ryan. It can mess up your interpreter.
>>> for _ in xrange(10): pass
...
>>> _
9
>>> 1+2
3
>>> _
9
And according to Python grammar, it is an acceptable variable name:
identifier ::= (letter|"_") (letter | digit | "_")*
You may be looking for
for _ in itertools.repeat(None, times): ...
this is THE fastest way to iterate times times in Python.
The general idiom for assigning to a value that isn't used is to name it _.
for _ in range(times):
do_stuff()
What everyone suggesting you to use _ isn't saying is that _ is frequently used as a shortcut to one of the gettext functions, so if you want your software to be available in more than one language then you're best off avoiding using it for other purposes.
import gettext
gettext.bindtextdomain('myapplication', '/path/to/my/language/directory')
gettext.textdomain('myapplication')
_ = gettext.gettext
# ...
print _('This is a translatable string.')
Here's a random idea that utilizes (abuses?) the data model (Py3 link).
class Counter(object):
def __init__(self, val):
self.val = val
def __nonzero__(self):
self.val -= 1
return self.val >= 0
__bool__ = __nonzero__ # Alias to Py3 name to make code work unchanged on Py2 and Py3
x = Counter(5)
while x:
# Do something
pass
I wonder if there is something like this in the standard libraries?
You can use _11 (or any number or another invalid identifier) to prevent name-colision with gettext. Any time you use underscore + invalid identifier you get a dummy name that can be used in for loop.
May be answer would depend on what problem you have with using iterator?
may be use
i = 100
while i:
print i
i-=1
or
def loop(N, doSomething):
if not N:
return
print doSomething(N)
loop(N-1, doSomething)
loop(100, lambda a:a)
but frankly i see no point in using such approaches
Instead of an unneeded counter, now you have an unneeded list.
Best solution is to use a variable that starts with "_", that tells syntax checkers that you are aware you are not using the variable.
x = range(5)
while x:
x.pop()
print "Work!"
I generally agree with solutions given above. Namely with:
Using underscore in for-loop (2 and more lines)
Defining a normal while counter (3 and more lines)
Declaring a custom class with __nonzero__ implementation (many more lines)
If one is to define an object as in #3 I would recommend implementing protocol for with keyword or apply contextlib.
Further I propose yet another solution. It is a 3 liner and is not of supreme elegance, but it uses itertools package and thus might be of an interest.
from itertools import (chain, repeat)
times = chain(repeat(True, 2), repeat(False))
while next(times):
print 'do stuff!'
In these example 2 is the number of times to iterate the loop. chain is wrapping two repeat iterators, the first being limited but the second is infinite. Remember that these are true iterator objects, hence they do not require infinite memory. Obviously this is much slower then solution #1. Unless written as a part of a function it might require a clean up for times variable.
We have had some fun with the following, interesting to share so:
class RepeatFunction:
def __init__(self,n=1): self.n = n
def __call__(self,Func):
for i in xrange(self.n):
Func()
return Func
#----usage
k = 0
#RepeatFunction(7) #decorator for repeating function
def Job():
global k
print k
k += 1
print '---------'
Job()
Results:
0
1
2
3
4
5
6
---------
7
If do_something is a simple function or can be wrapped in one, a simple map() can do_something range(some_number) times:
# Py2 version - map is eager, so it can be used alone
map(do_something, xrange(some_number))
# Py3 version - map is lazy, so it must be consumed to do the work at all;
# wrapping in list() would be equivalent to Py2, but if you don't use the return
# value, it's wastefully creating a temporary, possibly huge, list of junk.
# collections.deque with maxlen 0 can efficiently run a generator to exhaustion without
# storing any of the results; the itertools consume recipe uses it for that purpose.
from collections import deque
deque(map(do_something, range(some_number)), 0)
If you want to pass arguments to do_something, you may also find the itertools repeatfunc recipe reads well:
To pass the same arguments:
from collections import deque
from itertools import repeat, starmap
args = (..., my args here, ...)
# Same as Py3 map above, you must consume starmap (it's a lazy generator, even on Py2)
deque(starmap(do_something, repeat(args, some_number)), 0)
To pass different arguments:
argses = [(1, 2), (3, 4), ...]
deque(starmap(do_something, argses), 0)
We can use the while & yield, we can create our own loop function like this. Here you can refer to the official documentation.
def my_loop(start,n,step = 1):
while start < n:
yield start
start += step
for x in my_loop(0,15):
print(x)
#Return first n items of the iterable as a list
list(itertools.islice(iterable, n))
Taken from http://docs.python.org/2/library/itertools.html
If you really want to avoid putting something with a name (either an iteration variable as in the OP, or unwanted list or unwanted generator returning true the wanted amount of time) you could do it if you really wanted:
for type('', (), {}).x in range(somenumber):
dosomething()
The trick that's used is to create an anonymous class type('', (), {}) which results in a class with empty name, but NB that it is not inserted in the local or global namespace (even if a nonempty name was supplied). Then you use a member of that class as iteration variable which is unreachable since the class it's a member of is unreachable.
What about:
while range(some_number):
#do something