I have some code that looks like this.
condition = <expression>
while condition:
<some code>
I would like to be able to write that without having to write a separate statement to create the condition. E.g.,
while <create_condition(<expression>)>:
<some code>
Here are two possibilities that don't work, but that would scratch my itch.
with <expression> as condition:
<some code>
The problem with that is that it doesn't loop. If I embed a while inside the with I'm back where I started.
Define my own function to do this.
def looping_with(<expression>, <some code>):
<define looping_with>
The problem with this is that if <some code> is passed as a lambda expression it is limited to a single expression. None of the workarounds I've seen are attractive.
If <some code> is passed as an actual def one gets a syntax error. You can't pass a function definition as an argument to another function.
One could define the function elsewhere and then pass the function. But the point of with, while, and lambda is that the code itself, not a reference to the code, is embedded in context. (The original version of my code, which is not terrible, is better than that.)
Any suggestions would be appreciated.
UPDATE: (As Dave Beazley likes to say: You're going to hate this.)
I hesitate to offer this example, but this is something like what I'm trying to do.
class Container:
def __init__(self):
self.value = None
class Get_Next:
def __init__(self, gen):
self.gen = gen
def __call__(self, limit, container):
self.runnable_gen = self.gen(limit, container)
return self
def get_next(self):
try:
next(self.runnable_gen)
return True
except StopIteration:
return False
#Get_Next
def a_generator(limit, container):
i = 0
while i < limit:
container.value = i
yield
i += 1
container = Container()
gen = a_generator(5, container)
while gen.get_next():
print(container.value)
print('Done')
When run, the output is:
0
1
2
3
4
Done
P.S. Lest you think this is too far out, there is a very easy way to produce the same result. Remove the decorator from a_generator and then run:
for _ in a_generator(5, container):
print(container.value)
print('Done')
The result is the same.
The problem is that for _ in <something> is too ugly for me.
So, what I'm really looking for is a way to get the functionality of for _ in <something> with nicer syntax. The syntax should (a) indicate that we are establishing a context and (b) looping within that context. Hence the request for a combination of with and while.
You could a context manager class that would help in doing something like that:
class Condition:
def __init__(self, cond):
self.cond = cond
def __enter__(self):
return self
def __exit__(self, exc_type, exc_val, exc_tb):
pass
def __call__(self, *args, **kwargs):
return self.cond(*args, **kwargs)
with Condition(lambda x: x != 3) as condition:
a = 0
while condition(a):
print('a:', a)
a += 1
Output:
a: 0
a: 1
a: 2
Related
Assume I have two classes that use threads
class foo(threading.Thread):
def __init__(self):
threading.Thread.__init__(self,name="foo=>bar")
self.var1 = {}
def run(self):
while True
value, name = getvalue() // name is an string
self.var1[name] = value
bar(self)
class bar(threading.Thread):
def __init__(self,fooInstance):
threading.Thread.__init__(self,name="bar")
def run(self):
while True
arg = myfunction() // somefunction (not shown for simplicity)
val = myOtherfunction(fooInstance.var1[arg]) //other function
print(val)
f = foo()
f.start()
The variable var1 in foo will change over time and bar needs to be aware of these changes. It makes sense to me, but I wonder if there is something fundamental here that could fail eventually. is this correct in python?
The actual sharing part is the same question as "how do I share a value with another object?" without threads, and all the same solutions will work.
For example. you're already passing the foo instance into the bar initializer, so just get it from there:
class bar(threading.Thread):
def __init__(self,fooInstance):
threading.Thread.__init__(self,name="bar")
self.var1 = fooInstance.var1
But is this thread-safe?
Well, yes, but only because you never actually start the background thread. But I assume in your real code, you're going to have two threads running at the same time, both accessing that var1 value. In which case it's not thread-safe without some kind of synchronization. For example:
class foo(threading.Thread):
def __init__(self):
threading.Thread.__init__(self,name="foo=>bar")
self.var1 = {}
self.var1lock = threading.Lock()
class bar(threading.Thread):
def __init__(self,fooInstance):
threading.Thread.__init__(self,name="bar")
self.var1 = fooInstance.var1
self.var1lock = fooInstance.var1lock
And now, instead of this:
self.var1[name] = value
… you do this:
with self.var1lock:
self.var1[name] = value
And likewise, instead of this:
val = myOtherfunction(fooInstance.var1[arg]) //other function
… you do this:
with self.var1lock:
var1arg = var1[arg]
val = myOtherfunction(var1arg)
Or… as it turns out, in CPython, updating a value for a single key in a dict (only a builtin dict, not a subclass or custom mapping class!) has always been atomic, and probably always will be. If you want to rely on that fact, you can. But I'd only do that if the lock turned out to be a significant performance issue. And I'd comment every use of it to make it clear, too.
If you'd rather pass values instead of share them, the usual answer is queue.Queue or one of its relatives.
But this requires a redesign of your program. For example, maybe you want to pass each new/changed key-value pair over the queue. That would go something like this:
class foo(threading.Thread):
def __init__(self):
threading.Thread.__init__(self,name="foo=>bar")
self.var1 = {}
self.q = queue.Queue()
def run(self):
b = bar(self)
b.start()
while True:
value, name = getvalue() // name is an string
self.var1[name] = value
self.q.put((name, value))
class bar(threading.Thread):
def __init__(self,fooInstance):
threading.Thread.__init__(self,name="bar")
self.var1 = copy.deepcopy(fooInstance.var1)
self.q = fooInstance.q
def _checkq(self):
while True:
try:
key, val = self.q.get_nowait()
except queue.Empty:
break
else:
self.var1[key] = val
def run(self):
while True:
self._checkq()
arg = myfunction() // somefunction (not shown for simplicity)
val = myOtherfunction(fooInstance.var1[arg]) //other function
print(val)
I have a function of the form:
def my_func(my_list):
for i, thing in enumerate(my_list):
my_val = another_func(thing)
if i == 0:
# do some stuff
else:
if my_val == something:
return my_func(my_list[:-1])
# do some other stuff
The recursive part is getting called enough that I am getting a RecursionError, so I am trying to replace it with a while loop as explained here, but I can't work out how to reconcile this with the control flow statements in the function. Any help would be gratefully received!
There may be a good exact answer, but the most general (or maybe quick-and-dirty) way to switch from recursion to iteration is to manage the stack yourself. Just do manually what programming language does implicitly and have your own unlimited stack.
In this particular case there is tail recursion. You see, my_func recursive call result is not used by the caller in any way, it is immediately returned. What happens in the end is that the deepest recursive call's result bubbles up and is being returned as it is. This is what makes #outoftime's solution possible. We are only interested in into-recursion pass, as the return-from-recursion pass is trivial. So the into-recursion pass is replaced with iterations.
def my_func(my_list):
run = True
while run:
for i, thing in enumerate(my_list):
my_val = another_func(thing)
if i == 0:
# do some stuff
else:
if my_val == something:
my_list = my_list[:-1]
break
# do some other stuff
This is an iterative method.
Decorator
class TailCall(object):
def __init__(self, __function__):
self.__function__ = __function__
self.args = None
self.kwargs = None
self.has_params = False
def __call__(self, *args, **kwargs):
self.args = args
self.kwargs = kwargs
self.has_params = True
return self
def __handle__(self):
if not self.has_params:
raise TypeError
if type(self.__function__) is TailCaller:
return self.__function__.call(*self.args, **self.kwargs)
return self.__function__(*self.args, **self.kwargs)
class TailCaller(object):
def __init__(self, call):
self.call = call
def __call__(self, *args, **kwargs):
ret = self.call(*args, **kwargs)
while type(ret) is TailCall:
ret = ret.__handle__()
return ret
#TailCaller
def factorial(n, prev=1):
if n < 2:
return prev
return TailCall(factorial)(n-1, n * prev)
To use this decorator simply wrap your function with #TailCaller decorator and return TailCall instance initialized with required params.
I'd like to say thank you for inspiration to #o2genum and to Kyle Miller who wrote an excellent article about this problem.
Despite how good is to remove this limitation, probably, you have to be
aware of why this feature is not officially supported.
I have the following decorator that is supposed to wrap the implementation of test case functions within a try/except block and print the log if an exception occurs.
def print_log(test_case):
#wraps(test_case)
def run_test(self):
try:
test_case(self)
except:
Test_Loop.failure_teardown(self)
raise
return run_test
This however does not seem to work on one of my test cases that calls a yield generator
Please bear with me as this is a basic example:
class Test_Loop:
# ton of implementation here (e.g. initialization, etc)
def runIt(self, name, ip, port):
# code here
#print_log
def test_log_looper(self):
for l in self.links:
# initialize variables seen below and other stuff
for n in names:
# do stuff
for i in ips:
# do stuff
for p in ports:
yield self.runIt, l, n, i, p
From debugging, when the decorator is applied, it seems that it does not even enter the first loop. What am I doing wrong?
You need to iterate over your generator. Modify your decorator like this:
def print_log(test_case):
#wraps(test_case)
def run_test(self):
try:
for _ in test_case(self): pass
except:
Test_Loop.failure_teardown(self)
raise
return run_test
I have a nested function that I'm using as a callback in pyglet:
def get_stop_function(stop_key):
def stop_on_key(symbol, _):
if symbol == getattr(pyglet.window.key, stop_key):
pyglet.app.exit()
return stop_on_key
pyglet.window.set_handler('on_key_press', get_stop_function('ENTER'))
But then I run into problems later when I need to reference the nested function again:
pyglet.window.remove_handler('on_key_press', get_stop_function('ENTER'))
This doesn't work because of the way python treats functions:
my_stop_function = get_stop_function('ENTER')
my_stop_function is get_stop_function('ENTER') # False
my_stop_function == get_stop_function('ENTER') # False
Thanks to two similar questions I understand what is going on but I'm not sure what the workaround is for my case. I'm looking through the pyglet source code and it looks like pyglet uses equality to find the handler to remove.
So my final question is: how can I override the inner function's __eq__ method (or some other dunder) so that identical nested functions will be equal?
(Another workaround would be to store a reference to the function myself, but that is duplicating pyglet's job, will get messy with many callbacks, and anyways I'm curious about this question!)
Edit: actually, in the questions I linked above, it's explained that methods have value equality but not reference equality. With nested functions, you don't even get value equality, which is all I need.
Edit2: I will probably accept Bi Rico's answer, but does anyone know why the following doesn't work:
def get_stop_function(stop_key):
def stop_on_key(symbol, _):
if symbol == getattr(pyglet.window.key, stop_key):
pyglet.app.exit()
stop_on_key.__name__ = '__stop_on_' + stop_key + '__'
stop_on_key.__eq__ = lambda x: x.__name__ == '__stop_on_' + stop_key + '__'
return stop_on_key
get_stop_function('ENTER') == get_stop_function('ENTER') # False
get_stop_function('ENTER').__eq__(get_stop_function('ENTER')) # True
You could create a class for your stop functions and define your own comparison method.
class StopFunction(object):
def __init__(self, stop_key):
self.stop_key = stop_key
def __call__(self, symbol, _):
if symbol == getattr(pyglet.window.key, self.stop_key):
pyglet.app.exit()
def __eq__(self, other):
try:
return self.stop_key == other.stop_key
except AttributeError:
return False
StopFunciton('ENTER') == StopFunciton('ENTER')
# True
StopFunciton('ENTER') == StopFunciton('FOO')
# False
the solution is to keep a dictionary containing the generated functions around,
so that when you make the second call, you get the same object as in the first call.
That is, simply build some memoization logic, or use one of the libraries
existing with memoizing decorators:
ALL_FUNCTIONS = {}
def get_stop_function(stop_key):
if not stop_key in ALL_FUNCTIONS:
def stop_on_key(symbol, _):
if symbol == getattr(pyglet.window.key, stop_key):
pyglet.app.exit()
ALL_FUNCTIONS[stop_key] = stop_on_key
else:
stop_on_key = ALL_FUNCTIONS[stop_key]
return stop_on_key
You can generalize Bi Rico's solution to allow wrapping any functions up with some particular equality function pretty easily.
The first problem is defining what the equality function should check. I'm guessing for this case, you want the code to be identical (meaning functions created from the same def statement will be equal, but two functions created from character-for-character copies of the def statement will not), and the closures to be equal (meaning that if you call get_stop_function with two equal but non-identical stop_keys the functions will be equal), and nothing else to be relevant. But that's just a guess, and there are many other possibilities.
Then you just wrap a function the same way you'd wrap any other kind of object; just make sure __call__ is one of the things you delegate:
class EqualFunction(object):
def __init__(self, f):
self.f = f
def __eq__(self, other):
return (self.__code__ == other.__code__ and
all(x.cell_contents == y.cell_contents
for x, y in zip(self.__closure__, other.__closure__)))
def __getattr__(self, attr):
return getattr(self.f, attr)
def __call__(self, *args, **kwargs):
return self.f(*args, **kwargs)
If you want to support other dunder methods that aren't required to go through getattr (I don't think any of them are critical for functions, but I could be wrong…), either do it explicitly (as with __call__) or loop over them and add a generic wrapper to the type for each one.
To use the wrapper:
def make_f(i):
def f():
return i
return EqualFunction(f)
f1 = f(0)
f2 = f(0.0)
assert f1 == f2
Or, notice that EqualFunction actually works as a decorator, which may be more readable.
So, for your code:
def get_stop_function(stop_key):
#EqualFunction
def stop_on_key(symbol, _):
if symbol == getattr(pyglet.window.key, stop_key):
pyglet.app.exit()
return stop_on_key
I am maintaining a little library of useful functions for interacting with my company's APIs and I have come across (what I think is) a neat question that I can't find the answer to.
I frequently have to request large amounts of data from an API, so I do something like:
class Client(object):
def __init__(self):
self.data = []
def get_data(self, offset = 0):
done = False
while not done:
data = get_more_starting_at(offset)
self.data.extend(data)
offset += 1
if not data:
done = True
This works fine and allows me to restart the retrieval where I left off if something goes horribly wrong. However, since python functions are just regular objects, we can do stuff like:
def yo():
yo.hi = "yo!"
return None
and then we can interrogate yo about its properties later, like:
yo.hi => "yo!"
my question is: Can I rewrite my class-based example to pin the data to the function itself, without referring to the function by name. I know I can do this by:
def get_data(offset=0):
done = False
get_data.data = []
while not done:
data = get_more_starting_from(offset)
get_data.data.extend(data)
offset += 1
if not data:
done = True
return get_data.data
but I would like to do something like:
def get_data(offset=0):
done = False
self.data = [] # <===== this is the bit I can't figure out
while not done:
data = get_more_starting_from(offset)
self.data.extend(data) # <====== also this!
offset += 1
if not data:
done = True
return self.data # <======== want to refer to the "current" object
Is it possible to refer to the "current" object by anything other than its name?
Something like "this", "self", or "memememe!" is what I'm looking for.
I don't understand why you want to do this, but it's what a fixed point combinator allows you to do:
import functools
def Y(f):
#functools.wraps(f)
def Yf(*args):
return inner(*args)
inner = f(Yf)
return Yf
#Y
def get_data(f):
def inner_get_data(*args):
# This is your real get data function
# define it as normal
# but just refer to it as 'f' inside itself
print 'setting get_data.foo to', args
f.foo = args
return inner_get_data
get_data(1, 2, 3)
print get_data.foo
So you call get_data as normal, and it "magically" knows that f means itself.
You could do this, but (a) the data is not per-function-invocation, but per function (b) it's much easier to achieve this sort of thing with a class.
If you had to do it, you might do something like this:
def ybother(a,b,c,yrselflambda = lambda: ybother):
yrself = yrselflambda()
#other stuff
The lambda is necessary, because you need to delay evaluation of the term ybother until something has been bound to it.
Alternatively, and increasingly pointlessly:
from functools import partial
def ybother(a,b,c,yrself=None):
#whatever
yrself.data = [] # this will blow up if the default argument is used
#more stuff
bothered = partial(ybother, yrself=ybother)
Or:
def unbothered(a,b,c):
def inbothered(yrself):
#whatever
yrself.data = []
return inbothered, inbothered(inbothered)
This last version gives you a different function object each time, which you might like.
There are almost certainly introspective tricks to do this, but they are even less worthwhile.
Not sure what doing it like this gains you, but what about using a decorator.
import functools
def add_self(f):
#functools.wraps(f)
def wrapper(*args,**kwargs):
if not getattr(f, 'content', None):
f.content = []
return f(f, *args, **kwargs)
return wrapper
#add_self
def example(self, arg1):
self.content.append(arg1)
print self.content
example(1)
example(2)
example(3)
OUTPUT
[1]
[1, 2]
[1, 2, 3]