I've read several arguments comparing partial vs lambda, but most of them talked about how partial is more flexible (not limited to expressions) and gives info about the wrapped function. But I want to consider this from the caller's perspective. Here's my situation.
I have a function that takes a 1-argument modifier function. A request is passed into that modifier function to be modified:
def my_func(request, modifier):
modifier(request)
I'm also building some utilities that makes it easier to create parameterized modifier functions, e.g. adding/modifying URL params to the request. I thought of two ways of doing it, but not sure which one is better.
Option 1
def add_params(request, params):
for param in params:
# Manipulate the request with param.
This way, callers can use functools.partial to bind the params, like this:
modifier = functools.partial(add_params, params={'abc':'123'})
Option 2
def add_params(params):
def func(request):
for param in params:
# Modify request with param.
return func
Then callers use it like this:
modifier = add_params({'abc':'123'})
Question
If I don't care about function introspection, are there any downsides to using option 2? Would option 2 run into late binding issues? (Although my use case doesn't run into that). I really like how option 2 is easier for callers to use.
The two functions are completely isomorphic to each other from a mathematical perspective (though their efficiency may vary):
# Option 1
(Request, Params) -> None
# Option 2
Params -> (Request -> None)
For your purpose, I would say option 2 offers the most convenience since the function is already curried, so you can not only avoid partial but can also compose them easily:
import functools
def compose(*fs):
return functools.reduce(lambda f, g: lambda x: f(g(x)), fs)
modifier = compose(add_params({'abc':'123'}),
add_params({'def':'456'}))
If you ever want to call the function directly you can always do:
add_params({'abc':'123'})(request)
which is not really all that involved compared to option 1:
add_params(request, {'abc':'123'})
Late binding shouldn't pose an issue unless you use variables from outside the function, and if you do there's always a way to work around it.
Unfortunately option 2 has the disadvantage of being annoying to define, but this can be simplified using decorators:
def curry_request(f):
def wrapper(*args, **kwargs):
def inner(request):
f(request, *args, **kwargs)
return inner
return wrapper
#curry_request
def add_params(request, params):
# do something
def partial(func, *args, **keywords):
def newfunc(*fargs, **fkeywords):
newkeywords = keywords.copy()
newkeywords.update(fkeywords)
return func(*(args + fargs), **newkeywords)
newfunc.func = func
newfunc.args = args
newfunc.keywords = keywords
return newfunc
Compared with partial function implementation code,your option 2 is also good,i don't think it has any downsides in your situation.But functools.partial is a common way to result in a simplified signature,
if you want to ruturn a new partial function for another function,you can still invoke partial func.if you want to use option 2 model,you may need to implement a new function
Related
I have written the following decorator:
def partializable(fn):
def arg_partializer(*fixable_parameters):
def partialized_fn(dynamic_arg):
return fn(dynamic_arg, *fixable_parameters)
return partialized_fn
return arg_partializer
The purpose of this decorator is to break the function call into two calls. If I decorate the following:
#partializable
def my_fn(dyn, fix1, fix2):
return dyn + fix1 + fix2
I then can do:
core_accepting_dynamic_argument = my_fn(my_fix_1, my_fix_2)
final_result = core_accepting_dynamic_argument(my_dyn)
My problem is that the now decorated my_fn exhibits the following signature: my_fn(*fixable_parameters)
I want it to be: my_fn(fix1, fix2)
How can I accomplish this? I probably have to use wraps or the decorator module, but I need to preserve only part of the original signature and I don't know if that's possible.
Taking inspiration from https://stackoverflow.com/a/33112180/9204395, it's possible to accomplish this by manually altering the signature of arg_partializer, since only the signature of fn is known in the relevant scope and can be handled with inspect.
from inspect import signature
def partializable(fn):
def arg_partializer(*fixable_parameters):
def partialized_fn(dynamic_arg):
return fn(dynamic_arg, *fixable_parameters)
return partialized_fn
# Override signature
sig = signature(fn)
sig = sig.replace(parameters=tuple(sig.parameters.values())[1:])
arg_partializer.__signature__ = sig
return arg_partializer
This is not particularly elegant, but as I think about the problem I'm starting to suspect that this (or a conceptual equivalent) is the only possible way to pull this stunt. Feel free to contradict me.
This will seem trivial perhaps, but it is a condition that I run into fairly frequently and would like to find a more elegant way of writing this code. The method, while not terribly relevant to the question, takes a text value and an optional is_checked value to create a radio button (using dominate). In this case, I can't set 'checked' to None, or false - it either has to be there or not. It doesn't seem like I should have to write the 'input' line twice though, just to optionally add an argument.
def _get_radio_button(text: str, is_checked=False):
with label(text, cls="radio-inline") as lbl:
if is_checked:
input(text, type="radio", name="optradio", checked='checked')
else:
input(text, type="radio", name="optradio")
return lbl
This would be my second approach, but it is the same lines of code and less readable - though perhaps a tiny bit more DRY.
a = dict(type='radio', name='optradio')
if is_checked:
a['checked']='checked'
with label(text, cls="radio-inline") as lbl:
input(text, **a)
Question: How can I handle this code case with the fewest lines possible without sacrificing readability?
Your code looks fine, except obviously for the naming of a, which could be input_opts or something like that.
Another possibility to make it a bit clearer is to use direct keyword arguments for the common stuff and just inject the optional ones using **. When only one is optional, this can be quite short, e.g.:
checked_arg = {'checked': 'checked'} if is_checked else {}
with label(text, cls="radio-inline") as lbl:
input(text, type="radio", name="optradio", **checked_arg)
Only as concept :) You can decorate in this way own or alien (library) functions. Even more, you can make decorator as class (with __call__ method which will decorate underlying function) which can be parameterized with simple "morphisms" of underlying function arguments (they may be list of functions - as arguments of decorator class constructor). Also you can make more declarative style decorator and to inspect underlying function arguments (for default values, for example) - you are limited only by own fantasy :) So:
from functools import wraps
def adapt_gui_args(callable):
#wraps(callable)
def w(*args, **kwargs):
if kwargs.pop('is_checked', False): kwargs['checked'] = 'checked'
return callable(*args, **kwargs)
return w
# may be decorated with adapt_gui_args if it's your function
def input(*args, **kwargs):
print("args: ", args)
print("kwargs: ", kwargs)
# decorate input function outside its source body
input = adapt_gui_args(input)
def test(is_checked=False):
input(1, 2, type="radio", is_checked=is_checked)
test(False)
test(True)
I am attempting to integrate a very old system and a newer system at work. The best I can do is to utilize an RSS firehouse type feed the system utilizes. The goal is to use this RSS feed to make the other system perform certain actions when certain people do things.
My idea is to wrap a decorator around certain functions to check if the user (a user ID provided in the RSS feed) has permissions in the new system.
My current solution has a lot of functions that look like this, which are called based on an action field in the feed:
actions_dict = {
...
'action1': function1
}
actions_dict[RSSFEED['action_taken']](RSSFEED['user_id'])
def function1(user_id):
if has_permissions(user_id):
# Do this function
I want to create a has_permissions decorator that takes the user_id so that I can remove this redundant has_permissions check in each of my functions.
#has_permissions(user_id)
def function1():
# Do this function
Unfortunately, I am not sure how to write such a decorator. All the tutorials I see have the #has_permissions() line with a hardcoded value, but in my case it needs to be passed at runtime and will be different each time the function is called.
How can I achieve this functionality?
In your question, you've named both, the check of the user_id, as well as the wanted decorator has_permissions, so I'm going with an example where names are more clear: Let's make a decorator that calls the underlying (decorated) function when the color (a string) is 'green'.
Python decorators are function factories
The decorator itself (if_green in my example below) is a function. It takes a function to be decorated as argument (named function in my example) and returns a function (run_function_if_green in the example). Usually, the returned function calls the passed function at some point, thereby "decorating" it with other actions it might run before or after it, or both.
Of course, it might only conditionally run it, as you seem to need:
def if_green(function):
def run_function_if_green(color, *args, **kwargs):
if color == 'green':
return function(*args, **kwargs)
return run_function_if_green
#if_green
def print_if_green():
print('what a nice color!')
print_if_green('red') # nothing happens
print_if_green('green') # => what a nice color!
What happens when you decorate a function with the decorator (as I did with print_if_green, here), is that the decorator (the function factory, if_green in my example) gets called with the original function (print_if_green as you see it in the code above). As is its nature, it returns a different function. Python then replaces the original function with the one returned by the decorator.
So in the subsequent calls, it's the returned function (run_function_if_green with the original print_if_green as function) that gets called as print_if_green and which conditionally calls further to that original print_if_green.
Functions factories can produce functions that take arguments
The call to the decorator (if_green) only happens once for each decorated function, not every time the decorated functions are called. But as the function returned by the decorator that one time permanently replaces the original function, it gets called instead of the original function every time that original function is invoked. And it can take arguments, if we allow it.
I've given it an argument color, which it uses itself to decide whether to call the decorated function. Further, I've given it the idiomatic vararg arguments, which it uses to call the wrapped function (if it calls it), so that I'm allowed to decorate functions taking an arbitrary number of positional and keyword arguments:
#if_green
def exclaim_if_green(exclamation):
print(exclamation, 'that IS a nice color!')
exclaim_if_green('red', 'Yay') # again, nothing
exclaim_if_green('green', 'Wow') # => Wow that IS a nice color!
The result of decorating a function with if_green is that a new first argument gets prepended to its signature, which will be invisible to the original function (as run_function_if_green doesn't forward it). As you are free in how you implement the function returned by the decorator, it could also call the original function with less, more or different arguments, do any required transformation on them before passing them to the original function or do other crazy stuff.
Concepts, concepts, concepts
Understanding decorators requires knowledge and understanding of various other concepts of the Python language. (Most of which aren't specific to Python, but one might still not be aware of them.)
For brevity's sake (this answer is long enough as it is), I've skipped or glossed over most of them. For a more comprehensive speedrun through (I think) all relevant ones, consult e.g. Understanding Python Decorators in 12 Easy Steps!.
The inputs to decorators (arguments, wrapped function) are rather static in python. There is no way to dynamically pass an argument like you're asking. If the user id can be extracted from somewhere at runtime inside the decorator function however, you can achieve what you want..
In Django for example, things like #login_required expect that the function they're wrapping has request as the first argument, and Request objects have a user attribute that they can utilize. Another, uglier option is to have some sort of global object you can get the current user from (see thread local storage).
The short answer is no: you cannot pass dynamic parameters to decorators.
But... you can certainly invoke them programmatically:
First let's create a decorator that can perform a permission check before executing a function:
import functools
def check_permissions(user_id):
def decorator(f):
#functools.wraps(f)
def wrapper(*args, **kw):
if has_permissions(user_id):
return f(*args, **kw)
else:
# what do you want to do if there aren't permissions?
...
return wrapper
return decorator
Now, when extracting an action from your dictionary, wrap it using the decorator to create a new callable that does an automatic permission check:
checked_action = check_permissions(RSSFEED['user_id'])(
actions_dict[RSSFEED['action_taken']])
Now, when you call checked_action it will first check the permissions corresponding to the user_id before executing the underlying action.
You may easily work around it, example:
from functools import wraps
def some_function():
print("some_function executed")
def some_decorator(decorator_arg1, decorator_arg2):
def decorate(func):
#wraps(func)
def wrapper(*args, **kwargs):
print(decorator_arg1)
ret = func(*args, **kwargs)
print(decorator_arg2)
return ret
return wrapper
return decorate
arg1 = "pre"
arg2 = "post"
decorated = some_decorator(arg1, arg2)(some_function)
In [4]: decorated()
pre
some_function executed
post
I have a decorator function my_fun(I,k) and it is applied to a function add(x,y) as such
#my_fun(4,5)
def add(x,y): return x+y
I am new to Python would like to know if I am writing the my_fun function
How can I access x,y in the add method in my_fun?
How can I access the return value of add in the decorator function?
I am a little confused on syntax and concepts any explanation would be help.
A decorator consists of the decorator function and a function wrapper (and if you want additional arguments for the decorator another outer layer of function around it):
# Takes the arguments for the decorator and makes them accessible inside
def my_fun(decorator_argument1, decorator_argument2):
# Takes the function so that it can be wrapped.
def wrapfunc(func):
# Here we are actually going to wrap the function ... finally
def wrapper(*args, **kwargs):
# Call the function with the args and kwargs
res = func(*args, **kwargs)
# return this result
return res
# Replace the decorated function with the wrapper
return wrapper
# Return the wrapper for the function wrapper :-)
return wrapfunc
In your case if you only want to use the decorator with your function you don't need to bother with the *args, **kwargs and replace it by:
def wrapper(x, y):
# Here you can do stuff with x and y, i.e. print(x)
# Call the function with x and y
res = func(x, y)
# Here you can do stuff with the result, i.e. res = res * decorator_argument1
return res
I indicated the places where you can access x and y and the result.
If you want to predefine values for x and y a custom decorator is not the best way. You could use defaults:
def add(x=4,y=5): return x+y
add() # returns 9
add(2) # returns 7
add(5, 10) # returns 15
or if you want to fix an argument you should use functools.partial
If you're passing arguments to the decorator with #my_fun(4, 5), you need three levels of nested functions to implement the decorator in the simplest way. The outer level is the "decorator factory". It returns the middle level function, the decorator. The decorator gets called with the function it's decorating as an argument and needs to return the inner most nested function, the wrapper. The wrapper function is the one that gets called by the user.
def decorator_factory(deco_arg, deco_arg2): # name this whatever you want to use with #syntax
def decorator(func):
def wrapper(func_arg, func_arg2):
# This is a closure!
# In here you can write code using the arguments from the enclosing scpoes. e.g.:
return func(func_arg*deco_arg, func_arg2*deco_arg2) # uses args from all levels
return wrapper
return decorator
The inner functions here are closures. They can see the variables in the scope surrounding the place they were defined in, even after the functions those scope belonged to have finished running.
(Note, if you want your decorator to be able to decorate many different functions, you may want the wrapper function to accept *args and **kwargs and pass them along to func. The example above only works for functions that accept exactly two arguments. A limitation like that may be perfectly reasonable for some uses, but not always.)
I have written several functions that run sequentially, each one taking as its input the output of the previous function so in order to run it, I have to run this line of code
make_list(cleanup(get_text(get_page(URL))))
and I just find that ugly and inefficient, is there a better way to do sequential function calls?
Really, this is the same as any case where you want to refactor commonly-used complex expressions or statements: just turn the expression or statement into a function. The fact that your expression happens to be a composition of function calls doesn't make any difference (but see below).
So, the obvious thing to do is to write a wrapper function that composes the functions together in one place, so everywhere else you can make a simple call to the wrapper:
def get_page_list(url):
return make_list(cleanup(get_text(get_page(url))))
things = get_page_list(url)
stuff = get_page_list(another_url)
spam = get_page_list(eggs)
If you don't always call the exact same chain of functions, you can always factor out into the pieces that you frequently call. For example:
def get_clean_text(page):
return cleanup(get_text(page))
def get_clean_page(url):
return get_clean_text(get_page(url))
This refactoring also opens the door to making the code a bit more verbose but a lot easier to debug, since it only appears once instead of multiple times:
def get_page_list(url):
page = get_page(url)
text = get_text(page)
cleantext = cleanup(text)
return make_list(cleantext)
If you find yourself needing to do exactly this kind of refactoring of composed functions very often, you can always write a helper that generates the refactored functions. For example:
def compose1(*funcs):
#wraps(funcs[0])
def composed(arg):
for func in reversed(funcs):
arg = func(arg)
return arg
return composed
get_page_list = compose1(make_list, cleanup, get_text, get_page)
If you want a more complicated compose function (that, e.g., allows passing multiple args/return values around), it can get a bit complicated to design, so you might want to look around on PyPI and ActiveState for the various existing implementations.
You could try something like this. I always like separating train wrecks(the book "Clean Code" calls those nested functions train wrecks). This is easier to read and debug. Remember you probably spend twice as long reading your code than writing it so make it easier to read. You will thank yourself later.
url = get_page(URL)
url_text = get_text(url)
make_list(cleanup(url_text))
# you can also encapsulate that into its own function
def build_page_list_from_url(url):
url = get_page(URL)
url_text = get_text(url)
return make_list(cleanup(url_text))
Options:
Refactor: implement this series of function calls as one, aptly-named method.
Look into decorators. They're syntactic sugar for 'chaining' functions in this way. E.g. implement cleanup and make_list as a decorators, then decorate get_text with them.
Compose the functions. See code in this answer.
You could shorten constructs like that with something like the following:
class ChainCalls(object):
def __init__(self, *funcs):
self.funcs = funcs
def __call__(self, *args, **kwargs):
result = self.funcs[-1](*args, **kwargs)
for func in self.funcs[-2::-1]:
result = func(result)
return result
def make_list(arg): return 'make_list(%s)' % arg
def cleanup(arg): return 'cleanup(%s)' % arg
def get_text(arg): return 'get_text(%s)' % arg
def get_page(arg): return 'get_page(%r)' % arg
mychain = ChainCalls(make_list, cleanup, get_text, get_page)
print( mychain('http://is.gd') )
Output:
make_list(cleanup(get_text(get_page('http://is.gd'))))