I have some code that looks like this:
from functools import lru_cache
#lru_cache()
def get_cheese(type):
print('{}? We\'re all out.'.format(type))
return None
get_cheese(type='cheddar')
get_cheese('cheddar')
print(get_cheese.cache_info())
cache_info() reports that there were two misses - yet I called the function with the same argument.
It actually took some doing, but I figured out that it's because in one instance I was using the keyword arg, and the other I was using a positional argument.
But why?
The wrapper created by functools.lru_cache makes no attempt to inspect or copy the signature of the wrapped function. The Python version is defined as
def wrapper(*args, **kwargs):
...
key = make_key(args, kwds, typed)
...
and as you can see, it builds the key used in the cache based on args and kwargs, without knowing whether any positional or keyword arguments are equivalent. The C version similarly builds the key without caring about the original signature.
As for why it's designed that way? I'm not aware of the original rationale. It could have been deliberate, either for simplicity of implementation or to avoid the overhead of matching keyword arguments to positionals, or it could have been an oversight. If you think changing it would be worthwhile, you could bring it up on the issue tracker.
Related
I'm looking at the (presumably) built-in module called "typing.py", and I'm trying to understand what's going on in there. I'm specifically looking at the code below, where we see an input argument called "typed=False". What does that even mean?
As I understand it, "func=None" appears to mean "no function is allowed in the inputs" (correct me if I'm wrong), where "func" refers to the object-type "function" (which presumably most programmers are familiar with because it's a basic concept). But what about "typed=False"?
def _tp_cache(func=None, /, *, typed=False):
"""Internal wrapper caching __getitem__ of generic types with a fallback to
original function for non-hashable arguments.
"""
def decorator(func):
cached = functools.lru_cache(typed=typed)(func)
_cleanups.append(cached.cache_clear)
#functools.wraps(func)
def inner(*args, **kwds):
try:
return cached(*args, **kwds)
except TypeError:
pass # All real errors (not unhashable args) are raised below.
return func(*args, **kwds)
return inner
if func is not None:
return decorator(func)
return decorator
As I understand it, "func=None" appears to mean "no function is allowed in the inputs" (correct me if I'm wrong), where "func" refers to the object-type "function" (which presumably most programmers are familiar with because it's a basic concept). But what about "typed=False"?
None of that is correct.
func is a parameter name, not a type. =None means that it defaults to None if no value is provided. typed is another parameter name; =False means it defaults to False. The * and / in the parameters list indicate that func is a positional parameter (i.e. it's always just the first argument, the caller doesn't need to say func=) and typed is a keyword argument (i.e. it must always be specified as typed=SOMETHING by the caller).
Trying to understand Python code without having a basic understanding of Python syntax is going to be extremely difficult; it's not something that you can guess at as you've attempted to do here, especially if you're diving straight into decorator (higher-order) functions, which are a relatively advanced feature and require that you already have a very firm grasp of how basic functions work.
I recommend making your way through a Python tutorial, e.g. https://docs.python.org/3/tutorial/.
This has been a source of confusion and frustration for years now. Say you import a particularly poorly documented module and some method that you need to you only has **kwargs for its arguments, how are you supposed to know what keys that method is checking for?
def test(**kwargs):
if 'greeting' in kwargs:
print(kwargs['greeting'])
If i were to call text, how would i know that 'greeting is something the method was looking for?
test(greeting='hi)
Some simplistic cases the IDE can help out with, but most use cases seem to be out of the IDE's scope
Think of kwargs as a dictionary. There is no way to tell from the outside what key-value combinations the method will accept (in your case the test method is essentially a black box) but this is the point of having documentation. Without kwargs, some function headers would get extremely cluttered.
Use documentation!
The subprocess-module's docs is a good example. If you are using a newer version of python (3.7 or 3.6 with backport), consider using dataclasses as an alternative to kwargs, if it fits your usecase.
If it's not documented, your only recourse is to read the source.
Adding a **kwargs argument to a function is used when you don't want to explicitly define the arguments which must be named.
A trivial example:
If a function takes as an argument another function which is undetermined and may have different kwargs each time
def foo(func,**kwargs):
print(func)
return func(**kwargs)
You won't know what the function is explicitly looking for.
You can have in your example
def foo(greeting=None):
which shows the function is looking for greeting but it can be None
While I'm going through Python code and seeing functions called, I notice things like
functionCall(argument='something')
or
someclass.functionCall(argument='something')
I've played around with it and noticed you have to name that variable with the same name as the one in the scope of the function or class function itself. Is that just a convention (for useful naming) or is there more to it that I'm missing?
Those are just standard keyword arguments.
They are mainly useful when calling functions that usually assume default values, but the user is interested in passing a custom value without affecting the other defaults.
For example:
def foo(a='a', b='b', c='c'):
print a + b + c
Then:
>>> foo()
'abc'
>>> foo(c='x') # don't know or care about a and b's value
'abx'
This is called a keyword argument. Keyword arguments have a couple distinct advantages:
They can be specified in any order by the caller.
They have default values, which means the caller doesn't have to provide a value for each argument if the defaults are sensible.
They can make code more readable. For example, foo(width=100, height=200) is easier to understand than foo(100, 200). It is also easier to use, because you don't have to remember whether the function takes width,height or height,width.
It is good to have named arguments as arguments can be specified in any order by using named arguments.
Even required arguments (like object, which has no default value) can be named, and named arguments can appear in any order.
Also see This
Python's argument passing mechanisms are extremely flexible.
cons: too many arguments to a function. Solutions: split into multiple functions, pass some args together in a dictionary or object.
cons: bad variable names. Solution: give variables more descriptive names.
Or remember the correct order .. :)
class xyz:
def __init__ (self, a='1', b='2'):
print a,b
xyz(b=3,a=4)
xyz(a=5,b=6)
>>4 3
>>5 6
these are keyword parameters. They enable you to pass parameters in any order (useful for skipping optional parameters)
Please note though, calling functions that way comes with a little bit of overhead:
def test(a,b,c):
return a+b+c
def t1():
return test(1,2,3)
def t2():
return test(a=1, b=2, c=3)
timeit(t1)
0.23918700218200684
timeit(t2)
0.2716050148010254
I have a function that is taking an arbitrary set of arguments and then choosing the correct function to process them based on the types of the arguments.
My current approach is using a decorator on all of the processing functions that checks the types of the arguments, and then iterating through all of the functions until one accepts the arguments.
Something about this seems a little hacky to me, and as a relatively new python programmer, I was wondering if there was a more 'pythonic' method to doing this.
so, currently, I have something like this:
def function_router(*args):
for func in functions: #functions is a list of functions
try:
return func(*args)
except TypeError:
pass
#probably raise an exception if no function works
and each function in 'functions' would have a decorator like this:
def accepts(*types) :
def my_decorator(func):
def wrapped(*args, **kwargs):
for i in range(len(types)):
if not isinstance(args[i], types[i]):
raise TypeError('Type error, %s not instance of %s, it is %s' %(args[i],types[i], type(args[i])))
return func(*args, **kwargs)
return wrapped
return my_decorator
EDIT: Oh man, I actually really liked reading all of the solutions. The answer I chose was most effective for what I'm currently doing, but I learned something from all of the answers, so thank you all for your time.
Perhaps the right way to do this is to use keyword arguments, instead of relying on the types of the arguments. That way, you don't have to decorate the little functions, just name the arguments correctly. It will also let you take advantage of Python's duck typing.
It sounds like you're trying to describe multimethods, for which GvR has provided a nice recipe in the form of a decorator.
This might be a simple question:
Is there any difference between the two folowing:
def myfunc(a_list = [], **kwargs):
my_arg = kwargs.get('my_arg', None)
pass
and
def myfucn(a_list = [], my_arg = None):
pass
If not, which would be considered more pythonic?
Thanks,
-Matt
For a simple function, it's more Pythonic to explicitly define your arguments. Unless you have a legit requirement to accept any number of unknown or variable arguments, the **kwargs method is adding unnecessary complexity.
Bonus: Never initialize a list in the function definition! This can have unexpected results by causing it to persist because lists are mutable!
The first one can take virtually any keyword arguments provided (regardless of the fact that it only uses one of them), whereas the second can only take two. Neither is more Pythonic, you simply use the one appropriate for the task.
Also, the second is not "keyword arguments" but rather a "default value".
The second alternative allows my_arg to be passed as a positional rather than a keyword argument. I would consider it unpythonic to declare **kwargs when you don't actually use them for anything.