Develop Reference

Checking the return value of API-Functions Python

I am using an C++ API via Python using ctypes.CDLL:
api = CDLL(f'{path}/dll_name.dll')
This api has functions like open, remove, delete, select, read, write, etc... Whenever I use these functions it returns something. When I call these api functions via debugger one by one, I can of course see the return value.
However, since the return value of these functions is just some information of what happened after calling this function, it is not really necessary to save it in some variable, I simply call them and proceed.
Return values are integers between 0 to 15;
0: everything okay
1: license missing
2: wrong name
...
I would like to call 3 of these functions one after one with also checking if their return value is 0, otherwise raise some error. For that, the simple way would be something following, however it is also not that elegant, since I have to rewrite the same thing lots of time...:
status = api.select('something')
if not status:
pass
else:
raise SomeError(status)
status = api.open('something')
if not status:
pass
else:
raise SomeError(status)
...
Is there a way to handle this in a better way?

ctypes functions can assign a callable as an errcheck attribute that will post-process the return value:
errcheck
Assign a Python function or another callable to this
attribute. The callable will be called with three or more arguments:
callable(result, func, arguments)
result is what the foreign function
returns, as specified by the restype attribute.
func is the foreign function object itself, this allows reusing the
same callable object to check or post process the results of several
functions.
arguments is a tuple containing the parameters originally passed to
the function call, this allows specializing the behavior on the
arguments used.
The object that this function returns will be returned from the
foreign function call, but it can also check the result value and
raise an exception if the foreign function call failed.
Below is a simple C function that returns the error code passed to it, and a Python example that uses error check to raise exceptions if the return value is non-zero.
test.c
#ifdef _WIN32
# define API __declspec(dllexport)
#else
# define API
#endif
API int func(int retval) {
return retval;
}
test.py
import ctypes as ct
class FuncError(Exception): ...
class LicenseMissingError(FuncError): ...
class WrongNameError(FuncError): ...
def validate(result, func, arguments):
# requires Python 3.10 for match statement, use if/elif/else otherwise
match result:
case 0: return None # no error
case 1: raise LicenseMissingError()
case 2: raise WrongNameError()
case _: raise FuncError(result)
dll = ct.CDLL('./test')
dll.func.errcheck = validate
for i in range(4):
try:
print(dll.func(i))
except FuncError as e:
print(type(e), e)
Output:
None
<class '__main__.LicenseMissingError'>
<class '__main__.WrongNameError'>
<class '__main__.FuncError'> 3
Your example could look like the following, where any error would stop execution by raising an exception or use try/except to handle the exception gracefully in one place.
api = CDLL(f'{path}/dll_name.dll')
api.select.errcheck = validate
api.open.errcheck = validate
api.select('something')
api.open('something')

Why was Python decorator chaining designed to work backwards? What is the logic behind this order?

To start with, my question here is about the semantics and the logic behind why the Python language was designed like this in the case of chained decorators. Please notice the nuance how this is different from the question
How decorators chaining work?
Link: How decorators chaining work? It seems quite a number of other users had the same doubts, about the call order of chained Python decorators. It is not like I can't add a __call__ and see the order for myself. I get this, my point is, why was it designed to start from the bottom, when it comes to chained Python decorators?
E.g.
def first_func(func):
def inner():
x = func()
return x * x
return inner
def second_func(func):
def inner():
x = func()
return 2 * x
return inner
#first_func
#second_func
def num():
return 10
print(num())

Quoting the documentation on decorators:
The decorator syntax is merely syntactic sugar, the following two function definitions are semantically equivalent:
def f(arg):
...
f = staticmethod(f)
#staticmethod
def f(arg):
...
From this it follows that the decoration in
#a
#b
#c
def fun():
...
is equivalent to
fun = a(b(c(fun)))
IOW, it was designed like that because it's just syntactic sugar.
For proof, let's just decorate an existing function and not return a new one:
def dec1(f):
print(f"dec1: got {vars(f)}")
f.dec1 = True
return f
def dec2(f):
print(f"dec2: got {vars(f)}")
f.dec2 = True
return f
#dec1
#dec2
def foo():
pass
print(f"Fully decked out: {vars(foo)}")
prints out
dec2: got {}
dec1: got {'dec2': True}
Fully decked out: {'dec2': True, 'dec1': True}

TL;DR
g(f(x)) means applying f to x first, then applying g to the output.
Omit the parentheses, add # before and line break after each function name:
#g
#f
x
(Syntax only valid if x is the definition of a function/class.)
Abstract explanation
The reasoning behind this design decision becomes fairly obvious IMHO, if you remember what the decorator syntax - in its most abstract and general form - actually means. So I am going to try the abstract approach to explain this.
It is all about syntax
To be clear here, the distinguishing factor in the concept of the "decorator" is not the object underneath it (so to speak) nor the operation it performs. It is the special syntax and the restrictions for it. Thus, a decorator at its core is nothing more than feature of Python grammar.
The decorator syntax requires a target to be decorated. Initially (see PEP 318) the target could only be function definitions; later class definitions were also allowed to be decorated (see PEP 3129).
Minimal valid syntax
Syntactically, this is valid Python:
def f(): pass
#f
class Target: pass # or `def target: pass`
However, this will (perhaps unsuprisingly) cause a TypeError upon execution. As has been reiterated multiple times here and in other posts on this platform, the above is equivalent to this:
def f(): pass
class Target: pass
Target = f(Target)
Minimal working decorator
The TypeError stems from the fact that f lacks a positional argument. This is the obvious logical restriction imposed by what a decorator is supposed to do. Thus, to achieve not only syntactically valid code, but also have it run without errors, this is sufficient:
def f(x): pass
#f
class Target: pass
This is still not very useful, but it is enough for the most general form of a working decorator.
Decoration is just application of a function to the target and assigning the output to the target's name.
Chaining functions ⇒ Chaining decorators
We can ignore the target and what it is or does and focus only on the decorator. Since it merely stands for applying a function, the order of operations comes into play, as soon as we have more than one. What is the order of operation, when we chain functions?
def f(x): pass
def g(x): pass
class Target: pass
Target = g(f(Target))
Well, just like in the composition of purely mathematical functions, this implies that we apply f to Target first and then apply g to the result of f. Despite g appearing first (i.e. further left), it is not what is applied first.
Since stacking decorators is equivalent to nesting functions, it seems obvious to define the order of operation the same way. This time, we just skip the parentheses, add an # symbol in front of the function name and a line break after it.
def f(x): pass
def g(x): pass
#g
#f
class Target: pass
But, why though?
If after the explanation above (and reading the PEPs for historic background), the reasoning behind the order of operation is still not clear or still unintuitive, there is not really any good answer left, other than "because the devs thought it made sense, so get used to it".
PS
I thought I'd add a few things for additional context based on all the comments around your question.
Decoration vs. calling a decorated function
A source of confusion seems to be the distinction between what happens when applying the decorator versus calling the decorated function.
Notice that in my examples above I never actually called target itself (the class or function being decorated). Decoration is itself a function call. Adding #f above the target is calling the f and passing the target to it as the first positional argument.
A "decorated function" might not even be a function
The distinction is very important because nowhere does it say that a decorator actually needs to return a callable (function or class). f being just a function means it can return whatever it wants. This is again valid and working Python code:
def f(x): return 3.14
#f
def target(): return "foo"
try:
target()
except Exception as e:
print(repr(e))
print(target)
Output:
TypeError("'float' object is not callable")
3.14
Notice that the name target does not even refer to a function anymore. It just holds the 3.14 returned by the decorator. Thus, we cannot even call target. The entire function behind it is essentially lost immediately before it is even available to the global namespace. That is because f just completely ignores its first positional argument x.
Replacing a function
Expanding this further, if we want, we can have f return a function. Not doing that seems very strange, considering it is used to decorate a function. But it doesn't have to be related to the target at all. Again, this is fine:
def bar(): return "bar"
def f(x): return bar
#f
def target(): return "foo"
print(target())
print(target is bar)
Output:
bar
True
It comes down to convention
The way decorators are actually overwhelmingly used out in the wild, is in a way that still keeps a reference to the target being decorated around somewhere. In practice it can be as simple as this:
def f(x):
print(f"applied `f({x.__name__})`")
return
#f
def target(): return "foo"
Just running this piece of code outputs applied f(target). Again, notice that we don't call target here, we only called f. But now, the decorated function is still target, so we could add the call print(target()) at the bottom and that would output foo after the other output produced by f.
The fact that most decorators don't just throw away their target comes down to convention. You (as a developer) would not expect your function/class to simply be thrown away completely, when you use a decorator.
Decoration with wrapping
This is why real-life decorators typically either return the reference to the target at the end outright (like in the last example) or they return a different callable, but that callable itself calls the target, meaning a reference to the target is kept in that new callable's local namespace . These functions are what is usually referred to as wrappers:
def f(x):
print(f"applied `f({x.__name__})`")
def wrapper():
print(f"wrapper executing with {locals()=}")
return x()
return wrapper
#f
def target(): return "foo"
print(f"{target()=}")
print(f"{target.__name__=}")
Output:
applied `f(target)`
wrapper executing with locals()={'x': <function target at 0x7f1b2f78f250>}
target()='foo'
target.__name__='wrapper'
As you can see, what the decorator left us is wrapper, not what we originally defined as target. And the wrapper is what we call, when we write target().
Wrapping wrappers
This is the kind of behavior we typically expect, when we use decorators. And therefore it is not surprising that multiple decorators stacked together behave the way they do. The are called from the inside out (as explained above) and each adds its own wrapper around what it receives from the one applied before:
def f(x):
print(f"applied `f({x.__name__})`")
def wrapper_from_f():
print(f"wrapper_from_f executing with {locals()=}")
return x()
return wrapper_from_f
def g(x):
print(f"applied `g({x.__name__})`")
def wrapper_from_g():
print(f"wrapper_from_g executing with {locals()=}")
return x()
return wrapper_from_g
#g
#f
def target(): return "foo"
print(f"{target()=}")
print(f"{target.__name__=}")
Output:
applied `f(target)`
applied `g(wrapper_from_f)`
wrapper_from_g executing with locals()={'x': <function f.<locals>.wrapper_from_f at 0x7fbfc8d64f70>}
wrapper_from_f executing with locals()={'x': <function target at 0x7fbfc8d65630>}
target()='foo'
target.__name__='wrapper_from_g'
This shows very clearly the difference between the order in which the decorators are called and the order in which the wrapped/wrapping functions are called.
After the decoration is done, we are left with wrapper_from_g, which is referenced by our target name in global namespace. When we call it, wrapper_from_g executes and calls wrapper_from_f, which in turn calls the original target.

Python - When to use ()

Folks,
After much searching and reading I have come to the conclusion that methods use () and attributes don't.
Example using arr=np.arrange(25):
To find the size of the array I'd use: arr.size This is an attribute.
To find the max of the array I'd use: arr.max() This is a method.
To me, as an amateur python coder, I can't for the life of me tell what the actual difference is. (Note: I do understand that an attribute is found under _ _init_ _ while methods are not.) Is it really just the person who wrote the class randomly decided to put size as an attribute and max as method? Is there any way to tell while writing code to intuitively know when to use () and when not to without looking up the list of methods and attributes for each class?
Thanks for the help and sorry if I have any of the terms incorrect.

The full story is a little complicated (for all of it, see Python: the __getattribute__ method and descriptors and python __getattribute__ override and #property decorator and follow the links to the wiki on descriptor protocols), but in short, you write:
somevar.thing()
when you want to call the thing, and you write:
somevar.thing
when you want to use the value of the thing. This usage is the same as with functions and non-functions:
def f(arg):
print('f called, arg =', arg)
return 42
x = f('douglas adams')
print('f returned', x)
y = x
print('I just set y to x:', y)
y = f
print('this time I did not call', y)
which, when run (as Python3 or with from __future__ import print_function in Python2) prints:
f called, arg = douglas adams
f returned 42
I just set y to x: 42
this time I did not call <function f at ...>
If we try to do y = x(), it fails because we cannot call 42.
If you are going to define how the thing is to be used, define it as a method if it needs to be called, and as an instance attribute if it's just to be used (and/or set to some value). If you make the wrong decision—if you make it a used/set instance attribute, and it turns out later you need a function—you can work around it later with #property.
What's special about instance methods is that when you call them—or even when you don't—you get an extra self argument. The implementation by which this occurs is different in Python2 and Python3, but:
class K(object):
def method(self, arg):
print('method called, arg is', arg)
x = K()
x.method(42)
prints:
method called, arg is 42
Note that if we don't call it, we see this as a "bound method":
print('x.method is', x.method)
produces:
x.method is <bound method K.method of <__main__.K object at ...>>
If we look at K.method directly, the difference between Python2 and Python3 shows up:
$ python2 x.py
K.method is <unbound method K.method>
$ python3.6 x.py
K.method is <function K.method at ...>
but in the end it's all just descriptor protocols, with the CPython implementation able to do some short-cutting.

Function executes either the key is there or not in dictionary.get in python 2.7

I am working with dictionaries in Python and I am searching for a value using:
my_dictionary_object.get("key")
As we all know if the key is missing in dictionary, it will return None object.
So to save extra line and to make my code more interesting I am trying:
def my_function():
'''do some calculations'''
return missing_value_from_dictionary
Now here is the fun part; when I do
my_dictionary_object.get("key", my_function())
it executes the function whether the key is missing or not so I thought let's remove the brackets and I did:
my_dictionary_object.get("key", my_function)
and (just in case)
my_dictionary_object.get("key", lambda: my_function())
but the one without lambda didn't execute (because it was never called) same happened with the one with lambda.
TL;DR
My question is why is the function get executed if the key is present in the dictionary?
Is it something that I am doing wrong or something I am missing here?

In: my_dictionary_object.get("key", my_function()) the execution looks something like:
Evaluate the first argument ("key")
Evaluate the second argument which is an expression: my_function(). So let's call the function, and use the return value in its place. Very much like in: a = my_function(), python would call the function and put the returned value in its place.
Call the mydictionary_object.get(..) with the above two evaluated arguments.
In other words, dictionary.get("key", default) will simply return the 2nd argument if the key doesn't exist. If it is a lambda, lambda is returned. A lambda is an object. Note that in .get("key", my_function()), my_function() is not technically the second argument. The resultant value returned as a result of execution of that function is the second argument -- which explains where you're wrong, hopefully.
What you are looking for is actually captured in another container called defaultdict.
What you'd do is:
from collections import defaultdict
my_dictionary_object = defaultdict(my_function) # my_function should not take any argument.
my_dictionary_object["non-existing-key"] # Magic happens, see below.
What happens is, if the key (=x) doesn't exist, my_function is called without any arguments and the dictionary is updated with the value returned by the function against the key (=x). Something like:
if "key" not in dictionary:
dictionary["key"] = my_function()
return dictionary["key"]

Is there a Python equivalent to Ruby's respond_to?

Is a way to see if a class responds to a method in Python? like in ruby:
class Fun
def hello
puts 'Hello'
end
end
fun = Fun.new
puts fun.respond_to? 'hello' # true
Also is there a way to see how many arguments the method requires?

Hmmm .... I'd think that hasattr and callable would be the easiest way to accomplish the same goal:
class Fun:
def hello(self):
print 'Hello'
hasattr(Fun, 'hello') # -> True
callable(Fun.hello) # -> True
You could, of course, call callable(Fun.hello) from within an exception handling suite:
try:
callable(Fun.goodbye)
except AttributeError, e:
return False
As for introspection on the number of required arguments; I think that would be of dubious value to the language (even if it existed in Python) because that would tell you nothing about the required semantics. Given both the ease with which one can define optional/defaulted arguments and variable argument functions and methods in Python it seems that knowing the "required" number of arguments for a function would be of very little value (from a programmatic/introspective perspective).

Has method:
func = getattr(Fun, "hello", None)
if callable(func):
...
Arity:
import inspect
args, varargs, varkw, defaults = inspect.getargspec(Fun.hello)
arity = len(args)
Note that arity can be pretty much anything if you have varargs and/or varkw not None.

dir(instance) returns a list of an objects attributes.
getattr(instance,"attr") returns an object's attribute.
callable(x) returns True if x is callable.
class Fun(object):
def hello(self):
print "Hello"
f = Fun()
callable(getattr(f,'hello'))

I am no Ruby expert, so I am not sure if this answers your question. I think you want to check if an object contains a method. There are numerous ways to do so. You can try to use the hasattr() function, to see if an object hast the method:
hasattr(fun, "hello") #True
Or you can follow the python guideline don't ask to ask, just ask so, just catch the exception thrown when the object doesn't have the method:
try:
fun.hello2()
except AttributeError:
print("fun does not have the attribute hello2")