I have a model class with getter and setter methods, and the occasional static methods. I would like to enforce the usage of unicode strings as arguments for specific methods and using decorators was the first idea I had. Now I have something like this:
import types
class require_unicode(object):
def __init__(self, function):
self.f = function
def __call__(self, string):
if not isinstance(string, types.UnicodeType):
raise ValueError('String is not unicode')
return self.f(string)
class Foo(object):
something = 'bar'
#staticmethod
#require_unicode
def do_another(self, string):
return ' '.join(['baz', string])
#require_unicode
def set_something(self, string):
self.something = string
foo = Foo()
foo.set_something('ValueError is raised')
foo.set_something(u'argument count error')
foo.do_another('ValueError is raised')
foo.do_another(u'argument count error')
In the above code the method call inside decorator's __call__ fails due to wrong argument count (because the 'foo' object ref is missing?). Before doing something stupid I wanted to ask you guys. How this should be done?
I think, your problem is with the #staticmethod decorator, not with your require_unicode decorator. Staticmethods, unlike classmethods don't receive the reference to the class as the first argument, so your argument signature is wrong.
You must either change do_another to be a #classmethod, or remove self from the arguments.
EDIT: and, mind you, - #classmethod-decorated methods receive the class as the first argument, while instance methods receive the reference to the instance of the class (self). So it is a good idea to name the first argument to a classmethod "cls" or something, not "self" so it doesn't confuse anyone.
I would think that this was unpythonic - you should never check the type of your arguments, but instead check that they have the necessary methods and attributes. The simplest way to do this is by assuming they are there and getting an exception otherwise, but I guess you could do getattr too. Just don't check the type of things.
Another option is to use assertions. It depends on whether passing a non-unicode type into your methods should be considered a programming error that should be evident during development.
import types
class Foo:
def set_something(self, string):
assert isinstance(string, types.UnicodeType), 'String is not unicode'
self.something = string
This will raise an AssertionError exception whenever string is not of type unicode, but only when the Python interpretter is run in "deubg" mode. If you run Python with the -O option, the assert is efficiently ignored by the interpretter.
Related
I was watching Learn Python - Full Course for Beginners [Tutorial] on YouTube here.
At timestamp 4:11:54 the tutor explains what a class function is, however from my background in object oriented programming using other languages I thought the correct term would be method?
Now I am curious if there is a difference between a class function and method?
They are wrong. But, it's a minor confusion they made and doing video courses which involves speaking and typing can be certainly challenging. No big deal.
When the function belongs to a class, it's a method (a more specialized form of a function). When it's outside of a class it's a function.
How do I know they are wrong?
You use this syntax to create one in Python:
class SomeClass:
#classmethod
def the_method(cls, vars):
....
def instance_method(self, vars):
...
It's not a #classfunction decorator. It's a #classmethod decorator.
See the docs at https://docs.python.org/3/library/functions.html#classmethod
Method is the correct term for a function in a class. Methods and functions are pretty similar to each other. The only difference is that a method is called with an object and has the possibility to modify data of an object. Functions can modify and return data but they dont have an impact on objects.
Edit : Class function and method both mean the same thing although saying class function is not the right way to say it.
You dont need to create an instance with a classmethod. Formatter.format(“foo”) vs Formatter().format(“foo”).
That also means you cant store state/configuration on the instance because there is no instance. Also inheritance and polymorphism may not work as well with classmethods - i.e. if I was planning to bring them into play on a classmethod I’d be very cautious about actual behavior.
In practice you usually want to use regular methods, except on class methods that create instances (which are often called factory methods). If you really don't need an instance, maybe a standalone function would do the job just as well? Python does not require functions to live only on classes (like Java).
As far as terminology goes, don’t sweat it besides exam considerations. Method vs function are not that different except that one has an instance (or the class in a #classmethod) as first argument.
Typically, if it's indented under a class XXX: declaration, I'd call a method. If it was standalone, I'd call it a function.
OK, read further only if you don't mind getting confused a bit...
Besides the distinction between all these things is rather fluid in practice. classmethods can be called from instances, functions can be dynamically added to classes...
Here's some amusing stuff. Most of it is curiosity, except for the create_me classmethod which is the main reason to use classmethods as factories, but it shows the boundaries are fluid-ish.
This is not stuff you'd typically want to do, but it does walk through some of the finer distinctions on how methods/functions on classes can behave.
class Anc:
#classmethod
def format(cls, msg):
"I dont need an instance to work"
print(f"\n{cls}.format({msg.upper()})")
def __init__(self, keep=1) -> None:
self.keep = keep
def format_instance(self, msg):
"I do and it allows me to use configuration stored in it"
print(f"\n{self}.format_instance({msg[:self.keep]=})")
#classmethod
def create_me(cls, *args, **kwargs):
""" a factory is the main reason for class methods """
return cls(*args, **kwargs)
class Child1(Anc):
#classmethod
def format(cls, msg):
print(f"\n{cls}.format({msg.lower()})")
#staticmethod
def format_static(msg):
print(f"\nLonely static without cls or instance 😭{msg}😭")
class Child2(Anc):
def format_instance(self, msg):
"replace with stars"
print(f"\n{self}.format_instance({'*' * len(msg)})")
def wannabe_method(self, msg):
"is this a function or a method?"
print(f"\nwannabe_method({self},{msg=}).")
def wont_work_as_method():
"too many arguments when called as method"
print(f"\nwont_work_as_method()")
Anc.format("calling format as a classmethod. No instance needed!")
anc = Anc(keep=2)
anc.format("calling format through an instance. It'll automatically get the class as first argument!")
Child1.format("calling Child1's format, which does lower")
Child2.format("calling Child2's format, which will up on Anc.format")
Child1.format_static("this is a static")
child1 = Child1(keep=3)
child1.format_instance("this message will get truncated...")
child2 = Child2()
try:
child2.added_method("hey just got added!")
except (AttributeError,) as e:
print(f"\n❌as expected this fails {e} cuz missing method")
Child2.added_method = wannabe_method
child2.added_method("hey just got added! self gets the instance magically")
try:
wannabe_method("nope not gonna work")
except (TypeError,) as e:
print(f"\n❌as expected this fails {e} because it only got 1 argument")
wannabe_method("FAKE INSTANCE!", "hackish work")
Child2.wont_work = wont_work_as_method
try:
child2.wont_work()
except (TypeError,) as e:
print(f"\n❌ oh no! no place for self {e}")
child2_through_factory = Child2.create_me()
child2_through_factory.format_instance("My God, it's full of stars")
child2_through_factory.format("to uppercase")
child1_through_factory = Child1.create_me()
child1_through_factory.format("TO LOWERCASE")
output:
<class '__main__.Anc'>.format(CALLING FORMAT AS A CLASSMETHOD. NO INSTANCE NEEDED!)
<class '__main__.Anc'>.format(CALLING FORMAT THROUGH AN INSTANCE. IT'LL AUTOMATICALLY GET THE CLASS AS FIRST ARGUMENT!)
<class '__main__.Child1'>.format(calling child1's format, which does lower)
<class '__main__.Child2'>.format(CALLING CHILD2'S FORMAT, WHICH WILL UP ON ANC.FORMAT)
Lonely static without cls or instance 😭this is a static😭
<__main__.Child1 object at 0x10a824460>.format_instance(msg[:self.keep]='thi')
❌as expected this fails 'Child2' object has no attribute 'added_method' cuz missing method
wannabe_method(<__main__.Child2 object at 0x10a824280>,msg='hey just got added! self gets the instance magically').
❌as expected this fails wannabe_method() missing 1 required positional argument: 'msg' because it only got 1 argument
wannabe_method(FAKE INSTANCE!,msg='hackish work').
❌ oh no! no place for self wont_work_as_method() takes 0 positional arguments but 1 was given
<__main__.Child2 object at 0x10a824220>.format_instance(**************************)
<class '__main__.Child2'>.format(TO UPPERCASE)
<class '__main__.Child1'>.format(to lowercase)
I am writing a class for a neural network and I want to give it some form of customization, so that you can choose different cost functions and regularizations. For this I want to set them as default parameters in the __init__() method.
But when I pass MyClass.static_method in my example, the Interpreter then tells me that MyClass is not (yet) defined. Why is this and is there a nicer workaround than mine?
You can of course just set the static method as a default parameter, but then other problems arise. For example, if I want to access the functions name (which I actually want), I cannot use __name__ rightaway. I know how to do it another way, by accessing static_method.__func__.__name__. But this seems clumsy and as you get a staticmethod object, seems like its not intended to be used this way.
class MyClass:
#staticmethod
def static_method():
do_something()
def __init__(self, func=MyClass.static_method, func2=static_method):
self.name = func.__name__ #Does not work
self.name2 = func2.__func__.__name__ #Should work
I did expect for the MyClass.static_method to work, but the class does not seem to exist then. So, one last time, why?
The reason you're having problems with your static method usage as a default argument is due to a combination of two issues.
The first issue is that the default argument needs to be well defined when the def statement is run, not only when the function is called. That's because the default argument gets built into the function object, rather than being recalculated each time the function runs (this is the same reason why a mutable default argument like an empty list is often an error). Anyway, this is why you can't use MyClass.static_method as the default argument, since MyClass isn't defined yet when the function is being defined (the class object is only made after all its contents have been created).
The next issue is that a staticmethod object doesn't have all the same attributes and methods as a regular function. Normally this doesn't matter, as when you access it through a class object (e.g. MyClass.static_method once MyClass exists) or through an instance (e.g. self.static_method), it will be callable and have a __name__. But that's because you get the underlying function in those situations, rather than the staticmethod object itself. The staticmethod object itself is a descriptor, but not a callable.
So neither of these functions will work correctly:
class MyClass:
#staticmethod
def static_method():
pass
def foo(self, func=MyClass.static_method): # won't work because MyClass doesn't exist yet
pass
def bar(self, func=static_method): # this declaration will work (if you comment out foo)
name = func.__name__ # but this doesn't work when the bar() is called
func() # nor this, as func is the staticmethod object
What does work would be to use the actual function underlying the staticmethod object as the default:
def baz(self, func=static_method.__func__): # this works!
name = func.__name__
func()
This also works when you pass in some other function (or bound method), unlike the version of your code that used name = func.__func__.__name__.
DEFAULT = object()
class MyClass:
#staticmethod
def static_method():
do_something()
def __init__(self, func=DEFAULT, func2=DEFAULT):
self.name = self.static_method.__name__ if func is DEFAULT else func.__name__
self.name2 = self.static_method.__func__.__name__ if func2 is DEFAULT else func2.__func__.__name__
I guess??
Introduction
I have a Python class, which contains a number of methods. I want one of those methods to have a static counterpart—that is, a static method with the same name—which can handle more arguments. After some searching, I have found that I can use the #staticmethod decorator to create a static method.
Problem
For convenience, I have created a reduced test case which reproduces the issue:
class myclass:
#staticmethod
def foo():
return 'static method'
def foo(self):
return 'public method'
obj = myclass()
print(obj.foo())
print(myclass.foo())
I expect that the code above will print the following:
public method
static method
However, the code prints the following:
public method
Traceback (most recent call last):
File "sandbox.py", line 14, in <module>
print(myclass.foo())
TypeError: foo() missing 1 required positional argument: 'self'
From this, I can only assume that calling myclass.foo() tries to call its non-static counterpart with no arguments (which won't work because non-static methods always accept the argument self). This behavior baffles me, because I expect any call to the static method to actually call the static method.
I've tested the issue in both Python 2.7 and 3.3, only to receive the same error.
Questions
Why does this happen, and what can I do to fix my code so it prints:
public method
static method
as I would expect?
While it's not strictly possible to do, as rightly pointed out, you could always "fake" it by redefining the method on instantiation, like this:
class YourClass(object):
def __init__(self):
self.foo = self._instance_foo
#staticmethod
def foo():
print "Static!"
def _instance_foo(self):
print "Instance!"
which would produce the desired result:
>>> YourClass.foo()
Static!
>>> your_instance = YourClass()
>>> your_instance.foo()
Instance!
A similar question is here: override methods with same name in python programming
functions are looked up by name, so you are just redefining foo with an instance method. There is no such thing as an overloaded function in Python. You either write a new function with a separate name, or you provide the arguments in such a way that it can handle the logic for both.
In other words, you can't have a static version and an instance version of the same name. If you look at its vars you'll see one foo.
In [1]: class Test:
...: #staticmethod
...: def foo():
...: print 'static'
...: def foo(self):
...: print 'instance'
...:
In [2]: t = Test()
In [3]: t.foo()
instance
In [6]: vars(Test)
Out[6]: {'__doc__': None, '__module__': '__main__', 'foo': <function __main__.foo>}
Because attribute lookup in Python is something within the programmer's control, this sort of thing is technically possible. If you put any value into writing code in a "pythonic" way (using the preferred conventions and idioms of the python community), it is very likely the wrong way to frame a problem / design. But if you know how descriptors can allow you to control attribute lookup, and how functions become bound functions (hint: functions are descriptors), you can accomplish code that is roughly what you want.
For a given name, there is only one object that will be looked up on a class, regardless of whether you are looking the name up on an instance of the class, or the class itself. Thus, the thing that you're looking up has to deal with the two cases, and dispatch appropriately.
(Note: this isn't exactly true; if an instance has a name in its attribute namespace that collides with one in the namespace of its class, the value on the instance will win in some circumstances. But even in those circumstances, it won't become a "bound method" in the way that you probably would wish it to.)
I don't recommend designing your program using a technique such as this, but the following will do roughly what you asked. Understanding how this works requires a relatively deep understanding of python as a language.
class StaticOrInstanceDescriptor(object):
def __get__(self, cls, inst):
if cls is None:
return self.instance.__get__(self)
else:
return self.static
def __init__(self, static):
self.static = static
def instance(self, instance):
self.instance = instance
return self
class MyClass(object):
#StaticOrInstanceDescriptor
def foo():
return 'static method'
#foo.instance
def foo(self):
return 'public method'
obj = MyClass()
print(obj.foo())
print(MyClass.foo())
which does print out:
% python /tmp/sandbox.py
static method
public method
Ended up here from google so thought I would post my solution to this "problem"...
class Test():
def test_method(self=None):
if self is None:
print("static bit")
else:
print("instance bit")
This way you can use the method like a static method or like an instance method.
When you try to call MyClass.foo(), Python will complain since you did not pass the one required self argument. #coderpatros's answer has the right idea, where we provide a default value for self, so its no longer required. However, that won't work if there are additional arguments besides self. Here's a function that can handle almost all types of method signatures:
import inspect
from functools import wraps
def class_overload(cls, methods):
""" Add classmethod overloads to one or more instance methods """
for name in methods:
func = getattr(cls, name)
# required positional arguments
pos_args = 1 # start at one, as we assume "self" is positional_only
kwd_args = [] # [name:str, ...]
sig = iter(inspect.signature(func).parameters.values())
next(sig)
for s in sig:
if s.default is s.empty:
if s.kind == s.POSITIONAL_ONLY:
pos_args += 1
continue
elif s.kind == s.POSITIONAL_OR_KEYWORD:
kwd_args.append(s.name)
continue
break
#wraps(func)
def overloaded(*args, **kwargs):
# most common case: missing positional arg or 1st arg is not a cls instance
isclass = len(args) < pos_args or not isinstance(args[0], cls)
# handle ambiguous signatures, func(self, arg:cls, *args, **kwargs);
# check if missing required positional_or_keyword arg
if not isclass:
for i in range(len(args)-pos_args,len(kwd_args)):
if kwd_args[i] not in kwargs:
isclass = True
break
# class method
if isclass:
return func(cls, *args, **kwargs)
# instance method
return func(*args, **kwargs)
setattr(cls, name, overloaded)
class Foo:
def foo(self, *args, **kwargs):
isclass = self is Foo
print("foo {} method called".format(["instance","class"][isclass]))
class_overload(Foo, ["foo"])
Foo.foo() # "foo class method called"
Foo().foo() # "foo instance method called"
You can use the isclass bool to implement the different logic for class vs instance method.
The class_overload function is a bit beefy and will need to inspect the signature when the class is declared. But the actual logic in the runtime decorator (overloaded) should be quite fast.
There's one signature that this solution won't work for: a method with an optional, first, positional argument of type Foo. It's impossible to tell if we are calling the static or instance method just by the signature in this case. For example:
def bad_foo(self, other:Foo=None):
...
bad_foo(f) # f.bad_foo(None) or Foo.bad_foo(f) ???
Note, this solution may also report an incorrect isclass value if you pass in incorrect arguments to the method (a programmer error, so may not be important to you).
We can get a possibly more robust solution by doing the reverse of this: first start with a classmethod, and then create an instance method overload of it. This is essentially the same idea as #Dologan's answer, though I think mine is a little less boilerplatey if you need to do this on several methods:
from types import MethodType
def instance_overload(self, methods):
""" Adds instance overloads for one or more classmethods"""
for name in methods:
setattr(self, name, MethodType(getattr(self, name).__func__, self))
class Foo:
def __init__(self):
instance_overload(self, ["foo"])
#classmethod
def foo(self, *args, **kwargs):
isclass = self is Foo
print("foo {} method called:".format(["instance","class"][isclass]))
Foo.foo() # "foo class method called"
Foo().foo() # "foo instance method called"
Not counting the code for class_overload or instance_overload, the code is equally succinct. Often signature introspection is touted as the "pythonic" way to do these kinds of things. But I think I'd recommend using the instance_method solution instead; isclass will be correct for any method signature, including cases where you call with incorrect arguments (a programmer error).
Let's say I want to override a function like __int__ in a Python class so that I may do something like this.
class A(object):
def __init__(self):
self.__int__ = lambda: 1
a = A()
print int(a)
I expect that it would output "1" here instead of produce this error message
TypeError: int() argument must be a string or a number, not 'A'
When __int__ instead becomes a method built into the class it works as expected. Why? (This problem exists with any of the double underscore functions also)
That appears to be one more bit of magic in the __ magic methods. Unlike other methods, they're not looked up on the class instances when called implicitly.
It's well documented that they don't get resolved with the __getattribute__ magic method (and it would be a nice paradox if they did, since __getattribute__ would have to call itself to resolve its own name). But not checking the instances surprises me.
It's discussed a bit here, under the header "Special Method Lookup":
http://segfaulthunter.github.io/articles/biggestsurprise/
For instances of new-style classes, all special method lookup that is done implicitely is done in the class struct. Thus changing an instance's __str__ attribute does not effect the result of str(). Still, explicitely getting the attribute from the instance gives you the method in the instance.
I will be curious to see if anyone else can offer a more detailed explanation.
If we for the moment ignore that you are asking about special methods, then your code would look like this:
class A(object):
def __init__(self):
self.a_fun = lambda: 1
Is written more clearly like this:
class A(object):
def __init__(self):
self._int = 1
def a_fun(self):
return self._int
The resulting code isn't exactly the same, but close enough for it to not make much of a difference. The only difference is that the _int name has to be looked up as an attribute.
But if we now change it back to be a special method, it looks like this:
class A(object):
def __init__(self):
self.__int__ = lambda: 1
vs:
class A(object):
def __init__(self):
self._int = 1
def __int__(self):
return self._int
And now there is a very big difference: The second variant works, the first one doesn't. This is because special methods is always looked up on the class, not the instance. This is by design.
So instead of trying to be clever, just write what is clear and readable. In Python that tends to work best. ;-)
I'm teaching myself Python and I see the following in Dive into Python section 5.3:
By convention, the first argument of any Python class method (the reference to the current instance) is called self. This argument fills the role of the reserved word this in C++ or Java, but self is not a reserved word in Python, merely a naming convention. Nonetheless, please don't call it anything but self; this is a very strong convention.
Considering that self is not a Python keyword, I'm guessing that it can sometimes be useful to use something else. Are there any such cases? If not, why is it not a keyword?
No, unless you want to confuse every other programmer that looks at your code after you write it. self is not a keyword because it is an identifier. It could have been a keyword and the fact that it isn't one was a design decision.
As a side observation, note that Pilgrim is committing a common misuse of terms here: a class method is quite a different thing from an instance method, which is what he's talking about here. As wikipedia puts it, "a method is a subroutine that is exclusively associated either with a class (in which case it is called a class method or a static method) or with an object (in which case it is an instance method).". Python's built-ins include a staticmethod type, to make static methods, and a classmethod type, to make class methods, each generally used as a decorator; if you don't use either, a def in a class body makes an instance method. E.g.:
>>> class X(object):
... def noclass(self): print self
... #classmethod
... def withclass(cls): print cls
...
>>> x = X()
>>> x.noclass()
<__main__.X object at 0x698d0>
>>> x.withclass()
<class '__main__.X'>
>>>
As you see, the instance method noclass gets the instance as its argument, but the class method withclass gets the class instead.
So it would be extremely confusing and misleading to use self as the name of the first parameter of a class method: the convention in this case is instead to use cls, as in my example above. While this IS just a convention, there is no real good reason for violating it -- any more than there would be, say, for naming a variable number_of_cats if the purpose of the variable is counting dogs!-)
The only case of this I've seen is when you define a function outside of a class definition, and then assign it to the class, e.g.:
class Foo(object):
def bar(self):
# Do something with 'self'
def baz(inst):
return inst.bar()
Foo.baz = baz
In this case, self is a little strange to use, because the function could be applied to many classes. Most often I've seen inst or cls used instead.
I once had some code like (and I apologize for lack of creativity in the example):
class Animal:
def __init__(self, volume=1):
self.volume = volume
self.description = "Animal"
def Sound(self):
pass
def GetADog(self, newvolume):
class Dog(Animal):
def Sound(this):
return self.description + ": " + ("woof" * this.volume)
return Dog(newvolume)
Then we have output like:
>>> a = Animal(3)
>>> d = a.GetADog(2)
>>> d.Sound()
'Animal: woofwoof'
I wasn't sure if self within the Dog class would shadow self within the Animal class, so I opted to make Dog's reference the word "this" instead. In my opinion and for that particular application, that was more clear to me.
Because it is a convention, not language syntax. There is a Python style guide that people who program in Python follow. This way libraries have a familiar look and feel. Python places a lot of emphasis on readability, and consistency is an important part of this.
I think that the main reason self is used by convention rather than being a Python keyword is because it's simpler to have all methods/functions take arguments the same way rather than having to put together different argument forms for functions, class methods, instance methods, etc.
Note that if you have an actual class method (i.e. one defined using the classmethod decorator), the convention is to use "cls" instead of "self".