I am getting more and more familiar with static methods and class methods and I know the difference between each. But I ran into a problem today where I would kind of like to reference both self and cls in a method.
The only way I know how to accomplish this is to make a normal class method (not with #classmethod, but simply with def) and call the class with it's name explicitely like so:
class myClass:
def __init__():
self._ser.connect('COM5')
def ask(self, message: str) -> str:
return myClass.clean_output(self._ser.query(message))
#staticmethod
def clean_output(dirty_string: str):
clean_string = dirty_string.strip().replace(chr(4))
return clean_string
This example is an over-simplified version of the philosophy. I would like to call a clean or parse function on data I get back, like from a serial device. Is there any way to implement the ask method like so?:
def ask(self, message: str) -> str:
return cls.clean_output(self._ser.query(message))
Or is it ok that I'm calling it with myClass explicitly like that? If it is, when should programmers use #classmethod and when is it permissible to use the class name itself? Is using the #classmethod decorator only really needed when you expect subclassing to happen?
Just call the static method on self:
def ask(self, message: str) -> str:
return self.clean_output(self._ser.query(message))
It is available there too.
Attributes on a class are always available on the instances too (provided there is no attribute on the instance itself with the same name masking it). Methods (be they regular, static or class methods) are no exception, they too are just attributes. Their binding behaviour doesn't matter here.
Related
So I was messing around with a readonly-modifyable class pattern which is pretty common in java. It involves creating a base class containig readonly properties, and extending that class for a modifyable version. Usually there is a function readonly() or something simular to revert a modifyable-version back to a readonly-version of itself.
Unfortunately you cannot directly define a setter for a property defined in a super class in python, but you can simply redefine it as shown below. Mor interestingly In python you got the 'magic function' super returning a proxy-object of the parent/super class which allows for a cheecky readonly() implementation which feels really hacky, but as far as I can test it just works.
class Readonly:
_t:int
def __init__(self, t: int):
self._t = t
#property
def t(self) -> int:
return self._t
class Modifyable(Readonly):
def __init__(self, t: int):
super().__init__(t)
#property
def t(self) -> int:
return self._t # can also be super().t
#t.setter
def t(self, t):
self._t = t
def readonly(self) -> Readonly:
return super()
In the above pattern I can call the readonly function and obtain a proxy to the parent object without having to instantiate a readonly version. The only problem would be that when setting the readonly-attribute instead of throwing AttributeError: can't set attribute, it will throw a AttributeError: 'super' object has no attribute '<param-name>'
So here are the question for this:
Can this cause problems (exposing the super proxy-object outside the class itself)?
I tested this on python 3.8.5, but not sure if that is by accident and goes against the 'python-semantics' so to speak?
Is there a better/more desirable way to achieve this? (I have no idea if its even worth the ambiguous error message in ragards to performance for example)
I Would love to hear opinions and/or insights into this
I wrote a class that can handle integers with arbitrary precision (just for learning purposes). The class takes a string representation of an integer and converts it into an instance of BigInt for further calculations.
Often times you need the numbers Zero and One, so I thought it would be helpfull if the class could return these. I tried the following:
class BigInt():
zero = BigInt("0")
def __init__(self, value):
####yada-yada####
This doesn't work. Error: "name 'BigInt' is not defined"
Then I tried the following:
class BigInt():
__zero = None
#staticmethod
def zero():
if BigInt.__zero is None:
BigInt.__zero = BigInt('0')
return BigInt.__zero
def __init__(self, value):
####yada-yada####
This actually works very well. What I don't like is that zero is a method (and thus has to be called with BigInt.zero()) which is counterintuitive since it should just refer to a fixed value.
So I tried changing zero to become a property, but then writing BigInt.zero returns an instance of the class property instead of BigInt because of the decorator used. That instance cannot be used for calculations because of the wrong type.
Is there a way around this issue?
A static property...? We call a static property an "attribute". This is not Java, Python is a dynamically typed language and such a construct would be really overcomplicating matters.
Just do this, setting a class attribute:
class BigInt:
def __init__(self, value):
...
BigInt.zero = BigInt("0")
If you want it to be entirely defined within the class, do it using a decorator (but be aware it's just a more fancy way of writing the same thing).
def add_zero(cls):
cls.zero = cls("0")
return cls
#add_zero
class BigInt:
...
The question is contradictory: static and property don't go together in this way. Static attributes in Python are simply ones that are only assigned once, and the language itself includes a very large number of these. (Most strings are interred, all integers < a certain value are pre-constructed, etc. E.g. the string module.). Easiest approach is to statically assign the attributes after construction as wim illustrates:
class Foo:
...
Foo.first = Foo()
...
Or, as he further suggested, using a class decorator to perform the assignments, which is functionally the same as the above. A decorator is, effectively, a function that is given the "decorated" function as an argument, and must return a function to effectively replace the original one. This may be the original function, say, modified with some annotations, or may be an entirely different function. The original (decorated) function may or may not be called as appropriate for the decorator.
def preload(**values):
def inner(cls):
for k, v in values.items():
setattr(cls, k, cls(v))
return cls
return inner
This can then be used dynamically:
#preload(zero=0, one=1)
class Foo:
...
If the purpose is to save some time on common integer values, a defaultdict mapping integers to constructed BigInts could be useful as a form of caching and streamlined construction / singleton storage. (E.g. BigInt.numbers[27])
However, the problem of utilizing #property at the class level intrigued me, so I did some digging. It is entirely possible to make use of "descriptor protocol objects" (which the #property decorator returns) at the class level if you punt the attribute up the object model hierarchy, to the metaclass.
class Foo(type):
#property
def bar(cls):
print("I'm a", cls)
return 27
class Bar(metaclass=Foo):
...
>>> Bar.bar
I'm a <class '__main__.Bar'>
<<< 27
Notably, this attribute is not accessible from instances:
>>> Bar().bar
AttributeError: 'Bar' object has no attribute 'bar'
Hope this helps!
Assume you're writing an abstract class and one or more of its non-abstract class methods require the concrete class to have a specific class attribute; e.g., if instances of each concrete class can be constructed by matching against a different regular expression, you might want to give your ABC the following:
#classmethod
def parse(cls, s):
m = re.fullmatch(cls.PATTERN, s)
if not m:
raise ValueError(s)
return cls(**m.groupdict())
(Maybe this could be better implemented with a custom metaclass, but try to ignore that for the sake of the example.)
Now, because overriding of abstract methods & properties is checked at instance creation time, not subclass creation time, trying to use abc.abstractmethod to ensure concrete classes have PATTERN attributes won't work — but surely there should be something there to tell anyone looking at your code "I didn't forget to define PATTERN on the ABC; the concrete classes are supposed to define their own." The question is: Which something is the most Pythonic?
Pile of decorators
#property
#abc.abstractmethod
def PATTERN(self):
pass
(Assume Python 3.4 or higher, by the way.) This can be very misleading to readers, as it implies that PATTERN should be an instance property instead of a class attribute.
Tower of decorators
#property
#classmethod
#abc.abstractmethod
def PATTERN(cls):
pass
This can be very confusing to readers, as #property and #classmethod normally can't be combined; they only work together here (for a given value of "work") because the method is ignored once it's overridden.
Dummy value
PATTERN = ''
If a concrete class fails to define its own PATTERN, parse will only accept empty input. This option isn't widely applicable, as not all use cases will have an appropriate dummy value.
Error-inducing dummy value
PATTERN = None
If a concrete class fails to define its own PATTERN, parse will raise an error, and the programmer gets what they deserve.
Do nothing. Basically a more hardcore variant of #4. There can be a note in the ABC's docstring somewhere, but the ABC itself shouldn't have anything in the way of a PATTERN attribute.
Other???
You can use the __init_subclass__ method which was introduced in Python 3.6 to make customizing class creation easier without resorting to metaclasses. When defining a new class, it is called as the last step before the class object is created.
In my opinion, the most pythonic way to use this would be to make a class decorator that accepts the attributes to make abstract, thus making it explicit to the user what they need to define.
from custom_decorators import abstract_class_attributes
#abstract_class_attributes('PATTERN')
class PatternDefiningBase:
pass
class LegalPatternChild(PatternDefiningBase):
PATTERN = r'foo\s+bar'
class IllegalPatternChild(PatternDefiningBase):
pass
The traceback might be as follows, and occurs at subclass creation time, not instantiation time.
NotImplementedError Traceback (most recent call last)
...
18 PATTERN = r'foo\s+bar'
19
---> 20 class IllegalPatternChild(PatternDefiningBase):
21 pass
...
<ipython-input-11-44089d753ec1> in __init_subclass__(cls, **kwargs)
9 if cls.PATTERN is NotImplemented:
10 # Choose your favorite exception.
---> 11 raise NotImplementedError('You forgot to define PATTERN!!!')
12
13 #classmethod
NotImplementedError: You forgot to define PATTERN!!!
Before showing how the decorator is implemented, it is instructive to show how you could implement this without the decorator. The nice thing here is that if needed you could make your base class an abstract base class without having to do any work (just inherit from abc.ABC or make the metaclass abc.ABCMeta).
class PatternDefiningBase:
# Dear programmer: implement this in a subclass OR YOU'LL BE SORRY!
PATTERN = NotImplemented
def __init_subclass__(cls, **kwargs):
super().__init_subclass__(**kwargs)
# If the new class did not redefine PATTERN, fail *hard*.
if cls.PATTERN is NotImplemented:
# Choose your favorite exception.
raise NotImplementedError('You forgot to define PATTERN!!!')
#classmethod
def sample(cls):
print(cls.PATTERN)
class LegalPatternChild(PatternDefiningBase):
PATTERN = r'foo\s+bar'
Here is how the decorator could be implemented.
# custom_decorators.py
def abstract_class_attributes(*names):
"""Class decorator to add one or more abstract attribute."""
def _func(cls, *names):
""" Function that extends the __init_subclass__ method of a class."""
# Add each attribute to the class with the value of NotImplemented
for name in names:
setattr(cls, name, NotImplemented)
# Save the original __init_subclass__ implementation, then wrap
# it with our new implementation.
orig_init_subclass = cls.__init_subclass__
def new_init_subclass(cls, **kwargs):
"""
New definition of __init_subclass__ that checks that
attributes are implemented.
"""
# The default implementation of __init_subclass__ takes no
# positional arguments, but a custom implementation does.
# If the user has not reimplemented __init_subclass__ then
# the first signature will fail and we try the second.
try:
orig_init_subclass(cls, **kwargs)
except TypeError:
orig_init_subclass(**kwargs)
# Check that each attribute is defined.
for name in names:
if getattr(cls, name, NotImplemented) is NotImplemented:
raise NotImplementedError(f'You forgot to define {name}!!!')
# Bind this new function to the __init_subclass__.
# For reasons beyond the scope here, it we must manually
# declare it as a classmethod because it is not done automatically
# as it would be if declared in the standard way.
cls.__init_subclass__ = classmethod(new_init_subclass)
return cls
return lambda cls: _func(cls, *names)
I've been searching for something like this for quite a while, until yesterday I decided to dive into it. I like #SethMMorton's reply a lot, however 2 things are missing: allow a an abstract class to have a subclass that is abstract itself, and play nice with typehints and static typing tools such as mypy (which makes sense, since back in 2017 these were hardly a thing).
I started to set out to write a reply here with my own solution, however I realised I needed lots of tests and documentation, so I made it a proper python module: abstractcp.
Use (as of version 0.9.5):
class Parser(acp.Abstract):
PATTERN: str = acp.abstract_class_property(str)
#classmethod
def parse(cls, s):
m = re.fullmatch(cls.PATTERN, s)
if not m:
raise ValueError(s)
return cls(**m.groupdict())
class FooBarParser(Parser):
PATTERN = r"foo\s+bar"
def __init__(...): ...
class SpamParser(Parser):
PATTERN = r"(spam)+eggs"
def __init__(...): ...
See for full use the page on pypi or github.
Alternative Answer
Using dedicated class to annotate class variables
import abc
from typing import Generic, Set, TypeVar, get_type_hints
T = TypeVar('T')
class AbstractClassVar(Generic[T]):
pass
class Abstract(abc.ABC):
def __init_subclass__(cls) -> None:
def get_abstract_members(cls) -> Set[str]:
"""Gets a class's abstract members"""
abstract_members = set()
if cls is Abstract:
return abstract_members
for base_cls in cls.__bases__:
abstract_members.update(get_abstract_members(base_cls))
for (member_name, annotation) in get_type_hints(cls).items():
if getattr(annotation, '__origin__', None) is AbstractClassVar:
abstract_members.add(member_name)
return abstract_members
# Implementation checking for abstract class members
if Abstract not in cls.__bases__:
for cls_member in get_abstract_members(cls):
if not hasattr(cls, cls_member):
raise NotImplementedError(f"Wrong class implementation {cls.__name__} " +
f"with abstract class variable {cls_member}")
return super().__init_subclass__()
Usage
class Foo(Abstract):
foo_member: AbstractClassVar[str]
class UpperFoo(Foo):
# Everything should be implemented as intended or else...
...
Not Implementing the abstract class member foo_member will result in a NotImplementedError.
Answer was taken from my original answer to this question: enforcement for abstract properties in python3
I want to make a class that uses a strategy design pattern similar to this:
class C:
#staticmethod
def default_concrete_strategy():
print("default")
#staticmethod
def other_concrete_strategy():
print("other")
def __init__(self, strategy=C.default_concrete_strategy):
self.strategy = strategy
def execute(self):
self.strategy()
This gives the error:
NameError: name 'C' is not defined
Replacing strategy=C.default_concrete_strategy with strategy=default_concrete_strategy will work but, left as default, the strategy instance variable will be a static method object rather than a callable method.
TypeError: 'staticmethod' object is not callable
It will work if I remove the #staticmethod decorator, but is there some other way? I want the default parameter to be self documented so that others will immediately see an example of how to include a strategy.
Also, is there a better way to expose strategies rather than as static methods? I don't think that implementing full classes makes sense here.
No, you cannot, because the class definition has not yet completed running so the class name doesn't exist yet in the current namespace.
You can use the function object directly:
class C:
#staticmethod
def default_concrete_strategy():
print("default")
#staticmethod
def other_concrete_strategy():
print("other")
def __init__(self, strategy=default_concrete_strategy.__func__):
self.strategy = strategy
C doesn't exist yet when the methods are being defined, so you refer to default_concrete_strategy by the local name. .__func__ unwraps the staticmethod descriptor to access the underlying original function (a staticmethod descriptor is not itself callable).
Another approach would be to use a sentinel default; None would work fine here since all normal values for strategy are static functions:
class C:
#staticmethod
def default_concrete_strategy():
print("default")
#staticmethod
def other_concrete_strategy():
print("other")
def __init__(self, strategy=None):
if strategy is None:
strategy = self.default_concrete_strategy
self.strategy = strategy
Since this retrieves default_concrete_strategy from self the descriptor protocol is invoked and the (unbound) function is returned by the staticmethod descriptor itself, well after the class definition has completed.
I know that checking types in Python is bad and you should probably never do it. But I can't seem to find the disadvantage to this.
class O(object):
def __init__(self, name):
'''Can only be called in derived classes.'''
if type(self) is O:
message = "%(class)s cannot be instantiated, it must be derived."
raise TypeError, message % { "class" : O }
self.name = name
def fn(self):
'''Must be populated in derived classes.'''
raise NotImplementedError
Now if someone tries to instantiate O, a class I never meant to be instantiated, they know immediately.
Is this still bad form?
Look at Abstract Base Classes as they will provide more fine grained control over how the subclasses are instantiated if this is something that you really want to do.
All in all, this might be a valid use because you are not preventing me from passing whatever I want to your code but I still wouldn't consider it pythonic. You are telling me that I can't instantiate your class. What If I want to?
Using ABC's, it looks like:
import abc
class O(object):
__metaclass__ = abc.ABCMeta
def __init__(self, name):
self.name = name
#abc.abstractmethod
def fn(self):
pass
This has the advantage of not breaking super on the fn method. As you have it with the raise NotImplementedError, you are breaking super for multiple inheritance. If a class derives from two classes that subclass O and both call super (as they should to allow for multiple inheritance) then it will create the exception that you raise.
So now, you are not just telling me that I can't instantiate your class, you are telling me that I can't use multiple inheritance when subclassing from your class.
Don't check at all, we're are all adults here. Just add a note that O shouldn't be instantiated directly, either as a comment and/or in the documentation.
It's the same as if someone would call a method that requires an int as its parameter with a string instead. If the program crashes, they screwed it up. Otherwise you would need to add type checks to just about everything.
What is that you are trying to achieve with the above code.
In this case self is of type O and will always result in raising the exception.
Look at this snippet to understand it a little better
class O(object):
def __init__(self, name):
self.name = name
o = O("X")
print type(o)
print isinstance(o, O)
if type(o) is O:
print "Yes"
This will output
<class '__main__.O'>
True
Yes