Develop Reference

How do I use same getter and setter properties and functions for different attributes of a class the pythonic way?

I've got this class that I'm working on that stores Employees details.
I want all attributes to be protected and be set and gotten with specific logic, but not all in a unique way. I would like the same logic to apply to my _f_name and to my _l_name attributes, I would like the same logic perhaps to be applied to attributes that take in booleans and other general cases.
I've got this for the first attribute:
#property
def f_name(self):
return self.f_name
#f_name.setter
def f_name(self, f_name):
if f_name != str(f_name):
raise TypeError("Name must be set to a string")
else:
self._f_name = self._clean_up_string(f_name)
#f_name.deleter
def available(self):
raise AttributeError("Can't delete, you can only change this value.")
How can I apply the same functions and properites to other attributes?
Thaaaanks!

While it may seem like defining a subclass of property is possible, too many details of how a particular property work is left to the getter and setter to define, meaning it's more straightforward to define a custom property-like descriptor.
class CleanableStringProperty:
def __set_name__(self, owner, name):
self._private_name = "_" + name
self.name = name
def __get__(self, obj, objtype=None):
# Boilerplate to handle accessing the property
# via a class, rather than an instance of the class.
if obj is None:
return self
return getattr(obj, self._private_name)
def __set__(self, obj, value):
if not isinstance(value, str):
raise TypeError(f'{self.name} value must be a str')
setattr(obj, self._private_name, obj._clean_up_string(value))
def __delete__(self, obj):
raise AttributeError("Can't delete, you can only change this value.")
__set_name__ constructs the name of the instance attribute that the getter and setter will use. __get__ acts as the getter, using getattr to retrieve the constructed attribute name from the given object. __set__ validates and modifies the value before using setattr to set the constructed attribute name. __del__ simply raises an attribute error, independent of whatever object the caller is trying to remove the attribute from.
Here's a simple demonstration which causes all values assigned to the descriptor to be put into title case.
class Foo:
f_name = CleanableStringProperty()
l_name = CleanableStringProperty()
def __init__(self, first, last):
self.f_name = first
self.l_name = last
def _clean_up_string(self, v):
return v.title()
f = Foo("john", "doe")
assert f.f_name == "John"
assert f.l_name == "Doe"
try:
del f.f_name
except AttributeError:
print("Prevented first name from being deleted")
It would also be possible for the cleaning function, rather than being somethign that obj is expected to provide, to be passed as an argument to CleanableStringProperty itself. __init__ and __set__ would be modified as
def __init__(self, cleaner):
self.cleaner = cleaner
def __set__(self, obj, value):
if not isinstance(value, str):
raise TypeError(f'{self.name} value must be a str')
setattr(obj, self._private_name, self.cleaner(value))
and the descriptor would be initialized with
class Foo:
fname = CleanableStringProperty(str.title)
Note that Foo is no longer responsible for providing a cleaning method.

A property is just an implementation of a descriptor, so to create a custom property, you need an object with a __get__, __set__, and/or __delete__ method.
In your case, you could do something like this:
from typing import Any, Callable, Tuple
class ValidatedProperty:
def __set_name__(self, obj, name):
self.name = name
self.storage = f"_{name}"
def __init__(self, validation: Callable[[Any], Tuple[str, Any]]=None):
"""Initializes a ValidatedProperty object
Args:
validation (Callable[[Any], Tuple[str, Any]], optional): A Callable that takes the given value and returns an error string (empty string if no error) and the cleaned-up value. Defaults to None.
"""
self.validation = validation
def __get__(self, instance, owner):
return getattr(instance, self.storage)
def __set__(self, instance, value):
if self.validation:
error, value = self.validation(value)
if error:
raise ValueError(f"Error setting property {self.name}: {error}")
setattr(instance, self.storage, value)
def __delete__(self, instance):
raise AttributeError("Can't delete, you can only change this value.")
Let's define an example class to use this:
class User:
def __name_validation(value):
if not isinstance(value, str):
return (f"Expected string value, received {type(value).__name__}", None)
return ("", value.strip().title())
f_name = ValidatedProperty(validation=__name_validation)
l_name = ValidatedProperty(validation=__name_validation)
def __init__(self, fname, lname):
self.f_name = fname
self.l_name = lname
and test:
u = User("Test", "User")
print(repr(u.f_name)) # 'Test'
u.f_name = 123 # ValueError: Error setting property f_name: Expected string value, received int
u.f_name = "robinson " # Notice the trailing space
print(repr(u.f_name)) # 'Robinson'
u.l_name = "crusoe "
print(repr(u.l_name)) # 'Crusoe'

Python: code duplication on class attribute definition

I'm trying to implement a simple ORM in python. I'm facing a code duplication issue and I do not know how to solve it.
Here is a simplified example of a class in my project:
class Person:
TABLE_NAME = 'person'
FIELDS = [
('name', 'VARCHAR(50)'),
('age', 'INTEGER')
]
# CODE DUPLICATION: the two next lines shoudl be genereated with FIELDS not hard coded...
name: str
age: int
def __init__(self, **kwargs):
self.__dict__ = kwargs
#classmethod
def create_sql_table(cls):
# use TABLE_NAME and FIELDS to create sql table
pass
alice = Person(name='Alice', age=25)
print(alice.name)
If I remove the two lines name: strand age: int I lose auto-completion and I get a mypy error on the print line (Error: Person has no attribute name)
But If I keep it, I have code duplication (I write twice each field name).
Is there a way to avoid the code duplication (by generating this two lines using FIELDS variable for instance) ?
Or another way to implement this class that avoid code duplication (without mypy error and auto-completion loss) ?

You can use descriptors:
from typing import Generic, TypeVar, Any, overload, Union
T = TypeVar('T')
class Column(Generic[T]):
sql_type: str # the field type used for this column
def __init__(self) -> None:
self.name = '' # the name of the column
# this is called when the Person class (not the instance) is created
def __set_name__(self, owner: Any, name: str) -> None:
self.name = name # now contains the name of the attribute in the class
# the overload for the case: Person.name -> Column[str]
#overload
def __get__(self, instance: None, owner: Any) -> 'Column[T]': ...
# the overload for the case: Person().name -> str
#overload
def __get__(self, instance: Any, owner: Any) -> T: ...
# the implementation of attribute access
def __get__(self, instance: Any, owner: Any) -> Union[T, 'Column[T]']:
if instance is None:
return self
# implement your attribute access here
return getattr(instance, f'_{self.name}') # type: ignore
# the implementation for setting attributes
def __set__(self, instance: Any, value: T) -> None:
# maybe check here that the type matches
setattr(instance, f'_{self.name}', value)
Now we can create specializations for each column type:
class Integer(Column[int]):
sql_type = 'INTEGER'
class VarChar(Column[str]):
def __init__(self, size: int) -> None:
self.sql_type = f'VARCHAR({size})'
super().__init__()
And when you define the Person class we can use the column types
class Person:
TABLE_NAME = 'person'
name = VarChar(50)
age = Integer()
def __init__(self, **kwargs: Any) -> None:
for key, value in kwargs.items():
setattr(self, key, value)
#classmethod
def create_sql_table(cls) -> None:
print("CREATE TABLE", cls.TABLE_NAME)
for key, value in vars(cls).items():
if isinstance(value, Column):
print(key, value.sql_type)
Person.create_sql_table()
p = Person(age=10)
print(p.age)
p.age = 20
print(p.age)
This prints:
CREATE TABLE person
name VARCHAR(50)
age INTEGER
10
20
You should probably also create a base Model class that contains the __init__ and the class method of Person
You can also extend the Column class to allow nullable columns and add default values.
Mypy does not complain and can correctly infer the types for Person.name to str and Person.age to int.

Ok, I ended up with that
class Person:
# this is not full, you need to fill other types you use it with the correct relationship
types = {
str: 'VARCHAR(50)',
int: 'INTEGER',
} # you should extract that out if you use it elsewhere
TABLE_NAME = 'person'
# NOTE: the only annotated fields should be these. if you annotate anything else, It will break
name: str
age: int
def __init__(self, **kwargs):
self.__dict__ = kwargs
#property
def FIELDS(cls):
return [(key, cls.types[value]) for key, value in cls.__annotations__.items()]
alice = Person(name='Alice', age=25)
print(alice.FIELDS) # [('name', 'VARCHAR(50)'), ('age', 'INTEGER')]
And
>>> mypy <module>
>>> Success: no issues found in 1 source file

In the class Person try to add data type in constructor

How to assign from input to object from a class

I have multiple classes and I have instances from each class e.g: Student class. every instance (a student) has their own courses. Now when a user signs in (by input) I want to print their list of courses. Or even just their age to show that I have the correct object.
Is there a better way than eval() to get an object from class based on input
like the following example:
class Student:
def __init__(self, name, age):
self._name = name
self._age = age
blablue = Student('bla blue', '23')
name = input('enter your name')
name = name.split(' ')
stundent = eval(name[0] + name[1])
print(student)
print(student.age)
output:
enter your name: bla blue
<__main__.Foo object at 0x000001B2978C73C8>
23

I assume this is for educational purpose (production code would use a SQL database and some ORM):
try:
# python 2.x
input = raw_input
except NameError:
# python 3.x
pass
class AlreadyExists(ValueError):
pass
class DoesNotExist(LookupError):
pass
class FooCollection(object):
def __init__(self):
self._foos = {}
def add(self, foo):
if foo.name in self._foos:
raise AlreadyExists("Foo with name '{}' already exists".format(foo.name))
self.update(foo)
def update(self, foo):
self._foos[foo.name] = foo
def get(self, name):
try:
return self._foos[name]
except KeyError:
raise DoesNotExist("no Foo named '{}'".format(name))
class Foo(object):
def __init__(self, name, age):
self._name = name
self._age = age
# we at least need to be able to read the name
#property
def name(self):
return self._name
def __repr__(self):
return "Foo({}, {})".format(self._name, self._age)
def main():
foos = FooCollection()
blablue = Foo('bla blue', '23')
foos.add(blablue)
name = input('enter your name: ').strip()
try:
print("found {}".format(foos.get(name)))
except DoesNotExist as e:
print(e)
if ___name__ == "__main__":
main()
The principle here is to have a storage for your instances. I chose a dict for fast lookup with the Foo.name as key, in real life you'd probably want an opaque unique identifier for each instance and multiple indexes (i.e. one by id, one by name etc) - but actually in real life you would use a SQL database that already provide all those features in a much more optimized way ;-)
Also, I wrapped the dict in a dedicated class with its own interface. This allows to decouple the interface from the implementation (if you later decide you want more indexes than just name for example), and encapsulate domain logic too (i.e. checking you don't accidentally overwrite an existing Foo).

Python type checking system

I am trying to make custom type system in Python. Following is the code.
from inspect import Signature, Parameter
class Descriptor():
def __init__(self, name=None):
self.name = name
def __set__(self, instance, value):
instance.__dict__[self.name] = value
def __get__(self, instance, cls):
return instance.__dict__[self.name]
class Typed(Descriptor):
ty = object
def __set__(self, instance, value):
if not isinstance(value, self.ty):
raise TypeError('Expected %s' %self.ty)
super().__set__(instance, value)
class Integer(Typed):
ty = int
class Float(Typed):
ty = float
class String(Typed):
ty = str
class Positive(Descriptor):
def __set__(self, instance, value):
if value < 0:
raise ValueError('Expected >= 0')
super().__set__(instance, value)
class PosInteger(Integer, Positive):
pass
class Sized(Descriptor):
def __init__(self, *args, maxlen, **kwargs):
self.maxlen = maxlen
super().__init__(*args, **kwargs)
def __set__(self, instance, value):
if len(value) > self.maxlen:
raise ValueError('TooBig')
super().__set__(instance, value)
class SizedString(String, Sized):
pass
def make_signature(names):
return Signature([Parameter(name, Parameter.POSITIONAL_OR_KEYWORD) for name in names])
class StructMeta(type):
def __new__(cls, name, bases, clsdict):
fields = [key for key, value in clsdict.items() if isinstance(value, Descriptor)]
for name in fields:
#print(type(clsdict[name]))
clsdict[name].name = name
clsobj = super().__new__(cls, name, bases, clsdict)
sig = make_signature(fields)
setattr(clsobj, '__signature__', sig)
return clsobj
class Structure(metaclass = StructMeta):
def __init__(self, *args, **kwargs):
bound = self.__signature__.bind(*args, **kwargs)
for name, value in bound.arguments.items():
setattr(self, name, value)
Using the above type system, I got rid of all the boilerplate code and duplicate code that I would have to write in classes (mostly inside init) for checking types, validating values etc.
By using the code above, my classes would look as simple as this
class Stock(Structure):
name = SizedString(maxlen=9)
shares = PosInteger()
price = Float()
stock = Stock('AMZN', 100, 1600.0)
Till here things work fine. Now I want to extend this type checks functionality and create classes holding objects of another classes. For example price is now no longer a Float but its of type Price (i.e. another class Price).
class Price(Structure):
currency = SizedString(maxlen=3)
value = Float()
class Stock(Structure):
name = SizedString(maxlen=9)
shares = PosInteger()
price = Price() # This won't work.
This won't work because line "price = Price()" will make call to constructor of Price and would expect currency and value to be passed to the constructor because Price is a Structure and not a Descriptor. It throws "TypeError: missing a required argument: 'currency'".
But I want it to work and make it look like above because at the end of the day Price is also a type just like PosInteger but at the same time it has to be Structure too. i.e. Price should be inheriting from Structure but at the same time it has to be a descriptor too.
I can make it work by defining another class say "PriceType"
class Price(Structure):
currency = SizedString(maxlen=3)
value = Float()
class PriceType(Typed):
ty = Price
class Stock(Structure):
name = SizedString(maxlen=9)
shares = PosInteger()
price = PriceType()
stock = Stock('AMZN', 100, Price('INR', 2400.0))
But this looks a bit weird - Price and PriceType as two difference classes. Can someone help me understand if I can avoid creating PriceType class?
I am also losing out on a functionality to provide default values to fields.
For example, how can I keep default value of share field in Stock to 0 or default value of currency field in Price to 'USD'? i.e. something like below.
class Stock:
def __init__(name, price, shares=0)
class Price
def __init__(value, currency = 'USD')

A quick thing to do there is to have a simple function that will build the "PriceType" (and equivalents) when you declare the fields.
Since uniqueness of the descriptor classes themselves is not needed, and the relatively long time a class takes to be created is not an issue, since fields in a body class are only created at program-load time, you should be fine with:
def typefield(cls, *args, extra_checkers = (), **kwargs):
descriptor_class = type(
cls.__name__,
(Typed,) + extra_checkers,
{'ty': cls}
)
return descriptor_class(*args, **kwargs)
And now, code like this should just work:
class Stock(Structure):
name = SizedString(maxlen=9)
shares = PosInteger()
price = typefield(Price, "price")
(Also, note that Python 3.6+ have the __set_name__ method incorporated into the descriptor protocol - if you use this, you won't need to pass the field name as a parameter to the default descriptor __init__, and type field names twice)
update
In your comment, you seam to implicate want your Structure classes to work themselves as descriptors - that would not work well - the descriptors __get__ and __set__ methods are class methods - you want the fields to be populated with actual instances of your structures.
What can be done is to move the typefield method above to a class method in Structure, have it annotate the default parameters your want, and create a new intermediate descriptor class for these kind of fields that will automatically create an instance with the default values when it is read. Also, ty can simply be an instance attribute in the descriptor, so no need to create dynamic classes for the fields:
class StructField(Typed):
def __init__(self, *args, ty=None, def_args=(), def_kw=None, **kw):
self.def_args = def_args
self.def_kw = def_kw or {}
self.ty = ty
super().__init__(*args, **kw)
def __get__(self, instance, owner):
if self.name not in instance.__dict__:
instance.__dict__[self.name] = self.ty(*self.def_args, **self.def_kw)
return super().__get__(instance, owner)
...
class Structure(metaclass=StructMeta):
...
#classmethod
def field(cls, *args, **kw):
# Change the signature if you want extra parameters
# for the field, like extra validators, and such
return StructField(ty=cls, def_args=args, def_kw=kw)
...
class Stock(Structure):
...
price = Price.field("USD", 20.00)

How to raise exception for non-existent class member?

I have a method for automatically creating Python classes that wrap database tables, with class members that have the same name as the fields in the table. The class files look like this:
class CpsaUpldBuildChrgResultSet(Recordset):
def __init__(self, connection):
super().__init__(connection)
self.DefaultTableName = 'cpsa_upld_build_chrg_result'
self._keyFields.append('j_trans_seq')
self._keyFields.append('j_index')
#property
def j_trans_seq(self):
return self.GetValue('j_trans_seq')
#j_trans_seq.setter
def j_trans_seq(self, value):
self.SetKeyValue('j_trans_seq', value)
#property
def j_index(self):
return self.GetValue('j_index')
#j_index.setter
def j_index(self, value):
self.SetKeyValue('j_index', value)
I just found that if I try to set a value for a non-existent class member, such as J_TRANS_SEQ, no exception is thrown. Is there something I can add to this class so that an attempt to access a non-existent member would raise an exception?

You can add a __setattr__ method to your class that raises an AttributeError whenever an invalid attribute is assigned to. I'm not sure exactly how you'd want to determine which attributes are valid and which are not, but one approach might be something like this:
def __setattr__(self, name, value):
if hasattr(self, name):
super().__setattr__(name, value)
else:
raise AttributeError("{} object has no attribute {!r}".format(type(self), name))
This assumes that any attribute that can be looked up is also valid to be assigned to. It might break if your property's getters don't work unless the setter is called before the getter. It might also be too permissive, since it would allow setting of instance attributes that override class attributes (such as __init__). Another approach might be to check the name against a white-list of known attributes (but be sure to include the attributes that you need for the inherited class machinery, like DefaultTableName and _keyFields).

I think #Blckknght has the right idea, but left out some important details in his answer—such has how class attributes (class members) are set the first time, when they don't preexist, such as in the typical scenario when the class's __init__() method executes. Here's a more fully fleshed-out answer that works in Python 3 which addresses that deficiency.
It also shows how to minimize the coding of a bunch of repetitive properties.
class Recordset(object):
def __init__(self, connection):
print('Recordset.__init__({!r}) called'.format(connection))
def SetKeyValue(self, name, value):
print('SetKeyValue({!r}, {!r}) called'.format(name, value))
def GetValue(self, name):
print('GetValue({!r}) called'.format(name))
def fieldname_property(name):
storage_name = '_' + name
#property
def prop(self):
return self.GetValue(storage_name)
#prop.setter
def prop(self, value):
self.SetKeyValue(storage_name, value)
return prop
class CpsaUpldBuildChrgResultSet(Recordset):
# define properties for valid fieldnames
j_trans_seq = fieldname_property('j_trans_seq')
j_index = fieldname_property('j_index')
def __init__(self, connection):
super().__init__(connection)
self._setter('DefaultTableName', 'cpsa_upld_build_chrg_result')
def __setattr__(self, name, value):
if hasattr(self, name):
self._setter(name, value)
else:
raise AttributeError("No field named %r" % name)
def _setter(self, name, value):
"""Provides way to intentionally bypass overloaded __setattr__."""
super().__setattr__(name, value)
print('start')
db_table = CpsaUpldBuildChrgResultSet('SomeConnection')
print('assigning attributes...')
db_table.j_trans_seq = 42 # OK
db_table.j_index = 13 # OK
db_table.J_TRANS_SEQ = 99 # -> AttributeError: No field named 'J_TRANS_SEQ'
print('done')