I am designing an object-oriented data structure that shall be simple to handle from the user's perspective, e.g., by method chaining (aka Fluent interface). However, each change should only act temporarily on the object: within that chain, but not beyond that chain.
Here is a simplified example that does not work as intended:
class C:
def __init__(self, num):
self.sum = num
def add(self, num=0):
self.sum += num
return self
number = C(23)
number.add(7).add(12).sum
# 42
number.sum
# 42 (but I expect: 23)
In this case, .add() makes permanent changes to number. However, permanent changes should only be possible like this:
# This is the only way how a user should make permanent changes:
number = number.add(4).add(12)
In the temporary scope, I am looking for a way to get back to the old version of number after the chain is terminated. On the edge of despair, I can think of ugly solutions like "instance replication":
class C2:
def __init__(self, num):
self.sum = num
def add(self, num=0):
self2 = C2(self.sum) # or equivalently: self2 = copy.deepcopy(self)
self2.sum += num
return self2
number = C2(23)
number.add(7).add(12).sum
# 42
number.sum
# 23
However, the actuall classes and objects with which I am working contain a huge amount of data, attributes, methods, and even subclasses. So we should avoid copying the instance in every single method, besides the fact that it involves ugly code.
Are there ways to solve this problem, e.g. by (silently) creating a single copy only once at the first element of the chain? Or by destroying any changes made at the end of the chain? (Note that real-world "changes" involve many different, commutable methods other than just adding numbers)
An accepted solution should perform the necessary operations internally, i.e., without bothering the user interface:
# These are NOT accepted solutions:
number.copy().add(4).add(12)
number.add(4).add(12).undo()
If there no direct solution other than self-replication, the question would be: What is the most elegant way to do it that sustains code-readability and keeps memory usage low? E.g., decorating every class method by self-replicating function?
Instead of modyfing the object on which you call the method, return a modified copy:
class C:
def __init__(self, num):
self.sum = num
def add(self, num=0):
return C(self.sum + num)
number = C(23)
assert number.add(7).add(12).sum == 42
assert number.sum == 23
For details on memory handling in this solution, see comments of this posts. This solution is standard way of solving your problem.
Related
I am trying to modify an already defined class by changing an attribute's value. Importantly, I want this change to propagate internally.
For example, consider this class:
class Base:
x = 1
y = 2 * x
# Other attributes and methods might follow
assert Base.x == 1
assert Base.y == 2
I would like to change x to 2, making it equivalent to this.
class Base:
x = 2
y = 2 * x
assert Base.x == 2
assert Base.y == 4
But I would like to make it in the following way:
Base = injector(Base, x=2)
Is there a way to achieve this WITHOUT recompile the original class source code?
The effect you want to achieve belongs to the realm of "reactive programing" - a programing paradigm (from were the now ubiquitous Javascript library got its name as an inspiration).
While Python has a lot of mechanisms to allow that, one needs to write his code to actually make use of these mechanisms.
By default, plain Python code as the one you put in your example, uses the Imperative paradigm, which is eager: whenever an expression is encoutered, it is executed, and the result of that expression is used (in this case, the result is stored in the class attribute).
Python's advantages also can make it so that once you write a codebase that will allow some reactive code to take place, users of your codebase don't have to be aware of that, and things work more or less "magically".
But, as stated above, that is not free. For the case of being able to redefine y when x changes in
class Base:
x = 1
y = 2 * x
There are a couple paths that can be followed - the most important is that, at the time the "*" operator is executed (and that happens when Python is parsing the class body), at least one side of the operation is not a plain number anymore, but a special object which implements a custom __mul__ method (or __rmul__) in this case. Then, instead of storing a resulting number in y, the expression is stored somewhere, and when y is retrieved either as a class attribute, other mechanisms force the expression to resolve.
If you want this at instance level, rather than at class level, it would be easier to implement. But keep in mind that you'd have to define each operator on your special "source" class for primitive values.
Also, both this and the easier, instance descriptor approach using property are "lazily evaluated": that means, the value for y is calcualted when it is to be used (it can be cached if it will be used more than once). If you want to evaluate it whenever x is assigned (and not when y is consumed), that will require other mechanisms. Although caching the lazy approach can mitigate the need for eager evaluation to the point it should not be needed.
1 - Before digging there
Python's easiest way to do code like this is simply to write the expressions to be calculated as functions - and use the property built-in as a descriptor to retrieve these values. The drawback is small:
you just have to wrap your expressions in a function (and then, that function
in something that will add the descriptor properties to it, such as property). The gain is huge: you are free to use any Python code inside your expression, including function calls, object instantiation, I/O, and the like. (Note that the other approach requires wiring up each desired operator, just to get started).
The plain "101" approach to have what you want working for instances of Base is:
class Base:
x = 1
#property
def y(self):
return self.x * 2
b = Base()
b.y
-> 2
Base.x = 3
b.y
-> 6
The work of property can be rewritten so that retrieving y from the class, instead of an instance, achieves the effect as well (this is still easier than the other approach).
If this will work for you somehow, I'd recommend doing it. If you need to cache y's value until x actually changes, that can be done with normal coding
2 - Exactly what you asked for, with a metaclass
as stated above, Python'd need to know about the special status of your y attribute when calculcating its expression 2 * x. At assignment time, it would be already too late.
Fortunately Python 3 allow class bodies to run in a custom namespace for the attribute assignment by implementing the __prepare__ method in a metaclass, and then recording all that takes place, and replacing primitive attributes of interest by special crafted objects implementing __mul__ and other special methods.
Going this way could even allow values to be eagerly calculated, so they can work as plain Python objects, but register information so that a special injector function could recreate the class redoing all the attributes that depend on expressions. It could also implement lazy evaluation, somewhat as described above.
from collections import UserDict
import operator
class Reactive:
def __init__(self, value):
self._initial_value = value
self.values = {}
def __set_name__(self, owner, name):
self.name = name
self.values[owner] = self._initial_value
def __get__(self, instance, owner):
return self.values[owner]
def __set__(self, instance, value):
raise AttributeError("value can't be set directly - call 'injector' to change this value")
def value(self, cls=None):
return self.values.get(cls, self._initial_value)
op1 = value
#property
def result(self):
return self.value
# dynamically populate magic methods for operation overloading:
for name in "mul add sub truediv pow contains".split():
op = getattr(operator, name)
locals()[f"__{name}__"] = (lambda operator: (lambda self, other: ReactiveExpr(self, other, operator)))(op)
locals()[f"__r{name}__"] = (lambda operator: (lambda self, other: ReactiveExpr(other, self, operator)))(op)
class ReactiveExpr(Reactive):
def __init__(self, value, op2, operator):
self.op2 = op2
self.operator = operator
super().__init__(value)
def result(self, cls):
op1, op2 = self.op1(cls), self.op2
if isinstance(op1, Reactive):
op1 = op1.result(cls)
if isinstance(op2, Reactive):
op2 = op2.result(cls)
return self.operator(op1, op2)
def __get__(self, instance, owner):
return self.result(owner)
class AuxDict(UserDict):
def __init__(self, *args, _parent, **kwargs):
self.parent = _parent
super().__init__(*args, **kwargs)
def __setitem__(self, item, value):
if isinstance(value, self.parent.reacttypes) and not item.startswith("_"):
value = Reactive(value)
super().__setitem__(item, value)
class MetaReact(type):
reacttypes = (int, float, str, bytes, list, tuple, dict)
def __prepare__(*args, **kwargs):
return AuxDict(_parent=__class__)
def __new__(mcls, name, bases, ns, **kwargs):
pre_registry = {}
cls = super().__new__(mcls, name, bases, ns.data, **kwargs)
#for name, obj in ns.items():
#if isinstance(obj, ReactiveExpr):
#pre_registry[name] = obj
#setattr(cls, name, obj.result()
for name, reactive in pre_registry.items():
_registry[cls, name] = reactive
return cls
def injector(cls, inplace=False, **kwargs):
original = cls
if not inplace:
cls = type(cls.__name__, (cls.__bases__), dict(cls.__dict__))
for name, attr in cls.__dict__.items():
if isinstance(attr, Reactive):
if isinstance(attr, ReactiveExpr) and name in kwargs:
raise AttributeError("Expression attributes can't be modified by injector")
attr.values[cls] = kwargs.get(name, attr.values[original])
return cls
class Base(metaclass=MetaReact):
x = 1
y = 2 * x
And, after pasting the snippet above in a REPL, here is the
result of using injector:
In [97]: Base2 = injector(Base, x=5)
In [98]: Base2.y
Out[98]: 10
The idea is complicated with that aspect that Base class is declared with dependent dynamically evaluated attributes. While we can inspect class's static attributes, I think there's no other way of getting dynamic expression except for parsing the class's sourcecode, find and replace the "injected" attribute name with its value and exec/eval the definition again. But that's the way you wanted to avoid. (moreover: if you expected injector to be unified for all classes).
If you want to proceed to rely on dynamically evaluated attributes define the dependent attribute as a lambda function.
class Base:
x = 1
y = lambda: 2 * Base.x
Base.x = 2
print(Base.y()) # 4
I have run across a few examples of Python code that looks something like this:
class GiveNext :
list = ''
def __init__(self, list) :
GiveNext.list = list
def giveNext(self, i) :
retval = GiveNext.list[i]
return retval
class GiveABCs(GiveNext):
i = -1
def _init__(self, list) :
GiveNext.__init__(self, list)
def giveNext(self):
GiveABCs.i += 1
return GiveNext.giveNext(self, GiveABCs.i)
class Give123s(GiveNext):
i = -1
def _init__(self, list) :
GiveNext.__init__(self, list)
def giveNext(self):
Give123s.i += 1
return GiveNext.giveNext(self, Give123s.i)
for i in range(3):
print(GiveABCs('ABCDEFG').giveNext())
print(Give123s('12345').giveNext())
the output is: A 1 B 2 C 3
If I were more clever, I could figure out how to put the string literals inside the constructor...but that is not crucial right now.
My question is on the use of classes this way. Yes, an instance of the class gets created each time that that the call within the print() gets made. Yet the i's are 'permanent' in each class.
This strikes me as less of an object-oriented approach, and more of a way of using classes to accomplish encapsulation and/or a functional programming paradigm, since the instances are entirely transitory. In other words, an instance of the class is never instantiated for its own purposes; it is there only to allow access to the class-wide methods and variables within to do their thing, and then it is tossed away. In many cases, it seems like the class mechanism is used in a back-handed way, in order to leverage inheritance and name resolution/spacing: an instance of the class is never really required to be built or used, conceptually.
Is this standard Python form?
Bonus question: how would I put the string literals inside each class declaration? Right now, even if I change the _init__ for GiveABCs to
GiveNext.__init__(self, 'wxyz')
it completely ignores the 'wxyz' literal, and uses the 'ABCDEF' one - even though it is never mentioned...
Please don't learn Python with this code. As mentioned by others, this code goes against many Python principles.
One example: list is a Python builtin type. Don't overwrite it, especially not with a string instance!
The code also mixes class and instance variables and doesn't use super() in subclasses.
This code tries to simulate an iterator. So simply use an iterator:
give_abcs = iter('ABCDEFG')
give_123s = iter('12345')
for _ in range(3):
print(next(give_abcs))
print(next(give_123s))
# A
# 1
# B
# 2
# C
# 3
If you really want to fix the above code, you could use:
class GiveNext :
def __init__(self, iterable) :
self.i = - 1
self.iterable = iterable
def giveNext(self) :
self.i += 1
return self.iterable[self.i]
giveABCs = GiveNext('ABCDEFG')
give123s = GiveNext('12345')
for _ in range(3):
print(giveABCs.giveNext())
print(give123s.giveNext())
It outputs:
A
1
B
2
C
3
This code in the OP is an incredible amount of crap. Not only it is long, unreadable, misuses OO features, and does not use Python features at all (an iterator being a standard Python feature). Here is a suggestion for a more Pythonist approach:
giveABCs = iter('ABCDEFG')
give123s = iter('12345')
for i in range(3):
print(next(giveABCs))
print(next(give123s))
About your bonus question: I guess you are modifing the _init__() method of GiveABCs and Give123s. It is normal that whatever code you put in there has no effect, because the Python constructor is __init__() (with 2 leading underscores, not 1). So The constructor from GiveNext is not overloaded.
This question already has answers here:
Why do you need explicitly have the "self" argument in a Python method? [duplicate]
(10 answers)
Closed 6 years ago.
Consider this code:
class example(object):
def __init__ (): # No self
test() # No self
def test(x,y): # No self
return x+y
def test1(x,y): # No self
return x-y
print(example.test(10,5))
print(example.test1(10,5))
15
5
This works as expected. I believe I can write a whole program not using self. What am I missing? What is this self; why is it needed in some practical way?
I have read a lot about it - (stack, Python documentation), but I just don't understand why it's needed, since I can obviously create a program without it.
You can perfectly create a program without it. But then you'd be missing one of the key features of classes. If you can do without self, I'd argue you can do without classes and just do something purely with functions :)
Classes allow you to create objects which have a PROPERTY associated to them, and self allows you to access those values. So say you have a square.
g code:
class Square(object):
def __init__ (self, length, height):
self.length = length # THIS square's length, not others
self.height = height # THIS square's height, not other
def print_length_and_height(self):
print(self.length, self.height) # THIS square's length and height
square1 = Square(2,2)
square2 = Square(4,4)
square1.print_length_and_height() # 2 2
square2.print_length_and_height() # 4 4
Now, this example is quite silly, of course, but i think it shows what SELF specifically is for: it refers to the particular instance of an object.
By all means, if you don't see the point to it, just do away with classes and just use functions, there nothing wrong with that.
You haven't utilised a class or object properly. Cutting out the garbage code, your program reduces to:
def test(x,y): #No class
return x+y
def test1(x,y): #No class
return x-y
print(example.test(10,5))
print(example.test1(10,5))
Output:
15
5
Your "class" is no more useful than if you wrapped your program in the nested structures:
if True:
for i in range(1):
...
A proper object will have attributes (data fields) and functions that operate on that data (see below). Your code has an empty object; hence, you have nothing on which to operate, no need for self, and no need for a class at all.
Rather, use a class when you need to encapsulate a data representation and associated operations. Below, I've reused some of your code to make example do some trivial complex number work. There are many extensions and improvements to make in this; I kept it relatively close to your original work.
class example(object):
def __init__(self, a, b):
self.a = a
self.b = b
def __repr__(self):
sign = ' + ' if self.b >= 0 else ' - '
return str(self.a) + sign + str(abs(self.b)) + 'i'
def add(self, x):
self.a += x.a
self.b += x.b
def sub(self, x):
self.a -= x.a
self.b -= x.b
complex1 = example(10, 5)
complex2 = example(-3, 2)
complex1.add(complex2)
print(complex1)
complex2.sub(complex1)
print(complex2)
Output:
7 + 7i
-10 - 5i
Are you familiar with Object-Oriented Paradigm?
If you don't you should check it. Python is a Object-Oriented Language and self lets you define your object properties.
An example:
You have a class named Vehicle. A vehicle could be a bike, a car, even a plane. So something you can include is a name and a type.
class Vehicle():
def init(self, name, type): # Constructor
self.name = name
self.type = type
def info(self):
print("I'm a ")
print(self.name)
That's all, now you have a vehicle with name and type. Every instance of Vehicle would have a name and a type different or not and every intance can access its own variables. I'm sorry I can't explain it better. Firstable you need to know Object-Oriented Paradigm knowledge. Please comment my answer if you have doubts & I'll answer you or give a link where it comes explained better.
Is there a convention on how to have both a method and a function that do the same thing (or whether to do this at all)?
Consider, for example,
from random import choice
from collections import Counter
class MyDie:
def __init__(self, smallest, largest, how_many_rolls):
self.min = smallest
self.max = largest
self.number_of_rolls = how_many_rolls
def __call__(self):
return choice( range(self.min, self.max+1) )
def count_values(self):
return Counter([self() for n in range(self.number_of_rolls)])
def count_values(randoms_func, number_of_values):
return Counter([randoms_func() for n in range(number_of_values)])
where count_values is both a method and a function.
I think it's nice to have the method because the result "belongs to" the MyDie object. Also, the method can pull attributes from the MyDie object without having to pass them to count_values. On the other hand, it's nice to have the function in order to operate on functions other than MyDie, like
count_values(lambda: choice([3,5]) + choice([7,9]), 7)
Is it best to do this as above (where the code is repeated; assume the function is a longer piece of code, not just one line) or replace the count_values method with
def count_values(self):
return count_values(self, number_of_rolls)
or just get rid of the method all together and just have a function? Or maybe something else?
Here is an alternative that still allows you to encapsulate the logic in MyDie. Create a class method in MyDie
#staticmethod
def count_specified_values(random_func, number_of_values):
return Counter([randoms_func() for n in range(number_of_values)])
You also could add additional formal parameters to the constructor with default values that you could override to achieve the same functionality.
Suppose you have two classes, A and B. Class B is defined inside the class A. I want to access the variables and methods of the outer class while inside the inner class. The code here is a toy example but has the essentials of what I want to demonstrate:
class A:
a = 'even'
b = 'odd'
class B:
def __init__(self, n):
if n%2 == 0: self.num = a
if n%2 == 1: self.num = b
self.description = A.desc()
def __getitem__(self, i):
return self.B(i)
def desc(self):
return a + '-' + b
>>> c = A()
>>> d = c[4]
>>> TypeError: unbound method desc() must be called with A instance as first argument (got nothing instead)
Here the method desc does some work on the variables of the class A and produces output. Class A is initialized correctly and you can access the variables a and b, even from the inner scope, given that you don't define the description variable. However, I cannot find a way to call the outer scope class methods desc. Is it possible to use the method desc in B without instantiating class A?
Explanation on why I use such a pattern:
Variables a and b in my program are rather big. I only need to initialize them once. In addition, I don't want these variables to float around in the program but to be only accessible to the inner class. Adding to all these is the fact that I can use the A.__getitem__ to extract 'slices' of the big data when needed. So the outer class provides me with hiding/encapsulation of the data, the indexing operator (through __getitem__) and all the routines required for extraction of slices of data (here the method desc. The inner class, B, provides the bundling of useful information from the big data for each index. This, most likely, is not the optimal design for achieving the described task. I am open and eager to hear your opinion regarding the alternative patterns.
I can't see any reason for you to be using classes here, let alone nested ones. In any case, there is almost never a reason to nest classes in Python, since inner classes don't get any special access to the outer class.
However if you want to allow anything to access a method without instantiating the object, you can make it a classmethod:
#classmethod
def desc(self):
return a + '-' + b
But I can't see why you would do any of this. Also, nothing here is a closure.