I'm struggling to unit test the __init__ method on an enumeration I'm using. The difficulty is that I can't extend the enum (so can't implicitly call __init__). When I try to directly call it from my test it insists that the first parameter should be an instance of that enum. I can't do this, because I can't assume any of the enum's properties in advance.
With a contrived example of the problem, let's say I have an enum of even numbers:
class EvenNumbers(Enum):
two = 2
four = 4
I want to do some value checking so that no-one accidentally adds an odd number to this enum:
class EvenNumbers(Enum):
two = 2
four = 4
def __init__(self, value):
assert value % 2 == 0
Now I want to write a test to make sure that this works. Two approaches seem intuitive but don't work:
def test_approach_1():
'''
Try to extend the Enum with a bad value. This fails
because I can't extend an Enum.
'''
class BadEnum(EvenNumbers):
three = 3
def test_approach_2():
'''
Try to call the __init__ method directly. This fails
because I need to supply an instance of the enum as
the first parameter and I can't assume any values are
available.
'''
EvenNumbers.__init__(None, 3)
How should I unit test my __init__ method?
When testing an Enum that is a one-off (in other words, you only have one Enum class that is restricted to even numbers), then just test that Enum's values:
for enum in EvenNumbers:
self.assertTrue(enum.value % 2 == 0)
If you have behavior you want shared among many enumerations, or the behavior being tested has to do with the actual creation of the members, then make a base class with just the behavior (no members):
class EvenNumbers(Enum):
def __init__(self, value):
if value % 2 ! = 0:
raise ValueError('%d is not an even number' % value)
def double(self):
return self.value * 2
and then you can create test enumerations:
class TestEvenEnum(unittest.TestCase):
def test_bad_value(self):
with self.assertRaises(ValueError):
class NotEven(EvenNumbers):
one = 1
def test_good_value(self):
class YesEven(EvenNumbers):
two = 2
self.assertEqual(YesEven.two.double(), 4)
The solution I'm currently working with is to put the value checking logic into an empty enum:
class EvenNumbersBase(Enum):
def __init__(self, val): assert val % 2 == 0
class EvenNumbers(EvenNumbersBase):
two = 2
four = 4
This allows me to test that EvenNumbers inherits from EvenNumbersBase. I can subsequently try to extend EvenNumbersBase with different values to test the init method.
This feels like jumping through a few hoops. Can you suggest a better way?
Related
I am trying to modify an already defined class by changing an attribute's value. Importantly, I want this change to propagate internally.
For example, consider this class:
class Base:
x = 1
y = 2 * x
# Other attributes and methods might follow
assert Base.x == 1
assert Base.y == 2
I would like to change x to 2, making it equivalent to this.
class Base:
x = 2
y = 2 * x
assert Base.x == 2
assert Base.y == 4
But I would like to make it in the following way:
Base = injector(Base, x=2)
Is there a way to achieve this WITHOUT recompile the original class source code?
The effect you want to achieve belongs to the realm of "reactive programing" - a programing paradigm (from were the now ubiquitous Javascript library got its name as an inspiration).
While Python has a lot of mechanisms to allow that, one needs to write his code to actually make use of these mechanisms.
By default, plain Python code as the one you put in your example, uses the Imperative paradigm, which is eager: whenever an expression is encoutered, it is executed, and the result of that expression is used (in this case, the result is stored in the class attribute).
Python's advantages also can make it so that once you write a codebase that will allow some reactive code to take place, users of your codebase don't have to be aware of that, and things work more or less "magically".
But, as stated above, that is not free. For the case of being able to redefine y when x changes in
class Base:
x = 1
y = 2 * x
There are a couple paths that can be followed - the most important is that, at the time the "*" operator is executed (and that happens when Python is parsing the class body), at least one side of the operation is not a plain number anymore, but a special object which implements a custom __mul__ method (or __rmul__) in this case. Then, instead of storing a resulting number in y, the expression is stored somewhere, and when y is retrieved either as a class attribute, other mechanisms force the expression to resolve.
If you want this at instance level, rather than at class level, it would be easier to implement. But keep in mind that you'd have to define each operator on your special "source" class for primitive values.
Also, both this and the easier, instance descriptor approach using property are "lazily evaluated": that means, the value for y is calcualted when it is to be used (it can be cached if it will be used more than once). If you want to evaluate it whenever x is assigned (and not when y is consumed), that will require other mechanisms. Although caching the lazy approach can mitigate the need for eager evaluation to the point it should not be needed.
1 - Before digging there
Python's easiest way to do code like this is simply to write the expressions to be calculated as functions - and use the property built-in as a descriptor to retrieve these values. The drawback is small:
you just have to wrap your expressions in a function (and then, that function
in something that will add the descriptor properties to it, such as property). The gain is huge: you are free to use any Python code inside your expression, including function calls, object instantiation, I/O, and the like. (Note that the other approach requires wiring up each desired operator, just to get started).
The plain "101" approach to have what you want working for instances of Base is:
class Base:
x = 1
#property
def y(self):
return self.x * 2
b = Base()
b.y
-> 2
Base.x = 3
b.y
-> 6
The work of property can be rewritten so that retrieving y from the class, instead of an instance, achieves the effect as well (this is still easier than the other approach).
If this will work for you somehow, I'd recommend doing it. If you need to cache y's value until x actually changes, that can be done with normal coding
2 - Exactly what you asked for, with a metaclass
as stated above, Python'd need to know about the special status of your y attribute when calculcating its expression 2 * x. At assignment time, it would be already too late.
Fortunately Python 3 allow class bodies to run in a custom namespace for the attribute assignment by implementing the __prepare__ method in a metaclass, and then recording all that takes place, and replacing primitive attributes of interest by special crafted objects implementing __mul__ and other special methods.
Going this way could even allow values to be eagerly calculated, so they can work as plain Python objects, but register information so that a special injector function could recreate the class redoing all the attributes that depend on expressions. It could also implement lazy evaluation, somewhat as described above.
from collections import UserDict
import operator
class Reactive:
def __init__(self, value):
self._initial_value = value
self.values = {}
def __set_name__(self, owner, name):
self.name = name
self.values[owner] = self._initial_value
def __get__(self, instance, owner):
return self.values[owner]
def __set__(self, instance, value):
raise AttributeError("value can't be set directly - call 'injector' to change this value")
def value(self, cls=None):
return self.values.get(cls, self._initial_value)
op1 = value
#property
def result(self):
return self.value
# dynamically populate magic methods for operation overloading:
for name in "mul add sub truediv pow contains".split():
op = getattr(operator, name)
locals()[f"__{name}__"] = (lambda operator: (lambda self, other: ReactiveExpr(self, other, operator)))(op)
locals()[f"__r{name}__"] = (lambda operator: (lambda self, other: ReactiveExpr(other, self, operator)))(op)
class ReactiveExpr(Reactive):
def __init__(self, value, op2, operator):
self.op2 = op2
self.operator = operator
super().__init__(value)
def result(self, cls):
op1, op2 = self.op1(cls), self.op2
if isinstance(op1, Reactive):
op1 = op1.result(cls)
if isinstance(op2, Reactive):
op2 = op2.result(cls)
return self.operator(op1, op2)
def __get__(self, instance, owner):
return self.result(owner)
class AuxDict(UserDict):
def __init__(self, *args, _parent, **kwargs):
self.parent = _parent
super().__init__(*args, **kwargs)
def __setitem__(self, item, value):
if isinstance(value, self.parent.reacttypes) and not item.startswith("_"):
value = Reactive(value)
super().__setitem__(item, value)
class MetaReact(type):
reacttypes = (int, float, str, bytes, list, tuple, dict)
def __prepare__(*args, **kwargs):
return AuxDict(_parent=__class__)
def __new__(mcls, name, bases, ns, **kwargs):
pre_registry = {}
cls = super().__new__(mcls, name, bases, ns.data, **kwargs)
#for name, obj in ns.items():
#if isinstance(obj, ReactiveExpr):
#pre_registry[name] = obj
#setattr(cls, name, obj.result()
for name, reactive in pre_registry.items():
_registry[cls, name] = reactive
return cls
def injector(cls, inplace=False, **kwargs):
original = cls
if not inplace:
cls = type(cls.__name__, (cls.__bases__), dict(cls.__dict__))
for name, attr in cls.__dict__.items():
if isinstance(attr, Reactive):
if isinstance(attr, ReactiveExpr) and name in kwargs:
raise AttributeError("Expression attributes can't be modified by injector")
attr.values[cls] = kwargs.get(name, attr.values[original])
return cls
class Base(metaclass=MetaReact):
x = 1
y = 2 * x
And, after pasting the snippet above in a REPL, here is the
result of using injector:
In [97]: Base2 = injector(Base, x=5)
In [98]: Base2.y
Out[98]: 10
The idea is complicated with that aspect that Base class is declared with dependent dynamically evaluated attributes. While we can inspect class's static attributes, I think there's no other way of getting dynamic expression except for parsing the class's sourcecode, find and replace the "injected" attribute name with its value and exec/eval the definition again. But that's the way you wanted to avoid. (moreover: if you expected injector to be unified for all classes).
If you want to proceed to rely on dynamically evaluated attributes define the dependent attribute as a lambda function.
class Base:
x = 1
y = lambda: 2 * Base.x
Base.x = 2
print(Base.y()) # 4
I'm fairly new to python but I can't seem to find the answer to this anywhere.
I want to know how to be able to create a function that you can call on by attaching it to something else
for example .isdigit() has a period at the beginning and returns a 1 or 0 depending on if the variable is a number.
How can I do something similar, for instance: string.myfunc()?
You can create a class. For this example, .myfunc() will return 1 if the number is greater than 0, otherwise, it will return 0:
class s:
def __init__(self, value):
self.value = value
def myfunc(self):
if self.value > 0:
return 1
else:
return 0
num = s(20)
print(num.myfunc())
Output:
1
The .func() is specially used for objects belonging to a particular class. For example, .split() is specially built for strings.
You can also create them by creating a class and then defining a function in that class.
I have run across a few examples of Python code that looks something like this:
class GiveNext :
list = ''
def __init__(self, list) :
GiveNext.list = list
def giveNext(self, i) :
retval = GiveNext.list[i]
return retval
class GiveABCs(GiveNext):
i = -1
def _init__(self, list) :
GiveNext.__init__(self, list)
def giveNext(self):
GiveABCs.i += 1
return GiveNext.giveNext(self, GiveABCs.i)
class Give123s(GiveNext):
i = -1
def _init__(self, list) :
GiveNext.__init__(self, list)
def giveNext(self):
Give123s.i += 1
return GiveNext.giveNext(self, Give123s.i)
for i in range(3):
print(GiveABCs('ABCDEFG').giveNext())
print(Give123s('12345').giveNext())
the output is: A 1 B 2 C 3
If I were more clever, I could figure out how to put the string literals inside the constructor...but that is not crucial right now.
My question is on the use of classes this way. Yes, an instance of the class gets created each time that that the call within the print() gets made. Yet the i's are 'permanent' in each class.
This strikes me as less of an object-oriented approach, and more of a way of using classes to accomplish encapsulation and/or a functional programming paradigm, since the instances are entirely transitory. In other words, an instance of the class is never instantiated for its own purposes; it is there only to allow access to the class-wide methods and variables within to do their thing, and then it is tossed away. In many cases, it seems like the class mechanism is used in a back-handed way, in order to leverage inheritance and name resolution/spacing: an instance of the class is never really required to be built or used, conceptually.
Is this standard Python form?
Bonus question: how would I put the string literals inside each class declaration? Right now, even if I change the _init__ for GiveABCs to
GiveNext.__init__(self, 'wxyz')
it completely ignores the 'wxyz' literal, and uses the 'ABCDEF' one - even though it is never mentioned...
Please don't learn Python with this code. As mentioned by others, this code goes against many Python principles.
One example: list is a Python builtin type. Don't overwrite it, especially not with a string instance!
The code also mixes class and instance variables and doesn't use super() in subclasses.
This code tries to simulate an iterator. So simply use an iterator:
give_abcs = iter('ABCDEFG')
give_123s = iter('12345')
for _ in range(3):
print(next(give_abcs))
print(next(give_123s))
# A
# 1
# B
# 2
# C
# 3
If you really want to fix the above code, you could use:
class GiveNext :
def __init__(self, iterable) :
self.i = - 1
self.iterable = iterable
def giveNext(self) :
self.i += 1
return self.iterable[self.i]
giveABCs = GiveNext('ABCDEFG')
give123s = GiveNext('12345')
for _ in range(3):
print(giveABCs.giveNext())
print(give123s.giveNext())
It outputs:
A
1
B
2
C
3
This code in the OP is an incredible amount of crap. Not only it is long, unreadable, misuses OO features, and does not use Python features at all (an iterator being a standard Python feature). Here is a suggestion for a more Pythonist approach:
giveABCs = iter('ABCDEFG')
give123s = iter('12345')
for i in range(3):
print(next(giveABCs))
print(next(give123s))
About your bonus question: I guess you are modifing the _init__() method of GiveABCs and Give123s. It is normal that whatever code you put in there has no effect, because the Python constructor is __init__() (with 2 leading underscores, not 1). So The constructor from GiveNext is not overloaded.
So, I have defined the following class which should resemble a probability mass function. However, its logic seems broken and it will raise SUM_ERROR every time I try to initialize a new object.
class ProbabilityMass(dict):
class InvalidEntries(Exception):
pass
SUM_ERROR = InvalidEntries("all values must add upto '1'")
VAL_ERROR = InvalidEntries("negative values are not allowed")
def __init__(self, pm):
dict.__init__(pm)
# Input requirements
if not self.sumsUptoOne():
raise ProbabilityMass.SUM_ERROR
if not self.isNonnegative():
raise ProbabilityMass.VAL_ERROR
def isNonnegative(self):
return all(d < 0 for d in self.values())
def sumsUptoOne(self):
return sum(self.values()) == 1
How can I fix this?
Calling dict.__init__() does not initialize the class. The correct call to super should look like this:
def __init__(self, pm):
super(ProbabilityMass, self).__init__(pm)
# Input requirements
...
As a side note, your isNonnegative() method is also incorrect. Change it to:
def isNonnegative(self):
return all(d >= 0 for d in self.values())
Usually, when dict.__init__() is called, it is because you used dict(). When a class is called like a function, an instance is created, and the instance's .__init__() method is called with the arguments given to the class. Well, calling an instance method is the same thing as calling the class method with the instance as a first argument. Therefore, x = dict() is short for:
x = new dict instance
dict.__init__(x)
If you already have an instance of dict (or a subclass) that was not initialized, you can call __init__() yourself. You must, however, remember to pass the instance as the first argument:
dict.__init__(self, pm)
The more common way is to use the built-in super():
super(ProbabilityMass, self).__init__(pm)
In Python, how is it possible to reuse existing equal immutable objects (like is done for str)? Can this be done just by defining a __hash__ method, or does it require more complicated measures?
If you want to create via the class constructor and have it return a previously created object then you will need to provide a __new__ method (because by the time you get to __init__ the object has already been created).
Here is a simple example - if the value used to initialise has been seen before then a previously created object is returned rather than a new one created:
class Cached(object):
"""Simple example of immutable object reuse."""
def __init__(self, i):
self.i = i
def __new__(cls, i, _cache={}):
try:
return _cache[i]
except KeyError:
# you must call __new__ on the base class
x = super(Cached, cls).__new__(cls)
x.__init__(i)
_cache[i] = x
return x
Note that for this example you can use anything to initialise as long as it's hashable. And just to show that objects really are being reused:
>>> a = Cached(100)
>>> b = Cached(200)
>>> c = Cached(100)
>>> a is b
False
>>> a is c
True
There are two 'software engineering' solutions to this that don't require any low-level knowledge of Python. They apply in the following scenarios:
First Scenario: Objects of your class are 'equal' if they are constructed with the same constructor parameters, and equality won't change over time after construction. Solution: Use a factory that hashses the constructor parameters:
class MyClass:
def __init__(self, someint, someotherint):
self.a = someint
self.b = someotherint
cachedict = { }
def construct_myobject(someint, someotherint):
if (someint, someotherint) not in cachedict:
cachedict[(someint, someotherint)] = MyClass(someint, someotherint)
return cachedict[(someint, someotherint)]
This approach essentially limits the instances of your class to one unique object per distinct input pair. There are obvious drawbacks as well: not all types are easily hashable and so on.
Second Scenario: Objects of your class are mutable and their 'equality' may change over time. Solution: define a class-level registry of equal instances:
class MyClass:
registry = { }
def __init__(self, someint, someotherint, third):
MyClass.registry[id(self)] = (someint, someotherint)
self.someint = someint
self.someotherint = someotherint
self.third = third
def __eq__(self, other):
return MyClass.registry[id(self)] == MyClass.registry[id(other)]
def update(self, someint, someotherint):
MyClass.registry[id(self)] = (someint, someotherint)
In this example, objects with the same someint, someotherint pair are equal, while the third parameter does not factor in. The trick is to keep the parameters in registry in sync. As an alternative to update, you could override getattr and setattr for your class instead; this would ensure that any assignment foo.someint = y would be kept synced with your class-level dictionary. See an example here.
I believe you would have to keep a dict {args: object} of instances already created, then override the class' __new__ method to check in that dictionary, and return the relevant object if it already existed. Note that I haven't implemented or tested this idea. Of course, strings are handled at the C level.