I have quite complex system with python 2.7. Often in subclasses I need to override methods of parent class with just 'pass' like:
class Parent():
def some_method(self, data):
# do something to data here
....
class Child(Parent):
def some_method(self, data):
# Do not touch data, it should remain unchanged
pass
So my question: is there a way to test that method is overridden with a 'pass' statement? I mean an abstract test, given that it doesn't know anything about 'data': all it has is Child class, method name and that's it.
Of course, I can test that after running some_method data is not changed, but given the complexity of the system I have, there are too many places to check if something was affected, so I'm looking for an abstract solution.
UPD yes, this design is way far from perfect, I know, but can't help it, it's a legacy issue, so let's please leave the system design discussion aside except for one note: don't design your systems like this:)
So the solution I've come so far is (not totally abstract, but abstract enough for my case):
In test case call the subclass method intentionally passing it wrong-typed values, like None in my case:
def test_method_pass_statement(self):
test_class = Child()
try:
# If method isn't overridden, this should raise exception
# but check for your case if None doesn't raise exception
test_class.some_method(None)
except <Exception_type>:
# Customise error and message the way you want
raise
If the method is overridden with pass, it would consume whatever you pass it, given the number of arguments is correct.
Related
I want to define an abstract base class, called ParentClass. I want every child class of ParentClass to have a method called "fit" that defines a property "required". If someone tries to create a child class which does not have the "required" property within its fit method, I want an error to be created when the object's fit method is called. I am having trouble doing this.
The context is that I want to create a Parent class, abstract or otherwise, that requires its children to behave in a certain way and have certain properties so that I can trust them to behave in certain ways no matter who is creating children classes. I have found similar questions, but nothing precisely like what I am asking.
My naive attempt was something like the following:
class ParentClass(ABC):
#abstractmethod
def fit(self):
self.required = True
class ChildClass(ParentClass):
def __init__(self):
pass
def fit(self):
self.required = True
class ChildClass2(ParentClass):
def __init__(self):
pass
def fit(self):
self.not_essential = True
This doesn't work, but if possible I would like to refactor ParentClass in such a way that if someone runs:
>> b = ChildClass()
>> b.fit()
everything works fine, but if someone tries to run
>> b2 = ChildClass2()
>> b2.fit()
an error is thrown because the fit method of ChildClass2 doesn't define "required".
Is this possible in Python?
A related question is whether there is a better way to think about structuring my problem. Perhaps there is a better paradigm to achieve what I want? I understand that I can force child classes to have certain methods defined. A clunky way to achieve what I want is to have every property I want to be defined to be returned by a required method, but this feels very clunky, particularly if the number of properties I want to enforce as part of a standard becomes rather large.
Context and intentions: I want to use an object m_o of type My_object as a way of interfacing with another object called s_o of type Stubborn_object. For the sake of easy understanding, they should behave like if My_object inherited from Stubborn_object, in the way that calling an attribute that doesn't exist in My_object should call the attribute in Stubborn_object.
However, the tricky thing is that I wouldn't be asking this question if I could simply inherit My_object from Stubborn_object. It appears that I can't inherit from it, and, for many reasons, I also can't modify the code of the Stubborn_object class, so I have to use it as it is. Please note that trying to inherit isn't the issue of the question here. I know that other solutions exist for my practical problem, but I really want answers to stay on topic for many reasons. I suspect that other users can have different problems than mine and still be unable to inherit a class. Furthermore, not being able to inherit a class is not the only reason that could make someone read this question. In fact, it's quite a general Python object-oriented problem. I also believe the solution of my problem could be useful in other applications, like custom error handling within the object itself when an attribute is not found, or in thread management to lock the instance as soon as an attribute is called.
In addition to the problem of inheritance, let's suppose that I can't use conditions at higher levels to handle these cases, so everything has to be done inside My_object instance or its parents. That means that I can't use hasattr(m_o, attribute_name) to determine if I should call getattr(m_o, attribute_name) or getattr(s_o, attribute_name). This also means that any try/except blocks and other preconditions must be inside the My_object class or its parents. The point of this question is not about detecting exceptions when calling an attribute from outside the My_object instance. A try/catch block normally has to be outside the My_object class, and I previously stated that this can't be allowed.
For the sake of clarity and to provide a complete verifiable example, here is a sample code of the Stubborn_object class. I know that I said I can't inherit from Stubborn_object and the following code includes an inheritable class. Providing an example of an non-inheritable object would only bring confusion and it would'nt be really helpful to the question anyway, so here is a simple example of an inheritable object. The objective of this is to make an easy to understand question, so please just consider that you can't inherit from it:
class Stubborn_object:
def do_something(self):
print("do_something")
def action_to_override():
print("action_to_override")
def action_a(self):
print("action_a")
def action_b(self):
print("action_b")
Objective: Put it simply, I want my class My_object to detect all by itself that a lacking attribute has been called and run some instructions instead of throwing an AttributeError.
Current attempts: Right now, I manually redirect method calls to the Stubborn_object instance like so (it's successful, but not reliable nor scalable because of the use of hardcoding):
class My_object():
def __init__(self, s_o):
self.stubborn_object = s_o
def action_to_override(self):
# Do stuff. This method "overrides" the Stubborn_object.action_to_override method.
print("Here is stuff getting done instead of action_to_override")
def action_a(self):
return self.stubborn_object.action_a()
def action_b(self):
return self.stubborn_object.action_b()
s_o = Stubborn_object()
m_o = My_object(s_o)
m_o.action_to_override() # Executes Stubborn_object.do_something()
m_o.action_a() # Executes Stubborn_object.action_a()
m_o.action_b() # Executes Stubborn_object.action_b()
Executing this code along with the provided Stubborn_object code sample should print:
Here is stuff getting done instead of action_to_override
action_a
action_b
As you can see from methods action_a and action_b, I have to manually call the Stubborn_object methods from whithin the methods in My_object to mimic the attributes of Stubborn_object. This is ineficient, lacks of robustness and will throw an AttributeError exception if we attempt to make an action that wasn't included in the My_object code.
What if I wanted to automatically send method and attribute calls to the Stubborn_object instance without having to rewrite all of its method and attributes in My_object? I believe this can be achieved with detecting if a lacking attribute of My_object instance is called.
Expectations (or sort of): I am open to any solution that allows the My_object class or its parents to determine if the attribute is lacking or not, all within itself. So I believe I am ready to hear extremely original ideas, so go ahead.
On my part, I believe that something that uses parts of this code is the way to go, but it still lacks the "catch any called attribute" part:
class My_object():
def __init__(self, s_o):
# __init__ stays as it was.
self.stubborn_object = s_o
def action_to_override(self):
# This method also stays as it was.
# Do stuff. This method "overrides" the stubborn_object.action_to_override method.
print("Here is stuff getting done instead of action_to_override")
def run_me_when_method_is_not_found(self, method_name, **kwargs):
print("Method " + method_name + " not in 'My_object' class.")
return getattr(self.stubborn_object, method_name)(**kwargs)
So running those lines with the previous code sample
s_o = Stubborn_object()
m_o = My_object(s_o)
m_o.action_to_override() # Executes Stubborn_object.do_something()
m_o.action_a() # Executes Stubborn_object.action_a()
m_o.action_b() # Executes Stubborn_object.action_b()
will print
Here is stuff getting done instead of action_to_override
Method action_a not in 'My_object' class.
action_a
Method action_b not in 'My_object' class.
action_b
Some similar methods will have to be made for getters and setters, however, the idea stays the same. The thing is that this code lacks the ability to detect that an attribute is missing.
Question: How can I run the run_me_when_method_is_not_found when the method is not found in My_object? Especially, how can a My_object instance detect that the method doesn't exists in its class instead of throwing an AttributeError exception?
Thanks a lot.
Seems like overriding __getattribute__ will do exactly what you want: search for attribute in self.stubborn_object if it is missing in self. Put it into My_object class definition:
def __getattribute__(self, attr):
try:
return object.__getattribute__(self, attr)
except AttributeError:
return object.__getattribute__(self.stubborn_object, attr)
I have some working code (library) that, in some situations, I only need a small subset of its functional.
Thinking of a simpler case, the code (library) is a class that takes a few parameters when initializing.
For my limited use case, many of those parameters are not vital as they are not directly used in the internal calculation (some parameters are only used when I call particular methods of the object), while it is very hard to prepare those parameters properly.
So, I am wondering, if there is any easy way to know what parameters are essential without fully analyzing the library code (which is too complicated). For example, I may pass fake parameters to the api, And it would raise an exception only if they are actually used.
For example, I can pass in some_parameter = None for some_parameter that I guess won't be used. So whenever the library tries to access some_parameter.some_field an exception would be raised thus I can further look into the issue and replace it by the actually parameter. However, it would change the behavior of the library if the code itself accepts None as a parameter.
Are there any established approach to this problem? I don't mind false positive as I can always look into the problem and manually check if the usage of the fake parameters by the library is trivial.
For those suggestions on reading documentation and code, I don't have documentations! And the code is legacy code left by previous developers.
Update
#sapi:
Yes I would like to use the proxy pattern / object: I will further investigate on such topic.
"A virtual proxy is a placeholder for "expensive to create" objects. The real object is only created when a client first requests/accesses the object."
I am assuming all classes in question are new-style. This is always the case if you are using Python 3; in Python 2, they must extend from object. You can check a class with isinstance(MyClass, type). For the remainder of my answer, I will assume Python 3, since it was not specified. If you are using Python 2, make sure to extend from object where no other base class is specified.
If those conditions hold, you can write a descriptor that raises an exception whenever it is accessed:
class ParameterUsed(Exception):
pass
class UsageDescriptor:
def __init__(self, name):
super(UsageDescriptor, self).__init__()
self.name = name
def __get__(self, instance, owner):
raise ParameterUsed(self.name)
def __set__(self, instance, value):
# Ignore sets if the value is None.
if value is not None:
raise ParameterUsed(self.name)
def __delete__(self, instance):
# Ignore deletes.
pass
I will assume we are using this class as an example:
class Example:
def __init__(self, a, b):
self.a = a
self.b = b
def use_a(self):
print(self.a)
def use_b(self):
print(self.b)
If we want to see if a is used anywhere, extend the class and put an instance of our descriptor on the class:
class ExtExample(Example):
a = UsageDescriptor('a')
Now if we were to try to use the class, we can see which methods use a:
>>> example = ExtExample(None, None)
>>> example.use_a()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ParameterUsed: a
>>> example.use_b()
None
Here, we can see that use_a tried to use a (raising an exception because it did), but use_b did not (it completed successfully).
This approach works more generally than sapi’s does: in particular, sapi’s approach will only detect an attribute being accessed on the object. But there are plenty of things you can do that do not access attributes on that object. This approach, rather than detecting attributes being accessed on that object, detects the object itself being accessed.
Depending on what you're looking to achieve, you may be able to pass in a proxy object which throws an exception when accessed.
For example:
class ObjectUsedException(Exception):
pass
class ErrorOnUseProxy(object):
def __getattr__(self, name):
raise ObjectUsedException('Tried to access %s'%name)
Of course, that approach will fail in two pretty common situations:
if the library itself checks if the attribute exists (eg, to provide some default value)
if it's treated as a primitive (float, string etc), though you could modify this approach to take that into account
I belive the simplest and least intrusive way is to turn the parameters into properties:
class Foo(object):
def __init__(self):
pass
#property
def a(self):
print >>sys.stderr, 'Accesing parameter a'
return 1
bar = Foo()
print bar.a == 1
Will print True in stdout, and Accesing parameter a to stderr. You would have to tweak it to allow the class to change it.
I have the sense that this must be kind of a dumb question—nub here. So I'm open to an answer of the sort "This is ass-backwards, don't do it, please try this: [proper way]".
I'm using Python 2.7.5.
General Form of the Problem
This causes an infinite loop unless Thesaurus (an app-wide singleton) does not call Baseclass.__init__()
class Baseclass():
def __init__(self):
thes = Thesaurus()
#do stuff
class Thesaurus(Baseclass):
def __init__(self):
Baseclass.__init__(self)
#do stuff
My Specific Case
I have a base class that virtually every other class in my app extends (just some basic conventions for functionality within the app; perhaps should just be an interface). This base class is meant to house a singleton of a Thesaurus class that grants some flexibility with user input by inferring some synonyms (ie. {'yes':'yep', 'ok'}).
But since the subclass calls the superclass's __init__(), which in turn creates another subclass, loops ensue. Not calling the superclass's __init__() works just fine, but I'm concerned that's merely a lucky coincidence, and that my Thesaurus class may eventually be modified to require it's parent __init__().
Advice?
Well, I'm stopping to look at your code, and I'll just base my answer on what you say:
I have a base class that virtually every other class in my app extends (just some basic conventions for functionality within the app; perhaps should just be an interface).
this would be ThesaurusBase in the code below
This base class is meant to house a singleton of a Thesaurus class that grants some flexibility with user input by inferring some synonyms (ie. {'yes':'yep', 'ok'}).
That would be ThesaurusSingleton, that you can call with a better name and make it actually useful.
class ThesaurusBase():
def __init__(self, singleton=None):
self.singleton = singleton
def mymethod1(self):
raise NotImplementedError
def mymethod2(self):
raise NotImplementedError
class ThesaurusSingleton(ThesaurusBase):
def mymethod1(self):
return "meaw!"
class Thesaurus(TheraususBase):
def __init__(self, singleton=None):
TheraususBase.__init__(self, singleton)
def mymethod1(self):
return "quack!"
def mymethod2(self):
return "\\_o<"
now you can create your objects as follows:
singleton = ThesaurusSingleton()
thesaurus = Thesaurus(singleton)
edit:
Basically, what I've done here is build a "Base" class that is just an interface defining an expected behavior for all its children classes. The class ThesaurusSingleton (I know that's a terrible name) is also implementing that interface, because you said it had too and I did not want to discuss your design, you may always have good reasons for weird constraints.
And finally, do you really need to instantiate your singleton inside the class that is defining the singleton object? Though there may be some hackish way to do so, there's often a better design that avoids the "hackish" part.
What I think is that however you create your singleton, you should better do it explicitly. That's in the "Zen of python": explicit is better than implicit. Why? because then people reading your code (and that might be you in six months) will be able to understand what's happening and what you were thinking when you wrote that code. If you try to make things more implicit (like using sophisticated meta classes and weird self-inheritance) you may wonder what this code does in less than three weeks!
I'm not telling to avoid that kind of options, but to only use sophisticated stuff when you're out of simple ones!
Based on what you said I think the solution I gave can be a starting point. But as you focus on some obscure, yet not very useful hackish stuff instead of talking about your design, I can't be sure if my example is that appropriate, and hint you on the design.
edit2:
There's an another way to achieve what you say you want (but be sure that's really the design you want). You may want to use a class method that will act on the class itself (instead of the instances) and thus enable you to store a class-wide instance of itself:
>>> class ThesaurusBase:
... #classmethod
... def initClassWide(cls):
... cls._shared = cls()
...
>>> class T(ThesaurusBase):
... def foo(self):
... print self._shared
...
>>> ThesaurusBase.initClassWide()
>>> t = T()
>>> t.foo()
<__main__.ThesaurusBase instance at 0x7ff299a7def0>
and you can call the initClassWide method at the module level of where you declare ThesaurusBase, so whenever you import that module, it will have the singleton loaded (the import mechanism ensuring that python modules are run only once).
the short answer is:
do not instantiate an instance of a sub class from the super class constructor
longer answer:
if the motive you have to try to do this is the fact the Thesaurus is a singleton then you'll be better off exposing the singleton using a static method in the class (Thesaurus) and calling this method when you need the singleton
I am writing a small mocking class to do some tests.
But this class needs to support the idea of having nested attributes.
This example should provide some insight to the problem:
class Foo(object):
def __init__(self):
self.x = True
From the above class, we can have:
f = Foo()
f.x
I know I can add attributes falling back to __getattr__ to avoid an AttributeError, but what if I need something like this to be valid:
f = Foo()
f.x
f.x.y
f.x.y.z()
I know what to return if the object gets called as f.x.y.z() but I just need to find a way to get to z() that makes sense.
You can "mock anything" by returning, on each attribute access, another instance of the "mock anything" class (which must also be callable, if you want to have the .z() part work;-).
E.g.:
class MockAny(object):
# mock special methods by making them noops
def __init__(self, *a, **k): pass
# or returning fixed values
def __len__(self): return 0
# mock attributes:
def getattr(self, name):
return MockAny()
# make it callable, if you need to
def __call__(self, *a, **k):
return MockAny()
The alternative, of course, is to know what it is that you're mocking (by introspection, or by some form of "declarative description", or simply by coding mock for specific things;-) rather than take the catch-all approach; but, the latter is also feasible, as you see in the above (partial) example.
Personally, I'd recommend using an existing mocking framework such as pymox rather than reinventing this particular wheel (also, the source code for such frameworks can be more instructive than a reasonably terse response on SO, like this one;-).
If you are calling something like f.x.y.z() in your unit tests, the chances are you're trying to test too much. Each of these nested attributes should be covered by the unit tests for their particular classes.
Take another look at your Foo class and see if you can test its own behaviour in your unit tests.
Perhaps not the answer you were looking for, but hopefully one that will help in the long run.