I have the following code, which is a command line test
from cmd2 import Cmd
class App(Cmd, object):
def __init__(self, *args, **kwargs):
pass
def do_test(self, line):
'test'
print "parent test"
class App2():
def __init__(self, *args, **kwargs):
pass
def do_test2(self, line):
print "Test2"
app = App()
app.cmdloop()
Is there a possibility to extend the App class with extra functions?
I know there is the following solution
class App2(App):
....
app = App2()
app.cmdloop()
but in my case I would like to run only the App and extend it if it is possible.
In general this is not a good idea, because when people (including you in six months) read the code they will expect App to be the class they know about, and for an extended version of the class to have a different name. However, there's nothing preventing you from naming the subclass the same as the original:
class App(App):
# etc.
Python's smart enough to know that the App in parentheses means the one it already knows about, and then, since the new class has the same name, it replaces the original one. Don't worry, the new class contains a reference to the old one, so the old class doesn't go away entirely (if it did, nothing inherited by the subclass would work).
If the class came from some module that you've imported, you can even monkey-patch the replacement class back into the original module, so that all code that imports that module uses your replacement class. (Though I would recommend against it!)
import appmodule
class App(appmodule.App):
# etc.
appmodule.App = App
Of course, this gets tricky, because some modules may already have imported a reference to the original class, if you don't do this first thing in your script. And if other modules are also trying to patch the same class, all hell can break loose. Still, if you want to confuse yourself and those who will maintain your code, Python will let you do it!
It is worth noting that you can always augment the class dictionary, therefore extending it at runtime.
class App(...):
def __init__(self, a, b):
pass
def do_something(self, a):
pass
app_instance = App()
def do_something_else(self, b):
pass
App.do_something_else = do_something_else
app_instance.do_something_else('b')
You have to think how python does lookups at runtime. First looks at the instance of yourclass, then looks at the __mro__ (starting with type(yourclass)), and on up until it gets to object.
Since classes ARE objects, you can extend them by adding attributes, which will then be found during attribute lookups. Make sure you do this ONCE (eg, during an import of another file).
Here is a real example:
>>> class foo():
... pass
...
>>> x = foo()
>>>
>>> # Define a function and attach it
>>>
>>> def bar(self, a):
... print(a)
...
>>> foo.bar = bar
>>>
>>> x.bar('a')
a
this isnt an exact solution but it allows you to always access it by App name
rename App class to _App
then where you want to use it
from blah import _App as App
and when you extend it
from blah import App2 as App
Related
(Nota bene: This is heavily modified from the original question, to include details I erroneously elided.)
This is the (summarized) file (common.py) I'm testing. It contains a decorator (derived from the Decorum library) that calls a class method on another object(A): I want to patch out A, because that code makes an external call I'm not testing.
from decorum import Decorum
class A:
#classmethod
def c(cls):
pass
class ClassyDecorum(Decorum):
"""Hack to allow decorated instance methods of a class object to run with decorators.
Replace this once Decorum 1.0.4+ comes out.
"""
def __get__(self, instance, owner):
from functools import partial
return partial(self.call, instance)
class B(Decorum):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
def init(self, *args, **kwargs):
A.c()
return super().init(*args, **kwargs)
I'd like to #patch class A in my unittest, to isolate and check B.d()'s functionality. This is my unittest (located in test/test_common.py):
class BDecoratedClass(MagicMock):
#B
def dummy_func(self):
return "Success"
class TestB(TestCase):
#patch('unittest_experiment.A', autospec=True)
def test_d(self, mock_a):
b = BDecoratedClass()
b.dummy_func()
mock_a.c.assert_called_once_with() # Fails
Debugging the above, I see that A is never actually mocked: the code proceeds into A's code, so it makes sense that mock_a is never called, and thus the assertion fails. However, I'd like to properly monkey patch A. This approach works if I'm monkey patching an import that exists in common.py, but apparently not if the class is defined there?
Note that I think this is likely an issue of where I'm patching, that is #patch('common.A', autospec=True) should more likely be something like #patch('where.python.actually.finds.A.when.B.calls.A', autospec=True). But I'm very unclear on how to determine if that is the case, and if so, what the correct path is. For instance, #patch('BDecorated.common.A', autospec=True) does not work.
Thanks to #user2357112, I arrived at this solution. Caveat: I don't know if this is standard or 'best' practice, but it seems to work.
First, move the BDecoratedClass to it's own file in test/dummy.py. Then change the test to this:
class TestB(TestCase):
#patch('common.A', autospec=True)
def test_d(self, mock_a):
from test.dummy import BDecoratedClass
b = BDecoratedClass()
b.dummy_func()
mock_a.c.assert_called_once_with() # Succeeds
This forces the patch to execute prior to the import of the dummy class being decorated. It's a little weird because the import is inside the function, but for a test that seems fine.
Bigger Caveat:
This only works for the first test that imports something from the module where, in this case BDecoratedClass imports from. At that juncture everything else in the class has been loaded and cannot be patched.
It looks like patch substitutes the import and that's why it's too late at that point unless you work around it like you explained in your the answer. I found that using patch.object for individual methods also work. So something like:
class TestB(TestCase):
#patch.object(A, 'c')
def test_d(self, mock_c):
b = BDecoratedClass()
b.dummy_func()
mock_c.assert_called_once_with() # Succeeds
It's possible to use type in Python to create a new class object, as you probably know:
A = type('A', (object,), {})
a = A() # create an instance of A
What I'm curious about is whether there's any problem with creating different class objects with the same name, eg, following on from the above:
B = type('A', (object,), {})
In other words, is there an issue with this second class object, B, having the same name as our first class object, A?
The motivation for this is that I'd like to get a clean copy of a class to apply different decorators to without using the inheritance approach described in this question.
So I'd like to define a class normally, eg:
class Fruit(object):
pass
and then make a fresh copy of it to play with:
def copy_class(cls):
return type(cls.__name__, cls.__bases__, dict(cls.__dict__))
FreshFruit = copy_class(fruit)
In my testing, things I do with FreshFruit are properly decoupled from things I do to Fruit.
However, I'm unsure whether I should also be mangling the name in copy_class in order to avoid unexpected problems.
In particular, one concern I have is that this could cause the class to be replaced in the module's dictionary, such that future imports (eg, from module import Fruit return the copied class).
There is no reason why you can't have 2 classes with the same __name__ in the same module if you want to and have a good reason to do so.
e.g. In your example from module import Fruit -- python doesn't care at all about the __name__ of the class. It looks in the module's globals for Fruit and imports what it finds there.
Note that, in general, this approach isn't great if you're using super (although the same can be said for class decorators ...):
class A(Base):
def foo(self):
super(A, self).foo()
B = copy_class(A)
In this case, when B.foo is called, it will end up calling super(A, self) which could lead to funky behaviour in a number of circumstances. . .
I'm not really sure how best to explain what I want, so I'll just show some code:
class Stuffclass():
def add(self, x, y):
return x + y
def subtract(self, x, y):
return x - y
# imagine that there are 20-30 other methods in here (lol)
class MyClass:
def __init__(self):
self.st = Stuffclass()
def doSomething(self):
return self.st.add(1, 2)
m = MyClass()
m.doSomething() # will print 3
# Now, what I want to be able to do is:
print m.add(2, 3) # directly access the "add" method of MyClass.st
print m.subtract(10, 5) # directly access the "subtract" method of MyClass.st
m.SomeMethod() # execute function MyClass.st.SomeMethod
I know I could do something like this:
class MyClass:
def __init__(self):
self.st = Stuffclass()
self.add = self.st.add
self.subtract = self.st.subtract
...but this requires manually assigning all possible attributes.
I'm writing all the classes so I can guarantee no name collisions.
Making MyClass a subclass of Stuffclass won't work, because I actually am using this in a plugin-based application, where MyClass loads other code dynamically using import. This means MyClass can't subclass from the plugin, because the plugin could be anything that follows my API.
Advice please?
I believe that writing a getattr function for your class will let you do what you want.
Called when an attribute lookup has not found the attribute in the usual places (i.e. it is not an instance attribute nor is it found in the class tree for self). name is the attribute name. This method should return the (computed) attribute value or raise an AttributeError exception
So something as simple as:
def __getattr__(self, name):
if hasattr(self.st, name):
return getattr(self.st, name)
else:
raise AttributeError
should do roughly what you're after.
But, having answered (I think) the question you asked, I'm going to move on to the question I think you should have asked.
I actually am using this in a plugin-based application, where MyClass loads other code dynamically using import. This means MyClass can't subclass from the plugin, because the plugin could be anything that follows my API
I can see why MyClass can't be a subclass of StuffClass; but couldn't StuffClass be a subclass of MyClass? If you defined the inheritance that way, you'd have a guarantee what StuffClass implements all the basic stuff in MyClass, and also that your instances of StuffClass have all the extra methods defined in StuffClass.
From your mention that the plugins need to "follows my API", I'm assuming that might be a case where you need to ensure that the plugins implement a set of methods in order to conform with the API; but since the implementation of the methods is going to depend on the specifics of the plugin, you can't provide those functions in MyClass. In that case, it sounds as though defining an Abstract Base Class that your plugins are required to inherit from might be useful for you.
Use __getattr__ to delegate the calls to Stuffclass's instance:
class MyClass:
def __init__(self):
self.st = Stuffclass()
def __getattr__(self,attr):
return getattr(self.st,attr)
Demo:
>>> from so import *
>>> m = MyClass()
>>> m.add(1,2)
3
>>> m.subtract(100,2)
98
I am having trouble with this setup mainly because I am not sure what I actually want in order to solve this problem.
This is the setup
- main.py
- lib
- __init__.py
- index.py
- test.py
__init__.py has this code
import os
for module in os.listdir(os.path.dirname(__file__)+"/."):
if module == '__init__.py' or module[-3:] != '.py':
continue
__import__(module[:-3], locals(), globals())
del module
main.py has this code as of now
from lib.index import *
print User.__dict__
index.py has this code
class User(object):
def test(self):
return "hi"
pass
test.py has this code
class User(object):
def tes2(self):
return "hello"
When I execute main.py it successfully prints the method test from index.py but what I am trying to do is figure out a way where I can just create a file in the lib folder where that while has only one function in the format
class User(object):
def newFunction(self):
return abc
and this function should automatically be available for me in main.py
I am sure that this is not a hard thing to do but I honestly don't know what I want (what to search for to solve this) which is preventing me from researching the solution.
You can use a metaclass to customize class creation and add functions defined elsewhere:
import types
import os
import os.path
import imp
class PluginMeta(type):
def __new__(cls, name, bases, dct):
modules = [imp.load_source(filename, os.path.join(dct['plugindir'], filename))
for filename in os.listdir(dct['plugindir']) if filename.endswith('.py')]
for module in modules:
for name in dir(module):
function = getattr(module, name)
if isinstance(function, types.FunctionType):
dct[function.__name__] = function
return type.__new__(cls, name, bases, dct)
class User(metaclass=PluginMeta):
plugindir = "path/to/the/plugindir"
def foo(self):
print "foo"
user = User()
print dir(user)
Then in the plugin files, just create functions not classes:
def newFunction(self, abc):
self.abc = abc
return self.abc
And the metaclass will find them, turn them into methods, and attach them to your class.
Classes are objects, and methods are nothing more than attributes on class-objects.
So if you want to add a method to an existing class, outside the original class block, all that is is the problem of adding an attribute to an object, which I would hope you know how to do:
class User(object):
pass
def newFunction(self):
return 'foo'
User.newFunction = newFunction
agf's metaclass answer is basically a nifty automatic way of doing this, although it works by adding extra definitions to the class block before the class is created, rather than adding extra attributes to the class object afterwards.
That should be basically all you need to develop a framework in which things defined in one module are automatically added to a class defined elsewhere. But you still need to make a number of design decisions, such as:
If your externally-defined functions need auxiliary definitions, how do you determine what's supposed to get added to the class and what was just a dependency?
If you have more than one class you're extending this way, how do you determine what goes in which class?
At what point(s) in your program does the auto-extension happen?
Do you want to say in your class "this class has extensions defined elsewhere", or say in your extensions "this is an extension to a class defined elsewhere", or neither and somewhere bind extensions to classes externally from both?
Do you need to be able to have multiple versions of the "same" class with different extensions active at the same time?
A metaclass such as proposed by agf can be a very good way of implementing this sort of framework, because it lets you put all the complex code in one place while still "tagging" every class that doesn't work the way classes normally work. It does fix the answers to some of the questions I posed above, though.
here a working code we used in a project, I'm not sure it's the best way but it worked and there is almost no additional code to add to other files
cpu.py:
from cpu_base import CPU, CPUBase
import cpu_common
import cpu_ext
cpu_base.py:
def getClass():
return __cpu__
def setClass(CPUClass):
global __cpu__
__cpu__ = CPUClass
__classes__.append(CPUClass)
def CPU(*kw):
return __cpu__(*kw)
class CPUBase:
def __init__(self):
your_init_Stuff
# optionally a method classname_constructor to mimic __init__ for each one
for c in __classes__:
constructor = getattr(c, c.__name__ + '_constructor', None)
if constructor is not None:
constructor(self)
setClass(CPUBase)
cpu_common.py:
from cpu_base import getClass, setClass
class CPUCommon(getClass()):
def CPUCommon_constructor(self):
pass
setClass(CPUCommon)
cpu_ext.py:
from cpu_base import getClass, setClass
class CPUExt(getClass()):
pass
setClass(CPUExt)
to use the class import CPU from cpu.py
With a class in Python, how do I define a function to print every single instance of the class in a format defined in the function?
I see two options in this case:
Garbage collector
import gc
for obj in gc.get_objects():
if isinstance(obj, some_class):
dome_something(obj)
This has the disadvantage of being very slow when you have a lot of objects, but works with types over which you have no control.
Use a mixin and weakrefs
from collections import defaultdict
import weakref
class KeepRefs(object):
__refs__ = defaultdict(list)
def __init__(self):
self.__refs__[self.__class__].append(weakref.ref(self))
#classmethod
def get_instances(cls):
for inst_ref in cls.__refs__[cls]:
inst = inst_ref()
if inst is not None:
yield inst
class X(KeepRefs):
def __init__(self, name):
super(X, self).__init__()
self.name = name
x = X("x")
y = X("y")
for r in X.get_instances():
print r.name
del y
for r in X.get_instances():
print r.name
In this case, all the references get stored as a weak reference in a list. If you create and delete a lot of instances frequently, you should clean up the list of weakrefs after iteration, otherwise there's going to be a lot of cruft.
Another problem in this case is that you have to make sure to call the base class constructor. You could also override __new__, but only the __new__ method of the first base class is used on instantiation. This also works only on types that are under your control.
Edit: The method for printing all instances according to a specific format is left as an exercise, but it's basically just a variation on the for-loops.
You'll want to create a static list on your class, and add a weakref to each instance so the garbage collector can clean up your instances when they're no longer needed.
import weakref
class A:
instances = []
def __init__(self, name=None):
self.__class__.instances.append(weakref.proxy(self))
self.name = name
a1 = A('a1')
a2 = A('a2')
a3 = A('a3')
a4 = A('a4')
for instance in A.instances:
print(instance.name)
You don't need to import ANYTHING! Just use "self". Here's how you do this
class A:
instances = []
def __init__(self):
self.__class__.instances.append(self)
print('\n'.join(A.instances)) #this line was suggested by #anvelascos
It's this simple. No modules or libraries imported
Very nice and useful code, but it has a big problem: list is always bigger and it is never cleaned-up, to test it just add print(len(cls.__refs__[cls])) at the end of the get_instances method.
Here a fix for the get_instances method:
__refs__ = defaultdict(list)
#classmethod
def get_instances(cls):
refs = []
for ref in cls.__refs__[cls]:
instance = ref()
if instance is not None:
refs.append(ref)
yield instance
# print(len(refs))
cls.__refs__[cls] = refs
or alternatively it could be done using WeakSet:
from weakref import WeakSet
__refs__ = defaultdict(WeakSet)
#classmethod
def get_instances(cls):
return cls.__refs__[cls]
Same as almost all other OO languages, keep all instances of the class in a collection of some kind.
You can try this kind of thing.
class MyClassFactory( object ):
theWholeList= []
def __call__( self, *args, **kw ):
x= MyClass( *args, **kw )
self.theWholeList.append( x )
return x
Now you can do this.
object= MyClassFactory( args, ... )
print MyClassFactory.theWholeList
Python doesn't have an equivalent to Smallktalk's #allInstances as the architecture doesn't have this type of central object table (although modern smalltalks don't really work like that either).
As the other poster says, you have to explicitly manage a collection. His suggestion of a factory method that maintains a registry is a perfectly reasonable way to do it. You may wish to do something with weak references so you don't have to explicitly keep track of object disposal.
It's not clear if you need to print all class instances at once or when they're initialized, nor if you're talking about a class you have control over vs a class in a 3rd party library.
In any case, I would solve this by writing a class factory using Python metaclass support. If you don't have control over the class, manually update the __metaclass__ for the class or module you're tracking.
See http://www.onlamp.com/pub/a/python/2003/04/17/metaclasses.html for more information.
In my project, I faced a similar problem and found a simple solution that may also work for you in listing and printing your class instances. The solution worked smoothly in Python version 3.7; gave partial errors in Python version 3.5.
I will copy-paste the relevant code blocks from my recent project.
```
instances = []
class WorkCalendar:
def __init__(self, day, patient, worker):
self.day = day
self.patient = patient
self.worker= worker
def __str__(self):
return f'{self.day} : {self.patient} : {self.worker}'
In Python the __str__ method in the end, determines how the object will be interpreted in its string form. I added the : in between the curly brackets, they are completely my preference for a "Pandas DataFrame" kind of reading. If you apply this small __str__ function, you will not be seeing some machine-readable object type descriptions- which makes no sense for human eyes. After adding this __str__ function you can append your objects to your list and print them as you wish.
appointment= WorkCalendar("01.10.2020", "Jane", "John")
instances.append(appointment)
For printing, your format in __str__ will work as default. But it is also possible to call all attributes separately:
for instance in instances:
print(instance)
print(instance.worker)
print(instance.patient)
For detailed reading, you may look at the source: https://dbader.org/blog/python-repr-vs-str