class TaskInput:
def __init__(self):
self.cfg = my_config #### Question: How do I do this only once?
class TaskA(TaskInput):
def __init__(self):
pass
class TaskB (TaskInput):
def __init__(self):
pass
There are many tasks like TaskA, TaskB etc, they all are inherited from TaskInput.
Tasks also depend on something, let's say, a configuration which I only want to set ONCE.
The code has multiple Tasks classes, like TaskA, TaskB etc. They all depend on this common configuration.
One natural way would be to make this configuration a class member of TaskInput, ie, TaskInput.cfg = my_config, something that's initialized in __init__() of TaskInput.
However, if it's a member of TaskInput, it'll get executed multiple times, every time a new object of type TaskX is created as all those Tasks are inherited from TaskInput.
What's the best practice and best way to accomplish this in Python?
Make the configuration a class attribute by defining it on the class rather than in __init__.
class TaskInput:
cfg = my_config
It is now accessible as self.cfg on any instance of TaskInput or its children.
If the configuration isn't available when the class is defined (e.g. it's read later from a file), assign it on the class when it is available:
TaskInput.cfg = my_config
You have two choices as to how to handle writing the class definition in this situation.
Don't define cfg in the class definition at all, so you'll get a big juicy AttributeError if you try to access the configuration before it's available.
Define a default configuration which gets overwritten when the real configuration is available.
Generally I favor approach #1, since it "fails fast" (i.e. detects logic errors in your code when they occur rather than hiding them until something goes screwy later), but there are situations where you might need a default configuration to get things up and running before you can read the "real" configuration. In that case the default configuration should be the bare minimum possible, just what you need to get going.
I will not try to guess your need so I will assume you mean exactly what you said below, namely that you want a single initialization of a class member, but done through the creation of an instance.
a class member of TaskInput, ie, TaskInput.cfg = my_config, something
that's initialized in init() of TaskInput.
This can work, but not the way you did it. in your code you never created a class attribute, anything created with self is an instance attribute belonging to a single specific task instance so that:
from copy import deepcopy
class TaskInput:
_cfg = None # prefix with '_' to indicate it should be considered private
def __init__(self, my_config=None):
_cfg = _cfg or my_config
#property
def cfg(self):
""" If you want to privatize it a bit more,
make yourself a getter that returns a deep copy."""
return deepcopy(cfg)
Now, there basically is no such thing as true privatization in python and you will never be able to entirely prevent manipulation. In the example above, any child has direct read-write access to _cfg, so it would fall on us not to use it directly and pass by its accessors (__init__() and cfg()).
There's always a way to make things more difficult, like the following, using modules.
Project
├─ __init__.py
├─ settings.py
├─ module1.py
└─ module2.py
settings.py
cfg = None
module1.py
from copy import deepcopy
import settings
class A:
def __init__(self, cfg_=None):
settings.cfg = settings.cfg or cfg_
#property
def cfg(self):
return deepcopy(settings.cfg)
module2.py
""" The following classes won't be able to
overwrite the config without importing
from settings.py.
"""
from module1 import A
class B(A):
pass
class C(A):
def __init__(self):
super().__init__("foobar")
Giving these results:
b0 = B()
b0.cfg
# > None
b1 = B({"foo1": "bar1"})
b1.cfg
# > {'foo1': 'bar1'}
b2 = B({"foo1": "bar2", "foo3": "bar3"})
b2.cfg
# > {'foo1': 'bar1'}
try:
b2.cfg = 1234
except Exception as e:
print(type(e), e)
# > <class 'AttributeError'> can't set attribute
b2.cfg
# > {'foo1': 'bar1'}
c = C("asdf")
c.cfg
# > {'foo1': 'bar1'}
Which can be overkill of course and removes the actual ownership of the configuration from the class
Related
I am building a tool that takes directories as inputs and performs actions where necessary. These actions vary depeding on certain variables so I created a few class objects which help me with my needs in an organised fashion.
However, I hit a wall figuring out how to best design the following scenario.
For the sake of simplicity, let's assume there are only directories (no files). Also, the below is a heavily simplified example.
I have the following parent class:
# directory.py
from pathlib import Path
class Directory:
def __init__(self, absolute_path):
self.path = Path(absolute_path)
def content(self):
return [Directory(c) for c in self.path.iterdir()]
So, I have a method in the parent class that returns Directory instances for each directory inside the initial directory in absolute_path
What the above does, is hold all methods that can be performed on all directories. Now, I have a separate class that inherits from the above and adds further methods.
# special_directory.py
from directory import Directory
class SpecialDirectory(Directory):
def __init__(self, absolute_path):
super().__init__(absolute_path)
# More methods
I am using an Object Factory like approach to build one or the other based on a condition like so:
# directory_factory.py
from directory import Directory
from special_directory import SpecialDirectory
def pick(path):
return SpecialDirectory(path) if 'foo' in path else Directory(path)
So, if 'foo' exists in the path, it should be a SpecialDirectory instead allowing it to do everything Directory does plus more.
The problem I'm facing is with the content() method. Both should be able to do that but I don't want it to be limited to making a list of Directory instances. If any of its content has "foo*", it should be a SpecialDirectory.
Directory doesn't (and shouldn't) know about SpecialDirectory, so I tried importing and using the factory but it complains about some circular import (which makes sense).
I am not particularly stuck as I have come up with a temp fix, but it isn't pretty. So I was hoping I could get some tips as to what would be an effective and clean solution for this specific situation.
What you need is sometimes called a "virtual constructor" which is a way to allow subclasses to determine what type of class instance is created when calling the base class constructor. There's no such thing in Python (or C++ for that matter), but you can simulate them. Below is an example of a way of doing this.
Note this code is very similar to what's in my answer to the question titled Improper use of __new__ to generate classes? (which has more information about the technique). Also see the one to What exactly is a Class Factory?
from pathlib import Path
class Directory:
subclasses = []
#classmethod
def __init_subclass__(cls, **kwargs):
super().__init_subclass__(**kwargs)
cls.subclasses.append(cls)
def __init__(self, absolute_path):
self.path = Path(absolute_path)
def __new__(cls, path):
""" Create instance of appropriate subclass. """
for subclass in cls.subclasses:
if subclass.pick(path):
return object.__new__(subclass)
else:
return object.__new__(cls) # Default is this base class.
def content(self):
return [Directory(c) for c in self.path.iterdir()]
def __repr__(self):
classname = type(self).__name__
return f'{classname}(path={self.path!r})'
# More methods
...
class SpecialDirectory(Directory):
def __init__(self, absolute_path):
super().__init__(absolute_path)
#classmethod
def pick(cls, path):
return 'foo' in str(path)
# More methods
...
if __name__ == '__main__':
root = './_obj_factory_test'
d = Directory(root)
print(d.content())
I have a class and I need to test it
class ToBeTested():
def my_method():
pass
I have designed some unittests in python.
import unittest
class MyFirstUnitTest(unittest.TestCase):
def setUp(self):
# prepare some data
# run a method
# collect results
def test_one_assumption_about_results(self):
#self.assertEqual(...)
def test_another_assumption_about_results(self):
#self.assertEqual(...)
if __name__ == '__main__':
unittest.main()
I have designed this code looking at the examples.
Now I do not understand how to interface it the tesing class MyFirstUnitTest with the classe to be tested ToBeTested?
By inheritance?
class MyFirstUnitTest(unittest.TestCase,ToBeTested):
######
By creating a object of class ToBeTested as static member of
MyFirstUnitTest class definition?
class MyFirstUnitTest(unittest.TestCase):
mytestobj=ToBeTested()
def setUp(self):
By creating a object of class ToBeTested within each of the test
cases of MyFirstUnitTest class definition?
class MyFirstUnitTest(unittest.TestCase):
def setUp(self):
###
def test_one():
mytestobj=ToBeTested()
def test_two():
mytestobj=ToBeTested()
please suggest alternatives and why you would prefer any of them
There is a drawback to using inheritance and static member instead of creating a new object for every test: If tests change the state of the object, then the tests influence each other with their side effects. This gets even worse if the order of the test method calls are not guaranteed and changes (say, you rename a test and suddenly a different test case fails - these things have happened before!)
Additionally, having your test case inherit from the SUT (subject under test) means you only get to test one constructor.
I don't see any advantage to having the testcase inherit from the SUT, especially in python where there are no private members. If for some reason inheritance is necessary for the test (which sometimes IS the case), then having a seperate TestAdapter class that inherits and is instantiated by the test is the best practice.
There are other questions with answers which go more in depth:
https://softwareengineering.stackexchange.com/questions/366425/making-a-test-fixture-inherit-from-the-sut-class
https://softwareengineering.stackexchange.com/questions/154144/how-to-define-implementation-details
You can create an instance of your class(es) in your setup if it will not have to be reinstantiated.
And if you have to create a new instance in every test then just create a new instance in every test.
I have the following code, which is a command line test
from cmd2 import Cmd
class App(Cmd, object):
def __init__(self, *args, **kwargs):
pass
def do_test(self, line):
'test'
print "parent test"
class App2():
def __init__(self, *args, **kwargs):
pass
def do_test2(self, line):
print "Test2"
app = App()
app.cmdloop()
Is there a possibility to extend the App class with extra functions?
I know there is the following solution
class App2(App):
....
app = App2()
app.cmdloop()
but in my case I would like to run only the App and extend it if it is possible.
In general this is not a good idea, because when people (including you in six months) read the code they will expect App to be the class they know about, and for an extended version of the class to have a different name. However, there's nothing preventing you from naming the subclass the same as the original:
class App(App):
# etc.
Python's smart enough to know that the App in parentheses means the one it already knows about, and then, since the new class has the same name, it replaces the original one. Don't worry, the new class contains a reference to the old one, so the old class doesn't go away entirely (if it did, nothing inherited by the subclass would work).
If the class came from some module that you've imported, you can even monkey-patch the replacement class back into the original module, so that all code that imports that module uses your replacement class. (Though I would recommend against it!)
import appmodule
class App(appmodule.App):
# etc.
appmodule.App = App
Of course, this gets tricky, because some modules may already have imported a reference to the original class, if you don't do this first thing in your script. And if other modules are also trying to patch the same class, all hell can break loose. Still, if you want to confuse yourself and those who will maintain your code, Python will let you do it!
It is worth noting that you can always augment the class dictionary, therefore extending it at runtime.
class App(...):
def __init__(self, a, b):
pass
def do_something(self, a):
pass
app_instance = App()
def do_something_else(self, b):
pass
App.do_something_else = do_something_else
app_instance.do_something_else('b')
You have to think how python does lookups at runtime. First looks at the instance of yourclass, then looks at the __mro__ (starting with type(yourclass)), and on up until it gets to object.
Since classes ARE objects, you can extend them by adding attributes, which will then be found during attribute lookups. Make sure you do this ONCE (eg, during an import of another file).
Here is a real example:
>>> class foo():
... pass
...
>>> x = foo()
>>>
>>> # Define a function and attach it
>>>
>>> def bar(self, a):
... print(a)
...
>>> foo.bar = bar
>>>
>>> x.bar('a')
a
this isnt an exact solution but it allows you to always access it by App name
rename App class to _App
then where you want to use it
from blah import _App as App
and when you extend it
from blah import App2 as App
I am having trouble with this setup mainly because I am not sure what I actually want in order to solve this problem.
This is the setup
- main.py
- lib
- __init__.py
- index.py
- test.py
__init__.py has this code
import os
for module in os.listdir(os.path.dirname(__file__)+"/."):
if module == '__init__.py' or module[-3:] != '.py':
continue
__import__(module[:-3], locals(), globals())
del module
main.py has this code as of now
from lib.index import *
print User.__dict__
index.py has this code
class User(object):
def test(self):
return "hi"
pass
test.py has this code
class User(object):
def tes2(self):
return "hello"
When I execute main.py it successfully prints the method test from index.py but what I am trying to do is figure out a way where I can just create a file in the lib folder where that while has only one function in the format
class User(object):
def newFunction(self):
return abc
and this function should automatically be available for me in main.py
I am sure that this is not a hard thing to do but I honestly don't know what I want (what to search for to solve this) which is preventing me from researching the solution.
You can use a metaclass to customize class creation and add functions defined elsewhere:
import types
import os
import os.path
import imp
class PluginMeta(type):
def __new__(cls, name, bases, dct):
modules = [imp.load_source(filename, os.path.join(dct['plugindir'], filename))
for filename in os.listdir(dct['plugindir']) if filename.endswith('.py')]
for module in modules:
for name in dir(module):
function = getattr(module, name)
if isinstance(function, types.FunctionType):
dct[function.__name__] = function
return type.__new__(cls, name, bases, dct)
class User(metaclass=PluginMeta):
plugindir = "path/to/the/plugindir"
def foo(self):
print "foo"
user = User()
print dir(user)
Then in the plugin files, just create functions not classes:
def newFunction(self, abc):
self.abc = abc
return self.abc
And the metaclass will find them, turn them into methods, and attach them to your class.
Classes are objects, and methods are nothing more than attributes on class-objects.
So if you want to add a method to an existing class, outside the original class block, all that is is the problem of adding an attribute to an object, which I would hope you know how to do:
class User(object):
pass
def newFunction(self):
return 'foo'
User.newFunction = newFunction
agf's metaclass answer is basically a nifty automatic way of doing this, although it works by adding extra definitions to the class block before the class is created, rather than adding extra attributes to the class object afterwards.
That should be basically all you need to develop a framework in which things defined in one module are automatically added to a class defined elsewhere. But you still need to make a number of design decisions, such as:
If your externally-defined functions need auxiliary definitions, how do you determine what's supposed to get added to the class and what was just a dependency?
If you have more than one class you're extending this way, how do you determine what goes in which class?
At what point(s) in your program does the auto-extension happen?
Do you want to say in your class "this class has extensions defined elsewhere", or say in your extensions "this is an extension to a class defined elsewhere", or neither and somewhere bind extensions to classes externally from both?
Do you need to be able to have multiple versions of the "same" class with different extensions active at the same time?
A metaclass such as proposed by agf can be a very good way of implementing this sort of framework, because it lets you put all the complex code in one place while still "tagging" every class that doesn't work the way classes normally work. It does fix the answers to some of the questions I posed above, though.
here a working code we used in a project, I'm not sure it's the best way but it worked and there is almost no additional code to add to other files
cpu.py:
from cpu_base import CPU, CPUBase
import cpu_common
import cpu_ext
cpu_base.py:
def getClass():
return __cpu__
def setClass(CPUClass):
global __cpu__
__cpu__ = CPUClass
__classes__.append(CPUClass)
def CPU(*kw):
return __cpu__(*kw)
class CPUBase:
def __init__(self):
your_init_Stuff
# optionally a method classname_constructor to mimic __init__ for each one
for c in __classes__:
constructor = getattr(c, c.__name__ + '_constructor', None)
if constructor is not None:
constructor(self)
setClass(CPUBase)
cpu_common.py:
from cpu_base import getClass, setClass
class CPUCommon(getClass()):
def CPUCommon_constructor(self):
pass
setClass(CPUCommon)
cpu_ext.py:
from cpu_base import getClass, setClass
class CPUExt(getClass()):
pass
setClass(CPUExt)
to use the class import CPU from cpu.py
I have a group of test cases that all should have exactly the same test done, along the lines of "Does method x return the name of an existing file?"
I thought that the best way to do it would be a base class deriving from TestCase that they all share, and simply add the test to that class. Unfortunately, the testing framework still tries to run the test for the base class, where it doesn't make sense.
class SharedTest(TestCase):
def x(self):
...do test...
class OneTestCase(SharedTest):
...my tests are performed, and 'SharedTest.x()'...
I tried to hack in a check to simply skip the test if it's called on an object of the base class rather than a derived class like this:
class SharedTest(TestCase):
def x(self):
if type(self) != type(SharedTest()):
...do test...
else:
pass
but got this error:
ValueError: no such test method in <class 'tests.SharedTest'>: runTest
First, I'd like any elegant suggestions for doing this. Second, though I don't really want to use the type() hack, I would like to understand why it's not working.
You could use a mixin by taking advantage that the test runner only runs tests inheriting from unittest.TestCase (which Django's TestCase inherits from.) For example:
class SharedTestMixin(object):
# This class will not be executed by the test runner (it inherits from object, not unittest.TestCase.
# If it did, assertEquals would fail , as it is not a method that exists in `object`
def test_common(self):
self.assertEquals(1, 1)
class TestOne(TestCase, SharedTestMixin):
def test_something(self):
pass
# test_common is also run
class TestTwo(TestCase, SharedTestMixin):
def test_another_thing(self):
pass
# test_common is also run
For more information on why this works do a search for python method resolution order and multiple inheritance.
I faced a similar problem. I couldn't prevent the test method in the base class being executed but I ensured that it did not exercise any actual code. I did this by checking for an attribute and returning immediately if it was set. This attribute was only set for the Base class and hence the tests ran everywhere else but the base class.
class SharedTest(TestCase):
def setUp(self):
self.do_not_run = True
def test_foo(self):
if getattr(self, 'do_not_run', False):
return
# Rest of the test body.
class OneTestCase(SharedTest):
def setUp(self):
super(OneTestCase, self).setUp()
self.do_not_run = False
This is a bit of a hack. There is probably a better way to do this but I am not sure how.
Update
As sdolan says a mixin is the right way. Why didn't I see that before?
Update 2
(After reading comments) It would be nice if (1) the superclass method could avoid the hackish if getattr(self, 'do_not_run', False): check; (2) if the number of tests were counted accurately.
There is a possible way to do this. Django picks up and executes all test classes in tests, be it tests.py or a package with that name. If the test superclass is declared outside the tests module then this won't happen. It can still be inherited by test classes. For instance SharedTest can be located in app.utils and then used by the test cases. This would be a cleaner version of the above solution.
# module app.utils.test
class SharedTest(TestCase):
def test_foo(self):
# Rest of the test body.
# module app.tests
from app.utils import test
class OneTestCase(test.SharedTest):
...