Introduction
For Java, Dependency Injection works as pure OOP, i.e. you provide an interface to be implemented and in your framework code accept an instance of a class that implements the defined interface.
Now for Python, you are able to do the same way, but I think that method was too much overhead right in case of Python. So then how would you implement it in the Pythonic way?
Use Case
Say this is the framework code:
class FrameworkClass():
def __init__(self, ...):
...
def do_the_job(self, ...):
# some stuff
# depending on some external function
The Basic Approach
The most naive (and maybe the best?) way is to require the external function to be supplied into the FrameworkClass constructor, and then be invoked from the do_the_job method.
Framework Code:
class FrameworkClass():
def __init__(self, func):
self.func = func
def do_the_job(self, ...):
# some stuff
self.func(...)
Client Code:
def my_func():
# my implementation
framework_instance = FrameworkClass(my_func)
framework_instance.do_the_job(...)
Question
The question is short. Is there any better commonly used Pythonic way to do this? Or maybe any libraries supporting such functionality?
UPDATE: Concrete Situation
Imagine I develop a micro web framework, which handles authentication using tokens. This framework needs a function to supply some ID obtained from the token and get the user corresponding to that ID.
Obviously, the framework does not know anything about users or any other application specific logic, so the client code must inject the user getter functionality into the framework to make the authentication work.
See Raymond Hettinger - Super considered super! - PyCon 2015 for an argument about how to use super and multiple inheritance instead of DI. If you don't have time to watch the whole video, jump to minute 15 (but I'd recommend watching all of it).
Here is an example of how to apply what's described in this video to your example:
Framework Code:
class TokenInterface():
def getUserFromToken(self, token):
raise NotImplementedError
class FrameworkClass(TokenInterface):
def do_the_job(self, ...):
# some stuff
self.user = super().getUserFromToken(...)
Client Code:
class SQLUserFromToken(TokenInterface):
def getUserFromToken(self, token):
# load the user from the database
return user
class ClientFrameworkClass(FrameworkClass, SQLUserFromToken):
pass
framework_instance = ClientFrameworkClass()
framework_instance.do_the_job(...)
This will work because the Python MRO will guarantee that the getUserFromToken client method is called (if super() is used). The code will have to change if you're on Python 2.x.
One added benefit here is that this will raise an exception if the client does not provide a implementation.
Of course, this is not really dependency injection, it's multiple inheritance and mixins, but it is a Pythonic way to solve your problem.
The way we do dependency injection in our project is by using the inject lib. Check out the documentation. I highly recommend using it for DI. It kinda makes no sense with just one function but starts making lots of sense when you have to manage multiple data sources etc, etc.
Following your example it could be something similar to:
# framework.py
class FrameworkClass():
def __init__(self, func):
self.func = func
def do_the_job(self):
# some stuff
self.func()
Your custom function:
# my_stuff.py
def my_func():
print('aww yiss')
Somewhere in the application you want to create a bootstrap file that keeps track of all the defined dependencies:
# bootstrap.py
import inject
from .my_stuff import my_func
def configure_injection(binder):
binder.bind(FrameworkClass, FrameworkClass(my_func))
inject.configure(configure_injection)
And then you could consume the code this way:
# some_module.py (has to be loaded with bootstrap.py already loaded somewhere in your app)
import inject
from .framework import FrameworkClass
framework_instance = inject.instance(FrameworkClass)
framework_instance.do_the_job()
I'm afraid this is as pythonic as it can get (the module has some python sweetness like decorators to inject by parameter etc - check the docs), as python does not have fancy stuff like interfaces or type hinting.
So to answer your question directly would be very hard. I think the true question is: does python have some native support for DI? And the answer is, sadly: no.
Some time ago I wrote dependency injection microframework with a ambition to make it Pythonic - Dependency Injector. That's how your code can look like in case of its usage:
"""Example of dependency injection in Python."""
import logging
import sqlite3
import boto.s3.connection
import example.main
import example.services
import dependency_injector.containers as containers
import dependency_injector.providers as providers
class Platform(containers.DeclarativeContainer):
"""IoC container of platform service providers."""
logger = providers.Singleton(logging.Logger, name='example')
database = providers.Singleton(sqlite3.connect, ':memory:')
s3 = providers.Singleton(boto.s3.connection.S3Connection,
aws_access_key_id='KEY',
aws_secret_access_key='SECRET')
class Services(containers.DeclarativeContainer):
"""IoC container of business service providers."""
users = providers.Factory(example.services.UsersService,
logger=Platform.logger,
db=Platform.database)
auth = providers.Factory(example.services.AuthService,
logger=Platform.logger,
db=Platform.database,
token_ttl=3600)
photos = providers.Factory(example.services.PhotosService,
logger=Platform.logger,
db=Platform.database,
s3=Platform.s3)
class Application(containers.DeclarativeContainer):
"""IoC container of application component providers."""
main = providers.Callable(example.main.main,
users_service=Services.users,
auth_service=Services.auth,
photos_service=Services.photos)
Here is a link to more extensive description of this example - http://python-dependency-injector.ets-labs.org/examples/services_miniapp.html
Hope it can help a bit. For more information please visit:
GitHub https://github.com/ets-labs/python-dependency-injector
Docs http://python-dependency-injector.ets-labs.org/
Dependency injection is a simple technique that Python supports directly. No additional libraries are required. Using type hints can improve clarity and readability.
Framework Code:
class UserStore():
"""
The base class for accessing a user's information.
The client must extend this class and implement its methods.
"""
def get_name(self, token):
raise NotImplementedError
class WebFramework():
def __init__(self, user_store: UserStore):
self.user_store = user_store
def greet_user(self, token):
user_name = self.user_store.get_name(token)
print(f'Good day to you, {user_name}!')
Client Code:
class AlwaysMaryUser(UserStore):
def get_name(self, token):
return 'Mary'
class SQLUserStore(UserStore):
def __init__(self, db_params):
self.db_params = db_params
def get_name(self, token):
# TODO: Implement the database lookup
raise NotImplementedError
client = WebFramework(AlwaysMaryUser())
client.greet_user('user_token')
The UserStore class and type hinting are not required for implementing dependency injection. Their primary purpose is to provide guidance to the client developer. If you remove the UserStore class and all references to it, the code still works.
After playing around with some of the DI frameworks in python, I've found they have felt a bit clunky to use when comparing how simple it is in other realms such as with .NET Core. This is mostly due to the joining via things like decorators that clutter the code and make it hard to simply add it into or remove it from a project, or joining based on variable names.
I've recently been working on a dependency injection framework that instead uses typing annotations to do the injection called Simple-Injection. Below is a simple example
from simple_injection import ServiceCollection
class Dependency:
def hello(self):
print("Hello from Dependency!")
class Service:
def __init__(self, dependency: Dependency):
self._dependency = dependency
def hello(self):
self._dependency.hello()
collection = ServiceCollection()
collection.add_transient(Dependency)
collection.add_transient(Service)
collection.resolve(Service).hello()
# Outputs: Hello from Dependency!
This library supports service lifetimes and binding services to implementations.
One of the goals of this library is that it is also easy to add it to an existing application and see how you like it before committing to it as all it requires is your application to have appropriate typings, and then you build the dependency graph at the entry point and run it.
Hope this helps. For more information, please see
github: https://github.com/BradLewis/simple-injection
docs: https://simple-injection.readthedocs.io/en/latest/
pypi: https://pypi.org/project/simple-injection/
A very easy and Pythonic way to do dependency injection is importlib.
You could define a small utility function
def inject_method_from_module(modulename, methodname):
"""
injects dynamically a method in a module
"""
mod = importlib.import_module(modulename)
return getattr(mod, methodname, None)
And then you can use it:
myfunction = inject_method_from_module("mypackage.mymodule", "myfunction")
myfunction("a")
In mypackage/mymodule.py you define myfunction
def myfunction(s):
print("myfunction in mypackage.mymodule called with parameter:", s)
You could of course also use a class MyClass iso. the function myfunction. If you define the values of methodname in a settings.py file you can load different versions of the methodname depending on the value of the settings file. Django is using such a scheme to define its database connection.
I think that DI and possibly AOP are not generally considered Pythonic because of typical Python developers preferences, rather that language features.
As a matter of fact you can implement a basic DI framework in <100 lines, using metaclasses and class decorators.
For a less invasive solution, these constructs can be used to plug-in custom implementations into a generic framework.
There is also Pinject, an open source python dependency injector by Google.
Here is an example
>>> class OuterClass(object):
... def __init__(self, inner_class):
... self.inner_class = inner_class
...
>>> class InnerClass(object):
... def __init__(self):
... self.forty_two = 42
...
>>> obj_graph = pinject.new_object_graph()
>>> outer_class = obj_graph.provide(OuterClass)
>>> print outer_class.inner_class.forty_two
42
And here is the source code
Due to Python OOP implementation, IoC and dependency injection are not standard practices in the Python world. But the approach seems promising even for Python.
To use dependencies as arguments is a non-pythonic approach. Python is an OOP language with beautiful and elegant OOP model, that provides more straightforward ways to maintain dependencies.
To define classes full of abstract methods just to imitate interface type is weird too.
Huge wrapper-on-wrapper workarounds create code overhead.
I also don't like to use libraries when all I need is a small pattern.
So my solution is:
# Framework internal
def MetaIoC(name, bases, namespace):
cls = type("IoC{}".format(name), tuple(), namespace)
return type(name, bases + (cls,), {})
# Entities level
class Entity:
def _lower_level_meth(self):
raise NotImplementedError
#property
def entity_prop(self):
return super(Entity, self)._lower_level_meth()
# Adapters level
class ImplementedEntity(Entity, metaclass=MetaIoC):
__private = 'private attribute value'
def __init__(self, pub_attr):
self.pub_attr = pub_attr
def _lower_level_meth(self):
print('{}\n{}'.format(self.pub_attr, self.__private))
# Infrastructure level
if __name__ == '__main__':
ENTITY = ImplementedEntity('public attribute value')
ENTITY.entity_prop
EDIT:
Be careful with the pattern. I used it in a real project and it showed itself a not that good way. My post on Medium about my experience with the pattern.
Related
Python is a relatively new language for me. Unit Testing and Dependency Injection are something that I've been doing for a little while now, so I'm familiar with it from a C# perspective.
Recently, I wrote this piece of Python code:
import requests # my dependency: http://docs.python-requests.org/en/latest/
class someClass:
def __init__(self):
pass
def __do(self, url, datagram):
return requests.post(self, url, datagram)
And then I realized that I had just created a hard-coded dependency. Bleh.
I had considered changing my code to do "Constructor" Dependency Injection:
def __init__(self,requestLib=requests):
self.__request = requestLib
def __do(self, url, datagram):
return self.__request.post(self, url, datagram)
This now allows me to inject a fake/mock dependency for the sake of Unit Testing, but wasn't sure if this was considered Python-ic. So I'm appealing to the Python community for guidance.
What are some examples of Python-ic ways to do basic DI (mostly for the sake of writing Unit Tests that utilize Mocks/Fakes)?
ADDENDUM For anyone curious about the Mock answer, I decided to ask a separate question here: How does #mock.patch know which parameter to use for each mock object?
Don't do that. Just import requests as normal and use them as normal. Passing libraries as arguments to your constructors is a fun thing to do, but not very pythonic and unnecessary for your purposes. To mock things in unit tests, use mock library. In python 3 it is built into the standard library
https://docs.python.org/3.4/library/unittest.mock.html
And in python 2 you need to install it separately
https://pypi.python.org/pypi/mock
Your test code would look something like this (using python 3 version)
from unittest import TestCase
from unittest.mock import patch
class MyTest(TestCase):
#patch("mymodule.requests.post")
def test_my_code(self, mock_post):
# ... do my thing here...
While injecting the requests module can be a bit too much, it is a very good practice to have some dependencies as injectable.
After years using Python without any DI autowiring framework and Java with Spring I've come to realize that plain simple Python code often doesn't need a framework for dependency injection with autowiring (autowiring is what Guice and Spring both do in Java), i.e., just doing something like this may be enough:
def foo(dep = None): # great for unit testing!
...
This is pure dependency injection (quite simple) but without magical frameworks for automatically injecting them for you. The caller has to instantiate the dependency or you can do it like this:
def __init__(self, dep = None):
self.dep = dep or Dep()
As you go for bigger applications this approach won't cut it though. For that I've come up with injectable a micro-framework that wouldn't feel non-pythonic and yet would provide first class dependency injection autowiring.
Under the motto Dependency Injection for Humans™ this is what it looks like:
# some_service.py
class SomeService:
#autowired
def __init__(
self,
database: Autowired(Database),
message_brokers: Autowired(List[Broker]),
):
pending = database.retrieve_pending_messages()
for broker in message_brokers:
broker.send_pending(pending)
# database.py
#injectable
class Database:
...
# message_broker.py
class MessageBroker(ABC):
def send_pending(messages):
...
# kafka_producer.py
#injectable
class KafkaProducer(MessageBroker):
...
# sqs_producer.py
#injectable
class SQSProducer(MessageBroker):
...
I'm coming from the C# world, so my views may be a little skewed. I'm looking to do DI in Python, however I'm noticing a trend with libraries where they all appear to rely on a service locator. That is, you must tie your object creation to the framework, such as injectlib.build(MyClass) in order to get an instance of MyClass.
Here is an example of what I mean -
from injector import Injector, inject
class Inner(object):
def __init__(self):
self.foo = 'foo'
class Outer(object):
#inject(inner=Inner)
def __init__(self, inner=None):
if inner is None:
print('inner not provided')
self.inner = Inner()
else:
print('inner provided')
self.inner = inner
injector = Injector()
outer = Outer()
print(outer.inner.foo)
outer = injector.get(Outer)
print(outer.inner.foo)
Is there a way in Python to create a class while automatically inferring dependency types based on parameter names? So if I have a constructor parameter called my_class, then an instance of MyClass will be injected. Reason I ask is that I don't see how I could inject a dependency into a class that gets created automatically via a third party library.
To answer the question you explicitly asked: no, there's no built-in way in Python to automatically get a MyClass object from a parameter named my_class.
That said, neither "tying your object creation to the framework" nor the example code you gave seem terribly Pythonic, and this question in general is kind of confusing because DI in dynamic languages isn't really a big deal.
For general thoughts about DI in Python I'd say this presentation gives a pretty good overview of different approaches. For your specific question, I'll give two options based on what you might be trying to do.
If you're trying to add DI to your own classes, I would use paramaters with default values in the constructor, as that presentation shows. E.g:
import time
class Example(object):
def __init__(self, sleep_func=time.sleep):
self.sleep_func = sleep_func
def foo(self):
self.sleep_func(10)
print('Done!')
And then you could just pass in a dummy sleep function for testing or whatever.
If you're trying to manipulate a library's classes through DI, (not something I can really imagine a use case for, but seems like what you're asking) then I would probably just monkey patch those classes to change whatever needed changing. E.g:
import test_module
def dummy_sleep(*args, **kwargs):
pass
test_module.time.sleep = dummy_sleep
e = test_module.Example()
e.foo()
I have following class in Python:
class SDK(object):
URL = 'http://example.com'
def checkUrl(self, url):
#some code
class api:
def innerMethod(self, url):
data = self.checkUrl(url)
#rest of code
but when I try to access checkUrl from api, I get error. I try to call nested method by:
sdk = SDK()
sdk.api.innerMethod('http://stackoverflow.com')
Is there any simple way to call inner class methods, or (if not) structurize methods into inner objects? Any suggestion will be appreciated.
Edit:
class code:
class SDK(object):
def run(self, method, *param):
pass
class api:
def checkDomain(self, domain):
json = self.run('check', domain)
return json
run code:
sdk = SDK()
result = sdk.api().checkDomain('stackoverflow.com')
The SDK class is not a parent of the api class in your example, i.e. api does not inherit from SDK, they are merely nested.
Therefore the self object in your api.innerMethod method is only an instance of the api class and doesn't provide access to methods of the SDK class.
I strongly recommend getting more knowledgeable about object-oriented programming concepts and grasp what the issue is here. It will help you tremendously.
As for using modules to achieve something along these lines, you can, for example, pull everything from the SDK class to sdk.py file, which would be the sdk module.
sdk.py:
URL = 'http://example.com'
def checkUrl(url):
#some code
class api:
def innerMethod(self, url):
data = checkUrl(url)
#rest of code
main.py:
import sdk
api = sdk.api()
api.innerMethod('http://stackoverflow.com')
Or you may go even further and transform sdk to a package with api being a module inside it.
See https://docs.python.org/2/tutorial/modules.html for details on how to use modules and packages.
If you want a method to act as a classmethod, you have to tell python about it:
class SDK:
class api:
#classmethod
def foo(cls):
return 1
Then you have access like
SDK.api.foo()
Depending on what you're trying to do, this smells kind of un-pythonic. If it's just the namespace you care about, you'd typically use a module.
My Situation
I'm currently writing on a project in python which I want to use to learn a bit more about software architecture. I've read a few texts and watched a couple of talks about dependency injection and learned to love how clear constructor injection shows the dependencies of an object.
However, I'm kind of struggling how to get a dependency passed to an object. I decided NOT to use a DI framework since:
I don't have enough knowledge of DI to specify my requirements and thus cannot choose a framework.
I want to keep the code free of more "magical" stuff since I have the feeling that introducing a seldom used framework drastically decreases readability. (More code to read of which only a small part is used).
Thus, I'm using custom factory functions to create objects and explicitly pass their dependencies:
# Business and Data Objects
class Foo:
def __init__(self,bar):
self.bar = bar
def do_stuff(self):
print(self.bar)
class Bar:
def __init__(self,prefix):
self.prefix = prefix
def __str__(self):
return str(self.prefix)+"Hello"
# Wiring up dependencies
def create_bar():
return Bar("Bar says: ")
def create_foo():
return Foo(create_bar())
# Starting the application
f = create_foo()
f.do_stuff()
Alternatively, if Foo has to create a number of Bars itself, it gets the creator function passed through its constructor:
# Business and Data Objects
class Foo:
def __init__(self,create_bar):
self.create_bar = create_bar
def do_stuff(self,times):
for _ in range(times):
bar = self.create_bar()
print(bar)
class Bar:
def __init__(self,greeting):
self.greeting = greeting
def __str__(self):
return self.greeting
# Wiring up dependencies
def create_bar():
return Bar("Hello World")
def create_foo():
return Foo(create_bar)
# Starting the application
f = create_foo()
f.do_stuff(3)
While I'd love to hear improvement suggestions on the code, this is not really the point of this post. However, I feel that this introduction is required to understand
My Question
While the above looks rather clear, readable and understandable to me, I run into a problem when the prefix dependency of Bar is required to be identical in the context of each Foo object and thus is coupled to the Foo object lifetime. As an example consider a prefix which implements a counter (See code examples below for implementation details).
I have two Ideas how to realize this, however, none of them seems perfect to me:
1) Pass Prefix through Foo
The first idea is to add a constructor parameter to Foo and make it store the prefix in each Foo instance.
The obvious drawback is, that it mixes up the responsibilities of Foo. It controls the business logic AND provides one of the dependencies to Bar. Once Bar does not require the dependency any more, Foo has to be modified. Seems like a no-go for me. Since I don't really think this should be a solution, I did not post the code here, but provided it on pastebin for the very interested reader ;)
2) Use Functions with State
Instead of placing the Prefix object inside Foo this approach is trying to encapsulate it inside the create_foo function. By creating one Prefix for each Foo object and referencing it in a nameless function using lambda, I keep the details (a.k.a there-is-a-prefix-object) away from Foo and inside my wiring-logic. Of course a named function would work, too (but lambda is shorter).
# Business and Data Objects
class Foo:
def __init__(self,create_bar):
self.create_bar = create_bar
def do_stuff(self,times):
for _ in range(times):
bar = self.create_bar()
print(bar)
class Bar:
def __init__(self,prefix):
self.prefix = prefix
def __str__(self):
return str(self.prefix)+"Hello"
class Prefix:
def __init__(self,name):
self.name = name
self.count = 0
def __str__(self):
self.count +=1
return self.name+" "+str(self.count)+": "
# Wiring up dependencies
def create_bar(prefix):
return Bar(prefix)
def create_prefix(name):
return Prefix(name)
def create_foo(name):
prefix = create_prefix(name)
return Foo(lambda : create_bar(prefix))
# Starting the application
f1 = create_foo("foo1")
f2 = create_foo("foo2")
f1.do_stuff(3)
f2.do_stuff(2)
f1.do_stuff(2)
This approach seems much more useful to me. However, I'm not sure about common practices and thus fear that having state inside functions is not really recommended. Coming from a java/C++ background, I'd expect a function to be dependent on its parameters, its class members (if it's a method) or some global state. Thus, a parameterless function that does not use global state would have to return exactly the same value every time it is called. This is not the case here. Once the returned object is modified (which means that counter in prefix has been increased), the function returns an object which has a different state than it had when beeing returned the first time.
Is this assumption just caused by my restricted experience in python and do I have to change my mindset, i.e. don't think of functions but of something callable? Or is supplying functions with state an unintended misuse of lambda?
3) Using a Callable Class
To overcome my doubts on stateful functions I could use callable classes where the create_foo function of approach 2 would be replaced by this:
class BarCreator:
def __init__(self, prefix):
self.prefix = prefix
def __call__(self):
return create_bar(self.prefix)
def create_foo(name):
return Foo(BarCreator(create_prefix(name)))
While this seems a usable solution for me, it is sooo much more verbose.
Summary
I'm not absolutely sure how to handle the situation. Although I prefer number 2 I still have my doubts. Furthermore, I'm still hope that anyone comes up with a more elegant way.
Please comment, if there is anything you think is too vague or can be possibly misunderstood. I will improve the question as far as my abilities allow me to do :)
All examples should run under python2.7 and python3 - if you experience any problems, please report them in the comments and I'll try to fix my code.
If you want to inject a callable object but don't want it to have a complex setup -- if, as in your example, it's really just binding to a single input value -- you could try using functools.partial to provide a function <> value pair:
def factory_function(arg):
#processing here
return configurted_object_base_on_arg
class Consumer(object):
def __init__(self, injection):
self._injected = injection
def use_injected_value():
print self._injected()
injectable = functools.partial(factory_function, 'this is the configuration argument')
example = Consumer(injectable)
example.use_injected_value() # should return the result of your factory function and argument
As an aside, if you're creating a dependency injection setup like your option 3, you probably want to put the knwledge about how to do the configuration into a factory class rather than doing it inline as you're doing here. That way you can swap out factories if you want to choose between strategies. It's not functionally very different (unless the creation is more complex than this example and involves persistent state) but it's more flexible down the road if the code looks like
factory = FooBarFactory()
bar1 = factory.create_bar()
alt_factory = FooBlahFactory(extra_info)
bar2 = alt_factory.create_bar()
I'm writing a Python application which stores some data. For storing the data i've wrote a Connection class with abstract methods (using Python's abc module). This class is the super class all storage back-ends derive from. Each storage back-end has only one purpose, e.g. storing the data in plain text files or in a XML file.
All storage backends (inclusive the module where the super class is located) are in one package called 'data_handler'. And each back-end is in one module.
My application should be able the store data in multiple back-ends simultaneously and determinate at runtime which storage back-ends are available. To do this i had the idea to write a singleton class where each back-end have to register at their import. But this seems to be not so good in a dynamic language (please correct me if I misinterpret this). Another way could be the import of the package with import data_handler and then get the __file__ attribute of the package and search all Python files in the dir for subclasses of the super Connection class.
What method should I use, or are there other (maybe better) methods to do this.
Stefan
Is discovering the back-ends at runtime a strict requirement or would static
enumeration of them in the code do?
This feature will be nice to note have to edit the code when I add a new back-end
But should your application always write to all backends?
I will have a class where I can register available handler. And the data shall be written to each registered handler. But not all available handlers have to be registered.
Do not walk the filesystem (!) and scan the Python source code of the backends! That's an ugly hack at the best of times, and even worse here because you don't need anything like it at all! Registering all the classes on import is perfectly OK.
Store the backends in a class attribute instead of an instance attribute; that way, all Storage instances will look at the same set of backends:
>>> class Storage(object):
... backends = set()
...
... def register(self, backend):
... self.backends.add(backend)
...
Every backend can register itself by instantiating its own Storage, which has access to the class-level backends attribute:
>>> foo = Storage()
>>> foo.register("text")
>>> bar = Storage()
>>> bar.register("xml")
You can read this attribute by instantiating another Storage, which will read the same variable:
>>> baz = Storage()
>>> baz.backends
{'xml', 'text'}
You could even store the backend instances in a class attribute of Connection, and register each backend upon instantiation:
>>> class Connection(object,metaclass=abc.ABCMeta):
... #abc.abstractmethod
... def register(self, backend):
... pass
...
... backends = set()
...
>>> class TextBackend(Connection):
... def register(self):
... super().backends.add(self)
...
... def __init__(self):
... self.register()
...
>>> class XMLBackend(Connection):
... def register(self):
... super().backends.add(self)
...
... def __init__(self):
... self.register()
...
>>> foo = TextBackend()
>>> bar = XMLBackend()
>>> Connection.backends
{<__main__.XMLBackend object at 0x027ADAB0>, \
<__main__.TextBackend object at 0x027ADA50>}
If these backends are going to be distributed in various Python distributions, you might want to look at setuptools/distribute entry points. Here's an article on how you might use these for dynamic plugin finding services:
http://aroberge.blogspot.com/2008/12/plugins-part-6-setuptools-based.html
But should your application always write to all backends? If not, you could use (as usual) another layer of indirection, e.g.
storage = Storage()
storage.use(TextBackend, XMLBackend, YamlBackend)
storage.write(data)
Or something like this, with Storage being a dispatcher, which would just loop over the backends and call the appropriate serializer.
This is of course, very coarse.
you could use a function like this one:
def loadClass(fullclassname):
sepindex=fullclassname.rindex('.')
classname=fullclassname[sepindex+1:]
modname=fullclassname[:sepindex]
#dynmically import the class in the module
imod=__import__(modname,None,None,classname)
classtype=getattr(imod,classname)
return classtype
where fullclassname is the fully dotted qualifiant for the class you want load.
example (pseudo code,but the idea is there):
for package availability scanning, only perform some globbing , then for finding final class name, you may declare a Plugin class in each of your modules that has a getStorage()
#scan for modules , getPluginPackagesUnder (to be defined) returns the dotted name for all packages under a root path (using some globbing, listdir or whatever method)
pluginpackages=getPluginPackagesUnder("x/y/z")
storagelist=[]
for pgpck in plunginpackages:
pluginclass=loadClass("%s.Plugin"%pgpck)
storageinstance=Plugin().getStorage()
storagelist.append(storageinstance)
so, you can dynamically scan for your existing storage plugins