getattr(importlib.import_module(__name__), "Some Class")
This is my method.
But this is redundant.
Is there a simpler method?
I use django and I'm trying to pass a dict of model to other class whose name is same as model name.
Roughly, this is my code.
I want to do this without if statement.
instance_list = []
obj_list = [a,b,c,d]
for obj in obj_list:
dic = model_to_dict(obj)
if 'Soccer' == obj.__class__.__name__:
instance_list.append(Soccer(dic))
elif 'Tennis' == obj.__class__.__name__:
instance_list.append(Tennis(dic))
Another way to get something (not necessarily a class) from the current module by name:
this_module = sys.modules[__name__]
cls = getattr(this_module, "SomeClass")
If the point is to have a module function that returns a class from the same module by it's name, you could indeed rewrite it in a much less convoluted way:
# mymodule.py
class Foo(object):
pass
def get_class(name):
return globals()[name]
# main.py
import mymodule
print(mymodule.get_class("Foo"))
Note that depending on your use case it might be safer to use an explicit dict of classes that are allowed to be looked up by mymodule.get_class(). You may really want to consider this option if the class name comes from user inputs and you know exactly which classes should be allowed.
Also note that for Django models classes you could also use apps.get_model(app_name, model_name) which lets you get models from any app.
Related
I'm trying to store my_app.path.to.my_class in a database
I'm going use this to instantiate the appropriate class object using from django.utils.module_loading import import_string
I'm worried this scheme doesn't handle well the situation where I need to change the module name, or path. would there be a better scheme to handle this?
second, in order to store the class name for a class object, I think there's a way to get the fully qualified name for the class somehow.
i.e. Is there a way for me to generate the fully_qualified_class_name below?
class FooShipping(object):
fully_qualified_class_name = 'store.shipping.foo_shipping.FooShipping'
You can use the globals dictionary to do this:
MyClass = globals()[fully_qualified_class_name]
then initialize an object using that class:
my_object = MyClass()
Suppose I have a large number of classes defined by an import of a large library codebase, which I don't want to hack around with for reasons of maintainability. They all inherit from BaseClass, and BaseClass contains a method which I want to augment. I think the following is a workable solution
class MyMixin(object):
def method( self, args):
... # 1. a few lines of code copied from BaseClass's def of method
... # 2. some lines of my code that can't go before or after the copied code
... # 3. and the rest of the copied code
class MyAbcClass( MyMixin, AbcClass):
pass
# many similar lines
class MyZzzClass( MyMixin, ZzzClass):
pass
The question. Is there a way to take, say, a list of ("MyXxxClass", XxxClass) tuples, and write code that defines the MyXxxClasses? And is it sufficiently comprehensible that it beats the repetition in the above?
Use three-arg type to define the classes, then set them on the module's global dictionary:
todefine = [('MyAbcClass', AbcClass), ...]
for name, base in todefine:
globals()[name] = type(name, (MyMixin, base), {})
If the names to define follow the fixed pattern you gave (`"My" + base class name), you can repeat yourself even less by dynamically constructing the name to define:
todefine = [AbcClass, ...]
for base in todefine:
name = "My" + base.__name__
globals()[name] = type(name, (MyMixin, base), {})
And if you are trying to wrap all the classes from a given module, you can avoid even explicitly listing the classes by introspecting the module to generate todefine programmatically (if you know the module has or lacks __all__ you can just use the appropriate approach instead of trying one and defaulting to the other):
import inspect
try:
# For modules that define __all__, we want all exported classes
# even if they weren't originally defined in the module
todefine = filter(inspect.isclass, (getattr(somemodule, name) for name in somemodule.__all__))
except AttributeError:
# If __all__ not defined, heuristic approach; exclude private names
# defined with leading underscore, and objects that were imported from
# other modules (so if the module does from itertools import chain,
# we don't wrap chain)
todefine = (obj for name, obj in vars(somemodule).items() if not name.startswith('_') and inspect.isclass(obj) and inspect.getmodule(obj) is somemodule)
I'm not really sure how best to explain what I want, so I'll just show some code:
class Stuffclass():
def add(self, x, y):
return x + y
def subtract(self, x, y):
return x - y
# imagine that there are 20-30 other methods in here (lol)
class MyClass:
def __init__(self):
self.st = Stuffclass()
def doSomething(self):
return self.st.add(1, 2)
m = MyClass()
m.doSomething() # will print 3
# Now, what I want to be able to do is:
print m.add(2, 3) # directly access the "add" method of MyClass.st
print m.subtract(10, 5) # directly access the "subtract" method of MyClass.st
m.SomeMethod() # execute function MyClass.st.SomeMethod
I know I could do something like this:
class MyClass:
def __init__(self):
self.st = Stuffclass()
self.add = self.st.add
self.subtract = self.st.subtract
...but this requires manually assigning all possible attributes.
I'm writing all the classes so I can guarantee no name collisions.
Making MyClass a subclass of Stuffclass won't work, because I actually am using this in a plugin-based application, where MyClass loads other code dynamically using import. This means MyClass can't subclass from the plugin, because the plugin could be anything that follows my API.
Advice please?
I believe that writing a getattr function for your class will let you do what you want.
Called when an attribute lookup has not found the attribute in the usual places (i.e. it is not an instance attribute nor is it found in the class tree for self). name is the attribute name. This method should return the (computed) attribute value or raise an AttributeError exception
So something as simple as:
def __getattr__(self, name):
if hasattr(self.st, name):
return getattr(self.st, name)
else:
raise AttributeError
should do roughly what you're after.
But, having answered (I think) the question you asked, I'm going to move on to the question I think you should have asked.
I actually am using this in a plugin-based application, where MyClass loads other code dynamically using import. This means MyClass can't subclass from the plugin, because the plugin could be anything that follows my API
I can see why MyClass can't be a subclass of StuffClass; but couldn't StuffClass be a subclass of MyClass? If you defined the inheritance that way, you'd have a guarantee what StuffClass implements all the basic stuff in MyClass, and also that your instances of StuffClass have all the extra methods defined in StuffClass.
From your mention that the plugins need to "follows my API", I'm assuming that might be a case where you need to ensure that the plugins implement a set of methods in order to conform with the API; but since the implementation of the methods is going to depend on the specifics of the plugin, you can't provide those functions in MyClass. In that case, it sounds as though defining an Abstract Base Class that your plugins are required to inherit from might be useful for you.
Use __getattr__ to delegate the calls to Stuffclass's instance:
class MyClass:
def __init__(self):
self.st = Stuffclass()
def __getattr__(self,attr):
return getattr(self.st,attr)
Demo:
>>> from so import *
>>> m = MyClass()
>>> m.add(1,2)
3
>>> m.subtract(100,2)
98
What are the best practices for extending an existing Python module – in this case, I want to extend the python-twitter package by adding new methods to the base API class.
I've looked at tweepy, and I like that as well; I just find python-twitter easier to understand and extend with the functionality I want.
I have the methods written already – I'm trying to figure out the most Pythonic and least disruptive way to add them into the python-twitter package module, without changing this modules’ core.
A few ways.
The easy way:
Don't extend the module, extend the classes.
exttwitter.py
import twitter
class Api(twitter.Api):
pass
# override/add any functions here.
Downside : Every class in twitter must be in exttwitter.py, even if it's just a stub (as above)
A harder (possibly un-pythonic) way:
Import * from python-twitter into a module that you then extend.
For instance :
basemodule.py
class Ball():
def __init__(self,a):
self.a=a
def __repr__(self):
return "Ball(%s)" % self.a
def makeBall(a):
return Ball(a)
def override():
print "OVERRIDE ONE"
def dontoverride():
print "THIS WILL BE PRESERVED"
extmodule.py
from basemodule import *
import basemodule
def makeBalls(a,b):
foo = makeBall(a)
bar = makeBall(b)
print foo,bar
def override():
print "OVERRIDE TWO"
def dontoverride():
basemodule.dontoverride()
print "THIS WAS PRESERVED"
runscript.py
import extmodule
#code is in extended module
print extmodule.makeBalls(1,2)
#returns Ball(1) Ball(2)
#code is in base module
print extmodule.makeBall(1)
#returns Ball(1)
#function from extended module overwrites base module
extmodule.override()
#returns OVERRIDE TWO
#function from extended module calls base module first
extmodule.dontoverride()
#returns THIS WILL BE PRESERVED\nTHIS WAS PRESERVED
I'm not sure if the double import in extmodule.py is pythonic - you could remove it, but then you don't handle the usecase of wanting to extend a function that was in the namespace of basemodule.
As far as extended classes, just create a new API(basemodule.API) class to extend the Twitter API module.
Don't add them to the module. Subclass the classes you want to extend and use your subclasses in your own module, not changing the original stuff at all.
Here’s how you can directly manipulate the module list at runtime – spoiler alert: you get the module type from types module:
from __future__ import print_function
import sys
import types
import typing as tx
def modulize(namespace: tx.Dict[str, tx.Any],
modulename: str,
moduledocs: tx.Optional[str] = None) -> types.ModuleType:
""" Convert a dictionary mapping into a legit Python module """
# Create a new module with a trivially namespaced name:
namespacedname: str = f'__dynamic_modules__.{modulename}'
module = types.ModuleType(namespacedname, moduledocs)
module.__dict__.update(namespace)
# Inspect the new module:
name: str = module.__name__
doc: tx.Optional[str] = module.__doc__
contents: str = ", ".join(sorted(module.__dict__.keys()))
print(f"Module name: {name}")
print(f"Module contents: {contents}")
if doc:
print(f"Module docstring: {doc}")
# Add to sys.modules, as per import machinery:
sys.modules.update({ modulename : module })
# Return the new module instance:
return module
… you could then use such a function like so:
ns = {
'func' : lambda: print("Yo Dogg"), # these can also be normal non-lambda funcs
'otherfunc' : lambda string=None: print(string or 'no dogg.'),
'__all__' : ('func', 'otherfunc'),
'__dir__' : lambda: ['func', 'otherfunc'] # usually this’d reference __all__
}
modulize(ns, 'wat', "WHAT THE HELL PEOPLE")
import wat
# Call module functions:
wat.func()
wat.otherfunc("Oh, Dogg!")
# Inspect module:
contents = ", ".join(sorted(wat.__dict__.keys()))
print(f"Imported module name: {wat.__name__}")
print(f"Imported module contents: {contents}")
print(f"Imported module docstring: {wat.__doc__}")
… You could also create your own module subclass, by specifying types.ModuleType as the ancestor of your newly declared class, of course; I have never personally found this necessary to do.
(Also, you don’t have to get the module type from the types module – you can always just do something like ModuleType = type(os) after importing os – I specifically pointed out this one source of the type because it is non-obvious; unlike many of its other builtin types, Python doesn’t offer up access to the module type in the global namespace.)
The real action is in the sys.modules dict, where (if you are appropriately intrepid) you can replace existing modules as well as adding your new ones.
Say you have an older module called mod that you use like this:
import mod
obj = mod.Object()
obj.method()
mod.function()
# and so on...
And you want to extend it, without replacing it for your users. Easily done. You can give your new module a different name, newmod.py or place it by same name at a deeper path and keep the same name, e.g. /path/to/mod.py. Then your users can import it in either of these ways:
import newmod as mod # e.g. import unittest2 as unittest idiom from Python 2.6
or
from path.to import mod # useful in a large code-base
In your module, you'll want to make all the old names available:
from mod import *
or explicitly name every name you import:
from mod import Object, function, name2, name3, name4, name5, name6, name7, name8, name9, name10, name11, name12, name13, name14, name15, name16, name17, name18, name19, name20, name21, name22, name23, name24, name25, name26, name27, name28, name29, name30, name31, name32, name33, name34, name35, name36, name37, name38, name39
I think the import * will be more maintainable for this use-case - if the base module expands functionality, you'll seamlessly keep up (though you might shade new objects with the same name).
If the mod you are extending has a decent __all__, it will restrict the names imported.
You should also declare an __all__ and extend it with the extended module's __all__.
import mod
__all__ = ['NewObject', 'newfunction']
__all__ += mod.__all__
# if it doesn't have an __all__, maybe it's not good enough to extend
# but it could be relying on the convention of import * not importing
# names prefixed with underscores, (_like _this)
Then extend the objects and functionality as you normally would.
class NewObject(object):
def newmethod(self):
"""this method extends Object"""
def newfunction():
"""this function builds on mod's functionality"""
If the new objects provide functionality you intend to replace (or perhaps you are backporting the new functionality into an older code base) you can overwrite the names
May I suggest not to reinvent the Wheel here? I'm building a >6k line Twitter Client for 2 month now, at first I checked python-twitter too, but it's lagging a lot behind the recent API changes,, Development doesn't seem to be that active either, also there was(at least when I last checked) no support for OAuth/xAuth).
So after searching around a bit more I discovered tweepy:
http://github.com/joshthecoder/tweepy
Pros: Active development, OAauth/xAuth and up to date with the API.
Chances are high that what you need is already in there.
So I suggest going with that, it's working for me, the only thing I had to add was xAuth(that got merge back to tweepy :)
Oh an a shameless plug, if you need to parse Tweets and/or format them to HTML use my python version of the twitter-text-* libraries:
http://github.com/BonsaiDen/twitter-text-python
This thing is unittestetd an guaranteed to parse Tweets just like Twitter.com does it.
Define a new class, and instead of inherit it from the class you want to extend from the original module, add an instance of the original class as an attribute to your new class.
And here comes the trick: intercept all non-existing method calls on your new class and try to call it on the instance of the old class.
In your NewClass just define new or overridden methods as you like:
import originalmodule
class NewClass:
def __init__(self, *args, **kwargs):
self.old_class_instance = originalmodule.create_oldclass_instance(*args, **kwargs)
def __getattr__(self, methodname):
"""This is a wrapper for the original OldClass class.
If the called method is not part of this NewClass class,
the call will be intercepted and replaced by the method
in the original OldClass instance.
"""
def wrapper(*args, **kwargs):
return getattr(self.old_class_instance, methodname)(*args, **kwargs)
return wrapper
def new_method(self, arg1):
"""Does stuff with the OldClass instance"""
thing = self.old_class_instance.get_somelist(arg1)
# returns the first element only
return thing[0]
def overridden_method(self):
"""Overrides an existing method, if OldClass has a method with the same name"""
print("This message is coming from the NewClass and not from the OldClass")
In my case I used this solution when simple inheritance from the old class was not possible, because an instance had to be created not by its constructor, but with an init script from an other class/module. (It is the originalmodule.create_oldclass_instance in the example above.)
In Java you can define a new class inline using anonymous inner classes. This is useful when you need to rewrite only a single method of the class.
Suppose that you want create a subclass of OptionParser that overrides only a single method (for example exit()). In Java you can write something like this:
new OptionParser () {
public void exit() {
// body of the method
}
};
This piece of code creates a anonymous class that extends OptionParser and override only the exit() method.
There is a similar idiom in Python? Which idiom is used in these circumstances?
You can use the type(name, bases, dict) builtin function to create classes on the fly. For example:
op = type("MyOptionParser", (OptionParser,object), {"foo": lambda self: "foo" })
op().foo()
Since OptionParser isn't a new-style class, you have to explicitly include object in the list of base classes.
Java uses anonymous classes mostly to imitate closures or simply code blocks. Since in Python you can easily pass around methods there's no need for a construct as clunky as anonymous inner classes:
def printStuff():
print "hello"
def doit(what):
what()
doit(printStuff)
Edit: I'm aware that this is not what is needed in this special case. I just described the most common python solution to the problem most commonly by anonymous inner classes in Java.
You can accomplish this in three ways:
Proper subclass (of course)
a custom method that you invoke with the object as an argument
(what you probably want) -- adding a new method to an object (or replacing an existing one).
Example of option 3 (edited to remove use of "new" module -- It's deprecated, I did not know ):
import types
class someclass(object):
val = "Value"
def some_method(self):
print self.val
def some_method_upper(self):
print self.val.upper()
obj = someclass()
obj.some_method()
obj.some_method = types.MethodType(some_method_upper, obj)
obj.some_method()
Well, classes are first class objects, so you can create them in methods if you want. e.g.
from optparse import OptionParser
def make_custom_op(i):
class MyOP(OptionParser):
def exit(self):
print 'custom exit called', i
return MyOP
custom_op_class = make_custom_op(3)
custom_op = custom_op_class()
custom_op.exit() # prints 'custom exit called 3'
dir(custom_op) # shows all the regular attributes of an OptionParser
But, really, why not just define the class at the normal level? If you need to customise it, put the customisation in as arguments to __init__.
(edit: fixed typing errors in code)
Python doesn't support this directly (anonymous classes) but because of its terse syntax it isn't really necessary:
class MyOptionParser(OptionParser):
def exit(self, status=0, msg=None):
# body of method
p = MyOptionParser()
The only downside is you add MyOptionParser to your namespace, but as John Fouhy pointed out, you can hide that inside a function if you are going to do it multiple times.
Python probably has better ways to solve your problem. If you could provide more specific details of what you want to do it would help.
For example, if you need to change the method being called in a specific point in code, you can do this by passing the function as a parameter (functions are first class objects in python, you can pass them to functions, etc). You can also create anonymous lambda functions (but they're restricted to a single expression).
Also, since python is very dynamic, you can change methods of an object after it's been created object.method1 = alternative_impl1, although it's actually a bit more complicated, see gnud's answer
In python you have anonymous functions, declared using lambda statement. I do not like them very much - they are not so readable, and have limited functionality.
However, what you are talking about may be implemented in python with a completely different approach:
class a(object):
def meth_a(self):
print "a"
def meth_b(obj):
print "b"
b = a()
b.__class__.meth_a = meth_b
You can always hide class by variables:
class var(...):
pass
var = var()
instead of
var = new ...() {};
This is what you would do in Python 3.7
#!/usr/bin/env python3
class ExmapleClass:
def exit(self):
print('this should NOT print since we are going to override')
ExmapleClass= type('', (ExmapleClass,), {'exit': lambda self: print('you should see this printed only')})()
ExmapleClass.exit()
I do this in python3 usually with inner classes
class SomeSerializer():
class __Paginator(Paginator):
page_size = 10
# defining it for e.g. Rest:
pagination_class = __Paginator
# you could also be accessing it to e.g. create an instance via method:
def get_paginator(self):
return self.__Paginator()
as i used double underscore, this mixes the idea of "mangling" with inner classes, from outside you can still access the inner class with SomeSerializer._SomeSerializer__Paginator, and also subclasses, but SomeSerializer.__Paginator will not work, which might or might not be your whish if you want it a bit more "anonymous".
However I suggest to use "private" notation with a single underscore, if you do not need the mangling.
In my case, all I need is a fast subclass to set some class attributes, followed up by assigning it to the class attribute of my RestSerializer class, so the double underscore would denote to "not use it at all further" and might change to no underscores, if I start reusing it elsewhere.
Being perverse, you could use the throwaway name _ for the derived class name:
class _(OptionParser):
def exit(self):
pass # your override impl
Here is a more fancy way of doing Maciej's method.
I defined the following decorator:
def newinstance(*args, **kwargs):
def decorator(cls):
return cls(*args, **kwargs)
return decorator
The following codes are roughly equivalent (also works with args!)
// java
MyClass obj = new MyClass(arg) {
public void method() {
// body of the method
}
};
# python
#newinstance(arg)
class obj(MyClass):
def method(self):
pass # body of the method
You can use this code from within a class/method/function if you want to define an "inner" class instance.