I have a class which would be a container for a number of variables of different types. The collection is finite and not very large so I didn't use a dictionary. Is there a way to automate, or shorten the creation of variables based on whether or not they are requested (specified as True/False) in the constructor?
Here is what I have for example:
class test:
def __init__(self,a=False,b=False,c=False):
if a: self.a = {}
if b: self.b = 34
if c: self.c = "generic string"
For any of a,b,c that are true in the constructor they will be created in the object.
I have a collection of standard variables (a,b,c,d..) that some objects will have and some objects won't. The number of combinations is too large to create separate classes, but the number of variables isn't enough to have a dictionary for them in each class.
Is there any way in python to do something like this:
class test:
def __init__(self,*args):
default_values = {a:{},b:34,c:"generic string"}
for item in args:
if item: self.arg = default_values[arg]
Maybe there is a whole other way to do this?
EDIT:
To clarify this a class which represents different type of bounding boxes on a 2D surface. Depending on the function of the box it can have any of frame coordinates, internal cross coordinates, id, population statistics (attached to that box), and some other cached values for easy calculation.
I don't want to have each object as a dictionary because there are methods attached to it which allow it to export and modify its internal data and interact with other objects of the same type (similar to how strings interact with + - .join, etc.). I also don't want to have a dictionary inside each object because the call to that variable is inelegant:
print foo.info["a"]
versus
print foo.a
Thanks to ballsdotball I've come up with a solution:
class test:
def __init__(self, a=False, b=False,c =False):
default_values = {"a":{},"b":34,"c":"generic string"}
for k, v in default_values.iteritems():
if eval(k): setattr(self,k,v)
Maybe something like:
def __init__(self,*args,**kwargs):
default_values = {a:{},b:34,c:"generic string"}
for k,v in kwargs.iteritems():
try:
if not v is False:
setattr(self,k,default_values[k])
except Exception, e:
print "Argument has no default value.",e
But to be honest I would just put the default values in with the init arguments instead of having to test for them like that.
*Edited a couple times for syntax.
You can subclass dict (if you aren't using positional arguments):
class Test(dict):
def your_method(self):
return self['foo'] * 4
You can also override __getattr__ and __setattr__ if the self['foo'] syntax bothers you:
class Test(dict):
def __getattr__(self, key):
return dict.__getattr__(self, key)
def __setattr__(self, key, value):
return dict.__getattr__(self, key, value)
def your_method(self):
return self.foo * 4
Related
I have a problem with my base class. I started writing it after finding an answer on this site about more informative __repr__() methods. I added to it after finding a different answer on this site about using pprint() with my own classes. I tinkered with it a little more after finding a third answer on this site about making my classes unpackable with a ** operator.
I modified it again after seeing in yet another answer on this site that there was a distinction between merely giving it __getitem__(), __iter__(), and __len__() methods on the one hand, and actually making it a fully-qualified mapping by subclassing collections.abc.Mapping on the other. Further, I saw that doing so would remove the need for writing my own keys() method, as the Mapping would take care of that.
So I got rid of keys(), and a class method broke.
The problem
I have a method that iterates through my class' keys and values to produce one big string formatted as I'd like it. That class looks like this.
class MyObj():
def __init__(self, foo, bar):
self.foo = foo
self.bar = bar
def the_problem_method(self):
"""Method I'm getting divergent output for."""
longest = len(max((key for key in self.keys()), key=len))
key_width = longest + TAB_WIDTH - longest % TAB_WIDTH
return '\n'.join((f'{key:<{key_width}}{value}' for key, value in self))
Yes, that doesn't have the base class in it, but the MWE later on will account for that. The nut of it is that (key for key in self.keys()) part. When I have a keys() method written, I get the output I want.
def keys(self):
"""Get object attribute names."""
return self.__dict__.keys()
When I remove that to go with the keys() method supplied by collections.abc.Mapping, I get no space between key and value
The question
I can get the output I want by restoring the keys() method (and maybe adding values() and items() while I'm at it), but is that the best approach? Would it be better to go with the Mapping one and modify my class method to suit it? If so, how? Should I leave Mapping well enough alone until I know I need it?
This is my base class to be copied all aver creation and subclassed out the wazoo. I want to Get. It. Right.
There are already several considerations I can think of and many more of which I am wholly ignorant.
I use Python 3.9 and greater. I'll abandon 3.9 when conda does.
I want to keep my more-informative __repr__() methods.
I want pprint() to work, via the _dispatch table method with _format_dict_items().
I want to allow for duck typing my classes reliably.
I have not yet used type hinting, but I want to allow for using best practices there if I start.
Everything else I know nothing about.
The MWE
This has my problem class at the top and output stuff at the bottom. There are two series of classes building upon the previous ones.
The first are ever-more-inclusive base classes, and it is here that the difference between the instance with the keys() method and that without is shown. the class, BaseMap, subclasses the Mapping and has the __getitem__(), __iter__(), and __len__() methods. The next class up the chain, BaseMapKeys, subclasses that and adds the keys() method.
The second group, MapObj and MapKeysObj, are subclasses of the problem class that also subclass those different base classes respectively.
OK, maybe the WE isn't so M, but lots of things got me to this point and I don't want to neglect any.
import collections.abc
from pprint import pprint, PrettyPrinter
TAB_WIDTH = 3
class MyObj():
def __init__(self, foo, bar):
self.foo = foo
self.bar = bar
def the_problem_method(self):
"""Method I'm getting divergent output for."""
longest = len(max((key for key in self.keys()), key=len))
key_width = longest + TAB_WIDTH - longest % TAB_WIDTH
return '\n'.join((f'{key:<{key_width}}{value}' for key, value in self))
class Base(object):
"""Base class with more informative __repr__."""
def __repr__(self):
"""Object representation."""
params = (f'{key}={repr(value)}'
for key, value in self.__dict__.items())
return f'{repr(self.__class__)}({", ".join(params)})'
class BaseMap(Base, collections.abc.Mapping):
"""Enable class to be pprint-able, unpacked with **."""
def __getitem__(self, attr):
"""Get object attribute values."""
return getattr(self.__dict__, attr)
def __iter__(self):
"""Make object iterable."""
for attr in self.__dict__.keys():
yield attr, getattr(self, attr)
def __len__(self):
"""Get length of object."""
return len(self.__dict__)
class BaseMapKeys(BaseMap):
"""Overwrite KeysView output with what I thought it would be."""
def keys(self):
"""Get object attribute names."""
return self.__dict__.keys()
class MapObj(BaseMap, MyObj):
"""Problem class with collections.abc.Mapping."""
def __init__(self, foo, bar):
super().__init__(foo, bar)
class MapKeysObj(BaseMapKeys, MyObj):
"""Problem class with collections.abc.Mapping and keys method."""
def __init__(self, foo, bar):
super().__init__(foo, bar)
if isinstance(getattr(PrettyPrinter, '_dispatch'), dict):
# assume the dispatch table method still works
def pprint_basemap(printer, object, stream, indent, allowance, context,
level):
"""Implement pprint for subclasses of BaseMap class."""
write = stream.write
write(f'{object.__class__}(\n {indent * " "}')
printer._format_dict_items(object, stream, indent, allowance + 1,
context, level)
write(f'\n{indent * " "})')
map_classes = [MapObj, MapKeysObj]
for map_class in map_classes:
PrettyPrinter._dispatch[map_class.__repr__] = pprint_basemap
def print_stuff(map_obj):
print('pprint object:')
pprint(map_obj)
print()
print('print keys():')
print(map_obj.keys())
print()
print('print list(keys()):')
print(list(map_obj.keys()))
print()
print('print the problem method:')
print(map_obj.the_problem_method())
print('\n\n')
params = ['This is a really long line to force new line in pprint output', 2]
baz = MapObj(*params)
print_stuff(baz)
scoggs = MapKeysObj(*params)
print_stuff(scoggs)
In part of a program I'm developing I want to perform a linear regression with terms that are some function of a data set X. The exact model used is configurable by the user, particularly which terms (or sets of terms) to use. This involves generating the matrix X' where every row of X' is a function of the corresponding row of X. The columns of X' will be the predictors for my regression.
For example, say my data set is two-dimensional (X has 2 columns). If we denote x and x' as corresponding rows of X and X', then assuming x is two-dimensional x' might be something like
[ 1, x[0], x[1], x[0] * x[1], sqrt(x[0]), sqrt(x[1]), x[0]**2, x[1]**2 ]
You can see these terms come in groups. First is just a 1 (constant), then the untransformed data (linear), then the product of the two data elements (would be all pairwise products if x had more than two dimensions), then square roots and squares of the individual terms.
I need to define all these sets of terms somehow in python, such that each has a user-readable name, function to generate the terms, function to get the number of terms from the dimensions of the input, function to generate labels for the terms based on column labels for the data, etc. Conceptually these all feel like they should be instances of a TermSet class or something similar, but this doesn't quite work because their methods would need to differ. My first thought was to go with something like this:
termsets = {} # Keep track of sets
class SqrtTerms:
display = 'Square Roots' # user-readable name
#staticmethod
def size(d):
"""Number of terms based on input columns"""
return d
#staticmethod
def make(X):
"""Make the terms from the input data"""
return numpy.sqrt(X)
#staticmethod
def labels(columns):
"""List of term labels based off of data column labels"""
return ['sqrt(%s)' % c for c in columns]
termsets['sqrt'] = SqrtTerms # register class in dict
class PairwiseProductTerms:
display = 'Pairwise Products'
#staticmethod
def size(d):
return (d * (d-1)) / 2
#staticmethod
def make(X):
# Some more complicated code that spans multiple lines
...
#staticmethod
def labels(columns):
# Technically a one-liner but also more complicated
return ['(%s) * (%s)' % (columns[c1], columns[c2])
for c1 in range(len(columns)) for c2 in range(len(columns))
if c2 > c1]
termsets['pairprod'] = PairwiseProductTerms
This works: I can retrieve the classes from the dictionary, put the ones I want to use in a list, and call the appropriate methods on each. Still, creating classes with only static attributes and methods seems ugly and unpythonic. Another idea I came up with would be to create a class decorator that could be used like:
# Convert bound methods to static ones, assign "display" static
# attribute and add to dict with key "name"
#regression_terms(name='sqrt', display='Square Roots')
class SqrtTerms:
def size(d):
return d
def make(X):
return numpy.sqrt(X)
def labels(columns):
return ['sqrt(%s)' % c for c in columns]
This gives the same result but is cleaner and much nicer (for myself) to read and write (especially if I need a lot of these). However, the way things actually work under the hood is obscured and anyone else reading this might have a hard idea figuring out what is going on at first. I also thought of creating a metaclass for these but that sounds like overkill. Is there a better pattern I should use here?
Some people will always say that this is an abuse of the language. I say Python was designed to be abusable, and the ability to create DSLs that don't require parsers yet that don't look like lisp is one of its core strengths.
If you really have a lot of these, go with the metaclass. If you do that, in addition to having a term dictionary, you can have attributes that reference the terms, as well. It's really nice, because you can have code like this:
print Terms.termsets
print Terms.sqrt
print Terms.pairprod
print Terms.pairprod.size(5)
return results like this:
{'pairprod': <class '__main__.PairwiseProductTerms'>,
'sqrt': <class '__main__.SqrtTerms'>}
<class '__main__.SqrtTerms'>
<class '__main__.PairwiseProductTerms'>
10
The full code that can do that is here:
from types import FunctionType
class MetaTerms(type):
"""
This metaclass will let us create a Terms class.
Every subclass of the terms class will have its
methods auto-wrapped as static methods, and
will be added to the terms directory.
"""
def __new__(cls, name, bases, attr):
# Auto-wrap all methods as static methods
for key, value in attr.items():
if isinstance(value, FunctionType):
attr[key] = staticmethod(value)
# call types.__new__ to finish the job
return super(MetaTerms, cls).__new__(cls, name, bases, attr)
def __init__(cls, name, bases, attr):
# At __init__ time, the class has already been
# built, so any changes to the bases or attr
# will not be reflected in the cls.
# Call types.__init__ to finish the job
super(MetaTerms, cls).__init__(name, bases, attr)
# Add the class into the termsets.
if name != 'Terms':
cls.termsets[cls.shortname] = cls
def __getattr__(cls, name):
return cls.termsets[name]
class Terms(object):
__metaclass__ = MetaTerms
termsets = {} # Keep track of sets
class SqrtTerms(Terms):
display = 'Square Roots' # user-readable name
shortname = 'sqrt' # Used to find in Terms.termsets
def size(d):
"""Number of terms based on input columns"""
return d
def make(X):
"""Make the terms from the input data"""
return numpy.sqrt(X)
def labels(columns):
"""List of term labels based off of data column labels"""
return ['sqrt(%s)' % c for c in columns]
class PairwiseProductTerms(Terms):
display = 'Pairwise Products'
shortname = 'pairprod'
def size(d):
return (d * (d-1)) / 2
def make(X):
pass
def labels(columns):
# Technically a one-liner but also more complicated
return ['(%s) * (%s)' % (columns[c1], columns[c2])
for c1 in range(len(columns)) for c2 in range(len(columns))
if c2 > c1]
print Terms.termsets
print Terms.sqrt
print Terms.pairprod
print Terms.pairprod.size(5)
If you hide away the metaclass and the base Terms class in a separate module, then nobody has to look at it -- just from baseterm import Terms. You could also do some cool auto-discovery / auto-import where dumping modules in the right directory automatically adds them to your DSL.
With the metaclass, the feature set can easily grow organically as you find other things you would like your mini-language to do.
I'm working on a class describing an object that can be expressed in several "units", I'll say, to keep things simple. Let's say we're talking about length. (It's actually something more complicated.) What I would like is for the user to be able to input 1 and "inch", for example, and automatically get member variables in feet, meters, furlongs, what have you as well. I want the user to be able to input any of the units I am dealing in, and get member variables in all the other units. My thought was to do something like this:
class length:
#classmethod
def inch_to_foot(cls,inch):
# etc.
#classmethod
def inch_to_meter(cls,inch):
# etc.
I guess you get the idea. Then I would define a dictionary in the class:
from_to={'inch':{'foot':inch_to_foot,'meter':inch_to_meter, ...},
'furlong':{'foot':furlong_to_foot, ...},
#etc
}
So then I think I can write an __init__ method
def __init__(self,num,unit):
cls = self.__class__
setattr(self,unit,num)
for k in cls.from_to[unit].keys:
setattr(self,k,cls.from_to[unit][k](num)
But no go. I get the error "class method not callable". Any ideas how I can make this work? Any ideas for scrapping the whole thing and trying a different approach? Thanks.
If you move the from_to variable into __init__ and modify it to something like:
cls.from_to={'inch':{'foot':cls.inch_to_foot,'meter':cls.inch_to_meter, }}
then I think it works as you expect.
Unfortunately I can't answer why because i haven't used classmethods much myself, but I think it is something to do with bound vs unbound methods. Anyway, if you print the functions stored in to_from in your code vs the ones with my modification you'll see they are different (mine are bound, yours are classmethod objects)
Hope that helps somewhat!
EDIT: I've thought about it a bit more, I think the problem is because you are storing a reference to the functions before they have been bound to the class (not surprising that the binding happens once the rest of the class has been parsed). My advice would be to forget about storing a dictionary of function references, but to store (in some representation of your choice) strings that indicate the units you can change between. For instance you might choose a similar format, such as:
from_to = {'inch':['foot','meter']}
and then look up the functions during __init__ using getattr
E.G.:
class length:
from_to = {'inch':['foot','meter']}
def __init__(self,num,unit):
if unit not in self.from_to:
raise RuntimeError('unit %s not supported'%unit)
cls = self.__class__
setattr(self,unit,num)
for k in cls.from_to[unit]:
f = getattr(cls,'%s_to_%s'%(unit,k))
setattr(self,k,f(num))
#classmethod
def inch_to_foot(cls,inch):
return inch/12.0
#classmethod
def inch_to_meter(cls,inch):
return inch*2.54/100
a = length(3,'inches')
print a.meter
print a.foot
print length.inch_to_foot(3)
I don't think doing with an __init__() method would be a good idea. I once saw an interesting way to do it in the Overriding the __new__ method section of in the classic document titled Unifying types and classes in Python 2.2 by Guido van Rossum.
Here's some examples:
class inch_to_foot(float):
"Convert from inch to feet"
def __new__(cls, arg=0.0):
return float.__new__(cls, float(arg)/12)
class inch_to_meter(float):
"Convert from inch to meter"
def __new__(cls, arg=0.0):
return float.__new__(cls, arg*0.0254)
print inch_to_meter(5) # 0.127
Here's a completely different answer that uses a metaclass and requires the conversion functions to bestaticmethodsrather thanclassmethods-- which it turns into properties based on the target unit's name. If searches for the names of any conversion functions itself, eliminating the need to manually definefrom_totype tables.
One thing about this approach is that the conversion functions aren't even called unless indirect references are made to the units associated with them. Another is that they're dynamic in the sense that the results returned will reflect the current value of the instance (unlike instances of three_pineapples'lengthclass, which stores the results of calling them on the numeric value of the instance when it's initially constructed).
You've never said what version of Python you're using, so the following code is for Python 2.2 - 2.x.
import re
class MetaUnit(type):
def __new__(metaclass, classname, bases, classdict):
cls = type.__new__(metaclass, classname, bases, classdict)
# add a constructor
setattr(cls, '__init__',
lambda self, value=0: setattr(self, '_value', value))
# add a property for getting and setting the underlying value
setattr(cls, 'value',
property(lambda self: self._value,
lambda self, value: setattr(self, '_value', value)))
# add an identity property the just returns the value unchanged
unitname = classname.lower() # lowercase classname becomes name of unit
setattr(cls, unitname, property(lambda self: self._value))
# find conversion methods and create properties that use them
matcher = re.compile(unitname + r'''_to_(?P<target_unitname>\w+)''')
for name in cls.__dict__.keys():
match = matcher.match(name)
if match:
target_unitname = match.group('target_unitname').lower()
fget = (lambda self, conversion_method=getattr(cls, name):
conversion_method(self._value))
setattr(cls, target_unitname, property(fget))
return cls
Sample usage:
scalar_conversion_staticmethod = (
lambda scale_factor: staticmethod(lambda value: value * scale_factor))
class Inch(object):
__metaclass__ = MetaUnit
inch_to_foot = scalar_conversion_staticmethod(1./12.)
inch_to_meter = scalar_conversion_staticmethod(0.0254)
a = Inch(3)
print a.inch # 3
print a.meter # 0.0762
print a.foot # 0.25
a.value = 6
print a.inch # 6
print a.meter # 0.1524
print a.foot # 0.5
I am looking for a way to create a basic python "object" which I can externally assign attributes to.
Currently I am doing it the following way:
I define an empty class with
class C(object):
pass
and then I instantiate an object and assign attributes like this:
c = C()
c.attr = 2
Coming to my question
Is there a way to instantiate an empty class object, which I can then assign attributes like shown above without defining a class C?
Is there maybe an other better way to accomplish what I am after?
It looks like you are looking for a flexible container that has no methods and can take attributes with arbitrary names. That's a dict.
d = dict()
d['myattr'] = 42
If you prefer the attribute syntax that you get with a class (c.myattr = 42), then use a class just as per the code in your question.
Is there a way to instantiate an empty class object, which I can then assign attributes like shown above without defining a class C?
Yes:
>>> C = type("C", (object,), {})
>>> c = C()
>>> c.attr = 2
But as you can see, it's not much of an improvement, and the end result is the same -- it's just another way of creating the same class C.
Addendum:
You can make it prettier by "hiding" it in a function:
def attr_holder(cls=type("C", (object,), {})):
return cls()
c = attr_holder()
c.attr = 2
Though this is just reinventing the wheel -- replace the two line function with
class attr_holder(object):
pass
and it'll work exactly the same, and we've come full circle. So: go with what David or Reorx suggests.
I had come to the same question long ago, and then create this class to use in many of my projects:
class DotDict(dict):
"""
retrieve value of dict in dot style
"""
def __getattr__(self, key):
try:
return self[key]
except KeyError:
raise AttributeError('has no attribute %s' % key)
def __setattr__(self, key, value):
self[key] = value
def __delattr__(self, key):
try:
del self[key]
except KeyError:
raise AttributeError(key)
def __str__(self):
return '<DotDict %s >' % self.__to_dict()
def __to_dict(self):
return dict(self)
When I want a object to store data or want to retrieve value easily from a dict, I always use this class.
Additionally, it can help me serialize the attributes that I set in the object, and reversely get the original dict when needed.
So I think this may be a good solution in many situations, though other tricks look simple,
they are not very helpful further.
In Python, how is it possible to reuse existing equal immutable objects (like is done for str)? Can this be done just by defining a __hash__ method, or does it require more complicated measures?
If you want to create via the class constructor and have it return a previously created object then you will need to provide a __new__ method (because by the time you get to __init__ the object has already been created).
Here is a simple example - if the value used to initialise has been seen before then a previously created object is returned rather than a new one created:
class Cached(object):
"""Simple example of immutable object reuse."""
def __init__(self, i):
self.i = i
def __new__(cls, i, _cache={}):
try:
return _cache[i]
except KeyError:
# you must call __new__ on the base class
x = super(Cached, cls).__new__(cls)
x.__init__(i)
_cache[i] = x
return x
Note that for this example you can use anything to initialise as long as it's hashable. And just to show that objects really are being reused:
>>> a = Cached(100)
>>> b = Cached(200)
>>> c = Cached(100)
>>> a is b
False
>>> a is c
True
There are two 'software engineering' solutions to this that don't require any low-level knowledge of Python. They apply in the following scenarios:
First Scenario: Objects of your class are 'equal' if they are constructed with the same constructor parameters, and equality won't change over time after construction. Solution: Use a factory that hashses the constructor parameters:
class MyClass:
def __init__(self, someint, someotherint):
self.a = someint
self.b = someotherint
cachedict = { }
def construct_myobject(someint, someotherint):
if (someint, someotherint) not in cachedict:
cachedict[(someint, someotherint)] = MyClass(someint, someotherint)
return cachedict[(someint, someotherint)]
This approach essentially limits the instances of your class to one unique object per distinct input pair. There are obvious drawbacks as well: not all types are easily hashable and so on.
Second Scenario: Objects of your class are mutable and their 'equality' may change over time. Solution: define a class-level registry of equal instances:
class MyClass:
registry = { }
def __init__(self, someint, someotherint, third):
MyClass.registry[id(self)] = (someint, someotherint)
self.someint = someint
self.someotherint = someotherint
self.third = third
def __eq__(self, other):
return MyClass.registry[id(self)] == MyClass.registry[id(other)]
def update(self, someint, someotherint):
MyClass.registry[id(self)] = (someint, someotherint)
In this example, objects with the same someint, someotherint pair are equal, while the third parameter does not factor in. The trick is to keep the parameters in registry in sync. As an alternative to update, you could override getattr and setattr for your class instead; this would ensure that any assignment foo.someint = y would be kept synced with your class-level dictionary. See an example here.
I believe you would have to keep a dict {args: object} of instances already created, then override the class' __new__ method to check in that dictionary, and return the relevant object if it already existed. Note that I haven't implemented or tested this idea. Of course, strings are handled at the C level.