I'm trying to create a custom object that supports nested attributes.
I need to implement a specific kind of search.
If an attribute doesn't exist at the lowest level, I want to recurse and see if the attribute exists at a higher level.
I've spent all day trying to do this. The closest I've come is being able to print the attribute search path.
class MyDict(dict):
def __init__(self):
super(MyDict, self).__init__()
def __getattr__(self, name):
return self.__getitem__(name)
def __getitem__(self, name):
if name not in self:
print name
self[name] = MyDict()
return super(MyDict, self).__getitem__(name)
config = MyDict()
config.important_key = 'important_value'
print 'important key is: ', config.important_key
print config.random.path.to.important_key
Output:
important key is: important_value
random
path
to
important_key
{}
What I need to happen is instead to see if important_key exists at the lowest level (config.random.path.to), then go up a level (config.random.path) and only return None if it doesn't exist at the top level.
Do you think this is possible?
Thank you so much!
Yes, it's possible. In your search routine, recur to the end of the path, checking for the desired attribute. At the bottom level, return the attribute if found, None otherwise. At each non-terminal level, recur to the next level down.
if end of path # base case
if attribute exists here
return attribute
else
return None
else # some upper level
exists_lower = search(next level down)
if exists_lower
return exists_lower
else
if attribute exists here
return attribute
else
return None
Does this pseudo code get you moving toward a solution?
Related
I am trying to build some variable dependencies in Python. For example, if a = x, b = y and c = a + b, then if a or b changes the value of c should be automatically updated. I am aware the Python variables and values work on the basis of tags and have been trying to work around this using __setattr__. I seem to be having some trouble doing this, due to the cyclic dependency in __setattr__.
Consider this small code snippet:
class DelayComponents(object):
'''
Delay Components Class
'''
def __init__(self, **delays):
'''
Constructor
'''
self.prop_delay = round(float(delays['prop_delay']), 2)
self.trans_delay = round(float(delays['trans_delay']), 2)
self.proc_delay = round(float(delays['proc_delay']), 2)
self.queue_delay = round(float(delays['queue_delay']), 2)
self.delay = (self.prop_delay + self.proc_delay +
self.trans_delay + self.queue_delay)
def __setattr__(self, key, value):
self.__dict__[key] = value
if (key in ("prop_delay", "trans_delay",
"proc_delay", "queue_delay")):
self.delay = (self.prop_delay + self.proc_delay +
self.trans_delay + self.queue_delay)
This seems to serve the purpose well, but when I create an object of DelayComponents for the first time, since __setattr__ has been overridden and is called for each of the values being created, the if check inside __setattr__ throws an error saying the remaining three variables have not been found (which is true, since they have not yet been created).
How do I resolve this dependency?
Also, is there some way to accomplish the same with a dict? More specifically, if the three variables were actually key-value pairs in a dict, where the third key's value was the sum of the values of the first two keys, would it be possible to update the third value automatically when either of the first two changes?
Assuming that you want zero default values for the unset _delays (in both __init__ and __setattr__) you could do something like:
class DelayComponents(object):
'''
Delay Components Class
'''
ATTRS = ['prop_delay', 'trans_delay', 'proc_delay', 'queue_delay']
def __init__(self, **delays):
'''
Constructor
'''
for attr in self.ATTRS:
setattr(self, attr, round(float(delays.get(attr, 0)), 2))
# No point in setting delay here - it's already done!
def __setattr__(self, key, value):
super(DelayComponents, self).__setattr__(key, value)
# This avoids directly interacting with the __dict__
if key in self.ATTRS:
self.delay = sum(getattr(self, attr, 0) for attr in self.ATTRS)
In use:
>>> d = DelayComponents(prop_delay=1, trans_delay=2, proc_delay=3, queue_delay=4)
>>> d.delay
10.0
Should you want different defaults for different attributes, DelayComponents.ATTRS could be a dictionary {'attribute_name': default_value, ...}.
A much simpler alternative is to make delay a #property, that is calculated only as required:
class DelayComponents(object):
'''
Delay Components Class
'''
ATTRS = ['prop_delay', 'trans_delay', 'proc_delay', 'queue_delay']
def __init__(self, **delays):
'''
Constructor
'''
for attr in self.ATTRS:
setattr(self, attr, round(float(delays.get(attr, 0)), 2))
#property
def delay(self):
return sum(getattr(self, attr, 0) for attr in self.ATTRS)
To answer your sub-question: no, there's no way to do this with a vanilla dict; the values for keys aren't reevaluated based on changes to the values from which they're calculated.
Also, in all seriousness, there is no point to your current docstrings; you might as well leave them out entirely. They provide no information, and aren't compliant with PEP-257 either.
I have seen some answers to the question "how to build a binary tree", but the relate answers seem not to work!! They are based on algorithms more or less like this:
def insert(item, tree):
if (item < tree.entry):
if (tree.left != None):
insert(item, tree.left)
else:
tree.left = Tree(item)
else:
if (tree.right != None):
insert(item, tree.right)
else:
tree.right = Tree(item)
The code mentioned before has been written by Isaac1000, but other ones are very similar. The problem is that when tree.right or tree.left is passed to the function "insert" in the following calls:
insert(item, tree.left)
insert(item, tree.right)
the ones who wrote the code think to pass the reference, instead of a copy of the value, so that, tree.left or tree.right will not be really changed.
Finally, that function or similar just work at the first or zero level of the Tree, but not at the n level of the tree.
So, how to build a binary search tree without pointers?
P.S.
I said "without pointers" just because I know that python hasn't got pointers, but please tell me if I'm wrong
#qfiard
"If you pass a mutable object into a method, the method gets a reference to that same object and you can mutate it to your heart's delight, but if you rebind the reference in the method, the outer scope will know nothing about it, and after you're done, the outer reference will still point at the original object."(Blay Conrad)
That speech made me all clear. I have understood that my code didn't word because I was used to rebind tree.left. The following code was the code I have been using till now:
def insert(self, data):
if self is None:
self = Tree(data)
else:
if data < self.data:
Link.insert(self.left, data)
else:
Link.insert(self.right, data)
Finally, when I wrote self = Tree(data) I was trying to rebind an object and the outer scope didn't know nothing about it. Instead, using the procedure I have posted, when I use self.right or self.left I try to modify the object without rebinding, so that the outer scope remembers my changes.
Arguments in Python are passed by assignment, which is similar to passing arguments by reference for mutable types (lists, objects, ...). See this stackoverflow answer for more details on the subject: https://stackoverflow.com/a/986145/2887956 .
The code that you provided works therefore perfectly if you use objects to represent your trees. The following code outputs
<<None, 0, None>, 1, <None, 2, None>>
class Tree(object):
def __init__(self, entry):
self.entry = entry
self.left = None
self.right = None
def __str__(self):
return '<%s, %d, %s>' % (self.left, self.entry, self.right)
def insert(item, tree):
if item < tree.entry:
if tree.left is not None:
insert(item, tree.left)
else:
tree.left = Tree(item)
else:
if tree.right is not None:
insert(item, tree.right)
else:
tree.right = Tree(item)
root = Tree(1)
insert(0, root)
insert(2, root)
print root
Your algorithm would also work if you used lists to represent your trees, although you would need to modify it substantially.
I am developing a multiplayer game. When I use an object from inventory, it should update the user creature's stats with the values of the attributes of an object.
This is my code:
try:
obj = self._get_obj_by_id(self.query['ObjectID']).first()
# Get user's current creature
cur_creature = self.user.get_current_creature()
# Applying object attributes to user attributes
for attribute in obj.attributes:
cur_creature.__dict__[str(attribute.Name)] += attribute.Value
dbObjs.session.commit()
except (KeyError, AttributeError) as err:
self.query_failed(err)
Now, this doesn't commit things properly for some reason, so I tried:
cur_creature.Health = 100
logging.warning(cur_creature.Health)
dbObjs.session.commit()
Which works, but is not very convenient (since I would need a big if statement to updated the different stats of the creature)
So I tried:
cur_creature.__dict__['Health'] = 100
logging.warning(cur_creature.Health)
dbObjs.session.commit()
I get 100 in logs, but no changes, so I tried:
cur_creature.__dict__['Health'] = 100
cur_creature.Health = cur_creature.__dict__['Health']
logging.warning(cur_creature.Health)
dbObjs.session.commit()
Still '100' in logs, but no changes, so I tried:
cur_creature.__dict__['Health'] = 100
cur_creature.Health = 100
logging.warning(cur_creature.Health)
dbObjs.session.commit()
Which still writes 100 in the logs, but doesn't commit changes to the database.
Now, this is weird, since it only differ by the working version for the fact that it has this line on top:
cur_creature.__dict__['Health'] = 100
Summary: If I modify an attribute directly, commit works fine. Instead, if I modify an attribute through the class' dictionary, then, no matter how I modify it afterwards, it doesn't commit changes to the db.
Any ideas?
Thanks in advance
UPDATE 1:
Also, this updates Health in the db, but not Hunger:
cur_creature.__dict__['Hunger'] = 0
cur_creature.Health = 100
cur_creature.Hunger = 0
logging.warning(cur_creature.Health)
dbObjs.session.commit()
So just accessing the dictionary is not a problem for attributes in general, but modifying an attribute through the dictionary, prevents the changes to that attributes from being committed.
Update 2:
As a temporary fix, I've overridden the function __set_item__(self) in the class Creatures:
def __setitem__(self, key, value):
if key == "Health":
self.Health = value
elif key == "Hunger":
self.Hunger = value
So that the new code for 'use object' is:
try:
obj = self._get_obj_by_id(self.query['ObjectID']).first()
# Get user's current creature
cur_creature = self.user.get_current_creature()
# Applying object attributes to user attributes
for attribute in obj.attributes:
cur_creature[str(attribute.Name)] += attribute.Value
dbObjs.session.commit()
except (KeyError, AttributeError) as err:
self.query_failed(err)
Update 3:
By having a look at the suggestions in the answers, I settled down for this solution:
In Creatures
def __setitem__(self, key, value):
if key in self.__dict__:
setattr(self, key, value)
else:
raise KeyError(key)
In the other method
# Applying object attributes to user attributes
for attribute in obj.attributes:
cur_creature[str(attribute.Name)] += attribute.Value
The problem does not reside in SQLAlchemy but is due to Python's descriptors mechanism. Every Column attribute is a descriptor: this is how SQLAlchemy 'hooks' the attribute retrieval and modification to produce database requests.
Let's try with a simpler example:
class Desc(object):
def __get__(self, obj, type=None):
print '__get__'
def __set__(self, obj, value):
print '__set__'
class A(object):
desc = Desc()
a = A()
a.desc # prints '__get__'
a.desc = 2 # prints '__set__'
However, if you go through a instance dictionary and set another value for 'desc', you bypass the descriptor protocol (see Invoking Descriptors):
a.__dict__['desc'] = 0 # Does not print anything !
Here, we just created a new instance attribute called 'desc' with a value of 0. The Desc.__set__ method was never called, and in your case SQLAlchemy wouldn't get a chance to 'catch' the assignment.
The solution is to use setattr, which is exactly equivalent to writing a.desc:
setattr(a, 'desc', 1) # Prints '__set__'
Don't use __dict__. Use getattr and setattr to modify attributes by name:
for attribute in obj.attributes:
setattr(cur_creature,str(attribute.Name), getattr(cur_creature,str(attribute.Name)) + attribute.Value)
More info:
setattr
getattr
hasattr(obj, attribute) is used to check if an object has the specified attribute but given an attribute is there a way to know where (all) it is defined?
Assume that my code is getting the name of an attribute (or a classmethod) as string and I want to invoke classname.attribute but I don't have the classname.
One solution that comes to my mind is this
def finder(attr):
for obj in globals():
try:
if globals()[obj].__dict__[attr]:
return(globals()[obj])
except:
...
usage:
class Lime(object):
#classmethod
def lfunc(self):
print('Classic')
getattr(finder('lfunc'),'lfunc')() #Runs lfunc method of Lime class
I am quite sure that this is not the best (oe even proper way) to do it. Can someone please provide a better way.
It is always "possible". Wether it is desirable is another history.
A quick and dirty way to do it is to iterate linearly over all classes and check if any define the attribute you have. Of course, that is subject to conflicts, and it will yield the first class that has such a named attribute. If it exists in more than one, it is up to you to decide which you want:
def finder(attr):
for cls in object.__subclasses__():
if hasattr(cls, attr):
return cls
raise ValueError
Instead of searching in "globals" this searches all subclasses of "object" - thus the classes to be found don't need to be in the namespace of the module where the finder function is.
If your methods are unique in teh set of classes you are searching, though, maybe you could just assemble a mapping of all methods and use it to call them instead.
Let's suppose all your classes inehrit from a class named "Base":
mapper = {attr_name:getattr(cls, attr_name) for cls in base.__subclasses__() for attr_name, obj in cls.__dict__.items()
if isinstance(obj, classmethod) }
And you call them with mapper['attrname']()
This avoids a linear search at each method call and thus would be much better.
- EDIT -
__subclassess__ just find the direct subclasses of a class, not the inheritance tree - so it won't be usefull in "real life" - maybe it is in the specifc case the OP has in its hands.
If one needs to find things across a inheritance tree, one needs to recurse over the each subclass as well.
As for old-style classes: of course this won't work - that is one of the motives for which they are broken by default in new code.
As for non-class attributes: they can only be found inspecting instances anyway - so another method has to be thought of - does not seem to be the concern of the O.P. here.
This might help:
import gc
def checker(checkee, maxdepth = 3):
def onlyDict(ls):
return filter(lambda x: isinstance(x, dict), ls)
collection = []
toBeInspected = {}
tBI = toBeInspected
gc.collect()
for dic in onlyDict(gc.get_referrers(checkee)):
for item, value in dic.iteritems():
if value is checkee:
collection.append(item)
elif item != "checker":
tBI[item] = value
def _auxChecker(checkee, path, collection, checked, current, depth):
if current in checked: return
checked.append(current)
gc.collect()
for dic in onlyDict(gc.get_referents(current)):
for item, value in dic.iteritems():
currentPath = path + "." + item
if value is checkee:
collection.append(currentPath)
else:
try:
_auxChecker(checkee, currentPath, collection,
checked, value, depth + 1)
if depth < maxdepth else None
except TypeError:
continue
checked = []
for item, value in tBI.iteritems():
_auxChecker(checkee, item, collection, checked, value, 1)
return collection
How to use:
referrer = []
class Foo:
pass
noo = Foo()
bar = noo
import xml
import libxml2
import sys
import os
op = os.path
xml.foo = bar
foobar = noo
for x in checker(foobar, 5):
try:
y= eval(x)
referrer.append(x)
except:
continue
del x, y
ps: attributes of the checkee will not be further checked, for recursive or nested references to the checkee itself.
This should work in all circumstances, but still needs a lot of testing:
import inspect
import sys
def finder(attr, classes=None):
result = []
if classes is None:
# get all accessible classes
classes = [obj for name, obj in inspect.getmembers(
sys.modules[__name__])]
for a_class in classes:
if inspect.isclass(a_class):
if hasattr(a_class, attr):
result.append(a_class)
else:
# we check for instance attributes
if hasattr(a_class(), attr):
result.append(a_class)
try:
result += finder(attr, a_class.__subclasses__())
except:
# old style classes (that don't inherit from object) do not
# have __subclasses; not the best solution though
pass
return list(set(result)) # workaround duplicates
def main(attr):
print finder(attr)
return 0
if __name__ == "__main__":
sys.exit(main("some_attr"))
I have created a function that takes a value, does some calculations and return the different answers as an object. However when I try to parallelize the code, using pp, I get the following error.
File "trmm.py", line 8, in getattr
return self.header_array[name]
RuntimeError: maximum recursion depth exceeded while calling a Python object
Here is a simple version of what I am trying to do.
class DataObject(object):
"""
Class to handle data objects with several arrays.
"""
def __getattr__(self, name):
try:
return self.header_array[name]
except KeyError:
try:
return self.line[name]
except KeyError:
raise AttributeError("%s instance has no attribute '%s'" %(self.__class__.__name__, name))
def __setattr__(self, name, value):
if name in ('header_array', 'line'):
object.__setattr__(self, name, value)
elif name in self.line:
self.line[name] = value
else:
self.header_array[name] = value
class TrmmObject(DataObject):
def __init__(self):
DataObject.__init__(self)
self.header_array = {
'header': None
}
self.line = {
'longitude': None,
'latitude': None
}
if __name__ == '__main__':
import pp
ppservers = ()
job_server = pp.Server(2, ppservers=ppservers)
def get_monthly_values(value):
tplObj = TrmmObject()
tplObj.longitude = value
tplObj.latitude = value * 2
return tplObj
job1 = job_server.submit(get_monthly_values, (5,), (DataObject,TrmmObject,),("numpy",))
result = job1()
If I change return tplObj to return [tplObj.longitude, tplObj.latitude] there is no problem. However, as I said before this is a simple version, in reality this change would complicate the program a lot.
I am very grateful for any help.
You almost never need to use getattr and setattr, and it almost always ends up with something blowing up, and infinite recursions is a typical effect of that. I can't really see any reason for using them here either. Be explicit and use the line and header_array dictionaries directly.
If you want a function that looks up a value over all arrays, create a function for that and call it explicitly. Calling the function __getitem__ and using [] is explicit. :-)
(And please don't call a dictionary "header_array", it's confusing).