class TrafficData(object):
def __init__(self):
self.__data = {}
def __getitem__(self, epoch):
if not isinstance(epoch, int):
raise TypeError()
return self.__data.setdefault(epoch, ProcessTraffic())
def __iadd__(self, other):
for epoch, traffic in other.iteritems():
# these work
#existing = self[epoch]
#existing += traffic
# this does not
self[epoch] += traffic # here the exception is thrown
return self
In the above trimmed down code, I do not expect an item assignment, yet apparently one is occurring on the marked line, and throwing the following exception:
File "nethogs2.py", line 130, in __iadd__
self[epoch] += traffic
TypeError: 'TrafficData' object does not support item assignment
However if I instead use the preceding 2 commented out lines, no exception is thrown.
As I see it, the 2 should behave in the same way. self[epoch] returns a reference to an object, and it's modified in place through that objects __iadd__. What am I misunderstanding here? I frequently run into this problem when using dictionaries.
Update0
It's probably worth pointing out that the values in self.__data have __iadd__ defined, but not __add__, and I'd much prefer to modify the value in place if possible. I would also like to avoid creating a __setitem__ method.
Update1
Below is a test case demonstrating the problem, I've left the code above for existing answers.
class Value(object):
def __init__(self, initial=0):
self.a = initial
def __iadd__(self, other):
self.a += other
return self
def __str__(self):
return str(self.a)
class Blah(object):
def __init__(self):
self.__data = {}
def __getitem__(self, key):
return self.__data.setdefault(key, Value())
a = Blah()
b = a[1]
b += 1
print a[1]
a[1] += 2
print a[1]
What you are exactly doing in:
self[epoch] += traffic
is:
self[epoch] = self[epoch] + traffic
But you haven't defined __setitem__ method, so you can do that on self.
You also need:
def __setitem__(self, epoch, value):
self.__data[epoch] = value
or something similar.
It's probably worth pointing out that
the values in self.__data have
__iadd__ defined, but not __add__, and I'd much prefer to modify the value in
place if possible.
To add some precision to previous answers, under the circumstances you describe, self[epoch] += traffic translates exactly to:
self[epoch] = self[epoch].__iadd__(traffic)
So if all you want are the side effects of __iadd__, without the item-assignment part, your choices are limited to the workaround that you've already identified in the comments in the code you've posted, or calling __iadd__ yourself -- possibly through the operator module, though I believe operator.__iadd__(self[epoch], traffic) has no added value compared to the simpler self[epoch].__iadd__(traffic) (when self[epoch] does have a __iadd__ method).
The code:
self[epoch] += traffic
Is syntactic sugar for:
self[epoch] = self[epoch] + traffic
So the assignment is not unexpected it is the assignment in +=. So you also need to override the __setitem__() method.
Related
For example, I have a class with a field __x, which is a list:
class C():
def __init__(self, xx):
self.__x = xx
#property
def x(self):
return self.__x
#x.setter
def x(self, xx):
raise Exception("Attempt to change an immutable field")
I can prevent changes such as these:
c = C([1,2,3])
c.x = [3,2,1]
But how can I prevent a change such as this?
c.x.append(4)
In the final analysis, you cannot protect your objects from inspection and manipulation.
Also, always ask yourself "from whom, exactly?" when you want to "protect" data.
Sometimes it's just not worth the effort to code around users not reading the documentation.
That being said, you could consider return tuple(self.__x) in the getter.
On the other hand, if __x contains other mutable objects, that would not prevent a user from manipulating those inner objects. (return list(self.__x) would also return a shallow copy of the data, but with less implicit "hey, I'm supposed to be immutable!" signaling.)
Something you should definitely consider is to change self.__x = xx to self.__x = list(xx) in the __init__ method, such that users doing
var = []
c = C(var)
can't "easily" (or by mistake, and again, there could be mutable inner objects) change the state of c by mutating var.
The simplest approach would be to accept an iterable on __init__ and turn it to a tuple internally:
class C(object):
def __init__(self, iterable):
self._tuple = tuple(iterable)
#property
def x(self):
return self._tuple
#x.setter
def x(self, value):
raise RuntimeError('can\'t reset the x attribute.')
c = C([1, 2, 3])
# c.x = 'a' Will raise 'RuntimeError: can't reset the x attribute.'
print(c.x)
A design like this one makes any object instantiated from the class immutable, so that mutating operations should return new objects instead of changing the state of the current one.
Let's say for instance that you want to implement a function that increment by one each item in self.x. With this approach you need to write something similar to:
def increment_by_one(c):
return C(t+1 for t in c.x)
As there's a cost associated with creating and destroying objects the trade-offs between this approach (which prevents mutation of the x attribute) and the one suggested by #timgeb should be evaluated on your use-case.
I recently posted a question, and with the help of some of you and also my reading, I came to the the understanding that:
From my original question I had posted was - what is the meaning of for item in b. Although I knew how for i in something works, I didn't understand why we are using b as we put items (see code below) in b.volume not in b. How would python know what is in b if we never put stuff in it (but in b.volume instead).
class Box:
def __init__(self):
self.volume = []
self.index = -1
def add_item(self, item):
self.volume.append(item)
def __iter__(self):
return self
def __next__(self):
self.index +=1
if self.index >= len(self.volume):
raise StopIteration
return self.volume[self.index]
class Item:
def __init__(self, name, weight):
self.name = name
self.weight = weight
b = Box()
b.add_item(Item('Cat', 5))
b.add_item(Item('Nintendo Switch', 1))
b.add_item(Item('Potatoes', 2))
for i in b:
print('The {} weighs {} kg'.format(item.name.lower(), item.weight))
So instead of for i in b, I thought it should have been for i in b.volume.
Here is what I think is right:
I think when we call for i in b it calls __iter__ and __next__ method for the object b. It tell that go to the __iter__ and __next__ of b (like just to specify the object so it doesn't go to some other object's __iter__ method if defined)
So after for i in b, __iter__ method of the object b, is called and then the __next__ method. The __next__ method returns something, which happens to be the value of i (for i in b).
Also, Python here doesn't know how many iterations it will perform as there is no list generated for eq For i in range(5) generates a list of [0,1,2,3,4] to b iterated over.
It is done until stop iteration is raised.
doing for i in b.volume will work, but then would defeat the purpose of __iter__ and __next__.
Is this correct?
All of the stuff your think is with the code is right, but also just telling you that, this works with b.volume will also work, so:
for i in b.volume:
print('The {} weighs {} kg'.format(i.name.lower(), i.weight))
Also gives:
The cat weighs 5 kg
The nintendo switch weighs 1 kg
The potatoes weighs 2 kg
Note: (as in my example), item should be i in this case (in the last loop).
I was wondering if using the #property in python to update an attribute overwrites it or simply updates it? As the speed is very different in the 2 cases.
And in case it gets overwritten, what alternative can I use? Example:
class sudoku:
def __init__(self,puzzle):
self.grid={(i,j):puzzle[i][j] for i in range(9) for j in range(9)}
self.elements
self.forbidden=set()
#property
def elements(self):
self.rows=[[self.grid[(i,j)] for j in range(9)] for i in range(9)]
self.columns=[[self.grid[(i,j)] for i in range(9)] for j in range(9)]
self.squares={(i,j): [self.grid[(3*i+k,3*j+l)] for k in range(3) for l in range(3)] for i in range(3) for j in range(3) }
self.stack=[self.grid]
self.empty={k for k in self.grid.keys() if self.grid[k]==0}
Basically, I work with the grid method, and whenever I need to update the other attributes I call elements. I prefer to call it manually tho. The question, however, is that if I change self.grid[(i,j)], does python calculate each attribute from scratch because self.grid was changed or does it only change the i-th row, j-th column etc?
Thank you
edit: added example code
As is, your question is totally unclear - but anyway, since you don't seem to understand what a property is and how it works...
class Obj(object):
def __init__(self, x, y):
self.x = x
#property
def x(self):
return self._x / 2
#x.setter
def x(self, value):
self._x = value * 2
Here we have a class with a get/set ("binding") property x, backed by a protected attribute _x.
The "#property" syntax here is mainly syntactic sugar, you could actually write this code as
class Obj(object):
def __init__(self, x, y):
self.x = x
self.y = y
def get_x(self):
return self._x / 2
def set_x(self, value):
self._x = value * 2
x = property(fget=get_x, fset=set_x)
The only difference with the previous version being that the get_x and set_x functions remain available as methods. Then if we have an obj instance:
obj = Obj(2, 4)
Then
x = obj.x
is just a shortcut for
x = obj.get_x()
and
obj.x = 42
is just a shortcut for
obj.set_x(42)
How this "shortcut" works is fully documented here, with a whole chapter dedicated to the property type.
As you can see there's nothing magical here, and once you get (no pun intended) the descriptor protocol and how the property class uses it, you can answer the question by yourself.
Note that properties will ALWAYS add some overhead (vs plain attributes or direct method call) since you have more indirections levels and method calls invoked, so it's best to only use them when it really makes sense.
EDIT: now you posted your code, I confirm that you don't understand Python's "properties" - not only the technical side of it but even the basic concept of a "computed attribute".
The point of computed attributes in general (the builtin property type being just one generic implementation of) is to have the interface of a plain attribute (something you can get the value if with value = obj.attrname and eventually set the value of with obj.attrname = somevalue) but actually invoking a getter (and eventually a setter) behind the hood.
Your elements "property" while technically implemented as a read-only property, is really a method that initializes half a dozen attributes of your class, doesn't return anything (well it implicitely returns None) and which return value is actually never used (of course). This is definitly not what computed attributes are for. This should NOT be a property, it should be a plain function (with some explicit name such as "setup_elements" or whatever makes sense here).
# nb1 : classes names should be CamelCased
# nb2 : in Python 2x, you want to inherit from 'object'
class Sudoku(object):
def __init__(self,puzzle):
self.grid={(i,j):puzzle[i][j] for i in range(9) for j in range(9)}
self.setup_elements()
self.forbidden=set()
def setup_elements(self):
self.rows=[[self.grid[(i,j)] for j in range(9)] for i in range(9)]
self.columns=[[self.grid[(i,j)] for i in range(9)] for j in range(9)]
self.squares={(i,j): [self.grid[(3*i+k,3*j+l)] for k in range(3) for l in range(3)] for i in range(3) for j in range(3) }
self.stack=[self.grid]
self.empty={k for k, v in self.grid.items() if v==0}
Now to answer your question:
if I change self.grid[(i,j)], does python calculate each attribute from scratch because self.grid was changed
self.grid is a plain attribute, so just rebinding self.grid[(i, j)] doesn't make "python" calculate anything else, of course. None of your object's other attributes will be impacted. Actually Python (the interpreter) has no mind-reading ability and will only do exactly what you asked for, nothing less, nothing more, period.
or does it only change the i-th row, j-th column
This :
obj = Sudoku(some_puzzle)
obj.grid[(1, 1)] = "WTF did you expect ?"
will NOT (I repeat: "NOT") do anything else than assigning the literal string "WTF did you expect ?" to obj.grid[(1, 1)]. None of the other attributes will be updated in any way.
Now if your question was: "if I change something to self.grid and call self.setup_elements() after, will Python recompute all attributes or only update self.rows[xxx] and self.columns[yyy]", then the answer is plain simple: Python will do exactly what you asked for: it will execute self.setup_elements(), line after line, statement after statement. Plain and simple. No magic here, and the only thing you'll get from making it a property instead of a plain method is that you won't have to type the () after to invoke the method.
So if what you expected from making this elements() method a property was to have some impossible magic happening behind the scene to detect that you actually only wanted to recompute impacted elements, then bad news, this is not going to happen, and you will have to explicitely tell the interpreter how to do so. Computed attributes might be part of the solution here, but not by any magic - you will have to write all the code needed to intercept assignments to any of those attributes and recompute what needs to be recomputed.
Beware, since all those attributes are mutable containers, just wrapping each of them into properties won't be enough - consider this:
class Foo(object):
def __init__(self):
self._bar = {"a":1, "b": 2}
#property
def bar(self):
print("getting self._bar")
return self._bar
#bar.setter
def bar(self, value):
print("setting self._bar to {}".format(value))
self._bar = value
>>> f = Foo()
>>> f.bar
getting self._bar
{'a': 1, 'b': 2}
>>> f.bar['z'] = "WTF ?"
getting self._bar
>>> f.bar
getting self._bar
{'a': 1, 'b': 2, 'z': 'WTF ?'}
>>> bar = f.bar
getting self._bar
>>> bar
{'a': 1, 'b': 2, 'z': 'WTF ?'}
>>> bar["a"] = 99
>>> f.bar
getting self._bar
{'a': 99, 'b': 2, 'z': 'WTF ?'}
As you can see, we could mutate self._bar without the bar.setter function ever being invoked - because f.bar["x"] = "y" is actually NOT assigning to f.bar (which would need f.bar = "something else") but _getting_ thef._bardict thru theFoo.bargetter, then invokingsetitem()` on this dict.
So if you want to intercept something like f.bar["x"] = "y", you will also have to write some dict-like object that will intercept all mutators access on the dict itself ( __setitem__, but also __delitem__ etc) and notify f of those changes, and change your property so that it returns an instance of this dict-like objects instead.
I have a class that looks like this
Class myClass:
def __init__(self, key, value):
self.key = key
self.value = value
where key is a string and value is always a list of elements of myClass, possibly empty.
I want to define my own iter method that returns value.key for each value in values. I tried
def __iter__(self):
return self
def __next__(self):
try:
self.value.next().key
except:
raise StopIteration
but it's looping forever. What am I doing wrong?
Also if I want to have Python 2 / 3 compatibility, should I add the method
def next(self):
return self.__next__()
There's no reason for you to implement __next__. You can use __iter__ to return a generator which will do what you want.
class Pair(object):
def __init__(self, key, value):
self.key = key
self.value = value
def __iter__(self):
return (v.key for v in self.value)
# alternative iter function, that allows more complicated logic
def __iter__(self):
for v in self.value:
yield v.key
p = Pair("parent", [Pair("child0", "value0"), Pair("child1", "value1")])
assert list(p) == ["child0", "child1"]
This way of doing things is compatible with both python2 and python3 as the returned generator will have the required next function in python2, and __next__ in python3.
You need to extract and preserve an iterator on list self.value -- you can't just call next on a list, you need an iterator on such a list.
So, you need an auxiliary iterator class:
class myClassIter(object):
def __init__(self, theiter):
self.theiter = theiter
def __next__(self):
return next(self.theiter).key
next = __next__
which I've also made Py 2/3 compatible with the object base and appropriate aliasing.
Here, I'm assuming every item in the list has a key attribute (so the only expected exception is StopIteration, which you can just propagate). If that is not the case, and you want to just stop the iteration when an item is met without the attribite, the try/except is needed, but keep it tight! -- a crucial design aspect of good exception handling. I.e, if these are indeed your specs:
def __next__(self):
try: return next(self.theiter).key
except AttributeError: raise StopIteration
don't catch all exceptions -- only the ones you specifically expect!
Now, in myClass, you'll want:
def __iter__(self):
return myClassIter(iter(self.value))
This means that myClass is an iterable, not an iterator, so you can e.g properly have more than one loop on a myClass instance:
mc = myClass(somekey, funkylist)
for ka in mc:
for kb in mc:
whatever(ka, kb)
If mc was itself an iterator, the inner loop would exhaust it and the semantics of the nested loops would therefore be completely different.
If you do indeed want such completely different semantics (i.e you want mc to be itself an iterator, not just an iterable) then you must dispense with the auxiliary class (but still need to store the iterator on self.value as an instance attribute for myClass) -- that would be a strange, uncomfortable arrangement, but it is (just barely) possible that it is indeed the arrangement your application needs...
I'm new to Python so apologies in advance if this is a stupid question.
For an assignment I need to overload augmented arithmetic assignments(+=, -=, /=, *=, **=, %=) for a class myInt. I checked the Python documentation and this is what I came up with:
def __iadd__(self, other):
if isinstance(other, myInt):
self.a += other.a
elif type(other) == int:
self.a += other
else:
raise Exception("invalid argument")
self.a and other.a refer to the int stored in each class instance. I tried testing this out as follows, but each time I get 'None' instead of the expected value 5:
c = myInt(2)
b = myInt(3)
c += b
print c
Can anyone tell me why this is happening? Thanks in advance.
You need to add return self to your method. Explanation:
The semantics of a += b, when type(a) has a special method __iadd__, are defined to be:
a = a.__iadd__(b)
so if __iadd__ returns something different than self, that's what will be bound to name a after the operation. By missing a return statement, the method you posted is equivalent to one with return None.
Augmented operators in Python have to return the final value to be assigned to the name they are called on, usually (and in your case) self. Like all Python methods, missing a return statement implies returning None.
Also,
Never ever ever raise Exception, which is impossible to catch sanely. The code to do so would have to say except Exception, which will catch all exceptions. In this case you want ValueError or TypeError.
Don't typecheck with type(foo) == SomeType. In this (and virtually all) cases, isinstance works better or at least the same.
Whenever you make your own type, like myInt, you should name it with capital letters so people can recognize it as a class name.
Yes, you need "return self", it will look like this:
def __iadd__(self, other):
if isinstance(other, myInt):
self.a += other.a
return self
elif type(other) == int:
self.a += other
return self
else:
raise Exception("invalid argument")