class A:
def __get(self):
return self._x
def __set(self, y):
self._x = y
def __delete_x(self):
print('DELETING')
del self._x
x = property(__get,__set,__delete_x)
b = A()
# Here, when b is deleted, i'd like b.x to be deleted, i.e __delete_x()
# called (and for immediate consequence, "DELETING" printed)
del b
The semantics of the del statement don't really lend themselves to what you want here. del b simple removes the reference to the A object you just instantiated from the local scope frame / dictionary; this does not directly cause any operation to be performed on the object itself. If that was the last reference to the object, then the reference count dropping to zero, or the garbage collector collecting a cycle, may cause the object to be deallocated. You could observe this by adding a __del__ method to the object, or by adding a weakref callback that performs the desired actions.
Neither of the latter two solutions seems like a great idea, though; __del__ methods prevent the garbage collector from collecting any cycles involving the object; and while weakrefs do not suffer from this problem, in either case you may be running in a strange environment (such as during program shutdown), which may make it difficult to get done what you want to accomplish.
If you can expand on your exact use case, it may be that there is an entirely different approach to accomplishing your desired end goal, but it is difficult to speculate based on such a general and limited example.
To control what happens when an instance of class A goes away (whether by being deleted or garbage collected), you can implement special method __del__(self) in A. If you want to have your code involved when a specific attribute of that instance goes away, you can either wrap that attribute with a wrapper class which has __del__, or, probably better in most cases, use the weakref module (however, not all types are subject to being target of weak references, so you may also need some wrapping for this case).
Avoiding __del__ is generally preferable, if you possibly can, because it can interfere with garbage collection and thereby cause "memory leaks" if and when you have circular references.
An ugly way to do it would be :
def __del__(self):
for x in dir(self.__class__):
if type(getattr(self.__class__, x)) == property:
getattr(self.__class__, x).fdel(self)
Related
I saw a class in which a __del__ method is defined. This method is used to destroy an instance of the class. However, I cannot find a place where this method is used. How is this method used? Like that: obj1.del()?.
How do I call the __del__ method?
__del__ is a finalizer. It is called when an object is garbage collected which happens at some point after all references to the object have been deleted.
In a simple case this could be right after you say del x or, if x is a local variable, after the function ends. In particular, unless there are circular references, CPython (the standard Python implementation) will garbage collect immediately.*
However, this is an implementation detail of CPython. The only required property of Python garbage collection is that it happens after all references have been deleted, so this might not necessary happen right after and might not happen at all.
Even more, variables can live for a long time for many reasons, e.g. a propagating exception or module introspection can keep variable reference count greater than 0. Also, variable can be a part of cycle of references — CPython with garbage collection turned on breaks most, but not all, such cycles, and even then only periodically.
Since you have no guarantee it's executed, one should never put the code that you need to be run into __del__() — instead, this code belongs to the finally clause of a try statement or to a context manager in a with statement. However, there are valid use cases for __del__: e.g. if an object X references Y and also keeps a copy of Y reference in a global cache (cache['X -> Y'] = Y) then it would be polite for X.__del__ to also delete the cache entry.
If you know that the destructor provides (in violation of the above guideline) a required cleanup, you might want to call it directly, since there is nothing special about it as a method: x.__del__(). Obviously, you should only do so if you know it can be called twice. Or, as a last resort, you can redefine this method using
type(x).__del__ = my_safe_cleanup_method
* Reference:
CPython implementation detail: CPython currently uses a reference-counting scheme with (optional) delayed detection of cyclically linked garbage, which collects most objects as soon as they become unreachable [...] Other implementations act differently and CPython may change.
I wrote up the answer for another question, though this is a more accurate question for it.
How do constructors and destructors work?
Here is a slightly opinionated answer.
Don't use __del__. This is not C++ or a language built for destructors. The __del__ method really should be gone in Python 3.x, though I'm sure someone will find a use case that makes sense. If you need to use __del__, be aware of the basic limitations per http://docs.python.org/reference/datamodel.html:
__del__ is called when the garbage collector happens to be collecting the objects, not when you lose the last reference to an object and not when you execute del object.
__del__ is responsible for calling any __del__ in a superclass, though it is not clear if this is in method resolution order (MRO) or just calling each superclass.
Having a __del__ means that the garbage collector gives up on detecting and cleaning any cyclic links, such as losing the last reference to a linked list. You can get a list of the objects ignored from gc.garbage. You can sometimes use weak references to avoid the cycle altogether. This gets debated now and then: see http://mail.python.org/pipermail/python-ideas/2009-October/006194.html.
The __del__ function can cheat, saving a reference to an object, and stopping the garbage collection.
Exceptions explicitly raised in __del__ are ignored.
__del__ complements __new__ far more than __init__. This gets confusing. See http://www.algorithm.co.il/blogs/programming/python-gotchas-1-del-is-not-the-opposite-of-init/ for an explanation and gotchas.
__del__ is not a "well-loved" child in Python. You will notice that sys.exit() documentation does not specify if garbage is collected before exiting, and there are lots of odd issues. Calling the __del__ on globals causes odd ordering issues, e.g., http://bugs.python.org/issue5099. Should __del__ called even if the __init__ fails? See http://mail.python.org/pipermail/python-dev/2000-March/thread.html#2423 for a long thread.
But, on the other hand:
__del__ means you do not forget to call a close statement. See http://eli.thegreenplace.net/2009/06/12/safely-using-destructors-in-python/ for a pro __del__ viewpoint. This is usually about freeing ctypes or some other special resource.
And my pesonal reason for not liking the __del__ function.
Everytime someone brings up __del__ it devolves into thirty messages of confusion.
It breaks these items in the Zen of Python:
Simple is better than complicated.
Special cases aren't special enough to break the rules.
Errors should never pass silently.
In the face of ambiguity, refuse the temptation to guess.
There should be one – and preferably only one – obvious way to do it.
If the implementation is hard to explain, it's a bad idea.
So, find a reason not to use __del__.
The __del__ method, it will be called when the object is garbage collected. Note that it isn't necessarily guaranteed to be called though. The following code by itself won't necessarily do it:
del obj
The reason being that del just decrements the reference count by one. If something else has a reference to the object, __del__ won't get called.
There are a few caveats to using __del__ though. Generally, they usually just aren't very useful. It sounds to me more like you want to use a close method or maybe a with statement.
See the python documentation on __del__ methods.
One other thing to note: __del__ methods can inhibit garbage collection if overused. In particular, a circular reference that has more than one object with a __del__ method won't get garbage collected. This is because the garbage collector doesn't know which one to call first. See the documentation on the gc module for more info.
The __del__ method (note spelling!) is called when your object is finally destroyed. Technically speaking (in cPython) that is when there are no more references to your object, ie when it goes out of scope.
If you want to delete your object and thus call the __del__ method use
del obj1
which will delete the object (provided there weren't any other references to it).
I suggest you write a small class like this
class T:
def __del__(self):
print "deleted"
And investigate in the python interpreter, eg
>>> a = T()
>>> del a
deleted
>>> a = T()
>>> b = a
>>> del b
>>> del a
deleted
>>> def fn():
... a = T()
... print "exiting fn"
...
>>> fn()
exiting fn
deleted
>>>
Note that jython and ironpython have different rules as to exactly when the object is deleted and __del__ is called. It isn't considered good practice to use __del__ though because of this and the fact that the object and its environment may be in an unknown state when it is called. It isn't absolutely guaranteed __del__ will be called either - the interpreter can exit in various ways without deleteting all objects.
As mentioned earlier, the __del__ functionality is somewhat unreliable. In cases where it might seem useful, consider using the __enter__ and __exit__ methods instead. This will give a behaviour similar to the with open() as f: pass syntax used for accessing files. __enter__ is automatically called when entering the scope of with, while __exit__ is automatically called when exiting it. See this question for more details.
I have something like that:
a = [instance1, instance2, ...]
if I do a
del a[1]
instance2 is removed from list, but is instance2 desctructor method called?
I'm interested in this because my code uses a lot of memory and I need to free memory deleting instances from a list.
Coming from a language like c++ (as I did), this tends to be a subject many people find difficult to grasp when first learning Python.
The bottomline is this: when you do del XXX, you are never* deleting an object when you use del. You are only deleting an object reference. However, in practice, assuming there are no other references laying about to the instance2 object, deleting it from your list will free the memory as you desire.
If you don't understand the difference between an object and an object reference, read on.
Python: Pass by value, or pass by reference?
You are likely familiar with the concept of passing arguments to a function by reference, or by value. However, Python does things differently. Arguments are always passed by object reference. Read this article for a helpful explanation of what this means.
To summarize: this means that when you pass a variable to a function, you are not passing a copy of the value of the variable (pass by value), nor are you passing the object itself - i.e., the address of the value in memory. You are passing the name-object that indirectly refers to the value held in memory.
What does this have to do with del...?
Well, I'll tell you.
Say you do this:
def deleteit(thing):
del thing
mylist = [1,2,3]
deleteit(mylist)
...what do you think will happen? Has mylist been deleted from the global namespace?
The answer is NO:
assert mylist # No error
The reason is that when you do del thing in the deleteit function, you are only deleting a local object reference to the object. That object reference exists ONLY inside of the function. As a sidebar, you might ask: is it possible to delete an object reference from the global namespace while inside a function? The answer is yes, but you have to declare the object reference to be part of the global namespace first:
def deletemylist():
global mylist
del mylist
mylist = [1,2,3]
deletemylist()
assert mylist #ERROR, as expected
Putting it all together
Now to get back to your original question. When, in ANY namespace, you do this:
del XXX
...you have NOT deleted the object signified by XXX. You CAN'T do that. You have only deleted the object reference XXX, which refers to some object in memory. The object itself is managed by the Python memory manager. This is a very important distinction.
Note that as a consequence, when you override the __del__ method of some object, which gets called when the object is deleted (NOT the object reference!):
class MyClass():
def __del__(self):
print(self, "deleted")
super().__del__()
m = MyClass()
del m
...the print statement inside the __del__ method does not necessarily occur immediately after you do del m. It only occurs at the point in time the object itself is deleted, and that is not up to you. It is up to the Python memory manager. When all object references in all the namespaces have been deleted, the __del__ method will eventually be executed. But not necessarily immediately.
The same is true when you delete an object reference that is part of a list, like in the original example. When you do del a[1], only the object reference to the object signified by a[1] is deleted, and the __del__ method of that object may or may not be called immediately (though as stated before, it will eventually be called once there are no more references to the object, and the object is garbage collected by the memory manager).
As a result of this, it is not recommended that you put things in the __del__ method that you want to happen immediately upon del mything, because it may not happen that way.
*I believe it is never. Inevitably someone will likely downvote my answer and leave a comment discussing the exception to the rule. But whatevs.
No. Calling del on a list element only removes a reference to an object from the list, it doesn't do anything (explicitly) to the object itself. However: If the reference in the list was the last one referring to the object, the object can now be destroyed and recycled. I think that the "normal" CPython implementation will immediately destroy and recycle the object, other variants' behaviour can vary.
If your object is resource-heavy and you want to be sure that the resources are freed correctly, use the with() construct. It's very easy to leak resources when relying on destructors. See this SO post for more details.
I'm trying to create a class using a static List, which collects all new instances of an object class. The problem I'm facing, seems like as soon as i try to use a list the same way as for example an integer, i can't use the magic marker __del__ anymore.
My Example:
class MyClass(object):
count = 0
#instances = []
def __init__(self, a, b):
self.a = a
self.b = b
MyClass.count += 1
#MyClass.instances.append(self)
def __str__(self):
return self.__repr__()
def __repr__(self):
return "a: " + str(self.a) + ", b: " + str(self.b)
def __del__(self):
MyClass.count -= 1
#MyClass.instances.remove(self)
A = MyClass(1,'abc')
B = MyClass(2,'def')
print MyClass.count
del B
print MyClass.count
With comments I get the correct answer:
2
1
But without the comments - including now the static object list MyClass.instances I get the wrong answer:
2
2
It seems like MyClass can't reach its __del__ method anymore! How Come?
From the docs,
del x doesn’t directly call x.__del__() — the former decrements the reference
count for x by one, and the latter is only called when x‘s reference count
reaches zero.
When you uncomment,
instances = []
...
...
MyClass.instances.append(self)
You are storing a reference to the current Object in the MyClass.instances. That means, the reference count is internally incremented by 1. That is why __del__ is not getting called immediately.
To resolve this problem, explicitly remove the item from the list like this
MyClass.instances.remove(B)
del B
Now it will print
2
1
as expected.
There is one more way to fix this problem. That is to use weakref. From the docs,
A weak reference to an object is not enough to keep the object alive:
when the only remaining references to a referent are weak references,
garbage collection is free to destroy the referent and reuse its
memory for something else. A primary use for weak references is to
implement caches or mappings holding large objects, where it’s desired
that a large object not be kept alive solely because it appears in a
cache or mapping.
So, having a weakref will not postpone object's deletion. With weakref, this can be fixed like this
MyClass.instances.append(weakref.ref(self))
...
...
# MyClass.instances.remove(weakref.ref(self))
MyClass.instances = [w_ref for w_ref in MyClass.instances if w_ref() is None]
Instead of using remove method, we can call each of the weakref objects and if they return None, they are already dead. So, we filter them out with the list comprehension.
So, now, when you say del B, even though weakrefs exist for B, it will call __del__ (unless you have made some other variable point to the same object, like by doing an assigment).
From to http://docs.python.org/2.7/reference/datamodel.html#basic-customization I quote (paragraph in gray after object.__del__):
del x doesn’t directly call x.__del__() — the former decrements the reference count for x by one, and the latter is only called when x‘s reference count reaches zero.
Here you call del B but there is still an instance of B in MyClass.instances, so that B is still referenced and hence not destroyed, so that the __del__ function is not called.
If you call directly B.__del__(), it works.
__del__ is only called when no more instances are left.
You should consider putting only weak refs into the MyClass.instances list.
This can be achieved with import weakref and then
either using a WeakSet for the list
or putting weakref.ref(self) into the list.
__del__ is automatically called whenever the last "strict" reference is removed. The weakrefs disappear automatically.
But be aware that there are some caveats on __del__ mentioned in the docs.
__del__ is used when the garbage collector remove an object from the memory. If you add your object to MyClass.instances then the object is marked as "used" and the garbage collector will never try to remove it. And so __del__ is never called.
You'd better use an explicit function (MyClass.del_element()) because you can't really predict when __del__ will be called (even if you don't add it to a list).
What are the best practices and recommendations for using explicit del statement in python? I understand that it is used to remove attributes or dictionary/list elements and so on, but sometimes I see it used on local variables in code like this:
def action(x):
result = None
something = produce_something(x)
if something:
qux = foo(something)
result = bar(qux, something)
del qux
del something
return result
Are there any serious reasons for writing code like this?
Edit: consider qux and something to be something "simple" without a __del__ method.
I don't remember when I last used del -- the need for it is rare indeed, and typically limited to such tasks as cleaning up a module's namespace after a needed import or the like.
In particular, it's not true, as another (now-deleted) answer claimed, that
Using del is the only way to make sure
a object's __del__ method is called
and it's very important to understand this. To help, let's make a class with a __del__ and check when it is called:
>>> class visdel(object):
... def __del__(self): print 'del', id(self)
...
>>> d = visdel()
>>> a = list()
>>> a.append(d)
>>> del d
>>>
See? del doesn't "make sure" that __del__ gets called: del removes one reference, and only the removal of the last reference causes __del__ to be called. So, also:
>>> a.append(visdel())
>>> a[:]=[1, 2, 3]
del 550864
del 551184
when the last reference does go away (including in ways that don't involve del, such as a slice assignment as in this case, or other rebindings of names and other slots), then __del__ gets called -- whether del was ever involved in reducing the object's references, or not, makes absolutely no difference whatsoever.
So, unless you specifically need to clean up a namespace (typically a module's namespace, but conceivably that of a class or instance) for some specific reason, don't bother with del (it can be occasionally handy for removing an item from a container, but I've found that I'm often using the container's pop method or item or slice assignment even for that!-).
No.
I'm sure someone will come up with some silly reason to do this, e.g. to make sure someone doesn't accidentally use the variable after it's no longer valid. But probably whoever wrote this code was just confused. You can remove them.
When you are running programs handling really large amounts of data ( to my experience when the totals memory consumption of the program approaches something like 1GB) deleting some objects:
del largeObject1
del largeObject2
…
can give your program the necessary breathing room to function without running out of memory. This can be the easiest way to modify a given program, in case of a “MemoryError” runtime error.
Actually, I just came across a use for this. If you use locals() to return a dictionary of local variables (useful when parsing things) then del is useful to get rid of a temporary that you don't want to return.
I have some ctypes bindings, and for each body.New I should call body.Free. The library I'm binding doesn't have allocation routines insulated out from the rest of the code (they can be called about anywhere there), and to use couple of useful features I need to make cyclic references.
I think It'd solve if I'd find a reliable way to hook destructor to an object. (weakrefs would help if they'd give me the callback just before the data is dropped.
So obviously this code megafails when I put in velocity_func:
class Body(object):
def __init__(self, mass, inertia):
self._body = body.New(mass, inertia)
def __del__(self):
print '__del__ %r' % self
if body:
body.Free(self._body)
...
def set_velocity_func(self, func):
self._body.contents.velocity_func = ctypes_wrapping(func)
I also tried to solve it through weakrefs, with those the things seem getting just worse, just only largely more unpredictable.
Even if I don't put in the velocity_func, there will appear cycles at least then when I do this:
class Toy(object):
def __init__(self, body):
self.body.owner = self
...
def collision(a, b, contacts):
whatever(a.body.owner)
So how to make sure Structures will get garbage collected, even if they are allocated/freed by the shared library?
There's repository if you are interested about more details: http://bitbucket.org/cheery/ctypes-chipmunk/
What you want to do, that is create an object that allocates things and then deallocates automatically when the object is no longer in use, is almost impossible in Python, unfortunately. The del statement is not guaranteed to be called, so you can't rely on that.
The standard way in Python is simply:
try:
allocate()
dostuff()
finally:
cleanup()
Or since 2.5 you can also create context-managers and use the with statement, which is a neater way of doing that.
But both of these are primarily for when you allocate/lock in the beginning of a code snippet. If you want to have things allocated for the whole run of the program, you need to allocate the resource at startup, before the main code of the program runs, and deallocate afterwards. There is one situation which isn't covered here, and that is when you want to allocate and deallocate many resources dynamically and use them in many places in the code. For example of you want a pool of memory buffers or similar. But most of those cases are for memory, which Python will handle for you, so you don't have to bother about those. There are of course cases where you want to have dynamic pool allocation of things that are NOT memory, and then you would want the type of deallocation you try in your example, and that is tricky to do with Python.
If weakrefs aren't broken, I guess this may work:
from weakref import ref
pointers = set()
class Pointer(object):
def __init__(self, cfun, ptr):
pointers.add(self)
self.ref = ref(ptr, self.cleanup)
self.data = cast(ptr, c_void_p).value # python cast it so smart, but it can't be smarter than this.
self.cfun = cfun
def cleanup(self, obj):
print 'cleanup 0x%x' % self.data
self.cfun(self.data)
pointers.remove(self)
def cleanup(cfun, ptr):
Pointer(cfun, ptr)
I yet try it. The important piece is that the Pointer doesn't have any strong references to the foreign pointer, except an integer. This should work if ctypes doesn't free memory that I should free with the bindings. Yeah, it's basicly a hack, but I think it may work better than the earlier things I've been trying.
Edit: Tried it, and it seem to work after small finetuning my code. A surprising thing is that even if I got del out from all of my structures, it seem to still fail. Interesting but frustrating.
Neither works, from some weird chance I've been able to drop away cyclic references in places, but things stay broke.
Edit: Well.. weakrefs WERE broken after all! so there's likely no solution for reliable cleanup in python, except than forcing it being explicit.
In CPython, __del__ is a reliable destructor of an object, because it will always be called when the reference count reaches zero (note: there may be cases - like circular references of items with __del__ method defined - where the reference count will never reaches zero, but that is another issue).
Update
From the comments, I understand the problem is related to the order of destruction of objects: body is a global object, and it is being destroyed before all other objects, thus it is no longer available to them.
Actually, using global objects is not good; not only because of issues like this one, but also because of maintenance.
I would then change your class with something like this
class Body(object):
def __init__(self, mass, inertia):
self._bodyref = body
self._body = body.New(mass, inertia)
def __del__(self):
print '__del__ %r' % self
if body:
body.Free(self._body)
...
def set_velocity_func(self, func):
self._body.contents.velocity_func = ctypes_wrapping(func)
A couple of notes:
The change is only adding a reference to the global body object, that thus will live at least as much as all the objects derived from that class.
Still, using a global object is not good because of unit testing and maintenance; better would be to have a factory for the object, that will set the correct "body" to the class, and in case of unit test will easily put a mock object. But that's really up to you and how much effort you think makes sense in this project.