I want to know when the Obj.del() method
is going to be called.
def my_integers():
Obj = ExitObj()
i = 0
while(1):
yield i
i += 1
def test_fun():
for i in my_integers():
if i > 3:
break
anything_function()
test_fun()
I did a test and Obj appeared to be deleted just after the break statement: before the anything_function() out the loop.
Can I rely on this and give some tasks that I want to be done when the loop is left to the __ del__ method of the object defined inside the generator?
I want to know when the Obj.__del__() method is going to be called.
You can't. It might never. Finalizers in Python (or in any environment with an automated garbage collector scheme, really) are not guaranteed to run at all, and should only be used for last-resort cleanup. If you want predictable lifetime management, use with statement and context managers.
class Foo(object):
def __enter__(self):
print 'Entered with block'
def __exit__(self, *exc_info):
print 'Exited with block'
return False
with Foo():
pass
In general you can't rely on destructor calling order. Destructors are called when the garbage collector reclaims the object. This can happen in indefinite future, or not happen at all if your program dies on an exception.
If you want a deterministric life cycle for your object, consider creating it inside a #contextmanager-decorated function and employing the with statement.
Here is what Python Language reference has to say bout object.__del__(self),
x.__del__() — ... is only called when x‘s reference count reaches zero. Some common situations that may prevent the reference count of an object from going to zero include: circular references between objects; a reference to the object on the stack frame of a function that caught an exception ...
So you should not rely on __del__ for cleanup. Context manager (as Cat Plus Plus mentions above), is the right choice.
Related
I saw a class in which a __del__ method is defined. This method is used to destroy an instance of the class. However, I cannot find a place where this method is used. How is this method used? Like that: obj1.del()?.
How do I call the __del__ method?
__del__ is a finalizer. It is called when an object is garbage collected which happens at some point after all references to the object have been deleted.
In a simple case this could be right after you say del x or, if x is a local variable, after the function ends. In particular, unless there are circular references, CPython (the standard Python implementation) will garbage collect immediately.*
However, this is an implementation detail of CPython. The only required property of Python garbage collection is that it happens after all references have been deleted, so this might not necessary happen right after and might not happen at all.
Even more, variables can live for a long time for many reasons, e.g. a propagating exception or module introspection can keep variable reference count greater than 0. Also, variable can be a part of cycle of references — CPython with garbage collection turned on breaks most, but not all, such cycles, and even then only periodically.
Since you have no guarantee it's executed, one should never put the code that you need to be run into __del__() — instead, this code belongs to the finally clause of a try statement or to a context manager in a with statement. However, there are valid use cases for __del__: e.g. if an object X references Y and also keeps a copy of Y reference in a global cache (cache['X -> Y'] = Y) then it would be polite for X.__del__ to also delete the cache entry.
If you know that the destructor provides (in violation of the above guideline) a required cleanup, you might want to call it directly, since there is nothing special about it as a method: x.__del__(). Obviously, you should only do so if you know it can be called twice. Or, as a last resort, you can redefine this method using
type(x).__del__ = my_safe_cleanup_method
* Reference:
CPython implementation detail: CPython currently uses a reference-counting scheme with (optional) delayed detection of cyclically linked garbage, which collects most objects as soon as they become unreachable [...] Other implementations act differently and CPython may change.
I wrote up the answer for another question, though this is a more accurate question for it.
How do constructors and destructors work?
Here is a slightly opinionated answer.
Don't use __del__. This is not C++ or a language built for destructors. The __del__ method really should be gone in Python 3.x, though I'm sure someone will find a use case that makes sense. If you need to use __del__, be aware of the basic limitations per http://docs.python.org/reference/datamodel.html:
__del__ is called when the garbage collector happens to be collecting the objects, not when you lose the last reference to an object and not when you execute del object.
__del__ is responsible for calling any __del__ in a superclass, though it is not clear if this is in method resolution order (MRO) or just calling each superclass.
Having a __del__ means that the garbage collector gives up on detecting and cleaning any cyclic links, such as losing the last reference to a linked list. You can get a list of the objects ignored from gc.garbage. You can sometimes use weak references to avoid the cycle altogether. This gets debated now and then: see http://mail.python.org/pipermail/python-ideas/2009-October/006194.html.
The __del__ function can cheat, saving a reference to an object, and stopping the garbage collection.
Exceptions explicitly raised in __del__ are ignored.
__del__ complements __new__ far more than __init__. This gets confusing. See http://www.algorithm.co.il/blogs/programming/python-gotchas-1-del-is-not-the-opposite-of-init/ for an explanation and gotchas.
__del__ is not a "well-loved" child in Python. You will notice that sys.exit() documentation does not specify if garbage is collected before exiting, and there are lots of odd issues. Calling the __del__ on globals causes odd ordering issues, e.g., http://bugs.python.org/issue5099. Should __del__ called even if the __init__ fails? See http://mail.python.org/pipermail/python-dev/2000-March/thread.html#2423 for a long thread.
But, on the other hand:
__del__ means you do not forget to call a close statement. See http://eli.thegreenplace.net/2009/06/12/safely-using-destructors-in-python/ for a pro __del__ viewpoint. This is usually about freeing ctypes or some other special resource.
And my pesonal reason for not liking the __del__ function.
Everytime someone brings up __del__ it devolves into thirty messages of confusion.
It breaks these items in the Zen of Python:
Simple is better than complicated.
Special cases aren't special enough to break the rules.
Errors should never pass silently.
In the face of ambiguity, refuse the temptation to guess.
There should be one – and preferably only one – obvious way to do it.
If the implementation is hard to explain, it's a bad idea.
So, find a reason not to use __del__.
The __del__ method, it will be called when the object is garbage collected. Note that it isn't necessarily guaranteed to be called though. The following code by itself won't necessarily do it:
del obj
The reason being that del just decrements the reference count by one. If something else has a reference to the object, __del__ won't get called.
There are a few caveats to using __del__ though. Generally, they usually just aren't very useful. It sounds to me more like you want to use a close method or maybe a with statement.
See the python documentation on __del__ methods.
One other thing to note: __del__ methods can inhibit garbage collection if overused. In particular, a circular reference that has more than one object with a __del__ method won't get garbage collected. This is because the garbage collector doesn't know which one to call first. See the documentation on the gc module for more info.
The __del__ method (note spelling!) is called when your object is finally destroyed. Technically speaking (in cPython) that is when there are no more references to your object, ie when it goes out of scope.
If you want to delete your object and thus call the __del__ method use
del obj1
which will delete the object (provided there weren't any other references to it).
I suggest you write a small class like this
class T:
def __del__(self):
print "deleted"
And investigate in the python interpreter, eg
>>> a = T()
>>> del a
deleted
>>> a = T()
>>> b = a
>>> del b
>>> del a
deleted
>>> def fn():
... a = T()
... print "exiting fn"
...
>>> fn()
exiting fn
deleted
>>>
Note that jython and ironpython have different rules as to exactly when the object is deleted and __del__ is called. It isn't considered good practice to use __del__ though because of this and the fact that the object and its environment may be in an unknown state when it is called. It isn't absolutely guaranteed __del__ will be called either - the interpreter can exit in various ways without deleteting all objects.
As mentioned earlier, the __del__ functionality is somewhat unreliable. In cases where it might seem useful, consider using the __enter__ and __exit__ methods instead. This will give a behaviour similar to the with open() as f: pass syntax used for accessing files. __enter__ is automatically called when entering the scope of with, while __exit__ is automatically called when exiting it. See this question for more details.
Taking the following code for example, does return object from function lead to memory leak?
I'm very curious about what happens to the object handle after used by the function use_age.
class Demo(object):
def _get_mysql_handle(self):
handle = MySQLdb.connect(host=self.conf["host"],
port=self.conf["port"],
user=self.conf["user"],
passwd=self.conf["passwd"],
db=self.conf["db"])
return handle
def use_age(self):
cursor = self._get_mysql_handle().cursor()
if __name__ == "__main__":
demo = Demo()
demo.use_age()
No, that code won't lead to a memory leak.
CPython handles object lifetimes by reference counting. In your example the reference count drops back to 0 and the database connection object is deleted again.
The local name handle in _get_mysql_handle is one reference, it is dropped when _get_mysql_handle returns.
The stack holding the return value from self._get_mysql_handle() is another, it too is dropped when the expression result is completed.
.cursor() is a method, so it'll have another reference for the self argument to that method, until the method exits.
The return value from .cursor() probably stores a reference, it'll be dropped when the cursor itself is reaped. That then depends on the lifetime of the local cursor variable in the use_age() method. As a local it doesn't live beyond the use_age() function.
Other Python implementations use garbage collection strategies; Jython uses the Java runtime facilities, for example. The object may live a little longer, but won't 'leak'.
In Python versions < 3.4, you do need to watch out for creating circular references with custom classes that define a __del__ method. Those are the circular references that the gc module does not break. You can introspect such chains in the gc.garbage object.
Can anyone describe me why this code will print '2 1 0 done' instead of expected output '0 1 2 done'?
As i can understand, we have some anonymous variables creating during list comprehensions, and they are garbage-collected, using filo principle, on list comprehension uncoverage end.
But, they still are referenced in list aa, aren't they?
Why the second 'del a' is not calling del magic method in that case?
class A:
def __init__(self, name):
self.name = name
def __del__(self):
print self.name,
aa = [A(str(i)) for i in range(3)]
for a in aa:
del a
print 'done'
Also, advanced questions. Please look at http://codepad.org/niaUzGEy
Why there are 5 copies, 3 copies? Musn't this be 1 copy? Why 5 or 3? Dont know, thats why asking it ;)
Thanks for your time!
You are confusing the del statement and the __del__ method.
del a simply unbinds the name a from whatever object it referenced. The list referenced by aa is unchanged so the objects all continue to exist.
The __del__ method is only called after the last reference to an object has been destroyed. That could be after a call to __del__ but usually isn't.
You rarely need to use del. It would be much more common just to rebind aa and then all the objects it contains will be released, and if not otherwise referenced their __del__ methods will be called automatically.
Also, you rarely need to use __del__. For most purposes Python's management of objects will handle cleanup automatically. Adding a __del__ method to a class is generally a bad idea as it can interfere with the garbage collector, so rather paradoxically __del__ makes it more likely that your program will leak memory. Also Python won't guarantee whether __del__ is actually called on program exit, and if it does you may find global variables you cant to use no longer exist, nor will it guarantee to only call it once (though you have to jump through hoops to make it call it more than once).
In short, avoid using __del__ if you possibly can.
It prints done 2 1 0(CPython).
You don't delete list elements in a for loop. They are deleted on exit. As far as I know call order of __del__ is implementation-specific, so it can be different in another implementations(IronPython, Jython etc.)
I'm trying to understand this code:
class Person:
'''Represents a person '''
population = 0
def __init__(self,name):
//some statements and population += 1
def __del__(self):
//some statements and population -= 1
def sayHi(self):
'''grettings from person'''
print 'Hi My name is %s' % self.name
def howMany(self):
'''Prints the current population'''
if Person.population == 1:
print 'i am the only one here'
else:
print 'There are still %d guyz left ' % Person.population
rohan = Person('Rohan')
rohan.sayHi()
rohan.howMany()
sanju = Person('Sanjivi')
sanju.howMany()
del rohan # am i doing this correctly?
How does the destructor get invoked -- automatically or do I have to add something in the "main" program/class like above?
Output:
Initializing person data
******************************************
Initializing Rohan
******************************************
Population now is: 1
Hi My name is Rohan
i am the only one here
Initializing person data
******************************************
Initializing Sanjivi
******************************************
Population now is: 2
In case Person dies:
******************************************
Sanjivi Bye Bye world
there are still 1 people left
i am the only one here
In case Person dies:
******************************************
Rohan Bye Bye world
i am the last person on earth
Population now is: 0
If required I can paste the whole lesson as well. I'm learning from:
http://www.ibiblio.org/swaroopch/byteofpython/read/
Here is a slightly opinionated answer.
Don't use __del__. This is not C++ or a language built for destructors. The __del__ method really should be gone in Python 3.x, though I'm sure someone will find a use case that makes sense. If you need to use __del __, be aware of the basic limitations per http://docs.python.org/reference/datamodel.html:
__del__ is called when the garbage collector happens to be collecting the objects, not when you lose the last reference to an object and not when you execution del object.
__del__ is responsible for calling any __del__ in a superclass, though it is not
clear if this is in method resolution order (MRO) or just calling each superclass.
Having a __del__ means that the garbage collector gives up on detecting and cleaning any cyclic links, such as losing the last reference to a linked list. You can get a list of the objects ignored from gc.garbage. You can sometimes use weak references to avoid the cycle altogether. This gets debated now and then: see http://mail.python.org/pipermail/python-ideas/2009-October/006194.html.
The __del__ function can cheat, saving a reference to an object, and stopping the garbage collection.
Exceptions explicitly raised in __del__ are ignored.
__del__ complements __new__ far more than __init__. This gets confusing. See http://www.algorithm.co.il/blogs/index.php/programming/python/python-gotchas-1-del-is-not-the-opposite-of-init/ for an explanation and gotchas.
__del__ is not a "well-loved" child in Python. You will notice that sys.exit() documentation does not specify if garbage is collected before exiting, and there are lots of odd issues. Calling the __del__ on globals causes odd ordering issues, e.g., http://bugs.python.org/issue5099. Should __del__ called even if the __init__ fails? See http://mail.python.org/pipermail/python-dev/2000-March/thread.html#2423 for a long thread.
But, on the other hand:
__del__ means you do not forget to call a close statement. See http://eli.thegreenplace.net/2009/06/12/safely-using-destructors-in-python/ for a pro __del__ viewpoint. This is usually about freeing ctypes or some other special resource.
And my pesonal reason for not liking the __del__ function.
Everytime someone brings up __del__ it devolves into thirty messages of confusion.
It breaks these items in the Zen of Python:
Complex is better than complicated.
Special cases aren't special enough to break the rules.
Errors should never pass silently.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
If the implementation is hard to explain, it's a bad idea.
So, find a reason not to use __del__.
As I understood them from my early CPTS experiance:
Constructors: Constructors are mainly used in classes to initialze the class with values, and gives an oppurtunity to do some background work based on creation. If you pass in values during the creation of an object, this is where you can handle assignment of those values to variables within your class. (In this case, upon construction you are incrementing a variable that keeps track of population).
Destructors: Destructors cleanup a class. In python, due to the garbage collector it's not as important as languages that can leave hanging pointers (c++). (In this case you are decrementing the population variable on destruction of the object).
class A:
def __get(self):
return self._x
def __set(self, y):
self._x = y
def __delete_x(self):
print('DELETING')
del self._x
x = property(__get,__set,__delete_x)
b = A()
# Here, when b is deleted, i'd like b.x to be deleted, i.e __delete_x()
# called (and for immediate consequence, "DELETING" printed)
del b
The semantics of the del statement don't really lend themselves to what you want here. del b simple removes the reference to the A object you just instantiated from the local scope frame / dictionary; this does not directly cause any operation to be performed on the object itself. If that was the last reference to the object, then the reference count dropping to zero, or the garbage collector collecting a cycle, may cause the object to be deallocated. You could observe this by adding a __del__ method to the object, or by adding a weakref callback that performs the desired actions.
Neither of the latter two solutions seems like a great idea, though; __del__ methods prevent the garbage collector from collecting any cycles involving the object; and while weakrefs do not suffer from this problem, in either case you may be running in a strange environment (such as during program shutdown), which may make it difficult to get done what you want to accomplish.
If you can expand on your exact use case, it may be that there is an entirely different approach to accomplishing your desired end goal, but it is difficult to speculate based on such a general and limited example.
To control what happens when an instance of class A goes away (whether by being deleted or garbage collected), you can implement special method __del__(self) in A. If you want to have your code involved when a specific attribute of that instance goes away, you can either wrap that attribute with a wrapper class which has __del__, or, probably better in most cases, use the weakref module (however, not all types are subject to being target of weak references, so you may also need some wrapping for this case).
Avoiding __del__ is generally preferable, if you possibly can, because it can interfere with garbage collection and thereby cause "memory leaks" if and when you have circular references.
An ugly way to do it would be :
def __del__(self):
for x in dir(self.__class__):
if type(getattr(self.__class__, x)) == property:
getattr(self.__class__, x).fdel(self)