Develop Reference

ModuleNotFoundError: import of time halted; None in sys.modules occured when Object gets out of scope(__del__) [duplicate]

I saw a class in which a __del__ method is defined. This method is used to destroy an instance of the class. However, I cannot find a place where this method is used. How is this method used? Like that: obj1.del()?.
How do I call the __del__ method?

__del__ is a finalizer. It is called when an object is garbage collected which happens at some point after all references to the object have been deleted.
In a simple case this could be right after you say del x or, if x is a local variable, after the function ends. In particular, unless there are circular references, CPython (the standard Python implementation) will garbage collect immediately.*
However, this is an implementation detail of CPython. The only required property of Python garbage collection is that it happens after all references have been deleted, so this might not necessary happen right after and might not happen at all.
Even more, variables can live for a long time for many reasons, e.g. a propagating exception or module introspection can keep variable reference count greater than 0. Also, variable can be a part of cycle of references — CPython with garbage collection turned on breaks most, but not all, such cycles, and even then only periodically.
Since you have no guarantee it's executed, one should never put the code that you need to be run into __del__() — instead, this code belongs to the finally clause of a try statement or to a context manager in a with statement. However, there are valid use cases for __del__: e.g. if an object X references Y and also keeps a copy of Y reference in a global cache (cache['X -> Y'] = Y) then it would be polite for X.__del__ to also delete the cache entry.
If you know that the destructor provides (in violation of the above guideline) a required cleanup, you might want to call it directly, since there is nothing special about it as a method: x.__del__(). Obviously, you should only do so if you know it can be called twice. Or, as a last resort, you can redefine this method using
type(x).__del__ = my_safe_cleanup_method
* Reference:
CPython implementation detail: CPython currently uses a reference-counting scheme with (optional) delayed detection of cyclically linked garbage, which collects most objects as soon as they become unreachable [...] Other implementations act differently and CPython may change.

I wrote up the answer for another question, though this is a more accurate question for it.
How do constructors and destructors work?
Here is a slightly opinionated answer.
Don't use __del__. This is not C++ or a language built for destructors. The __del__ method really should be gone in Python 3.x, though I'm sure someone will find a use case that makes sense. If you need to use __del__, be aware of the basic limitations per http://docs.python.org/reference/datamodel.html:
__del__ is called when the garbage collector happens to be collecting the objects, not when you lose the last reference to an object and not when you execute del object.
__del__ is responsible for calling any __del__ in a superclass, though it is not clear if this is in method resolution order (MRO) or just calling each superclass.
Having a __del__ means that the garbage collector gives up on detecting and cleaning any cyclic links, such as losing the last reference to a linked list. You can get a list of the objects ignored from gc.garbage. You can sometimes use weak references to avoid the cycle altogether. This gets debated now and then: see http://mail.python.org/pipermail/python-ideas/2009-October/006194.html.
The __del__ function can cheat, saving a reference to an object, and stopping the garbage collection.
Exceptions explicitly raised in __del__ are ignored.
__del__ complements __new__ far more than __init__. This gets confusing. See http://www.algorithm.co.il/blogs/programming/python-gotchas-1-del-is-not-the-opposite-of-init/ for an explanation and gotchas.
__del__ is not a "well-loved" child in Python. You will notice that sys.exit() documentation does not specify if garbage is collected before exiting, and there are lots of odd issues. Calling the __del__ on globals causes odd ordering issues, e.g., http://bugs.python.org/issue5099. Should __del__ called even if the __init__ fails? See http://mail.python.org/pipermail/python-dev/2000-March/thread.html#2423 for a long thread.
But, on the other hand:
__del__ means you do not forget to call a close statement. See http://eli.thegreenplace.net/2009/06/12/safely-using-destructors-in-python/ for a pro __del__ viewpoint. This is usually about freeing ctypes or some other special resource.
And my pesonal reason for not liking the __del__ function.
Everytime someone brings up __del__ it devolves into thirty messages of confusion.
It breaks these items in the Zen of Python:
Simple is better than complicated.
Special cases aren't special enough to break the rules.
Errors should never pass silently.
In the face of ambiguity, refuse the temptation to guess.
There should be one – and preferably only one – obvious way to do it.
If the implementation is hard to explain, it's a bad idea.
So, find a reason not to use __del__.

The __del__ method, it will be called when the object is garbage collected. Note that it isn't necessarily guaranteed to be called though. The following code by itself won't necessarily do it:
del obj
The reason being that del just decrements the reference count by one. If something else has a reference to the object, __del__ won't get called.
There are a few caveats to using __del__ though. Generally, they usually just aren't very useful. It sounds to me more like you want to use a close method or maybe a with statement.
See the python documentation on __del__ methods.
One other thing to note: __del__ methods can inhibit garbage collection if overused. In particular, a circular reference that has more than one object with a __del__ method won't get garbage collected. This is because the garbage collector doesn't know which one to call first. See the documentation on the gc module for more info.

The __del__ method (note spelling!) is called when your object is finally destroyed. Technically speaking (in cPython) that is when there are no more references to your object, ie when it goes out of scope.
If you want to delete your object and thus call the __del__ method use
del obj1
which will delete the object (provided there weren't any other references to it).
I suggest you write a small class like this
class T:
def __del__(self):
print "deleted"
And investigate in the python interpreter, eg
>>> a = T()
>>> del a
deleted
>>> a = T()
>>> b = a
>>> del b
>>> del a
deleted
>>> def fn():
... a = T()
... print "exiting fn"
...
>>> fn()
exiting fn
deleted
>>>
Note that jython and ironpython have different rules as to exactly when the object is deleted and __del__ is called. It isn't considered good practice to use __del__ though because of this and the fact that the object and its environment may be in an unknown state when it is called. It isn't absolutely guaranteed __del__ will be called either - the interpreter can exit in various ways without deleteting all objects.

As mentioned earlier, the __del__ functionality is somewhat unreliable. In cases where it might seem useful, consider using the __enter__ and __exit__ methods instead. This will give a behaviour similar to the with open() as f: pass syntax used for accessing files. __enter__ is automatically called when entering the scope of with, while __exit__ is automatically called when exiting it. See this question for more details.

Call destructor of an element from a list

I have something like that:
a = [instance1, instance2, ...]
if I do a
del a[1]
instance2 is removed from list, but is instance2 desctructor method called?
I'm interested in this because my code uses a lot of memory and I need to free memory deleting instances from a list.

Coming from a language like c++ (as I did), this tends to be a subject many people find difficult to grasp when first learning Python.
The bottomline is this: when you do del XXX, you are never* deleting an object when you use del. You are only deleting an object reference. However, in practice, assuming there are no other references laying about to the instance2 object, deleting it from your list will free the memory as you desire.
If you don't understand the difference between an object and an object reference, read on.
Python: Pass by value, or pass by reference?
You are likely familiar with the concept of passing arguments to a function by reference, or by value. However, Python does things differently. Arguments are always passed by object reference. Read this article for a helpful explanation of what this means.
To summarize: this means that when you pass a variable to a function, you are not passing a copy of the value of the variable (pass by value), nor are you passing the object itself - i.e., the address of the value in memory. You are passing the name-object that indirectly refers to the value held in memory.
What does this have to do with del...?
Well, I'll tell you.
Say you do this:
def deleteit(thing):
del thing
mylist = [1,2,3]
deleteit(mylist)
...what do you think will happen? Has mylist been deleted from the global namespace?
The answer is NO:
assert mylist # No error
The reason is that when you do del thing in the deleteit function, you are only deleting a local object reference to the object. That object reference exists ONLY inside of the function. As a sidebar, you might ask: is it possible to delete an object reference from the global namespace while inside a function? The answer is yes, but you have to declare the object reference to be part of the global namespace first:
def deletemylist():
global mylist
del mylist
mylist = [1,2,3]
deletemylist()
assert mylist #ERROR, as expected
Putting it all together
Now to get back to your original question. When, in ANY namespace, you do this:
del XXX
...you have NOT deleted the object signified by XXX. You CAN'T do that. You have only deleted the object reference XXX, which refers to some object in memory. The object itself is managed by the Python memory manager. This is a very important distinction.
Note that as a consequence, when you override the __del__ method of some object, which gets called when the object is deleted (NOT the object reference!):
class MyClass():
def __del__(self):
print(self, "deleted")
super().__del__()
m = MyClass()
del m
...the print statement inside the __del__ method does not necessarily occur immediately after you do del m. It only occurs at the point in time the object itself is deleted, and that is not up to you. It is up to the Python memory manager. When all object references in all the namespaces have been deleted, the __del__ method will eventually be executed. But not necessarily immediately.
The same is true when you delete an object reference that is part of a list, like in the original example. When you do del a[1], only the object reference to the object signified by a[1] is deleted, and the __del__ method of that object may or may not be called immediately (though as stated before, it will eventually be called once there are no more references to the object, and the object is garbage collected by the memory manager).
As a result of this, it is not recommended that you put things in the __del__ method that you want to happen immediately upon del mything, because it may not happen that way.
*I believe it is never. Inevitably someone will likely downvote my answer and leave a comment discussing the exception to the rule. But whatevs.

No. Calling del on a list element only removes a reference to an object from the list, it doesn't do anything (explicitly) to the object itself. However: If the reference in the list was the last one referring to the object, the object can now be destroyed and recycled. I think that the "normal" CPython implementation will immediately destroy and recycle the object, other variants' behaviour can vary.

If your object is resource-heavy and you want to be sure that the resources are freed correctly, use the with() construct. It's very easy to leak resources when relying on destructors. See this SO post for more details.

How do constructors and destructors work?

I'm trying to understand this code:
class Person:
'''Represents a person '''
population = 0
def __init__(self,name):
//some statements and population += 1
def __del__(self):
//some statements and population -= 1
def sayHi(self):
'''grettings from person'''
print 'Hi My name is %s' % self.name
def howMany(self):
'''Prints the current population'''
if Person.population == 1:
print 'i am the only one here'
else:
print 'There are still %d guyz left ' % Person.population
rohan = Person('Rohan')
rohan.sayHi()
rohan.howMany()
sanju = Person('Sanjivi')
sanju.howMany()
del rohan # am i doing this correctly?
How does the destructor get invoked -- automatically or do I have to add something in the "main" program/class like above?
Output:
Initializing person data
******************************************
Initializing Rohan
******************************************
Population now is: 1
Hi My name is Rohan
i am the only one here
Initializing person data
******************************************
Initializing Sanjivi
******************************************
Population now is: 2
In case Person dies:
******************************************
Sanjivi Bye Bye world
there are still 1 people left
i am the only one here
In case Person dies:
******************************************
Rohan Bye Bye world
i am the last person on earth
Population now is: 0
If required I can paste the whole lesson as well. I'm learning from:
http://www.ibiblio.org/swaroopch/byteofpython/read/

Here is a slightly opinionated answer.
Don't use __del__. This is not C++ or a language built for destructors. The __del__ method really should be gone in Python 3.x, though I'm sure someone will find a use case that makes sense. If you need to use __del __, be aware of the basic limitations per http://docs.python.org/reference/datamodel.html:
__del__ is called when the garbage collector happens to be collecting the objects, not when you lose the last reference to an object and not when you execution del object.
__del__ is responsible for calling any __del__ in a superclass, though it is not
clear if this is in method resolution order (MRO) or just calling each superclass.
Having a __del__ means that the garbage collector gives up on detecting and cleaning any cyclic links, such as losing the last reference to a linked list. You can get a list of the objects ignored from gc.garbage. You can sometimes use weak references to avoid the cycle altogether. This gets debated now and then: see http://mail.python.org/pipermail/python-ideas/2009-October/006194.html.
The __del__ function can cheat, saving a reference to an object, and stopping the garbage collection.
Exceptions explicitly raised in __del__ are ignored.
__del__ complements __new__ far more than __init__. This gets confusing. See http://www.algorithm.co.il/blogs/index.php/programming/python/python-gotchas-1-del-is-not-the-opposite-of-init/ for an explanation and gotchas.
__del__ is not a "well-loved" child in Python. You will notice that sys.exit() documentation does not specify if garbage is collected before exiting, and there are lots of odd issues. Calling the __del__ on globals causes odd ordering issues, e.g., http://bugs.python.org/issue5099. Should __del__ called even if the __init__ fails? See http://mail.python.org/pipermail/python-dev/2000-March/thread.html#2423 for a long thread.
But, on the other hand:
__del__ means you do not forget to call a close statement. See http://eli.thegreenplace.net/2009/06/12/safely-using-destructors-in-python/ for a pro __del__ viewpoint. This is usually about freeing ctypes or some other special resource.
And my pesonal reason for not liking the __del__ function.
Everytime someone brings up __del__ it devolves into thirty messages of confusion.
It breaks these items in the Zen of Python:
Complex is better than complicated.
Special cases aren't special enough to break the rules.
Errors should never pass silently.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
If the implementation is hard to explain, it's a bad idea.
So, find a reason not to use __del__.

As I understood them from my early CPTS experiance:
Constructors: Constructors are mainly used in classes to initialze the class with values, and gives an oppurtunity to do some background work based on creation. If you pass in values during the creation of an object, this is where you can handle assignment of those values to variables within your class. (In this case, upon construction you are incrementing a variable that keeps track of population).
Destructors: Destructors cleanup a class. In python, due to the garbage collector it's not as important as languages that can leave hanging pointers (c++). (In this case you are decrementing the population variable on destruction of the object).

Deleting attributes when deleting instance

class A:
def __get(self):
return self._x
def __set(self, y):
self._x = y
def __delete_x(self):
print('DELETING')
del self._x
x = property(__get,__set,__delete_x)
b = A()
# Here, when b is deleted, i'd like b.x to be deleted, i.e __delete_x()
# called (and for immediate consequence, "DELETING" printed)
del b

The semantics of the del statement don't really lend themselves to what you want here. del b simple removes the reference to the A object you just instantiated from the local scope frame / dictionary; this does not directly cause any operation to be performed on the object itself. If that was the last reference to the object, then the reference count dropping to zero, or the garbage collector collecting a cycle, may cause the object to be deallocated. You could observe this by adding a __del__ method to the object, or by adding a weakref callback that performs the desired actions.
Neither of the latter two solutions seems like a great idea, though; __del__ methods prevent the garbage collector from collecting any cycles involving the object; and while weakrefs do not suffer from this problem, in either case you may be running in a strange environment (such as during program shutdown), which may make it difficult to get done what you want to accomplish.
If you can expand on your exact use case, it may be that there is an entirely different approach to accomplishing your desired end goal, but it is difficult to speculate based on such a general and limited example.

To control what happens when an instance of class A goes away (whether by being deleted or garbage collected), you can implement special method __del__(self) in A. If you want to have your code involved when a specific attribute of that instance goes away, you can either wrap that attribute with a wrapper class which has __del__, or, probably better in most cases, use the weakref module (however, not all types are subject to being target of weak references, so you may also need some wrapping for this case).
Avoiding __del__ is generally preferable, if you possibly can, because it can interfere with garbage collection and thereby cause "memory leaks" if and when you have circular references.

An ugly way to do it would be :
def __del__(self):
for x in dir(self.__class__):
if type(getattr(self.__class__, x)) == property:
getattr(self.__class__, x).fdel(self)

Using explicit del in python on local variables

What are the best practices and recommendations for using explicit del statement in python? I understand that it is used to remove attributes or dictionary/list elements and so on, but sometimes I see it used on local variables in code like this:
def action(x):
result = None
something = produce_something(x)
if something:
qux = foo(something)
result = bar(qux, something)
del qux
del something
return result
Are there any serious reasons for writing code like this?
Edit: consider qux and something to be something "simple" without a __del__ method.

I don't remember when I last used del -- the need for it is rare indeed, and typically limited to such tasks as cleaning up a module's namespace after a needed import or the like.
In particular, it's not true, as another (now-deleted) answer claimed, that
Using del is the only way to make sure
a object's __del__ method is called
and it's very important to understand this. To help, let's make a class with a __del__ and check when it is called:
>>> class visdel(object):
... def __del__(self): print 'del', id(self)
...
>>> d = visdel()
>>> a = list()
>>> a.append(d)
>>> del d
>>>
See? del doesn't "make sure" that __del__ gets called: del removes one reference, and only the removal of the last reference causes __del__ to be called. So, also:
>>> a.append(visdel())
>>> a[:]=[1, 2, 3]
del 550864
del 551184
when the last reference does go away (including in ways that don't involve del, such as a slice assignment as in this case, or other rebindings of names and other slots), then __del__ gets called -- whether del was ever involved in reducing the object's references, or not, makes absolutely no difference whatsoever.
So, unless you specifically need to clean up a namespace (typically a module's namespace, but conceivably that of a class or instance) for some specific reason, don't bother with del (it can be occasionally handy for removing an item from a container, but I've found that I'm often using the container's pop method or item or slice assignment even for that!-).

No.
I'm sure someone will come up with some silly reason to do this, e.g. to make sure someone doesn't accidentally use the variable after it's no longer valid. But probably whoever wrote this code was just confused. You can remove them.

When you are running programs handling really large amounts of data ( to my experience when the totals memory consumption of the program approaches something like 1GB) deleting some objects:
del largeObject1
del largeObject2
…
can give your program the necessary breathing room to function without running out of memory. This can be the easiest way to modify a given program, in case of a “MemoryError” runtime error.

Actually, I just came across a use for this. If you use locals() to return a dictionary of local variables (useful when parsing things) then del is useful to get rid of a temporary that you don't want to return.