>>> class S(object):
... def __init__(self):
... self.x = 1
... def x(self):
... return self.x
...
>>> s = S()
>>> s.x
1
>>> s.x()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'int' object is not callable
Why, in this example, is s.x a method, but also an integer? It seems to me that self.x = 1 should replace the def x(self): declaration of the attribute x during instantiation. Why is it that I can get and call, resulting in an integer and a method, respectively, the same attribute? My guess is that the variable look-up pattern in new-style classes is duck typed, so as to return the most relevant result to the caller. I would love to hear the whole story.
Python doesn't use separate spaces for callable and non-callable objects: a name is a name is a name. s.x, by Python rules, must refer to exactly the same object, whether you're going to call it or not. Another way of putting it: assuming that _aux is a name not otherwise used,
_aux = self.x
_aux()
and
self.x()
must have absolutely identical semantics in Python, the fact that in the former the intermediate value self.x is being bound to a name and called later notwithstanding.
Having single, "unified" namespaces for callables and non-callables has a huge number of advantages -- it makes name-lookup rules (for each of bare and qualified names) enormously simpler, for example, by decoupling them totally from the purpose to which the name being looked up is going to be put (be it immediately after the lookup's result, or later still), and also from the type (callable or non-callable) of whatever object turns up to be first referenced according to the lookup rules.
Especially considering how many different callable types Python has (functions, classes, instances of classes which define __call__, special types such as staticmethod and classmethod, ...!-), any other rule could only lead to total chaos. (Note also, for example, that even C++, a language which definitely is not afraid by complexity but which also lets class-instances be callable [[if the class overloads operator()]], uses a similar unified-namespace rule -- again, discriminating between callables and non-callables would be a totally unwarranted nightmare, were the rules any different in this regard!-).
It looks like you're having a misunderstanding of the error you're seeing. When your s object is instantiated, its constructor replaces the method x by an integer, so in the s object, x is an integer, not a function. Trying to call it as a method results in an exception being thrown.
Python is duck-typed in the sense that method calls are resolved at runtime - the compiler has no problem with s.x() because x might have been created as a method dynamically. However, when the interpreter actually calls x as a method, it notices x is an integer and can't be called, hence the TypeError.
I'm not sure what you think is going on, but there's nothing that tricky happening. When you assign self.x = 1, the method x is no longer accessible. From that point forward, s.x is only an integer -- attempts to call it as a method result in an exception, as you saw.
It seems that the x property is defined as a method in the class definition. However, actually instantiating an object overwrites that name with an integer - hence, the behavior observed. It's never actually two at once. So, this is basically some faulty code.
This is what your code is doing:
Create a class named S with 2 methods, __init__ and x
Create an instance of S and name it s
Call S.__init__ with s as parameter
Set s.x with the value 1
Print s.x
Print the result of calling s.x
Now, if you look in 2.1.1 you will see that you have overrided the method x with an integer, which means that you cannot call that again withing s (but it stills in S class)
If you have done that, and yet, need call x function, try it:
>>> class S(object):
... def __init__(self):
... self.x = 1
... def x(self):
... return self.x
...
>>> s = S()
>>> s.x
1
>>> S.x(s)
1
>>>
I just did it so you understand why you are losing the x as method, do it in the right way and avoid to have instances variables with the same name as class methods
Related
So, I was playing around with Python while answering this question, and I discovered that this is not valid:
o = object()
o.attr = 'hello'
due to an AttributeError: 'object' object has no attribute 'attr'. However, with any class inherited from object, it is valid:
class Sub(object):
pass
s = Sub()
s.attr = 'hello'
Printing s.attr displays 'hello' as expected. Why is this the case? What in the Python language specification specifies that you can't assign attributes to vanilla objects?
For other workarounds, see How can I create an object and add attributes to it?.
To support arbitrary attribute assignment, an object needs a __dict__: a dict associated with the object, where arbitrary attributes can be stored. Otherwise, there's nowhere to put new attributes.
An instance of object does not carry around a __dict__ -- if it did, before the horrible circular dependence problem (since dict, like most everything else, inherits from object;-), this would saddle every object in Python with a dict, which would mean an overhead of many bytes per object that currently doesn't have or need a dict (essentially, all objects that don't have arbitrarily assignable attributes don't have or need a dict).
For example, using the excellent pympler project (you can get it via svn from here), we can do some measurements...:
>>> from pympler import asizeof
>>> asizeof.asizeof({})
144
>>> asizeof.asizeof(23)
16
You wouldn't want every int to take up 144 bytes instead of just 16, right?-)
Now, when you make a class (inheriting from whatever), things change...:
>>> class dint(int): pass
...
>>> asizeof.asizeof(dint(23))
184
...the __dict__ is now added (plus, a little more overhead) -- so a dint instance can have arbitrary attributes, but you pay quite a space cost for that flexibility.
So what if you wanted ints with just one extra attribute foobar...? It's a rare need, but Python does offer a special mechanism for the purpose...
>>> class fint(int):
... __slots__ = 'foobar',
... def __init__(self, x): self.foobar=x+100
...
>>> asizeof.asizeof(fint(23))
80
...not quite as tiny as an int, mind you! (or even the two ints, one the self and one the self.foobar -- the second one can be reassigned), but surely much better than a dint.
When the class has the __slots__ special attribute (a sequence of strings), then the class statement (more precisely, the default metaclass, type) does not equip every instance of that class with a __dict__ (and therefore the ability to have arbitrary attributes), just a finite, rigid set of "slots" (basically places which can each hold one reference to some object) with the given names.
In exchange for the lost flexibility, you gain a lot of bytes per instance (probably meaningful only if you have zillions of instances gallivanting around, but, there are use cases for that).
As other answerers have said, an object does not have a __dict__. object is the base class of all types, including int or str. Thus whatever is provided by object will be a burden to them as well. Even something as simple as an optional __dict__ would need an extra pointer for each value; this would waste additional 4-8 bytes of memory for each object in the system, for a very limited utility.
Instead of doing an instance of a dummy class, in Python 3.3+, you can (and should) use types.SimpleNamespace for this.
It is simply due to optimization.
Dicts are relatively large.
>>> import sys
>>> sys.getsizeof((lambda:1).__dict__)
140
Most (maybe all) classes that are defined in C do not have a dict for optimization.
If you look at the source code you will see that there are many checks to see if the object has a dict or not.
So, investigating my own question, I discovered this about the Python language: you can inherit from things like int, and you see the same behaviour:
>>> class MyInt(int):
pass
>>> x = MyInt()
>>> print x
0
>>> x.hello = 4
>>> print x.hello
4
>>> x = x + 1
>>> print x
1
>>> print x.hello
Traceback (most recent call last):
File "<interactive input>", line 1, in <module>
AttributeError: 'int' object has no attribute 'hello'
I assume the error at the end is because the add function returns an int, so I'd have to override functions like __add__ and such in order to retain my custom attributes. But this all now makes sense to me (I think), when I think of "object" like "int".
https://docs.python.org/3/library/functions.html#object :
Note: object does not have a __dict__, so you can’t assign arbitrary attributes to an instance of the object class.
It's because object is a "type", not a class. In general, all classes that are defined in C extensions (like all the built in datatypes, and stuff like numpy arrays) do not allow addition of arbitrary attributes.
This is (IMO) one of the fundamental limitations with Python - you can't re-open classes. I believe the actual problem, though, is caused by the fact that classes implemented in C can't be modified at runtime... subclasses can, but not the base classes.
I have a test class:
class TestClass:
def someFunction(self, someInt):
self.i = someInt
return self.i
And was testing it with:
x = MyClass()
x.someFunction = 3
print(x.someFunction)
x.someFunction(4)
However, this leads to TypeError: 'int' object is not callable. I understand why I am getting this error--someFunction has been rebound to an integer (in this case, 3) and, well, an integer is not a function and therefore someFunction is no longer callable. But I am wondering what the reasoning is behind allowing a member function to be rebound just as if it is a data member? It seems prone to accidental errors. I am just starting to learn Python and feel like I'm missing a fundamental aspect of Python that this behavior would fit into.
This is allowed because of the way attribute lookup works in Python and this is by design. In Python, many things that are discouraged, forbidden or impossible in other languages, are allowed to leverage your use case (if used wisely). Of course, more power implies more responsibility.
After all, we're all consenting adults here.
Some background information on attribute resolution
Class instances start with an empty __dict__ attribute where all object attributes are stored. By accessing x.someFunction you are implicitly trying x.__dict__['someFunction']. If 'someFunction' does not exist in x.__dict__, the same is tried for the class of x, i.e. type(x).someFunction or better type(x).__dict__['someFunction'].
When your write x by doing x.someFunction = 3, what actually happens is x.__dict__['someFunction'] = 3, regardless of what the reading attribute access might return.
The only real (?) magic happens during method calls, where self is provided automatically, i.e. x.someFunction(4) is resolved to type(x).__dict__['someFunction'](x, 4) instead of type(x).__dict__['someFunction'](4) or x.__dict__['someFunction'](4). This is somewhat related to attribute access and may cause confusion.
So, you actually do not "rebind" the function, but hide the class attribute someFunction with the instance attribute someFunction of x. If you do
print(MyClass.someFunction)
print(MyClass().someFunction)
you will see that the method is still there. You can even restore the initial state with
del x.__dict__['someFunction']
Note: The things I described as resolution illustrate the concept. In reality the inner workings of python may be more subtle or complex, but they will have the same effect. For example, in Python 2, methods have been wrapped as so called unbound methods before being stored in the class dictionary. This has been dropped in Python 3, where the class dictionary contains the plain function.
Please execute your demo code in a python shell:
>>> class TestClass:
... def someFunction(self, someInt):
... self.i = someInt
... return self.i
...
>>> x = TestClass()
>>> x.someFunction = 3
>>> print(x.someFunction)
3
>>> x.someFunction(4)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'int' object is not callable
Please see the following content:
>>> type(x.someFunction)
<type 'instancemethod'>
>>> x.someFunction = 3
>>> type(x.someFunction)
<type 'int'>
If you try x.someFunction = 3, the instancemethod caled x.someFunction translates into a int.
In python
functions are threated as first-class objects
and variables are not typed.
Or just "Everything is an object" in python. So there is just no fundamental difference of a variable containing an integer vs a variable containing a function. Both, first-class functions and untyped variables have it's own pros and cons but allow you to e.g. simply make a list that contains integers, string, other objects as well as functions.
The reason you can assign anything to function variables is because they are implemented as objects. This enables you to return functions and pass functions as arguments, unlike in C.
The usefulness of this feature far surpasses the 'accidental errors'.
For example, you can do something like this:
# some code
def sum(a, b):
return a+b
def product(a, b):
return a*b
def ret_func(str):
if str=='add':
func = sum
elif str=='multiply':
func = product
return func
addition_result = ret_func('add')(x, y)
multiplication_result = ret_func('multiply')(x, y)
# some more code
I want
class MyClass(object):
_my_unique = ???? # if this were lisp, I would have written (cons nil nil) here
def myFunc (self, arg):
assert arg != _my_unique # this must never fail
...
What do use instead of ??? to ensure that the assert never fails?
(With Lisp, I could create _my_unique with (cons nil nil) and used eq in assert).
PS. Use case: I will put _my_unique in a dict, so I want it to be equal to itself, but I do not want it to be equal (in the dict collision sense) to anything passed in from the outside.
You can use object(), but this won't make the assert "never fail". It will still fail if you call myFunc and pass MyClass.my_unique as the object. Also, if you want to test whether it's the same object, use arg is not my_unique rather than !=.
You can use object().
Return a new featureless object. object is a base for all new style
classes. It has the methods that are common to all instances of new
style classes.
If what you're asking is "how do I make _my_unique unique for each instance of MyClass", you could simply create a new, empty object in the constructor.
For example:
>>> class MyClass(object):
... def __init__(self):
... self._my_unique = object()
...
>>> foo=MyClass()
>>> bar=MyClass()
>>> foo._my_unique
<object object at 0x100ce70c0>
>>> bar._my_unique
<object object at 0x100ce7090>
If you want to hide _my_unique, give it two underscores. This way, nobody can accidentally get at the value. It's not impossible, but they would need to work a little harder to get at the value. This is called name mangling, and you can read more about it here: http://docs.python.org/2/tutorial/classes.html#private-variables-and-class-local-references
>>> class MyClass(object):
... def __init__(self):
... self.__my_unique = object()
...
>>> foo=MyClass()
>>> foo.__my_unique
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'MyClass' object has no attribute '__my_unique'
This is a mildly confusing question, as it seems to mix a bunch of concepts without purpose. For one thing, most any object in Python is unique, but there may be multiple ways to reach them. The operator to check identities is not != (inequality) but is not. Unlike Java, Python does not require you to put things in a class, and does not implicitly look in self (which they know as implicit this). cons from Lisp is used to construct pairs which frequently form singly linked lists, a structure that's unusual in Python because we use dynamic reference arrays known as lists. Lists are mutable, and thus constructing one using list() or [] will produce a unique object.
So, while it is rather pointless, one way to write a function producing functions that perform the useless assert might be:
def createfunc():
mylist=[]
def subfunc(x):
assert x is not mylist
return subfunc
Each call to createfunc() returns a new function with its own mylist. However, the object being unique doesn't make it impossible to reach:
f=createfunc()
f(f.func_closure[0].cell_contents) # raises AssertionError
Regarding the PS, to be relevant to dict collissions, your object must also be hashable, which makes object() a better choice than list() for the unique object.
This works:
>>> def bar(x, y):
... print x, y
...
>>> bar(y=3, x=1)
1 3
And this works:
>>> class Foo(object):
... def bar(self, x, y):
... print x, y
...
>>> z = Foo()
>>> z.bar(y=3, x=1)
1 3
And even this works:
>>> Foo.bar(z, y=3, x=1)
1 3
But why doesn't this work in Python 2.x?
>>> Foo.bar(self=z, y=3, x=1)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: unbound method bar() must be called with Foo instance as first argument (got nothing instead)
This makes metaprogramming more difficult, because it requires special case handling. I'm curious if it's somehow necessary by Python's semantics or just an artifact of implementation.
z.bar is a bound method -- it already has an im_self attribute that becomes the first argument (conventionally named self) to the underlying function object, the bound method's im_func attribute. To override that you obviously need to re-bind im_self (edit: or call the im_func instead) -- whatever you do in terms of argument passing is not going to have any effect on it, of course. Yep, that's the documented way bound methods object work in Python (not just an implementation detail: every correct Python implementation has to do it exactly this way). So it's "necessary" in the sense that it's part of what makes Python exactly the language it is, as opposed to being a slighty or strongly different language. Of course you could design a different language that chooses to play by completely different rules, but -- it wouldn't be Python, of course.
Edit: the OP's edits clarified he's calling an unbound method, not a bound one. This still doesn't work, and the reason is clear from the error message the attempt gets:
TypeError: unbound method bar() must
be called with Foo instance as first
argument (got nothing instead)
The rule underlying this very clear error message is that the instance must be the first argument (so of course a positional one: named arguments have no ordering). The unbound method doesn't "know" (nor care) what that parameter's name may be (and the use of name self for it is only a convention, not a rule of the Python language): it only care about the unambiguous condition of "first argument" (among the positional ones, of course).
This obscure corner case could certainly be altered (with a Python 3.2 patch, if and when the language-changes "freeze" ends;-) by making unbound methods seriously more complicated: they'd have to introspect and save the first-argument's name at creation time, and check keyword arguments on each call just in case somebody's passing self by name instead of by position. I don't think this would break any existing, working code, it would only slow down just about every existing Python program. If you write and propose a patch implementing this complication, and get active on python-dev to advocate for it against the sure-to-come firestorm of opposition, you do no doubt stand a > 0 chance to ram it through -- good luck.
The rest of us, meanwhile, will keep getting the im_func attribute instead, as one absurdly-tiny extra step in what has to be a pretty complicated inded edifice of metaprogramming to warrant such a change -- it isn't a "special case" at all, compared with the horrid difficulties of adapting named-argument passing to builtins that don't take named arguments (and don't expose their "argument names" to easily allow the transformation of named arguments into positional ones (now that would be a windmill worth attacking, IMHO: of all callables, builtins are the worst to metaprogram about, because of that!-).
So, I was playing around with Python while answering this question, and I discovered that this is not valid:
o = object()
o.attr = 'hello'
due to an AttributeError: 'object' object has no attribute 'attr'. However, with any class inherited from object, it is valid:
class Sub(object):
pass
s = Sub()
s.attr = 'hello'
Printing s.attr displays 'hello' as expected. Why is this the case? What in the Python language specification specifies that you can't assign attributes to vanilla objects?
For other workarounds, see How can I create an object and add attributes to it?.
To support arbitrary attribute assignment, an object needs a __dict__: a dict associated with the object, where arbitrary attributes can be stored. Otherwise, there's nowhere to put new attributes.
An instance of object does not carry around a __dict__ -- if it did, before the horrible circular dependence problem (since dict, like most everything else, inherits from object;-), this would saddle every object in Python with a dict, which would mean an overhead of many bytes per object that currently doesn't have or need a dict (essentially, all objects that don't have arbitrarily assignable attributes don't have or need a dict).
For example, using the excellent pympler project (you can get it via svn from here), we can do some measurements...:
>>> from pympler import asizeof
>>> asizeof.asizeof({})
144
>>> asizeof.asizeof(23)
16
You wouldn't want every int to take up 144 bytes instead of just 16, right?-)
Now, when you make a class (inheriting from whatever), things change...:
>>> class dint(int): pass
...
>>> asizeof.asizeof(dint(23))
184
...the __dict__ is now added (plus, a little more overhead) -- so a dint instance can have arbitrary attributes, but you pay quite a space cost for that flexibility.
So what if you wanted ints with just one extra attribute foobar...? It's a rare need, but Python does offer a special mechanism for the purpose...
>>> class fint(int):
... __slots__ = 'foobar',
... def __init__(self, x): self.foobar=x+100
...
>>> asizeof.asizeof(fint(23))
80
...not quite as tiny as an int, mind you! (or even the two ints, one the self and one the self.foobar -- the second one can be reassigned), but surely much better than a dint.
When the class has the __slots__ special attribute (a sequence of strings), then the class statement (more precisely, the default metaclass, type) does not equip every instance of that class with a __dict__ (and therefore the ability to have arbitrary attributes), just a finite, rigid set of "slots" (basically places which can each hold one reference to some object) with the given names.
In exchange for the lost flexibility, you gain a lot of bytes per instance (probably meaningful only if you have zillions of instances gallivanting around, but, there are use cases for that).
As other answerers have said, an object does not have a __dict__. object is the base class of all types, including int or str. Thus whatever is provided by object will be a burden to them as well. Even something as simple as an optional __dict__ would need an extra pointer for each value; this would waste additional 4-8 bytes of memory for each object in the system, for a very limited utility.
Instead of doing an instance of a dummy class, in Python 3.3+, you can (and should) use types.SimpleNamespace for this.
It is simply due to optimization.
Dicts are relatively large.
>>> import sys
>>> sys.getsizeof((lambda:1).__dict__)
140
Most (maybe all) classes that are defined in C do not have a dict for optimization.
If you look at the source code you will see that there are many checks to see if the object has a dict or not.
So, investigating my own question, I discovered this about the Python language: you can inherit from things like int, and you see the same behaviour:
>>> class MyInt(int):
pass
>>> x = MyInt()
>>> print x
0
>>> x.hello = 4
>>> print x.hello
4
>>> x = x + 1
>>> print x
1
>>> print x.hello
Traceback (most recent call last):
File "<interactive input>", line 1, in <module>
AttributeError: 'int' object has no attribute 'hello'
I assume the error at the end is because the add function returns an int, so I'd have to override functions like __add__ and such in order to retain my custom attributes. But this all now makes sense to me (I think), when I think of "object" like "int".
https://docs.python.org/3/library/functions.html#object :
Note: object does not have a __dict__, so you can’t assign arbitrary attributes to an instance of the object class.
It's because object is a "type", not a class. In general, all classes that are defined in C extensions (like all the built in datatypes, and stuff like numpy arrays) do not allow addition of arbitrary attributes.
This is (IMO) one of the fundamental limitations with Python - you can't re-open classes. I believe the actual problem, though, is caused by the fact that classes implemented in C can't be modified at runtime... subclasses can, but not the base classes.