I want
class MyClass(object):
_my_unique = ???? # if this were lisp, I would have written (cons nil nil) here
def myFunc (self, arg):
assert arg != _my_unique # this must never fail
...
What do use instead of ??? to ensure that the assert never fails?
(With Lisp, I could create _my_unique with (cons nil nil) and used eq in assert).
PS. Use case: I will put _my_unique in a dict, so I want it to be equal to itself, but I do not want it to be equal (in the dict collision sense) to anything passed in from the outside.
You can use object(), but this won't make the assert "never fail". It will still fail if you call myFunc and pass MyClass.my_unique as the object. Also, if you want to test whether it's the same object, use arg is not my_unique rather than !=.
You can use object().
Return a new featureless object. object is a base for all new style
classes. It has the methods that are common to all instances of new
style classes.
If what you're asking is "how do I make _my_unique unique for each instance of MyClass", you could simply create a new, empty object in the constructor.
For example:
>>> class MyClass(object):
... def __init__(self):
... self._my_unique = object()
...
>>> foo=MyClass()
>>> bar=MyClass()
>>> foo._my_unique
<object object at 0x100ce70c0>
>>> bar._my_unique
<object object at 0x100ce7090>
If you want to hide _my_unique, give it two underscores. This way, nobody can accidentally get at the value. It's not impossible, but they would need to work a little harder to get at the value. This is called name mangling, and you can read more about it here: http://docs.python.org/2/tutorial/classes.html#private-variables-and-class-local-references
>>> class MyClass(object):
... def __init__(self):
... self.__my_unique = object()
...
>>> foo=MyClass()
>>> foo.__my_unique
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'MyClass' object has no attribute '__my_unique'
This is a mildly confusing question, as it seems to mix a bunch of concepts without purpose. For one thing, most any object in Python is unique, but there may be multiple ways to reach them. The operator to check identities is not != (inequality) but is not. Unlike Java, Python does not require you to put things in a class, and does not implicitly look in self (which they know as implicit this). cons from Lisp is used to construct pairs which frequently form singly linked lists, a structure that's unusual in Python because we use dynamic reference arrays known as lists. Lists are mutable, and thus constructing one using list() or [] will produce a unique object.
So, while it is rather pointless, one way to write a function producing functions that perform the useless assert might be:
def createfunc():
mylist=[]
def subfunc(x):
assert x is not mylist
return subfunc
Each call to createfunc() returns a new function with its own mylist. However, the object being unique doesn't make it impossible to reach:
f=createfunc()
f(f.func_closure[0].cell_contents) # raises AssertionError
Regarding the PS, to be relevant to dict collissions, your object must also be hashable, which makes object() a better choice than list() for the unique object.
Related
I am writing a custom class which extends the default python lists by adding some new functions such as shuffling , adding, multiplying etc.
The code goes something like this:
class xlist(list):
def __init__(self, original_list: list):
self._olist = original_list
def sumall(self) -> int:
sum = 0
for e in self._olist:
sum += e
return sum
...
But while doing some calculations I needed to get the type of a instance of a xlist. I want to do something like this :
>>> from xlist import xlist
>>> x = xlist([1, 2, 3])
>>> type(x)
When I do this I get <class 'xlist.xlist'> , but I want it to return list.
I am little confused about Metaclasses which seems to be able to solve the problem.
Any Help?
Why do you expect type(x) to return list if you're really creating an xlist? Your xlist inherits from list, so every xlist object is an instance of a list since it inherits from it all of its behaviour (and extends by adding some new functionality).
Note that:
x = xlist([1, 2, 3])
isinstance(x, list)
returns True. You might also want to have a look at Difference between type() and isinstance()
There are two ways for Python to check the class of an object - one is calling type and the other is checking the __class__ slot.
Most times both return the samething, but one can modify the class (for example, by customizing attribute access on the metaclass) so that __class__ will "lie" and Python code using myobject.__class__ will get the "false" information.
However, underneath, the "true" __class__ slot in the type object will always hold a reference to the real type - and this can't be falsified. Any C extension, and maybe even a few Python extensions, and the return to type(myobject) itself will see the real class.
Changing the contents of this slot actually changes the class of your instance. It is feasible from pure Python with a simple = attribution - but there are guards in place on this assignment to ensure it is only done across types that have a compatible memory layout. Forcing it to change to an incompatible type (Via an extension, or ctypes) will get your Python runtime to segfault.
All that said, there is no reason to lie about your class to users of your class -they should be able to "see" that the object the are holding is a xlist and not a list, and that xlists are also list objects, due to inheritance. Falsifying this information would be a rather bad practice. On the other hand, there are few calls, in Python stdlib itself, that require the underlying object to really be a list and won't accept subtypes (notoriously Python´s json.dumps serialization). That call has a native code path and won't be fooled by customizing access to __class__. However, the same call also has a Python only codepath that is triggered by setting some of the optional arguments (for example, by passing ident=4 on the call). If that is you are trying to achieve (fool some code that requires an strict list) you have to heck that, and if it is Python code, it is doable. In the specific case of json.dump, you'd be better monkeypatching the encoder to use a less strict check than to falsify your object - because I think the code there uses type for the checing.
So, with all of the above said, the "metaclass trick" to Falsify the return of .__class__ can be as simple as:
class xlist(list, metaclass=Meta):
def __init__(self, original_list: list):
self._olist = original_list
def sumall(self) -> int:
sum = 0
for e in self._olist:
sum += e
return sum
#property
def __class__(self):
return list
So, I was playing around with Python while answering this question, and I discovered that this is not valid:
o = object()
o.attr = 'hello'
due to an AttributeError: 'object' object has no attribute 'attr'. However, with any class inherited from object, it is valid:
class Sub(object):
pass
s = Sub()
s.attr = 'hello'
Printing s.attr displays 'hello' as expected. Why is this the case? What in the Python language specification specifies that you can't assign attributes to vanilla objects?
For other workarounds, see How can I create an object and add attributes to it?.
To support arbitrary attribute assignment, an object needs a __dict__: a dict associated with the object, where arbitrary attributes can be stored. Otherwise, there's nowhere to put new attributes.
An instance of object does not carry around a __dict__ -- if it did, before the horrible circular dependence problem (since dict, like most everything else, inherits from object;-), this would saddle every object in Python with a dict, which would mean an overhead of many bytes per object that currently doesn't have or need a dict (essentially, all objects that don't have arbitrarily assignable attributes don't have or need a dict).
For example, using the excellent pympler project (you can get it via svn from here), we can do some measurements...:
>>> from pympler import asizeof
>>> asizeof.asizeof({})
144
>>> asizeof.asizeof(23)
16
You wouldn't want every int to take up 144 bytes instead of just 16, right?-)
Now, when you make a class (inheriting from whatever), things change...:
>>> class dint(int): pass
...
>>> asizeof.asizeof(dint(23))
184
...the __dict__ is now added (plus, a little more overhead) -- so a dint instance can have arbitrary attributes, but you pay quite a space cost for that flexibility.
So what if you wanted ints with just one extra attribute foobar...? It's a rare need, but Python does offer a special mechanism for the purpose...
>>> class fint(int):
... __slots__ = 'foobar',
... def __init__(self, x): self.foobar=x+100
...
>>> asizeof.asizeof(fint(23))
80
...not quite as tiny as an int, mind you! (or even the two ints, one the self and one the self.foobar -- the second one can be reassigned), but surely much better than a dint.
When the class has the __slots__ special attribute (a sequence of strings), then the class statement (more precisely, the default metaclass, type) does not equip every instance of that class with a __dict__ (and therefore the ability to have arbitrary attributes), just a finite, rigid set of "slots" (basically places which can each hold one reference to some object) with the given names.
In exchange for the lost flexibility, you gain a lot of bytes per instance (probably meaningful only if you have zillions of instances gallivanting around, but, there are use cases for that).
As other answerers have said, an object does not have a __dict__. object is the base class of all types, including int or str. Thus whatever is provided by object will be a burden to them as well. Even something as simple as an optional __dict__ would need an extra pointer for each value; this would waste additional 4-8 bytes of memory for each object in the system, for a very limited utility.
Instead of doing an instance of a dummy class, in Python 3.3+, you can (and should) use types.SimpleNamespace for this.
It is simply due to optimization.
Dicts are relatively large.
>>> import sys
>>> sys.getsizeof((lambda:1).__dict__)
140
Most (maybe all) classes that are defined in C do not have a dict for optimization.
If you look at the source code you will see that there are many checks to see if the object has a dict or not.
So, investigating my own question, I discovered this about the Python language: you can inherit from things like int, and you see the same behaviour:
>>> class MyInt(int):
pass
>>> x = MyInt()
>>> print x
0
>>> x.hello = 4
>>> print x.hello
4
>>> x = x + 1
>>> print x
1
>>> print x.hello
Traceback (most recent call last):
File "<interactive input>", line 1, in <module>
AttributeError: 'int' object has no attribute 'hello'
I assume the error at the end is because the add function returns an int, so I'd have to override functions like __add__ and such in order to retain my custom attributes. But this all now makes sense to me (I think), when I think of "object" like "int".
https://docs.python.org/3/library/functions.html#object :
Note: object does not have a __dict__, so you can’t assign arbitrary attributes to an instance of the object class.
It's because object is a "type", not a class. In general, all classes that are defined in C extensions (like all the built in datatypes, and stuff like numpy arrays) do not allow addition of arbitrary attributes.
This is (IMO) one of the fundamental limitations with Python - you can't re-open classes. I believe the actual problem, though, is caused by the fact that classes implemented in C can't be modified at runtime... subclasses can, but not the base classes.
I have a test class:
class TestClass:
def someFunction(self, someInt):
self.i = someInt
return self.i
And was testing it with:
x = MyClass()
x.someFunction = 3
print(x.someFunction)
x.someFunction(4)
However, this leads to TypeError: 'int' object is not callable. I understand why I am getting this error--someFunction has been rebound to an integer (in this case, 3) and, well, an integer is not a function and therefore someFunction is no longer callable. But I am wondering what the reasoning is behind allowing a member function to be rebound just as if it is a data member? It seems prone to accidental errors. I am just starting to learn Python and feel like I'm missing a fundamental aspect of Python that this behavior would fit into.
This is allowed because of the way attribute lookup works in Python and this is by design. In Python, many things that are discouraged, forbidden or impossible in other languages, are allowed to leverage your use case (if used wisely). Of course, more power implies more responsibility.
After all, we're all consenting adults here.
Some background information on attribute resolution
Class instances start with an empty __dict__ attribute where all object attributes are stored. By accessing x.someFunction you are implicitly trying x.__dict__['someFunction']. If 'someFunction' does not exist in x.__dict__, the same is tried for the class of x, i.e. type(x).someFunction or better type(x).__dict__['someFunction'].
When your write x by doing x.someFunction = 3, what actually happens is x.__dict__['someFunction'] = 3, regardless of what the reading attribute access might return.
The only real (?) magic happens during method calls, where self is provided automatically, i.e. x.someFunction(4) is resolved to type(x).__dict__['someFunction'](x, 4) instead of type(x).__dict__['someFunction'](4) or x.__dict__['someFunction'](4). This is somewhat related to attribute access and may cause confusion.
So, you actually do not "rebind" the function, but hide the class attribute someFunction with the instance attribute someFunction of x. If you do
print(MyClass.someFunction)
print(MyClass().someFunction)
you will see that the method is still there. You can even restore the initial state with
del x.__dict__['someFunction']
Note: The things I described as resolution illustrate the concept. In reality the inner workings of python may be more subtle or complex, but they will have the same effect. For example, in Python 2, methods have been wrapped as so called unbound methods before being stored in the class dictionary. This has been dropped in Python 3, where the class dictionary contains the plain function.
Please execute your demo code in a python shell:
>>> class TestClass:
... def someFunction(self, someInt):
... self.i = someInt
... return self.i
...
>>> x = TestClass()
>>> x.someFunction = 3
>>> print(x.someFunction)
3
>>> x.someFunction(4)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'int' object is not callable
Please see the following content:
>>> type(x.someFunction)
<type 'instancemethod'>
>>> x.someFunction = 3
>>> type(x.someFunction)
<type 'int'>
If you try x.someFunction = 3, the instancemethod caled x.someFunction translates into a int.
In python
functions are threated as first-class objects
and variables are not typed.
Or just "Everything is an object" in python. So there is just no fundamental difference of a variable containing an integer vs a variable containing a function. Both, first-class functions and untyped variables have it's own pros and cons but allow you to e.g. simply make a list that contains integers, string, other objects as well as functions.
The reason you can assign anything to function variables is because they are implemented as objects. This enables you to return functions and pass functions as arguments, unlike in C.
The usefulness of this feature far surpasses the 'accidental errors'.
For example, you can do something like this:
# some code
def sum(a, b):
return a+b
def product(a, b):
return a*b
def ret_func(str):
if str=='add':
func = sum
elif str=='multiply':
func = product
return func
addition_result = ret_func('add')(x, y)
multiplication_result = ret_func('multiply')(x, y)
# some more code
>>> class S(object):
... def __init__(self):
... self.x = 1
... def x(self):
... return self.x
...
>>> s = S()
>>> s.x
1
>>> s.x()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'int' object is not callable
Why, in this example, is s.x a method, but also an integer? It seems to me that self.x = 1 should replace the def x(self): declaration of the attribute x during instantiation. Why is it that I can get and call, resulting in an integer and a method, respectively, the same attribute? My guess is that the variable look-up pattern in new-style classes is duck typed, so as to return the most relevant result to the caller. I would love to hear the whole story.
Python doesn't use separate spaces for callable and non-callable objects: a name is a name is a name. s.x, by Python rules, must refer to exactly the same object, whether you're going to call it or not. Another way of putting it: assuming that _aux is a name not otherwise used,
_aux = self.x
_aux()
and
self.x()
must have absolutely identical semantics in Python, the fact that in the former the intermediate value self.x is being bound to a name and called later notwithstanding.
Having single, "unified" namespaces for callables and non-callables has a huge number of advantages -- it makes name-lookup rules (for each of bare and qualified names) enormously simpler, for example, by decoupling them totally from the purpose to which the name being looked up is going to be put (be it immediately after the lookup's result, or later still), and also from the type (callable or non-callable) of whatever object turns up to be first referenced according to the lookup rules.
Especially considering how many different callable types Python has (functions, classes, instances of classes which define __call__, special types such as staticmethod and classmethod, ...!-), any other rule could only lead to total chaos. (Note also, for example, that even C++, a language which definitely is not afraid by complexity but which also lets class-instances be callable [[if the class overloads operator()]], uses a similar unified-namespace rule -- again, discriminating between callables and non-callables would be a totally unwarranted nightmare, were the rules any different in this regard!-).
It looks like you're having a misunderstanding of the error you're seeing. When your s object is instantiated, its constructor replaces the method x by an integer, so in the s object, x is an integer, not a function. Trying to call it as a method results in an exception being thrown.
Python is duck-typed in the sense that method calls are resolved at runtime - the compiler has no problem with s.x() because x might have been created as a method dynamically. However, when the interpreter actually calls x as a method, it notices x is an integer and can't be called, hence the TypeError.
I'm not sure what you think is going on, but there's nothing that tricky happening. When you assign self.x = 1, the method x is no longer accessible. From that point forward, s.x is only an integer -- attempts to call it as a method result in an exception, as you saw.
It seems that the x property is defined as a method in the class definition. However, actually instantiating an object overwrites that name with an integer - hence, the behavior observed. It's never actually two at once. So, this is basically some faulty code.
This is what your code is doing:
Create a class named S with 2 methods, __init__ and x
Create an instance of S and name it s
Call S.__init__ with s as parameter
Set s.x with the value 1
Print s.x
Print the result of calling s.x
Now, if you look in 2.1.1 you will see that you have overrided the method x with an integer, which means that you cannot call that again withing s (but it stills in S class)
If you have done that, and yet, need call x function, try it:
>>> class S(object):
... def __init__(self):
... self.x = 1
... def x(self):
... return self.x
...
>>> s = S()
>>> s.x
1
>>> S.x(s)
1
>>>
I just did it so you understand why you are losing the x as method, do it in the right way and avoid to have instances variables with the same name as class methods
So, I was playing around with Python while answering this question, and I discovered that this is not valid:
o = object()
o.attr = 'hello'
due to an AttributeError: 'object' object has no attribute 'attr'. However, with any class inherited from object, it is valid:
class Sub(object):
pass
s = Sub()
s.attr = 'hello'
Printing s.attr displays 'hello' as expected. Why is this the case? What in the Python language specification specifies that you can't assign attributes to vanilla objects?
For other workarounds, see How can I create an object and add attributes to it?.
To support arbitrary attribute assignment, an object needs a __dict__: a dict associated with the object, where arbitrary attributes can be stored. Otherwise, there's nowhere to put new attributes.
An instance of object does not carry around a __dict__ -- if it did, before the horrible circular dependence problem (since dict, like most everything else, inherits from object;-), this would saddle every object in Python with a dict, which would mean an overhead of many bytes per object that currently doesn't have or need a dict (essentially, all objects that don't have arbitrarily assignable attributes don't have or need a dict).
For example, using the excellent pympler project (you can get it via svn from here), we can do some measurements...:
>>> from pympler import asizeof
>>> asizeof.asizeof({})
144
>>> asizeof.asizeof(23)
16
You wouldn't want every int to take up 144 bytes instead of just 16, right?-)
Now, when you make a class (inheriting from whatever), things change...:
>>> class dint(int): pass
...
>>> asizeof.asizeof(dint(23))
184
...the __dict__ is now added (plus, a little more overhead) -- so a dint instance can have arbitrary attributes, but you pay quite a space cost for that flexibility.
So what if you wanted ints with just one extra attribute foobar...? It's a rare need, but Python does offer a special mechanism for the purpose...
>>> class fint(int):
... __slots__ = 'foobar',
... def __init__(self, x): self.foobar=x+100
...
>>> asizeof.asizeof(fint(23))
80
...not quite as tiny as an int, mind you! (or even the two ints, one the self and one the self.foobar -- the second one can be reassigned), but surely much better than a dint.
When the class has the __slots__ special attribute (a sequence of strings), then the class statement (more precisely, the default metaclass, type) does not equip every instance of that class with a __dict__ (and therefore the ability to have arbitrary attributes), just a finite, rigid set of "slots" (basically places which can each hold one reference to some object) with the given names.
In exchange for the lost flexibility, you gain a lot of bytes per instance (probably meaningful only if you have zillions of instances gallivanting around, but, there are use cases for that).
As other answerers have said, an object does not have a __dict__. object is the base class of all types, including int or str. Thus whatever is provided by object will be a burden to them as well. Even something as simple as an optional __dict__ would need an extra pointer for each value; this would waste additional 4-8 bytes of memory for each object in the system, for a very limited utility.
Instead of doing an instance of a dummy class, in Python 3.3+, you can (and should) use types.SimpleNamespace for this.
It is simply due to optimization.
Dicts are relatively large.
>>> import sys
>>> sys.getsizeof((lambda:1).__dict__)
140
Most (maybe all) classes that are defined in C do not have a dict for optimization.
If you look at the source code you will see that there are many checks to see if the object has a dict or not.
So, investigating my own question, I discovered this about the Python language: you can inherit from things like int, and you see the same behaviour:
>>> class MyInt(int):
pass
>>> x = MyInt()
>>> print x
0
>>> x.hello = 4
>>> print x.hello
4
>>> x = x + 1
>>> print x
1
>>> print x.hello
Traceback (most recent call last):
File "<interactive input>", line 1, in <module>
AttributeError: 'int' object has no attribute 'hello'
I assume the error at the end is because the add function returns an int, so I'd have to override functions like __add__ and such in order to retain my custom attributes. But this all now makes sense to me (I think), when I think of "object" like "int".
https://docs.python.org/3/library/functions.html#object :
Note: object does not have a __dict__, so you can’t assign arbitrary attributes to an instance of the object class.
It's because object is a "type", not a class. In general, all classes that are defined in C extensions (like all the built in datatypes, and stuff like numpy arrays) do not allow addition of arbitrary attributes.
This is (IMO) one of the fundamental limitations with Python - you can't re-open classes. I believe the actual problem, though, is caused by the fact that classes implemented in C can't be modified at runtime... subclasses can, but not the base classes.