According to the Python 2.7.12 Documentation,
When the attribute is a user-defined method object, a new method
object is only created if the class from which it is being retrieved
is the same as, or a derived class of, the class stored in the
original method object; otherwise, the original method object is
used as it is.
I'm trying to understand this paragraph by writing code:
# Parent: The class stored in the original method object
class Parent(object):
# func: The underlying function of original method object
def func(self):
pass
func2 = func
# Child: A derived class of Parent
class Child(Parent):
# Parent.func: The original method object
func = Parent.func
# AnotherClass: Another different class, neither subclasses nor subclassed
class AnotherClass(object):
# Parent.func: The original method object
func = Parent.func
a = Parent.func
b = Parent.func2
c = Child.func
d = AnotherClass.func
print b is a
print c is a
print d is a
The output is:
False
False
False
I expect it to be:
False
False
True
Every time you write Parent.func, you are getting a new unbound method object. So AnotherClass.func never changes, but Parent.func does. You can see this if you do something like this:
a = Parent.func
a2 = Parent.func
b = Parent.func2
c = Child.func
d = AnotherClass.func
d2 = AnotherClass.func
Then:
>>> a is a2
False
>>> d is d2
True
So your comments referring to "the original method object" are somewhat misleading. Each part of your code that evaluates Parent.func is getting a different method object. (Note that the quotation you gave does not apply to Parent.func, because func as written in Parent is not a method object; it is a function object. It is only when you access the attribute by writing Parent.func that you get a method object.)
I'm adding a bit here to clarify the quotation you gave.
I think you are misunderstanding what that quotation is saying. This may be because the quotation is somewhat vague about the distinction between "the attribute" and the value retrieved by getting that attribute.
Let me clarify by separating two notions. In an expression like something.attr I'll use "the raw value" to refer to the actual value stored in the class/object dictionary, i.e., the value of something.__dict__['attr']. I'll use "the evaluated value" to refer to the actual result of evaluating something.attr. The difference between the two is that the descriptor protocol is activated on the raw value to get the evaluated value.
The documentation you quoted is describing a situation where the raw value is a method object. This is the case in your definitions of Child and AnotherClass. The quotation does not apply to func inside Parent, because in that case the raw value is not a method; it is a function. In that case, the following paragraph of the documentation applies:
When a user-defined method object is created by retrieving a user-defined function object...
In other words, the documentation you quoted ("When the attribute is a user-defined method object") applies to cases like this:
obj = [a method object]
class Foo(object):
attr = obj
Your quoted documentation does not apply to the following case. In this case, the attribute is "a user-defined function object":
func = [a function object]
class Foo(object):
attr = func
In your example, there is no "original method object". You can only get a method object after the class is defined. The "original" object is a function object. When you write func in the class body, you are defining a function; it is only when you access Parent.func that you get a method object.
Related
Why do the following lines give me the same result?
str.upper('hello')
and
'hello'.upper()
I tried to do the same with list.append but got a TypeError.
list.append([1])
Is the str type in Python overloaded? How can this be achieved by writing a class/function? I would appreciate an example.
list.append takes two arguments - the list to modify and the element to append. So you need to do it like this:
ls = [1]
list.append(ls, 2)
which is equivalent to the much more popular:
ls.append(2)
str.upper and list.append are both functions.
str.upper takes one argument.
>>> str.upper('test')
'TEST'
list.append takes two arguments.
>>> my_list = []
>>> list.append(my_list, 1)
>>> my_list
[1]
str.upper and list.append (like other functions) are also non-data-descriptors with a __get__ method which in this context has two implications:
When you access the function through the class via the dot notation (str.upper, list.append) the function's __get__ method (i.e. string.upper.__get__ and list.append.__get__) is called but it returns just the function itself.
When you access the function through an instance (my_string.upper, my_list.append) the function's __get__ method is called and it will return a new callable acting like the original function, but with whatever was "in front of the dot" automatically passed as the first argument. .
That's why you need to pass 1 - 1 = 0 arguments when calling my_string.upper() and 2 - 1 = 1 argument when calling my_list.append(1).
>>> 'my_string'.upper()
'MY_STRING'
>>>
>>> my_list = []
>>> my_list.append(1)
>>> my_list
[1]
You could even get these modified callables (methods) by explicitly calling __get__ and passing the argument to be bound (what has been before the dot) as its argument.
>>> my_string = 'my_string'
>>> upper_maker = str.upper.__get__(my_string)
>>> upper_maker()
'MY_STRING'
>>>
>>> my_list = []
>>> appender = list.append.__get__(my_list)
>>> appender(1)
>>> my_list
[1]
Finally, here's a short example demonstrating how descriptor instances can detect whether they are being accessed via their owner-class or via an instance.
class Descriptor:
def __get__(self, instance, owner_class):
if instance is None:
print('accessed through class')
# list.append.__get__ would return list.append here
else:
print('accessed through instance')
# list.append.__get__ would build a new callable here
# that takes one argument x and that internally calls
# list.append(instance, x)
class Class:
attribute = Descriptor()
Class.attribute # prints 'accessed through class'
instance = Class()
instance.attribute # prints 'accessed through instance'
Quoting Dave Kirbys answer from Relationship between string module and str:
There is some overlap between the string module and the str type,
mainly for historical reasons. In early versions of Python str objects
did not have methods, so all string manipulation was done with
functions from the string module. When methods were added to the str
type (in Python 1.5?) the functions were left in the string module for
compatibility, but now just forward to the equivalent str method.
However the string module also contains constants and functions that
are not methods on str, such as formatting, character translation etc.
There is nothing at all magical going on with str (except that we have a nice syntactic shortcut to creating one using ""). You can write a class that behaves like str and list to see more clearly what is happening here.
class MyClass():
def __init__(self, arg):
self.val=str(arg)
def do_thing(self):
self.val = "asdf"
def do_thing_with_arg(self, arg):
self.val = "asdf " + str(arg)
def __repr__(self):
return self.val
my_thing = MyClass("qwerty")
# this is like 'hello'.upper()
my_thing.do_thing()
print(my_thing)
# it prints 'asdf'
my_thing = MyClass("qwerty")
# this is like str.upper('hello')
MyClass.do_thing(my_thing)
print(my_thing)
# it prints 'asdf'
my_thing = MyClass("qwerty")
# this is like my_list.append('qwerty')
my_thing.do_thing_with_arg('zxcv')
print(my_thing)
# it prints 'asdf zxcv'
my_thing = MyClass("qwerty")
# this is like list.append(my_list, 'qwerty')
MyClass.do_thing_with_arg(my_thing, 'zxcv')
print(my_thing)
# it prints 'asdf zxcv'
The short version is, you're invoking what looks like an "instance method" on a class, but you are supplying the instance ('self') yourself as the first argument to the function call.
I'm trying to understand how classes work a bit better "under the hood" of python.
If I create a class Foo like so
class Foo:
bar = True
Foo is then directly accessible, such as print(Foo) or print(Foo.bar)
However, if I dynamically create create a class and don't set it to a variable like so
type('Foo',(),{'bar':True})
If done in the interpreter it shows <class '__main__.Foo'>. However, when I try to print Foo it's undefined...NameError: name 'Foo' is not defined
Does this mean that when a class is created the "traditional" way (the first Foo class above), that python automatically sets a variable for the class of the same name? Sort of like this
# I realize this is not valid, just to convey the idea
Foo = class Foo:
bar = True
If so, then why doesn't python also create a variable named Foo set to class Foo when using type() to create it?
let's compare your problem with function statements and lambdas (because they play the same role here), consider this function f :
def f ():
return 1
the above snippet of code is not an expression at all, it is a python statement that creates a function named f returning 1 upon calling it.
let's now do the same thing, but in a different way :
f = lambda : 1
the above snippet of code is a python expression (an assignment) that assigns the symbol f to the lambda expression (which is our function) lambda : 1. if we didn't do the assignment, the lambda expression would be lost, it is the same as writing >>> 1 in the python REPL and then trying after that to reference it.
Using type with 3 argument is analogous to using the lambda to create a function. Without assignment the evaluated expression is garbage collected.
However, just you can still create an instance of the class, just like you can immediately call a lambda function.
>>> lambda x: True
<function <lambda> at 0x0000022FF95AB598>
>>> type('Test', (), {'x': True})
<class '__main__.Test'>
You can also create an instance of the class, just like you can immediately call a function
>>> t = type('Test', (), {'x': True})()
>>> t.x
True
>>> type('Test2', (), {'y': 123})().y
123
>>> (lambda x: True)(1000) # any input returns True
True
From documentation
class type(name, bases, dict)
With three arguments, return a new type object. This is essentially a dynamic form of the class statement. The name string is the class name and becomes the name attribute; the bases tuple itemizes the base classes and becomes the bases attribute; and the dict dictionary is the namespace containing definitions for class body and becomes the dict attribute. For example, the following two statements create identical type objects:
class X(object):
a = 1
X = type('X', (object,), dict(a=1))
So yes, I think you have the right idea. type() does create a class but a dynamic form.
I think you're making this too complicated. If you don't assign a value / object to a symbol, it is always "lost". Doesn't matter if the value / object is a class or something else. Example:
x = 2 + 2
That assigns the value 4 to the symbol x. Compare to:
2 + 2
The operation is carried out but the result 4 isn't assigned to a symbol.
Exact situation you have with classes.
I've just get to know about Class and User-Defined-Function in Python and currently practicing with them.
Sometimes I got really confused whether an inputted variable or instance attribute is directly used or been copied as a local variable that only works during the call of function.
For example:
class test1(object):
def __init__(self, a):
self.a = a
class test2(object):
def __init__(self, test):
self.a = test.a
self.add_d()
def add_d(self):
self.a += 'd'
print self.a
class test3(object):
def __init__(self, test):
self.fc = test
self.add_d()
def add_d(self):
self.fc.a += 'd'
print self.fc.a
And:
In [36]: t = test1('abc')
In [37]: test2(t)
abcd
Out[37]: <__main__.test2 at 0xce5bcf98>
In [38]: t.a
Out[38]: 'abc'
It didn't change attribute "a" of instance "t" from class test1.
In [39]: test3(t)
abcd
Out[39]: <__main__.test3 at 0xfc3c9e8>
In [40]: t.a
Out[40]: 'abcd'
It changed attribute "a" of instance "t" from class test1.
Usually, I used functions for several scenario:
Aim to alter (mutate) the input variables from calling a function.
The return value is the only thing desired from the function call, and inputted variables should be unchanged.
Passing an Axes of matplotlib and arrange plotting commands in the function. (This would always work, never went wrong.)
Things would be pretty troublesome if I made unwanted changes to raw data and vice versa. So what is the basic concept here? Also, is there any habits that we should develop to prevent this kind of mistake?
I think that you are asking why was the test1 instance modified in the second example but not modified in the first?
In the first one after:
self.a = test.a
self.a refers to test.a which is the same immutable string object. Now when:
self.a += 'd'
executes it rebinds the value of the expression self.a + d to self.a, i.e. a new string object is created. Because strings are immutable the original can not be altered, so a new one must be created. At this point the attribute a in the test2 object refers to a different object than the a attribute in the test1 object, which remains unchanged.
In the second example after
self.fc = test
self.fc refers to the object test which is the same object that the variable t refers to. In terms of variable references this is the same as the first case, however, the difference is that this object is mutable so when
self.fc.a += 'd'
executes a new string is created and bound to the a attribute of the object - but self.fc still refers to the same object. Hence t.a is changed.
I've tried to explain this in terms of your example. The crucial point is that Python variables refer to objects, and there can be multiple references to the same object. Changes to the object can be made via any of the referring variables, and that change will be visible regardless of which variable is used to view it. Copies of objects are not made during assignment, however, the reference can be changed if a new object is created in order to preserve the immutability of certain objects.
I recommend that you give this a read Facts and myths about Python names and values.
What I do not understand is b = Bar(a). What does it do? How is Bar taking a as an argument?
Won't that mean Bar inherits from a? What is Bar.Foo1 = Foo? Does it mean Foo1 is an instance of class Foo()? How do we access Foo1 when it itself is an object? What is the meaning of b.arg.variable? Doesn't it mean that b has a method arg which has a variable called variable? The following code is from this answer
I just could not find parsing objects as an argument to another class.
class Foo (object):
#^class name #^ inherits from object
bar = "Bar" #Class attribute.
def __init__(self):
# #^ The first variable is the class instance in methods.
# # This is called "self" by convention, but could be any name you want.
self.variable="Foo" #instance attribute.
print self.variable, self.bar #<---self.bar references class attribute
self.bar = " Bar is now Baz" #<---self.bar is now an instance attribute
print self.variable, self.bar
def method(self,arg1,arg2):
#This method has arguments. You would call it like this : instance.method(1,2)
print "in method (args):",arg1,arg2
print "in method (attributes):", self.variable, self.bar
a=Foo() # this calls __init__ (indirectly), output:
# Foo bar
# Foo Bar is now Baz
print a.variable # Foo
a.variable="bar"
a.method(1,2) # output:
# in method (args): 1 2
# in method (attributes): bar Bar is now Baz
Foo.method(a,1,2) #<--- Same as a.method(1,2). This makes it a little more explicit what the argument "self" actually is.
class Bar(object):
def __init__(self,arg):
self.arg=arg
self.Foo1=Foo()
b=Bar(a)
b.arg.variable="something"
print a.variable # something
print b.Foo1.variable # Foo
"What I do not understand is b = Bar(a). What does it do?"
b = Bar(a) does two things. First, it creates an object of class Bar (with any class variables and methods attached). Then, it runs __init__ with the first argument (self) pointing to the object that was just created, and with a as the second argument (arg). While running __init__, as one of the commands in that method, it sets self.arg to point to the object pointed to by arg (i.e. to the object pointed to by the variable a). Finally, the variable b is set to refer to the object that was created.
It may help to think this way: a variable in Python is really just a pointer that points to an object. You can have more than one variable pointing to the same object. In this case, a and b.arg both point to the same object.
I found this sort of thing confusing too, at first. I had seen the advice to think of variables as separate concepts from the object they point to and ignored it as unnecessarily complicating things, but I had to go back to accepting that way of thinking in order to make sense of things. People do often use the variable as a name to refer to the object it points to; you just have to know when to take this literally or not.
"Wont that mean Bar inherit from a?"
No. If a is a class, then class Bar(a) would mean that Bar inherits from a. But in
b = Bar(a), a is an object being passed as an argument to __init__.
"What is Bar.Foo1 = Foo?"
Sorry -- I don't see that in the example code you gave.
"What is the meaning of b.arg.variable?"
b is an object (i mean, b refers to an object) and b.arg is one of the attributes of that object. Methods and variables are different types of attributes. In this case, b.arg is a variable pointing to an object. The object referred to by b.arg has attribute variable, which is a variable.
b.arg refers to the same object that a refers to, therefore b.arg.variable is the same variable as a.variable. It not only points to the same object, but actually is the same variable. The object it points to is the string "something".
#Brenbarn: I think that's what quirius meant by "Wont that mean Bar inherit from a?".
Here is a simpler example. Suppose you have these classes:
class Foo(object):
pass
class Bar(object):
def __init__(self, arg):
self.arg = arg
Here are two things you could do:
# Option 1: pass in an integer
>>> b = Bar(1)
>>> b.arg
1
# Option 2: pass in a Foo object
>>> a = Foo()
>>> b = Bar(a)
>>> b.arg
<__main__.Foo object at 0x0000000002A440F0>
There is no difference in how the two cases are handled. Passing in a (a Foo object) is no different from passing in an integer. All that Bar does is store the value that is passed in, and it can store it just the same whether it is a Foo or an int. When you call Bar(something), it is entirely up to the Bar class how to handle the object that is passed in. The type of the passed-in object is not involved except insofar as Bar chooses to explicitly involve it (e.g., by calling methods on the passed in object).
The way I usually declare a class variable to be used in instances in Python is the following:
class MyClass(object):
def __init__(self):
self.a_member = 0
my_object = MyClass()
my_object.a_member # evaluates to 0
But the following also works. Is it bad practice? If so, why?
class MyClass(object):
a_member = 0
my_object = MyClass()
my_object.a_member # also evaluates to 0
The second method is used all over Zope, but I haven't seen it anywhere else. Why is that?
Edit: as a response to sr2222's answer. I understand that the two are essentially different. However, if the class is only ever used to instantiate objects, the two will work he same way. So is it bad to use a class variable as an instance variable? It feels like it would be but I can't explain why.
The question is whether this is an attribute of the class itself or of a particular object. If the whole class of things has a certain attribute (possibly with minor exceptions), then by all means, assign an attribute onto the class. If some strange objects, or subclasses differ in this attribute, they can override it as necessary. Also, this is more memory-efficient than assigning an essentially constant attribute onto every object; only the class's __dict__ has a single entry for that attribute, and the __dict__ of each object may remain empty (at least for that particular attribute).
In short, both of your examples are quite idiomatic code, but they mean somewhat different things, both at the machine level, and at the human semantic level.
Let me explain this:
>>> class MyClass(object):
... a_member = 'a'
...
>>> o = MyClass()
>>> p = MyClass()
>>> o.a_member
'a'
>>> p.a_member
'a'
>>> o.a_member = 'b'
>>> p.a_member
'a'
On line two, you're setting a "class attribute". This is litterally an attribute of the object named "MyClass". It is stored as MyClass.__dict__['a_member'] = 'a'. On later lines, you're setting the object attribute o.a_member to be. This is completely equivalent to o.__dict__['a_member'] = 'b'. You can see that this has nothing to do with the separate dictionary of p.__dict__. When accessing a_member of p, it is not found in the object dictionary, and deferred up to its class dictionary: MyClass.a_member. This is why modifying the attributes of o do not affect the attributes of p, because it doesn't affect the attributes of MyClass.
The first is an instance attribute, the second a class attribute. They are not the same at all. An instance attribute is attached to an actual created object of the type whereas the class variable is attached to the class (the type) itself.
>>> class A(object):
... cls_attr = 'a'
... def __init__(self, x):
... self.ins_attr = x
...
>>> a1 = A(1)
>>> a2 = A(2)
>>> a1.cls_attr
'a'
>>> a2.cls_attr
'a'
>>> a1.ins_attr
1
>>> a2.ins_attr
2
>>> a1.__class__.cls_attr = 'b'
>>> a2.cls_attr
'b'
>>> a1.ins_attr = 3
>>> a2.ins_attr
2
Even if you are never modifying the objects' contents, the two are not interchangeable. The way I understand it, accessing class attributes is slightly slower than accessing instance attributes, because the interpreter essentially has to take an extra step to look up the class attribute.
Instance attribute
"What's a.thing?"
Class attribute
"What's a.thing? Oh, a has no instance attribute thing, I'll check its class..."
I have my answer! I owe to #mjgpy3's reference in the comment to the original post. The difference comes if the value assigned to the class variable is MUTABLE! THEN, the two will be changed together. The members split when a new value replaces the old one
>>> class MyClass(object):
... my_str = 'a'
... my_list = []
...
>>> a1, a2 = MyClass(), MyClass()
>>> a1.my_str # This is the CLASS variable.
'a'
>>> a2.my_str # This is the exact same class variable.
'a'
>>> a1.my_str = 'b' # This is a completely new instance variable. Strings are not mutable.
>>> a2.my_str # This is still the old, unchanged class variable.
'a'
>>> a1.my_list.append('w') # We're changing the mutable class variable, but not reassigning it.
>>> a2.my_list # This is the same old class variable, but with a new value.
['w']
Edit: this is pretty much what bukzor wrote. They get the best answer mark.