Develop Reference

Functions, methods, and how many arguments do I have to give them?

Why do the following lines give me the same result?
str.upper('hello')
and
'hello'.upper()
I tried to do the same with list.append but got a TypeError.
list.append([1])
Is the str type in Python overloaded? How can this be achieved by writing a class/function? I would appreciate an example.

list.append takes two arguments - the list to modify and the element to append. So you need to do it like this:
ls = [1]
list.append(ls, 2)
which is equivalent to the much more popular:
ls.append(2)

str.upper and list.append are both functions.
str.upper takes one argument.
>>> str.upper('test')
'TEST'
list.append takes two arguments.
>>> my_list = []
>>> list.append(my_list, 1)
>>> my_list
[1]
str.upper and list.append (like other functions) are also non-data-descriptors with a __get__ method which in this context has two implications:
When you access the function through the class via the dot notation (str.upper, list.append) the function's __get__ method (i.e. string.upper.__get__ and list.append.__get__) is called but it returns just the function itself.
When you access the function through an instance (my_string.upper, my_list.append) the function's __get__ method is called and it will return a new callable acting like the original function, but with whatever was "in front of the dot" automatically passed as the first argument. .
That's why you need to pass 1 - 1 = 0 arguments when calling my_string.upper() and 2 - 1 = 1 argument when calling my_list.append(1).
>>> 'my_string'.upper()
'MY_STRING'
>>>
>>> my_list = []
>>> my_list.append(1)
>>> my_list
[1]
You could even get these modified callables (methods) by explicitly calling __get__ and passing the argument to be bound (what has been before the dot) as its argument.
>>> my_string = 'my_string'
>>> upper_maker = str.upper.__get__(my_string)
>>> upper_maker()
'MY_STRING'
>>>
>>> my_list = []
>>> appender = list.append.__get__(my_list)
>>> appender(1)
>>> my_list
[1]
Finally, here's a short example demonstrating how descriptor instances can detect whether they are being accessed via their owner-class or via an instance.
class Descriptor:
def __get__(self, instance, owner_class):
if instance is None:
print('accessed through class')
# list.append.__get__ would return list.append here
else:
print('accessed through instance')
# list.append.__get__ would build a new callable here
# that takes one argument x and that internally calls
# list.append(instance, x)
class Class:
attribute = Descriptor()
Class.attribute # prints 'accessed through class'
instance = Class()
instance.attribute # prints 'accessed through instance'

Quoting Dave Kirbys answer from Relationship between string module and str:
There is some overlap between the string module and the str type,
mainly for historical reasons. In early versions of Python str objects
did not have methods, so all string manipulation was done with
functions from the string module. When methods were added to the str
type (in Python 1.5?) the functions were left in the string module for
compatibility, but now just forward to the equivalent str method.
However the string module also contains constants and functions that
are not methods on str, such as formatting, character translation etc.

There is nothing at all magical going on with str (except that we have a nice syntactic shortcut to creating one using ""). You can write a class that behaves like str and list to see more clearly what is happening here.
class MyClass():
def __init__(self, arg):
self.val=str(arg)
def do_thing(self):
self.val = "asdf"
def do_thing_with_arg(self, arg):
self.val = "asdf " + str(arg)
def __repr__(self):
return self.val
my_thing = MyClass("qwerty")
# this is like 'hello'.upper()
my_thing.do_thing()
print(my_thing)
# it prints 'asdf'
my_thing = MyClass("qwerty")
# this is like str.upper('hello')
MyClass.do_thing(my_thing)
print(my_thing)
# it prints 'asdf'
my_thing = MyClass("qwerty")
# this is like my_list.append('qwerty')
my_thing.do_thing_with_arg('zxcv')
print(my_thing)
# it prints 'asdf zxcv'
my_thing = MyClass("qwerty")
# this is like list.append(my_list, 'qwerty')
MyClass.do_thing_with_arg(my_thing, 'zxcv')
print(my_thing)
# it prints 'asdf zxcv'
The short version is, you're invoking what looks like an "instance method" on a class, but you are supplying the instance ('self') yourself as the first argument to the function call.

Avoiding creating new method object

According to the Python 2.7.12 Documentation,
When the attribute is a user-defined method object, a new method
object is only created if the class from which it is being retrieved
is the same as, or a derived class of, the class stored in the
original method object; otherwise, the original method object is
used as it is.
I'm trying to understand this paragraph by writing code:
# Parent: The class stored in the original method object
class Parent(object):
# func: The underlying function of original method object
def func(self):
pass
func2 = func
# Child: A derived class of Parent
class Child(Parent):
# Parent.func: The original method object
func = Parent.func
# AnotherClass: Another different class, neither subclasses nor subclassed
class AnotherClass(object):
# Parent.func: The original method object
func = Parent.func
a = Parent.func
b = Parent.func2
c = Child.func
d = AnotherClass.func
print b is a
print c is a
print d is a
The output is:
False
False
False
I expect it to be:
False
False
True

Every time you write Parent.func, you are getting a new unbound method object. So AnotherClass.func never changes, but Parent.func does. You can see this if you do something like this:
a = Parent.func
a2 = Parent.func
b = Parent.func2
c = Child.func
d = AnotherClass.func
d2 = AnotherClass.func
Then:
>>> a is a2
False
>>> d is d2
True
So your comments referring to "the original method object" are somewhat misleading. Each part of your code that evaluates Parent.func is getting a different method object. (Note that the quotation you gave does not apply to Parent.func, because func as written in Parent is not a method object; it is a function object. It is only when you access the attribute by writing Parent.func that you get a method object.)
I'm adding a bit here to clarify the quotation you gave.
I think you are misunderstanding what that quotation is saying. This may be because the quotation is somewhat vague about the distinction between "the attribute" and the value retrieved by getting that attribute.
Let me clarify by separating two notions. In an expression like something.attr I'll use "the raw value" to refer to the actual value stored in the class/object dictionary, i.e., the value of something.__dict__['attr']. I'll use "the evaluated value" to refer to the actual result of evaluating something.attr. The difference between the two is that the descriptor protocol is activated on the raw value to get the evaluated value.
The documentation you quoted is describing a situation where the raw value is a method object. This is the case in your definitions of Child and AnotherClass. The quotation does not apply to func inside Parent, because in that case the raw value is not a method; it is a function. In that case, the following paragraph of the documentation applies:
When a user-defined method object is created by retrieving a user-defined function object...
In other words, the documentation you quoted ("When the attribute is a user-defined method object") applies to cases like this:
obj = [a method object]
class Foo(object):
attr = obj
Your quoted documentation does not apply to the following case. In this case, the attribute is "a user-defined function object":
func = [a function object]
class Foo(object):
attr = func
In your example, there is no "original method object". You can only get a method object after the class is defined. The "original" object is a function object. When you write func in the class body, you are defining a function; it is only when you access Parent.func that you get a method object.

Are functions first class objects in python?

I am learning a tutorial on python.It is explaining how functions are first class objects in Python.
def foo():
pass
print(foo.__class__)
print(issubclass(foo.__class__,object))
The output that I get for the above code is
<type 'function'>
True
This program is supposed to demonstrate that functions are first class objects in python? My questions are as follows.
How does the above code prove that functions are fist class objects?
What are the attributes of a first class object?
what does function.__class__ signify? It returns a tuple <type,function> which doesn't mean much?

Here's what Guido says about first class objects in his blog:
One of my goals for Python was to make it so that all objects were "first class." By this, I meant that I wanted all objects that could be named in the language (e.g., integers, strings, functions, classes, modules, methods, etc.) to have equal status. That is, they can be assigned to variables, placed in lists, stored in dictionaries, passed as arguments, and so forth.
The whole blog post is worth reading.
In the example you posted, the tutorial may be making the point that first class objects are generally descendents of the "object" class.

First-class simply means that functions can be treated as a value -- that is you can assign them to variables, return them from functions, as well as pass them in as a parameter. That is you can do code like:
>>> def say_hi():
print "hi"
>>> def say_bye():
print "bye"
>>> f = say_hi
>>> f()
hi
>>> f = say_bye
>>> f()
bye
This is useful as you can now assign functions to variables like any ordinary variable:
>>> for f in (say_hi, say_bye):
f()
hi
bye
Or write higher order functions (that take functions as parameters):
>>> def call_func_n_times(f, n):
for i in range(n):
f()
>>> call_func_n_times(say_hi, 3)
hi
hi
hi
>>> call_func_n_times(say_bye, 2)
bye
bye
About __class__ in python tells what type of object you have. E.g., if you define an list object in python: a = [1,2,3], then a.__class__ will be <type 'list'>. If you have a datetime (from datetime import datetime and then d = datetime.now(), then the type of d instance will be <type 'datetime.datetime'>. They were just showing that in python a function is not a brand new concept. It's just an ordinary object of <type 'function'>.

You proved that functions are first class objects because you were allowed to pass foo as an argument to a method.
The attributes of first class objects was nicely summarised in this post: https://stackoverflow.com/a/245208/3248346
Depending on the language, this can
imply:
being expressible as an anonymous literal value
being storable in variables
being storable in data structures
having an intrinsic identity (independent of any given name)
being comparable for equality with other entities
being passable as a parameter to a procedure/function
being returnable as the result of a procedure/function
being constructible at runtime
being printable
being readable
being transmissible among distributed processes
being storable outside running processes

Regarding your third question, <type 'function'> isn't a tuple. Python's tuple notation is (a,b), not angle brackets.
foo.__class__ returns a class object, that is, an object which represents the class to which foo belongs; class objects happen to produce descriptive strings in the interpreter, in this case telling you that the class of foo is the type called 'function'. (Classes and types are basically the same in modern Python.)
It doesn't mean a whole lot other than that, like any other object, functions have a type:
>>> x = 1
>>> x.__class__
<type 'int'>
>>> y = "bar"
>>> y.__class__
<type 'str'>
>>> def foo(): pass
...
>>> foo.__class__
<type 'function'>

Regarding your comment to #I.K.s answer, f_at_2() in the following would be the method.
def f_at_2(f):
return f(2)
def foo(n):
return n ** n
def bar(n):
return n * n
def baz(n):
return n / 2
funcs = [foo, bar, baz]
for f in funcs:
print f.func_name, f_at_2(f)
...
>>>
foo 4
bar 4
baz 1
>>>
A method is a function of/in a class, but the concept also applies to a function (outside of a class). The functions (as objects) are contained in a data structure and passed to another object.

confusing about python class and instance variables

I saw the following Python documentation which says that "define variables in a Class" will be class variables:
"Programmer's note: Variables defined in the class definition are
class variables; they are shared by all instances. "
but as I wrote sample code like this:
class CustomizedMethods(object):
class_var1 = 'foo'
class_var2 = 'bar'
cm1 = CustomizedMethods()
cm2 = CustomizedMethods()
print cm1.class_var1, cm1.class_var2 #'foo bar'
print cm2.class_var1, cm2.class_var2 #'foo bar'
cm2.class_var1, cm2.class_var2 = 'bar','for'
print cm1.class_var1, cm1.class_var2 #'foo bar' #here not changed as my expectation
print cm2.class_var1, cm2.class_var2 #'bar foo' #here has changed but they seemed to become instance variables.
I'm confused since what I tried is different from Python's official documentation.

When you assign an attribute on the instance, it is assigned on the instance, even if it previously existed on the class. At first, class_var1 and class_var2 are indeed class attributes. But when you do cm1.class_var1 = "bar", you are not changing this class attribute. Rather, you are creating a new attribute, also called class_var1, but this one is an instance attribute on the instance cm1.
Here is another example showing the difference, although it still may be a bit tough to grasp:
>>> class A(object):
... var = []
>>> a = A()
>>> a.var is A.var
True
>>> a.var = []
>>> a.var is A.var
False
At first, a.var is A.var is true (i.e., they are the same object): since a doesn't have it's own attribute called var, trying to access that goes through to the class. After you give a its own instance attribute, it is no longer the same as the one on the class.

You're assigning attributes on the instances, so yes, they become instance variables at that point. Python looks for attributes on whatever object you specify, then if it can't find them there, looks up the inheritance chain (to the class, the class's parents, etc.). So the attribute you assign on the instance "shadows" or "hides" the class's attribute of the same name.

Strings are immutable, so the difference between a class and instance variable isn't as noticable. For immutable variables in a class definition, the main thing to notice is less use of memory (i.e., if you have 1,000 instances of CustomizedMethods, there's still only one instance of the string "foo" stored in memory.)
However, using mutable variables in a class can introduce subtle bugs if you don't know what you're doing.
Consider:
class CustomizedMethods(object):
class_var = {}
cm1 = CustomizedMethods()
cm2 = CustomizedMethods()
cm1.class_var['test'] = 'foo'
print cm2.class_var
'foo'
cm2.class_var['test'] = 'bar'
print cm1.class_var
'bar'

When you reassigned the cm2 variables, you created new instance variables that "hid" the class variables.
>>> CustomizedMethods.class_var1 = 'one'
>>> CustomizedMethods.class_var2 = 'two'
>>> print cm1.class_var1, cm1.class_var2
one two
>>> print cm2.class_var1, cm2.class_var2
bar for

Try to
print cm1.__dict__
print cm2.__dict__
it will be enlightning...
When you ask cm2 for an attribute it first looks among the attributes of the instance (if one matches the name) and then if there is no matching attribute among the class attributes.
So class_var1 and class_var2 are the names of the class attributes.
Try also the following:
cm2.__class__.class_var1 = "bar_foo"
print cm1.class_var1
what do you expect?

Best of two ways to declare a class variable in Python

The way I usually declare a class variable to be used in instances in Python is the following:
class MyClass(object):
def __init__(self):
self.a_member = 0
my_object = MyClass()
my_object.a_member # evaluates to 0
But the following also works. Is it bad practice? If so, why?
class MyClass(object):
a_member = 0
my_object = MyClass()
my_object.a_member # also evaluates to 0
The second method is used all over Zope, but I haven't seen it anywhere else. Why is that?
Edit: as a response to sr2222's answer. I understand that the two are essentially different. However, if the class is only ever used to instantiate objects, the two will work he same way. So is it bad to use a class variable as an instance variable? It feels like it would be but I can't explain why.

The question is whether this is an attribute of the class itself or of a particular object. If the whole class of things has a certain attribute (possibly with minor exceptions), then by all means, assign an attribute onto the class. If some strange objects, or subclasses differ in this attribute, they can override it as necessary. Also, this is more memory-efficient than assigning an essentially constant attribute onto every object; only the class's __dict__ has a single entry for that attribute, and the __dict__ of each object may remain empty (at least for that particular attribute).
In short, both of your examples are quite idiomatic code, but they mean somewhat different things, both at the machine level, and at the human semantic level.
Let me explain this:
>>> class MyClass(object):
... a_member = 'a'
...
>>> o = MyClass()
>>> p = MyClass()
>>> o.a_member
'a'
>>> p.a_member
'a'
>>> o.a_member = 'b'
>>> p.a_member
'a'
On line two, you're setting a "class attribute". This is litterally an attribute of the object named "MyClass". It is stored as MyClass.__dict__['a_member'] = 'a'. On later lines, you're setting the object attribute o.a_member to be. This is completely equivalent to o.__dict__['a_member'] = 'b'. You can see that this has nothing to do with the separate dictionary of p.__dict__. When accessing a_member of p, it is not found in the object dictionary, and deferred up to its class dictionary: MyClass.a_member. This is why modifying the attributes of o do not affect the attributes of p, because it doesn't affect the attributes of MyClass.

The first is an instance attribute, the second a class attribute. They are not the same at all. An instance attribute is attached to an actual created object of the type whereas the class variable is attached to the class (the type) itself.
>>> class A(object):
... cls_attr = 'a'
... def __init__(self, x):
... self.ins_attr = x
...
>>> a1 = A(1)
>>> a2 = A(2)
>>> a1.cls_attr
'a'
>>> a2.cls_attr
'a'
>>> a1.ins_attr
1
>>> a2.ins_attr
2
>>> a1.__class__.cls_attr = 'b'
>>> a2.cls_attr
'b'
>>> a1.ins_attr = 3
>>> a2.ins_attr
2

Even if you are never modifying the objects' contents, the two are not interchangeable. The way I understand it, accessing class attributes is slightly slower than accessing instance attributes, because the interpreter essentially has to take an extra step to look up the class attribute.
Instance attribute
"What's a.thing?"
Class attribute
"What's a.thing? Oh, a has no instance attribute thing, I'll check its class..."

I have my answer! I owe to #mjgpy3's reference in the comment to the original post. The difference comes if the value assigned to the class variable is MUTABLE! THEN, the two will be changed together. The members split when a new value replaces the old one
>>> class MyClass(object):
... my_str = 'a'
... my_list = []
...
>>> a1, a2 = MyClass(), MyClass()
>>> a1.my_str # This is the CLASS variable.
'a'
>>> a2.my_str # This is the exact same class variable.
'a'
>>> a1.my_str = 'b' # This is a completely new instance variable. Strings are not mutable.
>>> a2.my_str # This is still the old, unchanged class variable.
'a'
>>> a1.my_list.append('w') # We're changing the mutable class variable, but not reassigning it.
>>> a2.my_list # This is the same old class variable, but with a new value.
['w']
Edit: this is pretty much what bukzor wrote. They get the best answer mark.