Develop Reference

Why do you not have to return objects in Python?

I am a newbie to Python and programming in general and I have just gotten into OOP. In Python, whatever is defined in a function's namespace is only valid while inside that namespace; once outside that space things like variables are forgotten:
def foo():
a = 3
print(a)
NameError: name 'a' is not defined
As far as I know, besides returning data from functions, any information inside the function is lost at the end of the function call, and therein lies my question. Take the following code:
class Example:
def __init__(self, name):
self.name = name
def foo(self):
self.name = 'John'
bar = Example('Jake')
bar.foo()
print(bar.name)
'John'
My question is: why don't you have to return objects after methods? In normal functions any variables are forgotten, but in methods it seems data is actually appended to the object itself, such as the foo() method being able to reference self.name despite self.name first being referenced in another method. Is this correct? Or is there a better technical explanation?

First, you don't have to return a value in a method, not in python, and not in many other programming languages. for example:
def append_a(lst):
lst.append('a')
bob = []
append_a(bob)
print(bob)
['a']
Above we do not return anything in the function, but we use it to modify an existing data structure, this is very common almost anywhere.
Secondly, in your second example, you created an instance of the class Example, when you look at self.something you are looking at a member of the class, unlike other languages, where often members are only declared once, in python you can dynamically add members.
Thus when looking at bar.name you are looking at a member of the class, its value on the instance bar. If you would look at a different instance, the value will be different.
class Example:
def __init__(self, name):
self.name = name
def foo(self):
self.name = 'John'
bar = Example('Jake')
bob = Example('Bob')
bar.foo()
print(bar.name)
print(bob.name)
John
Bob

To understand this you will need to understand how self works. You can learn more here:
Understanding self in python
In a nutshell, self refers to the calling object. Invoking self.variable refers to the variable associated with the calling object. Python is smart enough to create one if it doesn't exist.
Calling self.variable inside a class is the same as calling object.variable with your object reference
Consider the following example to prove this:
class Example:
def print_x(self):
print(self.x)
obj = Example()
obj.x = 5; # Create a new attribute of the object and assign it a value 5
print(obj.x) # Outputs 5
obj.print_x() # Outputs 5
In your example, I've added a couple of print statements to help you understand the state of the program during the execution:
class Example:
def __init__(self, name):
print(dir(self)) # Printing object contents before initializing name
self.name = name # New attribute 'name' created
print(dir(self)) # Printing object contents after initializing name
def foo(self):
print("Before foo, self.name = "+ self.name)
self.name = 'John'
print("After foo, self.name = "+ self.name)
bar = Example('Jake')
bar.foo()
print(bar.name)
The output of the above code is
['__doc__', '__init__', '__module__', 'foo']
['__doc__', '__init__', '__module__', 'foo', 'name']
Before foo, self.name = Jake
After foo, self.name = John
John
I will walk you through this code. When we first create bar, the __init__() method is called. Here we print the contents of the object using dir(self). The output ['__doc__', '__init__', '__module__', 'foo'] indicates that the object has only one member, the 'foo' method.
Now we create a new attribute called 'name' and assign it the value 'Jake'. Thus the object now has another member, the 'name' attribute as seen by the output of the next dir(self) ['__doc__', '__init__', '__module__', 'foo', 'name']
Now we call the foo method and print the value before and after the method. Before the name is changed in foo, the value of name associated with the object is "Jake". However, after the name is changed, the value of self.name is "John". This is indicated by
Before foo, self.name = Jake
After foo, self.name = John`
We next verify that the change made by changing self.name has indeed changed the value of name in bar by printing bar.name which gives us the expected output, John
Now coming back to your question, self.name is not an ordinary variable inside some method that is lost when we are out of scope. self can be used essentially anywhere inside the class to refer to the calling object (bar in this case). It is used to manipulate this calling object. Now since bar is within the scope, we are able to print its name attribute.
In normal functions any variables are forgotten but in methods it
seems data is actually appended to the object itself
self manipulates the attributes in the object and is not limited to the scope of the method.

There isn't a lot of difference between methods and functions (See this thread for details)
Though one important distinction, that you might have immediately noticed, is that methods have self as their first argument and we say that method foo is "bound" to instance bar of class Example, meaning simply that foo will be called with its first argument (self) as the instance (bar) itself
With this knowledge lets see what the following function does:
class Example:
def __init__(self, name):
self.name = name
def foo(self):
self.name = 'John'
bar = Example('Jake')
In method init, you assign name to self. But what is self? It is bar itself, so calling init can be thought of as doing
bar.name = 'Jake'
Then when you called method foo
bar.foo()
You equivalently did
bar.name = 'John'
So, it should not be surprising when finally the output of the following was 'John'
print(bar.name) # John
About your query regarding methods not having to return anything, it is not quite so. Methods and functions may or may not return anything as (See this answer). But in this context, what is happening is an object passed to a method is being manipulated (self, which is bar, is being assigned a name) and because the object is alive after the method call completes, we can observe the changes done by the method (ie, we can see that bar.name is changed to 'John')
This works works with functions as well:
def another_foo(self):
self.name = 'Mark'
baz = Example('Jake')
another_foo(baz)
print(baz.name) # Mark
You see, this function did not return anything either. It worked just like the method foo by manipulating its argument. In fact, you could add it to the Example class and use it like a method
Example.another_foo = another_foo
new_bar = Example('Jake')
print(new_bar.name) # Jake
new_bar.another_foo()
print(new_bar.name) # Mark

Welcome to Stack Overflow. You are thinking in the right direction. I'll start at a more basic level so that other beginner viewers can understand what is going on.
Think of Example as a template for creating new objects of a certain kind -- meaning the objects will all have the same attributes (also known as properties) and functionalities. For example, by drawing an analogy with real-life objects, all cars have the attribute "speed" and the functionality "accelerate".
The attribute values will be specific to the objects. For example, one car will have 0 mph speed and another 70 mph.
The attribute values at any point describe the state of an object. Methods, which you can think of as the object's functionalities, allow the ability to change an object's state.
bar is an object that you created using the template (that is, class) Example. If you have to describe this object's state, you'd tell us the values of its attributes. In this case, the object has only one attribute: name.
Now, this is the important part: name is not just any variable, it's an attribute of any object you make from the class Example. Think of name as always attached to the object. That's why you wrote self.name = 'John' and not name = 'John'. In the latter case, name would not be a part of the object bar, and no other method in the class would have access to the name variable.
So, to summarize, when you have created an object out of a class, think of the object as having various attributes. All the methods, or functionalities, of the object will have access to all those attributes. The values of the attributes would describe, at any point, the object's state at that moment. It is through that object's methods that one would change its state.
Finally, here's a great tool to visualize what happens at each line of your code: pythontutor.com

Well, this is what is called as 'scoping' in OOP. And though the difference between classes and functions might seem subtle, it makes a big difference.
Simply said, functions rely on global and local variables. Global variables are objects that you define in the global workspace. This is like having a workbench and getting to use a common object across your functions and classes that you define. And so, for example, if you modify your function foo() to include an argument, like so:
a = 3 # This is globally defined
def foo(a):
a = 3 * a
return a
print(foo(a))
9
Changing the global value of a to something like 2 will give you 6 as the answer. We 'return' an object because it is locally defined within the scope of a function. In this example above, foo() performs the local operation 3 * a and defines it locally to a itself and returns it. And so you can also skip the global variable a when executing foo() and define it locally within the function scope:
print(foo(6)) # Locally defined
18
print(a)
3 # Returns global variable
Classes on the other hand need to be declared and that has been very nicely explained by members (see above).

How to access "__" (double underscore) variables in methods added to a class

Background
I wish to use a meta class in order to add helper methods based on the original class. If the method I wish to add uses self.__attributeName I get an AttributeError (because of name mangling) but for an existing identical method this isn't a problem.
Code example
Here is a simplified example
# Function to be added as a method of Test
def newfunction2(self):
"""Function identical to newfunction"""
print self.mouse
print self._dog
print self.__cat
class MetaTest(type):
"""Metaclass to process the original class and
add new methods based on the original class
"""
def __new__(meta, name, base, dct):
newclass = super(MetaTest, meta).__new__(
meta, name, base, dct
)
# Condition for adding newfunction2
if "newfunction" in dct:
print "Found newfunction!"
print "Add newfunction2!"
setattr(newclass, "newfunction2", newfunction2)
return newclass
# Class to be modified by MetaTest
class Test(object):
__metaclass__ = MetaTest
def __init__(self):
self.__cat = "cat"
self._dog = "dog"
self.mouse = "mouse"
def newfunction(self):
"""Function identical to newfunction2"""
print self.mouse
print self._dog
print self.__cat
T = Test()
T.newfunction()
T.newfunction2() # AttributeError: 'Test' object has no attribute '__cat'
Question
Is there a way of adding newfunction2 that could use self.__cat?
(Without renaming self.__cat to self._cat.)
And maybe something more fundamental, why isn't self.__cat being treated in the same way for both cases since newfunction2 is now part of Test?

Name mangling happens when the methods in a class are compiled. Attribute names like __foo are turned in to _ClassName__foo, where ClassName is the name of the class the method is defined in. Note that you can use name mangling for attributes of other objects!
In your code, the name mangling in newfunction2 doesn't work because when the function is compiled, it's not part of the class. Thus the lookups of __cat don't get turned into __Test_cat the way they did in Test.__init__. You could explicitly look up the mangled version of the attribute name if you want, but it sounds like you want newfunction2 to be generic, and able to be added to multiple classes. Unfortunately, that doesn't work with name mangling.
Indeed, preventing code not defined in your class from accessing your attributes is the whole reason to use name mangling. Usually it's only worth bothering with if you're writing a proxy or mixin type and you don't want your internal-use attributes to collide with the attributes of the class you're proxying or mixing in with (which you won't know in advance).

To answer both of your questions:
You will need to change self.__cat when you need to call it from newfunction2 to self._Test__cat thanks to the name mangling rule.
Python documentation:
This mangling is done without regard to the syntactic position of the
identifier, as long as it occurs within the definition of a class.
Let me brake it down for you, it's saying that it doesn't matter where your interpreter is reading when it encounters a name mangled name. The name will only be mangled if it occurs in the definition of a class, which in your case, it's not. Since it's not directly "under" a class definition. So when it reads self.__cat, it's keeping it at self.__cat, not going to textually replace it with self._Test__cat since it isn't defined inside theTest class.

You can use <Test instance>._Test__cat to access the __cat attribute from the Test class. (where <Test instance> is replaced by self or any other instance of the Test class)
learn more in the Python doc

class B:
def __init__(self):
self.__private = 0
def __private_method(self):
'''A private method via inheritance'''
return ('{!r}'.format(self))
def internal_method(self):
return ('{!s}'.format(self))
class C(B):
def __init__(self):
super().__init__()
self.__private = 1
def __private_method(self):
return 'Am from class C'
c = C()
print(c.__dict__)
b = B()
print(b.__dict__)
print(b._B__private)
print(c._C__private_method())

Python Class Script Running Errors?

I am trying to make a class using Python 2.7.9 and it keeps running into errors.
Here is my script:
class Hero():
def __init__(self, name):
self.health=50
def eat(self, food):
if(food=='apple'):
self.health+=10
self.name=Jeff
Jeff=Hero('Jeff')
def introduce(self, name):
print Jeff.name
def checkAtt():
print Jeff.health
introduce()
It keeps saying name 'Jeff' is not defined.

Your code has numerous problems. The first, that is causing the specific error, is that you attempt to assign:
self.name = Jeff
before you have defined either self or Jeff. self is conventionally only used inside instance methods (like your Hero.eat), where it is the name of the first parameter.
Secondly, your Hero.__init__ doesn't actually assign the name parameter to the name attribute; it should look like:
class Hero(object): # Note inheritance from 'object' for new-style class
def __init__(self, name):
self.name = name # Note assignment of instance attribute
self.health = 50
...
jeff = Hero("Jeff") will call Hero.__init__, creating new Hero instance, setting its name attribute to "Jeff" (and health attribute to 50) and assigning that instance to the name jeff.
Thirdly, you have two standalone functions (introduce and checkAtt) that should probably also be instance methods:
def Hero(object):
...
def introduce(self):
print self.name
...
jeff = Hero("Jeff")
jeff.introduce() # equivalent to 'Hero.introduce(jeff)'
or, if remaining as standalone functions, take a single parameter, the Hero instance to operate on (which should not be called self, again by convention) - there is not much point writing a function that only works if it's run in a scope where the name Jeff is available!
class Hero(object):
...
def introduce(hero):
print hero.name
jeff = Hero("Jeff")
introduce(jeff)
Note the indentation in these two different cases - it is very important in Python. Also, note the different ways of calling introduce depending on whether it's an instance method or a function.
I suggest you read the tutorial on classes and the style guide.

Why does Python seem to treat instance variables as shared between objects?

I was working on a simple script today when I noticed a strange quirk in the way Python treats instance variables.
Say we have a simple object:
class Spam(object):
eggs = {}
def __init__(self, bacon_type):
self.eggs["bacon"] = bacon_type
def __str__(self):
return "My favorite type of bacon is " + self.eggs["bacon"]
And we create two instances of this object with separate arguments:
spam1 = Spam("Canadian bacon")
spam2 = Spam("American bacon")
print spam1
print spam2
The results are puzzling:
My favorite type of bacon is American bacon
My favorite type of bacon is American bacon
It seems like the "eggs" dictionary is shared between all the different "Spam" instances -- either that or it is overwritten every time a new instance is created. This isn't really a problem in every day life, since we can solve it by declaring the instance variable in the initialization function:
class Spam(object):
def __init__(self, bacon_type):
self.eggs = {}
self.eggs["bacon"] = bacon_type
def __str__(self):
return "My favorite type of bacon is " + self.eggs["bacon"]
spam1 = Spam("Canadian bacon")
spam2 = Spam("American bacon")
print spam1
print spam2
With the code written this way, the result is what we expect:
My favorite type of bacon is Canadian bacon
My favorite type of bacon is American bacon
So while I'm not held up by this behavior, I don't understand why Python works this way. Can anyone shed some light on this?

As Ignacio has posted, variables which are assigned to at class scope in Python are class variables. Basically, in Python, a class is just a list of statements under a class statement. Once that list of statements finishes executing, Python scoops up any variables that were created during the course of that execution and makes a class out of them. If you want an instance variable, you actually do have to assign it to the instance.
On another note: it sounds like you may be coming to this from a Java (or Java-like) perspective. So perhaps you know that because Java requires variables to be explicitly declared, it needs to have instance variable declarations at class scope.
class Foo {
String bar;
public Foo() {
this.bar = "xyz";
}
}
Note that only the declaration is at class scope. In other words, the memory allocated for that variable is part of the class "template," but the actual value of the variable is not.
Python doesn't have any need for variable declarations. So in the Python translation, you just drop the declaration.
class Foo:
# String bar; <-- useless declaration is useless
def __init__(self):
self.bar = "xyz"
The memory will be allocated when it's needed; only the assignment is actually written out. And that goes in the constructor, just like in Java.

That's not an instance variable, that's a class variable. The fact that it is being accessed via an instance is irrelevant; it's still the same object.

Unlike in a compiled language, the class statement in Python is actually code that executes, and creates a class object (not an instance!) in memory.
Any symbols that are defined when the class block runs belong to the class itself. This includes variables, such as the eggs variable in your first example, as well as the __init__ and __str__ methods that you define. All of those are created when the class is defined, and they are all part of the class.
Instance variables are not created until you actually create an instance of the object, and the __init__ method is run, and they have to be attributes of self.
So, when the python interpreter executes
class Spam(object):
eggs = {}
def __init__(self):
<stuff>
def __str__(self):
<other stuff>
it is actually building a class object at run time. It executes the code "eggs={}", and it executes the two def statements, and it builds a class that has three attributes: eggs, __init__ and __str__.
Later, when it executes
spam1 = Spam()
Then it creates a new instance, and runs its __init__ method. The __init__ method itself, of course, belongs to the class; it is shared between all instances, just like the eggs attribute.
The instance itself gets passed in as the self parameter, and anything you define on it belong to that instance alone. That's why self has to be passed into every class method -- in python, the methods actually belong to the class itself, and self is the only way that you have to refer to the instance.

Passing a variable to a method in a class

I'm new to using classes and I'm trying to pass a variable to one of the methods inside of my class. How do I do it?
Here's an example of what I'm trying to accomplish:
class a_class():
def a_method(txt):
print txt
instance = a_class()
instance.a_method('hello world!)
P.S. I don't understand the whole self and __blah__ concepts yet, and I will avoid them at this point if I don't have to use them.

When writing an instance method for a class in Python- which looks exactly like what you've just coded up- you can't avoid using self. The first parameter to an instance method in Python is always the object the method is being called on. self is not a reserved word in Python- just the traditional name for that first parameter.
To quote from the official Python tutorial, chapter 9:
[...] the special thing about methods is that the object is passed as the first argument of the function. In our example, the call x.f() is exactly equivalent to MyClass.f(x). In general, calling a method with a list of n arguments is equivalent to calling the corresponding function with an argument list that is created by inserting the method’s object before the first argument.
Therefore, you need to define two parameters for your method. The first is always self- at least that is the conventional name- and the second is your actual parameter. So your code snippet should be:
class a_class(object):
def a_method(self, txt):
print txt
instance = a_class()
instance.a_method('hello world!')
Note that the class explicitly inherits from object (I'm not sure empty parentheses there are legal). You can also provide no inheritance, which is identical for most purposes, but different in some details of the behavior of the type system; the inheritance from object defines a_class as a new-style class rather than an old-style class, which is irrelevant for most purposes but probably worth being aware of.

You need to have
class a_class():
def a_method(self,txt):
print txt
The first variable of a class method always contains a reference to the object no matter what variable name you use. (Unless you are using it as a static method).

Instance Methods in Python must be provided the instance (given as self) as the first parameter in the method signature.
class a_class():
def a_method(self,txt):
print txt
That should be what you're looking for. Additionally, if you were to interact with a member variable you'd want to do something like this:
class a_class():
name = "example"
def a_method(self,txt):
print txt
print self.name

The self concept and the use of __init__ really isn't that confusing and it is essential to writing good Python code. __init__ is called on instantiation of a class, and simply include a self parameter in every class method, you can then use self to reference the instance of the class.
class a_class():
def __init__(self):
self.count = 0
def a_method(self, txt):
self.count += 1
print str(self.count), txt
instance = a_class()
instance.a_method('hello world!')
# prints "1 hello world!"
instance.a_method('hello again!')
# prints "2 hello again!"