Develop Reference

What is the difference between assign and declare a variable in python? [duplicate]

I want to clarify how variables are declared in Python.
I have seen variable declaration as
class writer:
path = ""
sometimes, there is no explicit declaration but just initialization using __init__:
def __init__(self, name):
self.name = name
I understand the purpose of __init__, but is it advisable to declare variable in any other functions?
How can I create a variable to hold a custom type?
class writer:
path = "" # string value
customObj = ??

Okay, first things first.
There is no such thing as "variable declaration" or "variable initialization" in Python.
There is simply what we call "assignment", but should probably just call "naming".
Assignment means "this name on the left-hand side now refers to the result of evaluating the right-hand side, regardless of what it referred to before (if anything)".
foo = 'bar' # the name 'foo' is now a name for the string 'bar'
foo = 2 * 3 # the name 'foo' stops being a name for the string 'bar',
# and starts being a name for the integer 6, resulting from the multiplication
As such, Python's names (a better term than "variables", arguably) don't have associated types; the values do. You can re-apply the same name to anything regardless of its type, but the thing still has behaviour that's dependent upon its type. The name is simply a way to refer to the value (object). This answers your second question: You don't create variables to hold a custom type. You don't create variables to hold any particular type. You don't "create" variables at all. You give names to objects.
Second point: Python follows a very simple rule when it comes to classes, that is actually much more consistent than what languages like Java, C++ and C# do: everything declared inside the class block is part of the class. So, functions (def) written here are methods, i.e. part of the class object (not stored on a per-instance basis), just like in Java, C++ and C#; but other names here are also part of the class. Again, the names are just names, and they don't have associated types, and functions are objects too in Python. Thus:
class Example:
data = 42
def method(self): pass
Classes are objects too, in Python.
So now we have created an object named Example, which represents the class of all things that are Examples. This object has two user-supplied attributes (In C++, "members"; in C#, "fields or properties or methods"; in Java, "fields or methods"). One of them is named data, and it stores the integer value 42. The other is named method, and it stores a function object. (There are several more attributes that Python adds automatically.)
These attributes still aren't really part of the object, though. Fundamentally, an object is just a bundle of more names (the attribute names), until you get down to things that can't be divided up any more. Thus, values can be shared between different instances of a class, or even between objects of different classes, if you deliberately set that up.
Let's create an instance:
x = Example()
Now we have a separate object named x, which is an instance of Example. The data and method are not actually part of the object, but we can still look them up via x because of some magic that Python does behind the scenes. When we look up method, in particular, we will instead get a "bound method" (when we call it, x gets passed automatically as the self parameter, which cannot happen if we look up Example.method directly).
What happens when we try to use x.data?
When we examine it, it's looked up in the object first. If it's not found in the object, Python looks in the class.
However, when we assign to x.data, Python will create an attribute on the object. It will not replace the class' attribute.
This allows us to do object initialization. Python will automatically call the class' __init__ method on new instances when they are created, if present. In this method, we can simply assign to attributes to set initial values for that attribute on each object:
class Example:
name = "Ignored"
def __init__(self, name):
self.name = name
# rest as before
Now we must specify a name when we create an Example, and each instance has its own name. Python will ignore the class attribute Example.name whenever we look up the .name of an instance, because the instance's attribute will be found first.
One last caveat: modification (mutation) and assignment are different things!
In Python, strings are immutable. They cannot be modified. When you do:
a = 'hi '
b = a
a += 'mom'
You do not change the original 'hi ' string. That is impossible in Python. Instead, you create a new string 'hi mom', and cause a to stop being a name for 'hi ', and start being a name for 'hi mom' instead. We made b a name for 'hi ' as well, and after re-applying the a name, b is still a name for 'hi ', because 'hi ' still exists and has not been changed.
But lists can be changed:
a = [1, 2, 3]
b = a
a += [4]
Now b is [1, 2, 3, 4] as well, because we made b a name for the same thing that a named, and then we changed that thing. We did not create a new list for a to name, because Python simply treats += differently for lists.
This matters for objects because if you had a list as a class attribute, and used an instance to modify the list, then the change would be "seen" in all other instances. This is because (a) the data is actually part of the class object, and not any instance object; (b) because you were modifying the list and not doing a simple assignment, you did not create a new instance attribute hiding the class attribute.

This might be 6 years late, but in Python 3.5 and above, you can give a hint about a variable type like this:
variable_name: type_name
or this:
variable_name # type: shinyType
This hint has no effect in the core Python interpreter, but many tools will use it to aid the programmer in writing correct code.
So in your case(if you have a CustomObject class defined), you can do:
customObj: CustomObject
See this or that for more info.

There's no need to declare new variables in Python. If we're talking about variables in functions or modules, no declaration is needed. Just assign a value to a name where you need it: mymagic = "Magic". Variables in Python can hold values of any type, and you can't restrict that.
Your question specifically asks about classes, objects and instance variables though. The idiomatic way to create instance variables is in the __init__ method and nowhere else — while you could create new instance variables in other methods, or even in unrelated code, it's just a bad idea. It'll make your code hard to reason about or to maintain.
So for example:
class Thing(object):
def __init__(self, magic):
self.magic = magic
Easy. Now instances of this class have a magic attribute:
thingo = Thing("More magic")
# thingo.magic is now "More magic"
Creating variables in the namespace of the class itself leads to different behaviour altogether. It is functionally different, and you should only do it if you have a specific reason to. For example:
class Thing(object):
magic = "Magic"
def __init__(self):
pass
Now try:
thingo = Thing()
Thing.magic = 1
# thingo.magic is now 1
Or:
class Thing(object):
magic = ["More", "magic"]
def __init__(self):
pass
thing1 = Thing()
thing2 = Thing()
thing1.magic.append("here")
# thing1.magic AND thing2.magic is now ["More", "magic", "here"]
This is because the namespace of the class itself is different to the namespace of the objects created from it. I'll leave it to you to research that a bit more.
The take-home message is that idiomatic Python is to (a) initialise object attributes in your __init__ method, and (b) document the behaviour of your class as needed. You don't need to go to the trouble of full-blown Sphinx-level documentation for everything you ever write, but at least some comments about whatever details you or someone else might need to pick it up.

For scoping purpose, I use:
custom_object = None

Variables have scope, so yes it is appropriate to have variables that are specific to your function. You don't always have to be explicit about their definition; usually you can just use them. Only if you want to do something specific to the type of the variable, like append for a list, do you need to define them before you start using them. Typical example of this.
list = []
for i in stuff:
list.append(i)
By the way, this is not really a good way to setup the list. It would be better to say:
list = [i for i in stuff] # list comprehension
...but I digress.
Your other question.
The custom object should be a class itself.
class CustomObject(): # always capitalize the class name...this is not syntax, just style.
pass
customObj = CustomObject()

As of Python 3, you can explicitly declare variables by type.
For instance, to declare an integer one can do it as follows:
x: int = 3
or:
def f(x: int):
return x
see this question for more detailed info about it:
Explicitly declaring a variable type in Python

Is there a way to list the attributes of a class without instantiating an object?

In Python 3.5, say I have:
class Foo:
def __init__(self, bar, barbar):
self.bar = bar
self.barbar = barbar
I want to get the list ["bar", "barbar"] from the class.
I know I can do:
foo = Foo(1, 2)
foo.__dict__.keys()
Is there a way to get ["bar", "barbar"] without instantiating an object?

No because the attributes are dynamic (so called instance attributes). Consider the following,
class Foo:
def __init__( self ):
self.bar = 1
def twice( self ):
self.barbar = 2
f = Foo()
print( list(f.__dict__.keys() ) )
f.twice()
print( list(f.__dict__.keys() ) )
In the first print, only f.bar was set, so that's the only attributes that's shown when printing the attribute keys. But after calling f.twice(), you create a new attribute to f and now printing it show both bar and barbar.

Warning -
The following isn't foolproof in always providing 100% correct data. If you end up having something like self.y = int(1) in your __init__, you will end up including the int in your collection of attributes, which is not a wanted result for your goals. Furthermore, if you happen to add a dynamic attribute somewhere in your code like Foo.some_attr = 'pork', then you will never see that either. Be aware of what it is that you are inspecting at what point of your code, and understand why you have and don't have those inclusions in your result. There are probably other "breakages" that will not give you the full 100% expectation of what are all the attributes associated with this class, but nonetheless, the following should give you something that you might be looking for.
However, I strongly suggest you take the advice of the other answers here and the duplicate that was flagged that explains why you can't/should not do this.
The following is a form of solution you can try to mess around with:
I will expand on the inspect answer.
However, I do question (and probably would advice against) the validity of doing something like this in production-ready code. For investigative purposes, sure, knock yourself out.
By using the inspect module as indicated already in one of the other answers, you can use the getmembers method which you can then iterate through the attributes and inspect the appropriate data you wish to investigate.
For example, you are questioning the dynamic attributes in the __init__
Therefore, we can take this example to illustrate:
from inspect import getmembers
class Foo:
def __init__(self, x):
self.x = x
self.y = 1
self.z = 'chicken'
members = getmembers(Foo)
for member in members:
if '__init__' in member:
print(member[1].__code__.co_names)
Your output will be a tuple:
('x', 'y', 'z')
Ultimately, as you inspect the class Foo to get its members, there are attributes you can further investigate as you iterate each member. Each member has attributes to further inspect, and so on. For this particular example, we focus on __init__ and inspect the __code__ (per documentation: The __code__ object representing the compiled function body) attribute which has an attribute called co_names which provides a tuple of members as indicated above with the output of running the code.

Try classname.annotations.keys()

As Lærne mentioned, attributes declared inside of functions (like __init__), are dynamic. They effectively don't exist until the __init__ function is called.
However, there is a way to do what you want.
You can create class attributes, like so:
class Foo:
bar = None
barbar = None
def __init__(self, bar, barbar):
self.bar = bar
self.barbar = barbar
And you can access those attributes like this:
[var for var in vars(Foo).keys() if not var.startswith('__')]
Which gives this result:
['bar', 'barbar']

What happens when we edit(append, remove...) a list and can we execute actions each time a list is edited

I would like to know if there is a way to create a list that will execute some actions each time I use the method append(or an other similar method).
I know that I could create a class that inherits from list and overwrite append, remove and all other methods that change content of list but I would like to know if there is an other way.
By comparison, if I want to print 'edited' each time I edit an attribute of an object I will not execute print("edited") in all methods of the class of that object. Instead, I will only overwrite __setattribute__.
I tried to create my own type which inherits of list and overwrite __setattribute__ but that doesn't work. When I use myList.append __setattribute__ isn't called. I would like to know what's realy occured when I use myList.append ? Is there some magic methods called that I could overwrite ?
I know that the question have already been asked there : What happens when you call `append` on a list?. The answer given is just, there is no answer... I hope it's a mistake.
I don't know if there is an answer to my request so I will also explain you why I'm confronted to that problem. Maybe I can search in an other direction to do what I want. I have got a class with several attributes. When an attribute is edited, I want to execute some actions. Like I explain before, to do this I am use to overwrite __setattribute__. That works fine for most of attributes. The problem is lists. If the attribute is used like this : myClass.myListAttr.append(something), __setattribute__ isn't called while the value of the attribute have changed.
The problem would be the same with dictionaries. Methods like pop doesn't call __setattribute__.

If I understand correctly, you would want something like Notify_list that would call some method (argument to the constructor in my implementation) every time a mutating method is called, so you could do something like this:
class Test:
def __init__(self):
self.list = Notify_list(self.list_changed)
def list_changed(self,method):
print("self.list.{} was called!".format(method))
>>> x = Test()
>>> x.list.append(5)
self.list.append was called!
>>> x.list.extend([1,2,3,4])
self.list.extend was called!
>>> x.list[1] = 6
self.list.__setitem__ was called!
>>> x.list
[5, 6, 2, 3, 4]
The most simple implementation of this would be to create a subclass and override every mutating method:
class Notifying_list(list):
__slots__ = ("notify",)
def __init__(self,notifying_method, *args,**kw):
self.notify = notifying_method
list.__init__(self,*args,**kw)
def append(self,*args,**kw):
self.notify("append")
return list.append(self,*args,**kw)
#etc.
This is obviously not very practical, writing the entire definition would be very tedious and very repetitive, so we can create the new subclass dynamically for any given class with functions like the following:
import functools
import types
def notify_wrapper(name,method):
"""wraps a method to call self.notify(name) when called
used by notifying_type"""
#functools.wraps(method)
def wrapper(*args,**kw):
self = args[0]
# use object.__getattribute__ instead of self.notify in
# case __getattribute__ is one of the notifying methods
# in which case self.notify will raise a RecursionError
notify = object.__getattribute__(self, "_Notify__notify")
# I'd think knowing which method was called would be useful
# you may want to change the arguments to the notify method
notify(name)
return method(*args,**kw)
return wrapper
def notifying_type(cls, notifying_methods="all"):
"""creates a subclass of cls that adds an extra function call when calling certain methods
The constructor of the subclass will take a callable as the first argument
and arguments for the original class constructor after that.
The callable will be called every time any of the methods specified in notifying_methods
is called on the object, it is passed the name of the method as the only argument
if notifying_methods is left to the special value 'all' then this uses the function
get_all_possible_method_names to create wrappers for nearly all methods."""
if notifying_methods == "all":
notifying_methods = get_all_possible_method_names(cls)
def init_for_new_cls(self,notify_method,*args,**kw):
self._Notify__notify = notify_method
namespace = {"__init__":init_for_new_cls,
"__slots__":("_Notify__notify",)}
for name in notifying_methods:
method = getattr(cls,name) #if this raises an error then you are trying to wrap a method that doesn't exist
namespace[name] = notify_wrapper(name, method)
# I figured using the type() constructor was easier then using a meta class.
return type("Notify_"+cls.__name__, (cls,), namespace)
unbound_method_or_descriptor = ( types.FunctionType,
type(list.append), #method_descriptor, not in types
type(list.__add__),#method_wrapper, also not in types
)
def get_all_possible_method_names(cls):
"""generates the names of nearly all methods the given class defines
three methods are blacklisted: __init__, __new__, and __getattribute__ for these reasons:
__init__ conflicts with the one defined in notifying_type
__new__ will not be called with a initialized instance, so there will not be a notify method to use
__getattribute__ is fine to override, just really annoying in most cases.
Note that this function may not work correctly in all cases
it was only tested with very simple classes and the builtin list."""
blacklist = ("__init__","__new__","__getattribute__")
for name,attr in vars(cls).items():
if (name not in blacklist and
isinstance(attr, unbound_method_or_descriptor)):
yield name
Once we can use notifying_type creating Notify_list or Notify_dict would be as simple as:
import collections
mutating_list_methods = set(dir(collections.MutableSequence)) - set(dir(collections.Sequence))
Notify_list = notifying_type(list, mutating_list_methods)
mutating_dict_methods = set(dir(collections.MutableMapping)) - set(dir(collections.Mapping))
Notify_dict = notifying_type(dict, mutating_dict_methods)
I have not tested this extensively and it quite possibly contains bugs / unhandled corner cases but I do know it worked correctly with list!

Python syntax for namedtuple

I see that the Python syntax for a namedtuple is:
Point = namedtuple('Point', ['x', 'y'])
Why isn't it simpler like so:
Point = namedtuple(['x','y'])
Its less verbose,

In general, objects don't know what variables they are assigned to:
# Create three variables referring to an OrderedPair class
tmp = namedtuple('OrderedPair', ['x','y']) # create a new class with metadata
Point = tmp # assign the class to a variable
Coordinate = tmp # assign the class to another var
That's a problem for named tuples. We have to pass in the class name to the namedtuple() factory function so that the class can be given a useful name, docstring, and __repr__ all of which have the class name inside it.
These reason it seems strange to you is that normal function and class definitions are handled differently. Python has special syntax for def and class that not only creates functions and classes, but it assigns their metadata (name and docstring) and assigns the result to a variable.
Consider what def does:
def square(x):
'Return a value times itself'
return x * x
The keyword def takes care of several things for you (notice that the word "square" will be used twice):
tmp = lambda x: x*x # create a function object
tmp.__name__ = 'square' # assign its metadata
tmp.__doc__ = 'Return a value times itself'
square = tmp # assign the function to a variable
The same is also true for classes. The class keyword takes care of multiple actions that would otherwise repeat the class name:
class Dog(object):
def bark(self):
return 'Woof!'
The underlying steps repeat the class name (notice that the word "Dog" is used twice):
Dog = type('Dog', (object,), {'bark': lambda self: 'Woof'})
Named tuples don't have the advantage of a special keyword like def or class so it has to do the first to steps itself. The final step of assigning to a variable belongs to you. If you think about it, the named tuple way is the norm in Python while def and class are the exception:
survey_results = open('survey_results') # is this really a duplication?
company_db = sqlite3.connect('company.db') # is this really a duplication?
www_python_org = urllib.urlopen('http://www.python.org')
radius = property(radius)
You are not the first to notice this. PEP 359 that suggested we add a new keyword, make, that could allow any callable to gain the auto-assignment capabilities of def, class, and import.
make <callable> <name> <tuple>:
<block>
would be translated into the assignment:
<name> = <callable>("<name>", <tuple>, <namespace>)
In the end, Guido didn't like the "make" proposal because it caused more problems than it solved (after all, it only saves you from making a single variable assignment).
Hope that helps you see why the class name is written twice. It isn't really duplication. The string form of the class name is used to assign metadata when the object is created, and the separate variable assignment just gives you a way to refer to that object. While they are usually the same name, they don't have to be :-)

namedtuple is a factory, returning a class. Consider only expression:
namedtuple(['x','y'])
What would be the name of class returned by this expression?

The class should have a name and know it. And it doesn't see the variable you assign it to, so it can't use that. Plus you could call it something else or even nothing at all:
c = namedtuple('Point', ['x', 'y'])
do_something_with_this(namedtuple('Point', ['x', 'y']))
Speaking of simpler syntax, you can also write it like this:
namedtuple('Point', 'x y')

Because namedtuple is a function that returns a class. To do that, it is actually rendering a string template and calling eval. To build the string, it needs all the arguments beforehand.
You need to include the relevant context as arguments to namedtuple for that to happen. If you don't provide the class name argument, it would need to guess. Programming languages don't like to guess.
With the rules of the Python language, the namedtuple function within this expression..
>>> Point = namedtuple(['x','y'])
..doesn't have access to variable name (Point) that the result is stored in once the expression has been executed. It only has access to the elements of the list provided as its argument (and variables that have been defined earlier).

When do we need Python Import Statements?

A piece of code works that I don't see why. It shouldn't work from my understanding. The problem is illustrated easily below:
"Main.py"
from x import * #class x is defined
from y import * #class y is defined
xTypeObj = x()
yTypeObj = y()
yTypeObj.func(xTypeObj)
"x.py"
class x(object):
def __init__...
...
def functionThatReturnsAString(self):
return "blah"
"y.py"
#NO IMPORT STATEMENT NEEDED?? WHY
class y(object):
def __init__...
...
def func(self, objOfTypeX):
print(objOfTypeX.functionThatReturnsAString())
My question is why do I NOT need to have an import statement in "y.py" of the type
from x import functionThatReturnAString()
How does it figure out how to call this method?

Python is an object-oriented programming language. In such a language, values are objects, and objects can have methods.
The functionThatReturnsAString function is a method on a class, and objOfTypeX is an instance of that class. Instances of a class carry with them all the methods of it's class.
This is why, for example, list objects in python have an .append() method:
>>> alist = []
>>> alist.append(1)
>>> alist
[1]
The list class has a .append() method, and you do not need to import that method to be able to call it. All you need is a reference to a list instance.
Technically speaking, a python list is a type, but that distinction does not matter here. On the whole, types are the same things as classes, for the purpose of this discussion.
Please do go and read the Python Tutorial, it explains classes in a later chapter (but you may want to skim through the first set of chapters first).

Python is a dynamically typed language. Unlike statically typed languages like C++ and Java calls to methods aren't bound until they are actually executed, thus why importing the module were that method is defined is not necessary. This has several implications:
Methods (and data members) can be added to and removed from an instance at runtime, so two instances of class Foo can actually have different methods even though they are of the same type.
Methods (and data members) can be added to and removed from a class at runtime, which will impact all current instances as well as new instances.
Bases classes can be added and removed to a class at runtime.
Note that this is not an exhaustive list of all of the difference between dynamically typed langauges and statically types languages.

Function yTypeObj.func is called from main.py where the class is imported. Therefore the object may be constructed and passed to the function, with all of its methods (functionThatReturnAString is a method of objOfTypeX).