Why is super used so much in PySide/PyQt? - python

Short version (tl;dr)
I am learning PySide, and most online tutorials use super to initialize UI elements. Is this important (i.e., more scalable), or is it a matter of taste?
Clarification: as I make more clear in the detailed version, this is not another generic thread asking when to use super (this has been done before). Rather, given the number of PySide tutorials that use super instead of <class>.__init__, I am trying to figure out if using super is standard in PySide applications? If so, is it because the circumstances where super is called for (involving resolving inheritances) come up a lot specifically in the use of PySide/PyQt? Or is it a matter of taste.
Detailed version
I am new to Python, and presently learning PySide using Zets tutorial (http://zetcode.com/gui/pysidetutorial/firstprograms/). The second example in the tutorial includes:
from PySide import QtGui
class Example(QtGui.QWidget):
def __init__(self):
super(Example, self).__init__()
self.initUI()
def initUI(self):
self.setGeometry(300,300,250,150)
self.setWindowTitle("PySide 101: Window the First!")
self.show()
app=QtGui.QApplication(sys.argv)
ex=Example()
sys.exit(app.exec_())
This works fine, but I have never used super. Hence, I rewrote the above code, successfully replacing super with more standard explicit invocation of the parent class:
QtGui.QWidget.__init__(self)
But as I search the web for PySide tutorials (e.g., http://qt-project.org/wiki/PySide-Newbie-Tutorials), they all include calls to super. My question is: should I use super for PySide scripting?
It seems that super seems most helpful when you have inheritance diamonds, that it tends to resolve instances of multiple inheritance in a reasonable way. Is super used a lot with PySide because there is a preponderance of cases of such diamonds that I will confront with more realistic complicated examples? [Edit: No: see my answer below.]
Why am I even asking? Why not just use super and be done with it?
I am asking because the book I am using to learn Python (Learning Python, by Lutz) spends over 20 pages on the topic of super, and explicitly cautions against using it. He suggests that new Python users go with the more traditional, explicit route before messing with it (e.g., see page 832, and pages 1041-1064 of Learning Python, 5th Edition). He basically paints it as a nonPythonic, arcane, rarely actually needed, new style that you should treat with great caution when just starting out, and thinks it is overused by experienced users.
Further, looking at the source code of two major PySide/PyQt based projects (Spyder and pyqtgraph), neither uses super. One (Spyder) explicitly tells contributors to avoid using it for compatibility reasons (http://code.google.com/p/spyderlib/wiki/NoteForContributors).
Note I link to a closely related post below, but the answer there discusses more generally when you would want to use super (when you have multiple inheritance). My question is whether PySide scripting justifies, or even requires, the use of super in the long term, or whether it is more Pythonic, and better for compatibility reasons, to explicitly name parent classes? Or is it a matter of taste?
If it is frowned upon (as my beginner book suggests) why is it so ubiquitous in PySide tutorials aimed at beginners? If it makes any difference, it seems the people writing these tutorials are seasoned Java programmers, or catering to such programmers. I am not.
Related topics
http://www.riverbankcomputing.com/pipermail/pyqt/2008-January/018320.html
Different ways of using __init__ for PyQt4
Understanding Python super() with __init__() methods

There is nothing wrong with instantiating parent classes in the traditional way, and some things to be said in favor of it. That said, using super simplifies the creation of subclasses, and the future modifications of one's PySide code, and people seem to have latched onto the latter as the overriding factor. This is not specific to Pyside, but to object-oriented programming in Python more generally (as pointed out in Kos's excellent answer).
The potential for simplification of code modification comes because within PySide it is necessary for things to work to define subclasses based on other QtGui objects (e.g., QtQui.QMainWindow and QtGui.QWidget). Further, people tend to futz around with their parent classes enough so that it seems easier to just use super, so that you you don't have to update your __init__ method every time you change parents.
So it isn't a matter of using super to help resolve cases of multiple inheritance, the case where most people agree it is probably best suited. Rather, it is a matter of doing less work within __init__ in case your parent class changes in the future.
Here are the responses from each author, both of whom wrote what I consider to be good PySide tutorials:
Author 1:
I think it is a matter of taste. Earlier tutorials (PyQt and PySide)
used .init and later I switched to super(). I personally
prefer super().
Author 2:
The reason people use super instead of .init(...) is to
avoid making changes if you change what the parent class is, e.g. if
you switch from QVBoxLayout to QHBoxLayout, you only have to change it
in the class definition line, rather than in the init method as
well.
So there you have it. Not these benefits aren't really specific to PySide, but to writing subclasses/inheritance more generally.
I'm not sure what Lutz, who seems very hesitant to endorse the use of super, would say (perhaps using super violates the 'Explicit is better than implicit' maxim).
Update Four Years Later
In retrospect, this debate is sort of over and this question is almost quaint (this was my first question at SO). While there used to be debate about the use of super, these debates are sort of over. Especially in Python 3 super's convenience has proven itself and just makes your code easier to maintain. Because in Qt/Pyside/PyQt framework the use of inheritance from more abstract Qt classes is ubiquitous, this is no small feature. Sure, you will need to be careful when you have crazy lattices of inheritance, but frankly since I asked this question, I have literally never run into this problem, and I currently use super in all my code. It arguably violates the "explicit is better than implicit" maxim, but "Simple is better than complex" and "practicality beats purity" are the overriding factors here (the practical aspect here is "maintainability counts").

Hm, nice one. But IMO it's only barely related to Qt/ PySide.
First, how are these two different? If you have simple inheritance (perhaps not counting "mixins"), then there's no difference in behaviour. A cosmetic difference remains- you don't need to name your base class again - but you do have to name the same class.
The differences start when you have multiple inheritance. Then a chain of super() calls for this hierarchy:
A
/ \
X Y
\ /
Z
can easily proceed like this through super() calls:
A
\
X --- Y
\
Z
without the need of X and Y knowing each other. This relates to the concept of method resolution order that allows a style of programming called "cooperative multiple inheritance" in the Python docs.
If there's a method Foo and implementations in X and Y of that method build upon A's implementation, then Z is easily able to rely on both X and Y without them knowing even about each other. This, however, has an important precondition: Foo has the same (or at least [compatible]) signature in every class, as specified by A's interface where it's initially defined.
The __init__ method is special: technically it works in exactly the same way with super, but! but (more often than not) for subclasses it has a totally different signature. If the subclass' __init__ looks different, then super won't give you anything over explicit base call because you aren't able to use cooperative multitasking anyway.
Note: Python is very atypical in this respect: in most OO languages constructors belong to the class, rather than the instances; in other words most languages rely on their equivalents of __new__ and don't have __init__ at all.
Note 2: I have never seen any real code that would rely of cooperative multiple inheritance. (Single inheritance easily makes enough spaghetti for me ;-))
Also, some good reading:
Method resolution order - what's it all about
Python's Super is nifty, but you can't use it
[

Related

PEP 8 and Python decorators

I'm working on a project where I have a class with some 10ish decorators on it, where I'm using them to provide validation on some methods and attributes (if it's relevant, the validation is very general, and re-used elsewhere on other classes). I've been advised by a friend who is a lot more of a PEP8 stickler than I am that this is bad form---however, I cannot find a cite for that, or even good advice on what constitutes a good use of a decorator versus a bad or non-essential one.
My own introduction to using decorators came from Flask, where they provide routing information, and can be stacked a couple deep. Could someone provide cited information on proper versus improper decorator usage, and better alternatives?
What matters in Python is code readability. 10 or so decorators might be pretty hard to keep track of, while 2 o 3 are easier to read. You don't need a convention for every character you type: sometimes, you just have to go for what is easier to read
As Martijn Pieters said in comments, decorators are just functions applied on the function defined below. To keep a script readable I would recommend to keep not more than 3 or 4 decorators. But the PEP8 doesn't give any advise about the number of decorators you should stack, because it isn't the goal of the PEP :)
If you have more than that, it's a architecture/app-design problem, not a coding style issue. You may need to redefine your architecture, espcially if you apply the same suite of decorators on every functions.
I would say - purely as a matter of opinion and from a readability point that more than a couple of decorators is not a good idea - especially since the ordering can be critical. You would be better of seeing if there are any base classes - possibly with decorators of their own that you can logically split your class into in an inheritance hierarchy.
If it is a sensible separation then the decorators will match what the classes are doing/contain, e.g. if you have a parameter class validation with a validation decorator and that inherits from a string_parameter class that has string validation, etc., You will end up with more understandable classes and decorators.
Note that you can also stack decorators - i.e. a decorator that decorates a decorator.

understanding proper python code organization [duplicate]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
Much of my programming background is in Java, and I'm still doing most of my programming in Java. However, I'm starting to learn Python for some side projects at work, and I'd like to learn it as independent of my Java background as possible - i.e. I don't want to just program Java in Python. What are some things I should look out for?
A quick example - when looking through the Python tutorial, I came across the fact that defaulted mutable parameters of a function (such as a list) are persisted (remembered from call to call). This was counter-intuitive to me as a Java programmer and hard to get my head around. (See here and here if you don't understand the example.)
Someone also provided me with this list, which I found helpful, but short. Anyone have any other examples of how a Java programmer might tend to misuse Python...? Or things a Java programmer would falsely assume or have trouble understanding?
Edit: Ok, a brief overview of the reasons addressed by the article I linked to to prevent duplicates in the answers (as suggested by Bill the Lizard). (Please let me know if I make a mistake in phrasing, I've only just started with Python so I may not understand all the concepts fully. And a disclaimer - these are going to be very brief, so if you don't understand what it's getting at check out the link.)
A static method in Java does not translate to a Python classmethod
A switch statement in Java translates to a hash table in Python
Don't use XML
Getters and setters are evil (hey, I'm just quoting :) )
Code duplication is often a necessary evil in Java (e.g. method overloading), but not in Python
(And if you find this question at all interesting, check out the link anyway. :) It's quite good.)
Don't put everything into classes. Python's built-in list and dictionaries will take you far.
Don't worry about keeping one class per module. Divide modules by purpose, not by class.
Use inheritance for behavior, not interfaces. Don't create an "Animal" class for "Dog" and "Cat" to inherit from, just so you can have a generic "make_sound" method.
Just do this:
class Dog(object):
def make_sound(self):
return "woof!"
class Cat(object):
def make_sound(self):
return "meow!"
class LolCat(object):
def make_sound(self):
return "i can has cheezburger?"
The referenced article has some good advice that can easily be misquoted and misunderstood. And some bad advice.
Leave Java behind. Start fresh. "do not trust your [Java-based] instincts". Saying things are "counter-intuitive" is a bad habit in any programming discipline. When learning a new language, start fresh, and drop your habits. Your intuition must be wrong.
Languages are different. Otherwise, they'd be the same language with different syntax, and there'd be simple translators. Because there are not simple translators, there's no simple mapping. That means that intuition is unhelpful and dangerous.
"A static method in Java does not translate to a Python classmethod." This kind of thing is really limited and unhelpful. Python has a staticmethod decorator. It also has a classmethod decorator, for which Java has no equivalent.
This point, BTW, also included the much more helpful advice on not needlessly wrapping everything in a class. "The idiomatic translation of a Java static method is usually a module-level function".
The Java switch statement in Java can be implemented several ways. First, and foremost, it's usually an if elif elif elif construct. The article is unhelpful in this respect. If you're absolutely sure this is too slow (and can prove it) you can use a Python dictionary as a slightly faster mapping from value to block of code. Blindly translating switch to dictionary (without thinking) is really bad advice.
Don't use XML. Doesn't make sense when taken out of context. In context it means don't rely on XML to add flexibility. Java relies on describing stuff in XML; WSDL files, for example, repeat information that's obvious from inspecting the code. Python relies on introspection instead of restating everything in XML.
But Python has excellent XML processing libraries. Several.
Getters and setters are not required in Python they way they're required in Java. First, you have better introspection in Python, so you don't need getters and setters to help make dynamic bean objects. (For that, you use collections.namedtuple).
However, you have the property decorator which will bundle getters (and setters) into an attribute-like construct. The point is that Python prefers naked attributes; when necessary, we can bundle getters and setters to appear as if there's a simple attribute.
Also, Python has descriptor classes if properties aren't sophisticated enough.
Code duplication is often a necessary evil in Java (e.g. method overloading), but not in Python. Correct. Python uses optional arguments instead of method overloading.
The bullet point went on to talk about closure; that isn't as helpful as the simple advice to use default argument values wisely.
One thing you might be used to in Java that you won't find in Python is strict privacy. This is not so much something to look out for as it is something not to look for (I am embarrassed by how long I searched for a Python equivalent to 'private' when I started out!). Instead, Python has much more transparency and easier introspection than Java. This falls under what is sometimes described as the "we're all consenting adults here" philosophy. There are a few conventions and language mechanisms to help prevent accidental use of "unpublic" methods and so forth, but the whole mindset of information hiding is virtually absent in Python.
The biggest one I can think of is not understanding or not fully utilizing duck typing. In Java you're required to specify very explicit and detailed type information upfront. In Python typing is both dynamic and largely implicit. The philosophy is that you should be thinking about your program at a higher level than nominal types. For example, in Python, you don't use inheritance to model substitutability. Substitutability comes by default as a result of duck typing. Inheritance is only a programmer convenience for reusing implementation.
Similarly, the Pythonic idiom is "beg forgiveness, don't ask permission". Explicit typing is considered evil. Don't check whether a parameter is a certain type upfront. Just try to do whatever you need to do with the parameter. If it doesn't conform to the proper interface, it will throw a very clear exception and you will be able to find the problem very quickly. If someone passes a parameter of a type that was nominally unexpected but has the same interface as what you expected, then you've gained flexibility for free.
The most important thing, from a Java POV, is that it's perfectly ok to not make classes for everything. There are many situations where a procedural approach is simpler and shorter.
The next most important thing is that you will have to get over the notion that the type of an object controls what it may do; rather, the code controls what objects must be able to support at runtime (this is by virtue of duck-typing).
Oh, and use native lists and dicts (not customized descendants) as far as possible.
The way exceptions are treated in Python is different from
how they are treated in Java. While in Java the advice
is to use exceptions only for exceptional conditions this is not
so with Python.
In Python things like Iterator makes use of exception mechanism to signal that there are no more items.But such a design is not considered as good practice in Java.
As Alex Martelli puts in his book Python in a Nutshell
the exception mechanism with other languages (and applicable to Java)
is LBYL (Look Before You Leap) :
is to check in advance, before attempting an operation, for all circumstances that might make the operation invalid.
Where as with Python the approach is EAFP (it's easier to Ask for forgiveness than permission)
A corrollary to "Don't use classes for everything": callbacks.
The Java way for doing callbacks relies on passing objects that implement the callback interface (for example ActionListener with its actionPerformed() method). Nothing of this sort is necessary in Python, you can directly pass methods or even locally defined functions:
def handler():
print("click!")
button.onclick(handler)
Or even lambdas:
button.onclick(lambda: print("click!\n"))

basic Python(ic) inheritance question

Quick question about Python(ic) inheritence: what is the difference between the idiom posted here (super(ChildClass,self).method(args)) vs. the one on the Python docs site (ParentClass.method(self,[args]))? Is one more Pythonic than the other?
Using super(ChildClass, self).method(args) allows you to walk the method resolution order and -- if everyone but the last parent uses super -- call every class in the hierarchy exactly once. (Not that super only works with new-style classes.)
Using ParentClass.method(self, args) calls one specific class. It does not work when involved in multiple inheritance.
This article provides some description of the issue and clarifies a lot of issues for some people. I don't agree with all of its conclusions, but it provides good examples and discussion.

Is it a good idea to use super() in Python?

Or should I just explicitly reference the superclasses whose methods I want to call?
It seems brittle to repeat the names of super classes when referencing their constructors, but this page http://fuhm.net/super-harmful/ makes some good arguments against using super().
The book Expert Python Programming has discussed the topic of "super pitfalls" in chapter 3. It is worth reading. Below is the book's conclusion:
Super usage has to be consistent: In a class hierarchy, super should be used everywhere or nowhere. Mixing super and classic calls is a confusing practice. People tend to avoid super, for their code to be more explicit.
Edit: Today I read this part of the book again. I'll copy some more sentences, since super usage is tricky:
Avoid multiple inheritance in your code.
Be consistent with its usage and don't mix new-style and
old-style.
Check the class hierarchy before calling its methods in
your subclass.
You can use super, but as the article says, there are drawbacks. As long as you know them, there is no problem with using the feature. It's like people saying "use composition, not inheritance" or "never use global variables". If the feature exists, there is a reason. Just be sure to understand the why and the what and use them wisely.
super() tries to solve for you the problem of multiple inheritance; it's hard to replicate its semantics and you certainly shouldn't create any new semantics unless you're completely sure.
For single inheritance, there's really no difference between
class X(Y):
def func(self):
Y.func(self)
and
class X(Y):
def func(self):
super().func()
so I guess that's just the question of taste.
I like super() more because it allows you to change the inherited class (for example when you're refactoring and add an intermediate class) without changing it on all the methods.
The problem people have with super is more a problem of multiple inheritance. So it is a little unfair to blame super. Without super multiple inheritance is even worse. Michele Simionato nicely wrapped this up in his blog article on super:
On the other hand, one may wonder if
all super warts aren't hints of some
serious problem underlying. It may
well be that the problem is not with
super, nor with cooperative methods:
the problem may be with multiple
inheritance itself.
So the main lesson is that you should try to avoid multiple inheritance.
In the interest of consistency I always use super, even if for single inheritance it does not really matter (apart from the small advantage of not having to know the parent class name). In Python 3+ super is more convenient, so there one should definitely use super.
Yes, you should use super() over other methods. This is now the standard object inheritance model in Python 3.
Just stick to keyword arguments in your __init__ methods and you shouldn't have too many problems. Additionally you can use **kwargs to support additional parameters that are not defined in levels of the inheritance chain.
I agree that it is brittle, but no less so than using the name of the inherited class.

Is anyone using meta-meta-classes / meta-meta-meta-classes in Python/ other languages?

I recently discovered metaclasses in python.
Basically a metaclass in python is a class that creates a class. There are many useful reasons why you would want to do this - any kind of class initialisation for example. Registering classes on factories, complex validation of attributes, altering how inheritance works, etc. All of this becomes not only possible but simple.
But in python, metaclasses are also plain classes. So, I started wondering if the abstraction could usefully go higher, and it seems to me that it can and that:
a metaclass corresponds to or implements a role in a pattern (as in GOF pattern languages).
a meta-metaclass is the pattern itself (if we allow it to create tuples of classes representing abstract roles, rather than just a single class)
a meta-meta-metaclass is a pattern factory, which corresponds to the GOF pattern groupings, e.g. Creational, Structural, Behavioural. A factory where you could describe a case of a certain type of problem and it would give you a set of classes that solved it.
a meta-meta-meta-metaclass (as far as I could go), is a pattern factory factory, a factory to which you could perhaps describe the type of your problem and it would give you a pattern factory to ask.
I have found some stuff about this online, but mostly not very useful. One problem is that different languages define metaclasses slightly differently.
Has anyone else used metaclasses like this in python/elsewhere, or seen this used in the wild, or thought about it? What are the analogues in other languages? E.g. in C++ how deep can the template recursion go?
I'd very much like to research it further.
This reminds me of the eternal quest some people seem to be on to make a "generic implementation of a pattern." Like a factory that can create any object (including another factory), or a general-purpose dependency injection framework that is far more complex to manage than simply writing code that actually does something.
I had to deal with people intent on abstraction to the point of navel-gazing when I was managing the Zend Framework project. I turned down a bunch of proposals to create components that didn't do anything, they were just magical implementations of GoF patterns, as though the pattern were a goal in itself, instead of a means to a goal.
There's a point of diminishing returns for abstraction. Some abstraction is great, but eventually you need to write code that does something useful.
Otherwise it's just turtles all the way down.
To answer your question: no.
Feel free to research it further.
Note, however, that you've conflated design patterns (which are just ideas) with code (which is an implementation.)
Good code often reflects a number of interlocking design patterns. There's no easy way for formalize this. The best you can do is a nice picture, well-written docstrings, and method names that reflect the various design patterns.
Also note that a meta-class is a class. That's a loop. There's no higher level of abstractions. At that point, it's just intent. The idea of meta-meta-class doesn't mean much -- it's a meta-class for meta-classes, which is silly but technically possible. It's all just a class, however.
Edit
"Are classes that create metaclasses really so silly? How does their utility suddenly run out?"
A class that creates a class is fine. That's pretty much it. The fact that the target class is a meta class or an abstract superclass or a concrete class doesn't matter. Metaclasses make classes. They might make other metaclasses, which is weird, but they're still just metaclasses making classes.
The utility "suddenly" runs out because there's no actual thing you need (or can even write) in a metaclass that makes another metaclass. It isn't that it "suddenly" becomes silly. It's that there's nothing useful there.
As I seed, feel free to research it. For example, actually write a metaclass that builds another metaclass. Have fun. There might be something useful there.
The point of OO is to write class definitions that model real-world entities. As such, a metaclass is sometimes handy to define cross-cutting aspects of several related classes. (It's a way to do some Aspect-Oriented Programming.) That's all a metaclass can really do; it's a place to hold a few functions, like __new__(), that aren't proper parts of the class itself.
During the History of Programming Languages conference in 2007, Simon Peyton Jones commented that Haskell allows meta programming using Type Classes, but that its really turtles all the way down. You can meta-meta-meta-meta etc program in Haskell, but that he's never heard of anyone using more than 3 levels of indirection.
Guy Steele pointed out that its the same thing in Lisp and Scheme. You can do meta-programming using backticks and evals (you can think of a backtick as a Python lambda, kinda), but he's never seen more than 3 backticks used.
Presumably they have seen more code than you or I ever has, so its only a slight exaggeration to say that no-one has ever gone beyond 3 levels of meta.
If you think about it, most people don't ever use meta-programming, and two levels is pretty hard to wrap your head around. I would guess that three is nearly impossible, and the that last guy to try four ended up in an asylum.
Since when I first understood metaclasses in Python, I kept wondering "what could be done with a meta-meta class?". This is at least 10 years ago - and now, just a couple months ago, it became clear for me that there is one mechanism in Python class creation that actually involves a "meta-meta" class. And therefore, it is possible to try to imagine some use for that.
To recap object instantiation in Python: Whenever one instantiates an object in Python by "calling" its class with the same syntax used for calling an ordinary function, the class's __new__ and __init__. What "orchestrates" the calling of these methods on the class is exactly the class'metaclass' __call__ method. Usually when one writes a metaclass in Python, either the __new__ or __init__ method of the metaclass is customized.
So, it turns out that by writing a "meta-meta" class one can customize its __call__ method and thus control which parameters are passed and to the metaclass's __new__ and __init__ methods, and if some other code is to be called before of after those. What turns out in the end is that metcalsses themselves are usually hardcoded and one needs just a few, if any, even in very large projects. So any customization that might be done at the "meta meta" call is usually done directly on the metaclass itself.
And them, there are those other less frequent uses for Python metaclasses - one can customize an __add__ method in a metaclass so that the classes they define are "addable", and create a derived class having the two added classes as superclasses. That mechanism is perfectly valid with metaclasses as well - therefore, so just we "have some actual code", follows an example of "meta-meta" class that allows one to compose "metaclasses" for a class just by adding them on class declaration:
class MM(type):
def __add__(cls, other):
metacls = cls.__class__
return metacls(cls.__name__ + other.__name__, (cls, other), {})
class M1(type, metaclass=MM):
def __new__(metacls, name, bases, namespace):
namespace["M1"] = "here"
print("At M1 creation")
return super().__new__(metacls, name, bases, namespace)
class M2(type, metaclass=MM):
def __new__(metacls, name, bases, namespace):
namespace["M2"] = "there"
print("At M2 creation")
return super().__new__(metacls, name, bases, namespace)
And we can see that working on the interactive console:
In [22]: class Base(metaclass = M1 + M2):
...: pass
...:
At M1 creation
At M2 creation
Note that as different metaclasses in Python are usually difficult to combine, this can actually be useful by allowing a user-made metaclass to be combined with a library's or stdlib one, without this one having to be explicitly declared as parent of the former:
In [23]: import abc
In [24]: class Combined(metaclass=M1 + abc.ABCMeta):
...: pass
...:
At M1 creation
The class system in Smalltalk is an interesting one to study. In Smalltalk, everything is an object and every object has a class. This doesn't imply that the hierarchy goes to infinity. If I remember correctly, it goes something like:
5 -> Integer -> Integer class -> Metaclass -> Metaclass class -> Metaclass -> ... (it loops)
Where '->' denotes "is an instance of".

Categories

Resources