PEP 8 and Python decorators - python

I'm working on a project where I have a class with some 10ish decorators on it, where I'm using them to provide validation on some methods and attributes (if it's relevant, the validation is very general, and re-used elsewhere on other classes). I've been advised by a friend who is a lot more of a PEP8 stickler than I am that this is bad form---however, I cannot find a cite for that, or even good advice on what constitutes a good use of a decorator versus a bad or non-essential one.
My own introduction to using decorators came from Flask, where they provide routing information, and can be stacked a couple deep. Could someone provide cited information on proper versus improper decorator usage, and better alternatives?

What matters in Python is code readability. 10 or so decorators might be pretty hard to keep track of, while 2 o 3 are easier to read. You don't need a convention for every character you type: sometimes, you just have to go for what is easier to read

As Martijn Pieters said in comments, decorators are just functions applied on the function defined below. To keep a script readable I would recommend to keep not more than 3 or 4 decorators. But the PEP8 doesn't give any advise about the number of decorators you should stack, because it isn't the goal of the PEP :)
If you have more than that, it's a architecture/app-design problem, not a coding style issue. You may need to redefine your architecture, espcially if you apply the same suite of decorators on every functions.

I would say - purely as a matter of opinion and from a readability point that more than a couple of decorators is not a good idea - especially since the ordering can be critical. You would be better of seeing if there are any base classes - possibly with decorators of their own that you can logically split your class into in an inheritance hierarchy.
If it is a sensible separation then the decorators will match what the classes are doing/contain, e.g. if you have a parameter class validation with a validation decorator and that inherits from a string_parameter class that has string validation, etc., You will end up with more understandable classes and decorators.
Note that you can also stack decorators - i.e. a decorator that decorates a decorator.

Related

Better design in order to avoid violating Liskov Substitution principle

I am running into an issue with Liskov Substitution Principle and am not quite sure what would be the best way to go around it.
Code in question
class BaseModel:
def run(self, base_model_input: BaseModelInput) -> BaseModelOutput:
"""Throws NotImplemented or #abstractmethod"""
pass
class SpecificModel(BaseModel):
def run(self, specific_input: SpecificModelInput) -> SpecificModelOutput:
# do things...
I understand well why is this not a great code, and why it violates the Liskov Substitution Principle. I am wondering how to design my system better to avoid this problem in the first place.
Fundamentally I have a BaseModel class that acts like an interface, providing some methods like run that the extending classes must implement. But extending classes also deal with specific input/output, that are also extensions of the base input/output classes (that is SpecificModelInput inherits from BaseModelInput and adds some fields and functionality, same with output)
What would be a better approach here?
As I stated in the comments:If you can really constrain the inputs and outputs to subtypes of a base-input and base-output, I don't see any problem with this model
If by violation you mean that on restricting the input options, the subclass can no longer b used everywhere the baseclass can, I'd say it is a case where you are taking the Liskov Substitution principle as dogma where it should be very good advice.
See, the Liskov..principle is a good guideline, and a bit simplistic, as it will give you a really good feeling of what inheritance should be.
But in real-world terms, restricting input parameters (and attribute types) is not a concern in all cases. Anyway, having a concrete example will serve us well: think of an abstract "Vehicle" class with a "board" method which allows you to board "Transportables" - and concrete "Transportables" and some os the allowable subclasses of "Transportable" are Persons, Dogs, Grocery Bag, Pianos, and Elephants. If your "Vehicle" class is a "Ship" - it can take all of those. If your vehicle class is a Car, some of those are out.
That seems to be your use case.
So, of course, you are violating the principle here. The wording in most places explaining it is that "if you substitute an instance of the subclass anywhere the superclass appears, the program should not break".
So, the most correct thing to do, is to modify the superclass contracts in a way that any input may or may not work, and allow it to raise a runtime exception (or return an object signalizing an error) even if the input is valid.
Everyone calling that method should handle the "did not work" state - with the example above it is easy to see that if I have a method in an unrelated class that calls vehicle.board, it should be responsible to see it is not trying to put an Elephant inside a car. If everywhere the method is called these checks are made, the principle holds!
If engineering this is overkill for whatever task you have, I'd say "...practicality beats purity" in this case, and simply set the annotations to silence the static type checker.
Of course, telling that to the static type checker is another thing - I think the use of Generics as pointed in the answer linked in the comments could do: How do I annotate the type of a parameter of an abstractmethod, when the parameter can have any type derived from a specific base type?

Why is super used so much in PySide/PyQt?

Short version (tl;dr)
I am learning PySide, and most online tutorials use super to initialize UI elements. Is this important (i.e., more scalable), or is it a matter of taste?
Clarification: as I make more clear in the detailed version, this is not another generic thread asking when to use super (this has been done before). Rather, given the number of PySide tutorials that use super instead of <class>.__init__, I am trying to figure out if using super is standard in PySide applications? If so, is it because the circumstances where super is called for (involving resolving inheritances) come up a lot specifically in the use of PySide/PyQt? Or is it a matter of taste.
Detailed version
I am new to Python, and presently learning PySide using Zets tutorial (http://zetcode.com/gui/pysidetutorial/firstprograms/). The second example in the tutorial includes:
from PySide import QtGui
class Example(QtGui.QWidget):
def __init__(self):
super(Example, self).__init__()
self.initUI()
def initUI(self):
self.setGeometry(300,300,250,150)
self.setWindowTitle("PySide 101: Window the First!")
self.show()
app=QtGui.QApplication(sys.argv)
ex=Example()
sys.exit(app.exec_())
This works fine, but I have never used super. Hence, I rewrote the above code, successfully replacing super with more standard explicit invocation of the parent class:
QtGui.QWidget.__init__(self)
But as I search the web for PySide tutorials (e.g., http://qt-project.org/wiki/PySide-Newbie-Tutorials), they all include calls to super. My question is: should I use super for PySide scripting?
It seems that super seems most helpful when you have inheritance diamonds, that it tends to resolve instances of multiple inheritance in a reasonable way. Is super used a lot with PySide because there is a preponderance of cases of such diamonds that I will confront with more realistic complicated examples? [Edit: No: see my answer below.]
Why am I even asking? Why not just use super and be done with it?
I am asking because the book I am using to learn Python (Learning Python, by Lutz) spends over 20 pages on the topic of super, and explicitly cautions against using it. He suggests that new Python users go with the more traditional, explicit route before messing with it (e.g., see page 832, and pages 1041-1064 of Learning Python, 5th Edition). He basically paints it as a nonPythonic, arcane, rarely actually needed, new style that you should treat with great caution when just starting out, and thinks it is overused by experienced users.
Further, looking at the source code of two major PySide/PyQt based projects (Spyder and pyqtgraph), neither uses super. One (Spyder) explicitly tells contributors to avoid using it for compatibility reasons (http://code.google.com/p/spyderlib/wiki/NoteForContributors).
Note I link to a closely related post below, but the answer there discusses more generally when you would want to use super (when you have multiple inheritance). My question is whether PySide scripting justifies, or even requires, the use of super in the long term, or whether it is more Pythonic, and better for compatibility reasons, to explicitly name parent classes? Or is it a matter of taste?
If it is frowned upon (as my beginner book suggests) why is it so ubiquitous in PySide tutorials aimed at beginners? If it makes any difference, it seems the people writing these tutorials are seasoned Java programmers, or catering to such programmers. I am not.
Related topics
http://www.riverbankcomputing.com/pipermail/pyqt/2008-January/018320.html
Different ways of using __init__ for PyQt4
Understanding Python super() with __init__() methods
There is nothing wrong with instantiating parent classes in the traditional way, and some things to be said in favor of it. That said, using super simplifies the creation of subclasses, and the future modifications of one's PySide code, and people seem to have latched onto the latter as the overriding factor. This is not specific to Pyside, but to object-oriented programming in Python more generally (as pointed out in Kos's excellent answer).
The potential for simplification of code modification comes because within PySide it is necessary for things to work to define subclasses based on other QtGui objects (e.g., QtQui.QMainWindow and QtGui.QWidget). Further, people tend to futz around with their parent classes enough so that it seems easier to just use super, so that you you don't have to update your __init__ method every time you change parents.
So it isn't a matter of using super to help resolve cases of multiple inheritance, the case where most people agree it is probably best suited. Rather, it is a matter of doing less work within __init__ in case your parent class changes in the future.
Here are the responses from each author, both of whom wrote what I consider to be good PySide tutorials:
Author 1:
I think it is a matter of taste. Earlier tutorials (PyQt and PySide)
used .init and later I switched to super(). I personally
prefer super().
Author 2:
The reason people use super instead of .init(...) is to
avoid making changes if you change what the parent class is, e.g. if
you switch from QVBoxLayout to QHBoxLayout, you only have to change it
in the class definition line, rather than in the init method as
well.
So there you have it. Not these benefits aren't really specific to PySide, but to writing subclasses/inheritance more generally.
I'm not sure what Lutz, who seems very hesitant to endorse the use of super, would say (perhaps using super violates the 'Explicit is better than implicit' maxim).
Update Four Years Later
In retrospect, this debate is sort of over and this question is almost quaint (this was my first question at SO). While there used to be debate about the use of super, these debates are sort of over. Especially in Python 3 super's convenience has proven itself and just makes your code easier to maintain. Because in Qt/Pyside/PyQt framework the use of inheritance from more abstract Qt classes is ubiquitous, this is no small feature. Sure, you will need to be careful when you have crazy lattices of inheritance, but frankly since I asked this question, I have literally never run into this problem, and I currently use super in all my code. It arguably violates the "explicit is better than implicit" maxim, but "Simple is better than complex" and "practicality beats purity" are the overriding factors here (the practical aspect here is "maintainability counts").
Hm, nice one. But IMO it's only barely related to Qt/ PySide.
First, how are these two different? If you have simple inheritance (perhaps not counting "mixins"), then there's no difference in behaviour. A cosmetic difference remains- you don't need to name your base class again - but you do have to name the same class.
The differences start when you have multiple inheritance. Then a chain of super() calls for this hierarchy:
A
/ \
X Y
\ /
Z
can easily proceed like this through super() calls:
A
\
X --- Y
\
Z
without the need of X and Y knowing each other. This relates to the concept of method resolution order that allows a style of programming called "cooperative multiple inheritance" in the Python docs.
If there's a method Foo and implementations in X and Y of that method build upon A's implementation, then Z is easily able to rely on both X and Y without them knowing even about each other. This, however, has an important precondition: Foo has the same (or at least [compatible]) signature in every class, as specified by A's interface where it's initially defined.
The __init__ method is special: technically it works in exactly the same way with super, but! but (more often than not) for subclasses it has a totally different signature. If the subclass' __init__ looks different, then super won't give you anything over explicit base call because you aren't able to use cooperative multitasking anyway.
Note: Python is very atypical in this respect: in most OO languages constructors belong to the class, rather than the instances; in other words most languages rely on their equivalents of __new__ and don't have __init__ at all.
Note 2: I have never seen any real code that would rely of cooperative multiple inheritance. (Single inheritance easily makes enough spaghetti for me ;-))
Also, some good reading:
Method resolution order - what's it all about
Python's Super is nifty, but you can't use it
[

Is python #decorator related to the decorator design pattern?

I might seem stupid here. However, I was writing some python code, and this thing struck me. In python there are things called decorators which are denoted by # and used "around" functions like:
class PlotABC(metaclass=ABCMeta):
#property
def figure(self):
if self._figure is None: self.__create_figure()
return self._figure
#abstractmethod
def _prepare_series(self):
pass
I also know slivers of design patterns and I know there are patterns called decorators. Then once I thought, "Hey maybe the name similarity is not a bizarre coincidence."
I have read wiki: Decorator pattern and the great answer How to make a chain of function decorators? (almost whole).
It seems to me that python's decorator semantics are really powerful and can serve to implement the design pattern. As they allow to wrap around functions and methods. Though, I am still confused, since I am inexperienced with design patterns. I also do not know a language with such a dedicated mechanism for them.
Could someone experienced in python and design patterns say if the python semantics were purposely created to create decorator patterns? Is it something more, less, or does it mean something different?
And maybe throw some light how declaring a method abstract or a property is decorating it?
No, it's not a bizarre coincidence, and no, python #decorators do not implement the GOF Decorator pattern.
The GOF (Gang of Four, so named after the four authors of the book "Design Patterns") decorator pattern adds "decorations" to change the behaviour of an object. It does this by creating "decorator" classes that can be applied to undecorated base classes.
Python decorators adds a "decoration" to a function or a class. These decorators are essentially functions that wrap or modify the function/class.
Both are similar in that you can re-use the decorator to add the same functionality several times, but the decorator pattern requires the decorators to be designed together with and for the classes that you can decorate.
Python decorators instead just do their work on any class of function.
A "pattern" is a workaround for a missing language feature. "Decoration" is a term which pre-existed the Gang Of Four patterns book in programming - it has been used to describe any number of operations that involve adding some information or behaviour onto something else.
Strictly speaking, this isn't a way of implementing the decorator pattern, because it doesn't involve the creation of a class which composes other classes (except co-incidentally that that is one thing you can do with a decorator). It does, however, together with python's other features, render it largely obsolete.
In fact, I'd add that most "patterns" are a way of faking dynamic or reflective behaviour in C++, Java, and the like. You'll find very little excitement outside of those language communities. In Python decorators and metaclasses provide a way to encode all kinds of transformations - I'd guess that a majority of patterns which aren't trivial code be substantially replaced with specific decorators or metaclasses in Python.

understanding proper python code organization [duplicate]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
Much of my programming background is in Java, and I'm still doing most of my programming in Java. However, I'm starting to learn Python for some side projects at work, and I'd like to learn it as independent of my Java background as possible - i.e. I don't want to just program Java in Python. What are some things I should look out for?
A quick example - when looking through the Python tutorial, I came across the fact that defaulted mutable parameters of a function (such as a list) are persisted (remembered from call to call). This was counter-intuitive to me as a Java programmer and hard to get my head around. (See here and here if you don't understand the example.)
Someone also provided me with this list, which I found helpful, but short. Anyone have any other examples of how a Java programmer might tend to misuse Python...? Or things a Java programmer would falsely assume or have trouble understanding?
Edit: Ok, a brief overview of the reasons addressed by the article I linked to to prevent duplicates in the answers (as suggested by Bill the Lizard). (Please let me know if I make a mistake in phrasing, I've only just started with Python so I may not understand all the concepts fully. And a disclaimer - these are going to be very brief, so if you don't understand what it's getting at check out the link.)
A static method in Java does not translate to a Python classmethod
A switch statement in Java translates to a hash table in Python
Don't use XML
Getters and setters are evil (hey, I'm just quoting :) )
Code duplication is often a necessary evil in Java (e.g. method overloading), but not in Python
(And if you find this question at all interesting, check out the link anyway. :) It's quite good.)
Don't put everything into classes. Python's built-in list and dictionaries will take you far.
Don't worry about keeping one class per module. Divide modules by purpose, not by class.
Use inheritance for behavior, not interfaces. Don't create an "Animal" class for "Dog" and "Cat" to inherit from, just so you can have a generic "make_sound" method.
Just do this:
class Dog(object):
def make_sound(self):
return "woof!"
class Cat(object):
def make_sound(self):
return "meow!"
class LolCat(object):
def make_sound(self):
return "i can has cheezburger?"
The referenced article has some good advice that can easily be misquoted and misunderstood. And some bad advice.
Leave Java behind. Start fresh. "do not trust your [Java-based] instincts". Saying things are "counter-intuitive" is a bad habit in any programming discipline. When learning a new language, start fresh, and drop your habits. Your intuition must be wrong.
Languages are different. Otherwise, they'd be the same language with different syntax, and there'd be simple translators. Because there are not simple translators, there's no simple mapping. That means that intuition is unhelpful and dangerous.
"A static method in Java does not translate to a Python classmethod." This kind of thing is really limited and unhelpful. Python has a staticmethod decorator. It also has a classmethod decorator, for which Java has no equivalent.
This point, BTW, also included the much more helpful advice on not needlessly wrapping everything in a class. "The idiomatic translation of a Java static method is usually a module-level function".
The Java switch statement in Java can be implemented several ways. First, and foremost, it's usually an if elif elif elif construct. The article is unhelpful in this respect. If you're absolutely sure this is too slow (and can prove it) you can use a Python dictionary as a slightly faster mapping from value to block of code. Blindly translating switch to dictionary (without thinking) is really bad advice.
Don't use XML. Doesn't make sense when taken out of context. In context it means don't rely on XML to add flexibility. Java relies on describing stuff in XML; WSDL files, for example, repeat information that's obvious from inspecting the code. Python relies on introspection instead of restating everything in XML.
But Python has excellent XML processing libraries. Several.
Getters and setters are not required in Python they way they're required in Java. First, you have better introspection in Python, so you don't need getters and setters to help make dynamic bean objects. (For that, you use collections.namedtuple).
However, you have the property decorator which will bundle getters (and setters) into an attribute-like construct. The point is that Python prefers naked attributes; when necessary, we can bundle getters and setters to appear as if there's a simple attribute.
Also, Python has descriptor classes if properties aren't sophisticated enough.
Code duplication is often a necessary evil in Java (e.g. method overloading), but not in Python. Correct. Python uses optional arguments instead of method overloading.
The bullet point went on to talk about closure; that isn't as helpful as the simple advice to use default argument values wisely.
One thing you might be used to in Java that you won't find in Python is strict privacy. This is not so much something to look out for as it is something not to look for (I am embarrassed by how long I searched for a Python equivalent to 'private' when I started out!). Instead, Python has much more transparency and easier introspection than Java. This falls under what is sometimes described as the "we're all consenting adults here" philosophy. There are a few conventions and language mechanisms to help prevent accidental use of "unpublic" methods and so forth, but the whole mindset of information hiding is virtually absent in Python.
The biggest one I can think of is not understanding or not fully utilizing duck typing. In Java you're required to specify very explicit and detailed type information upfront. In Python typing is both dynamic and largely implicit. The philosophy is that you should be thinking about your program at a higher level than nominal types. For example, in Python, you don't use inheritance to model substitutability. Substitutability comes by default as a result of duck typing. Inheritance is only a programmer convenience for reusing implementation.
Similarly, the Pythonic idiom is "beg forgiveness, don't ask permission". Explicit typing is considered evil. Don't check whether a parameter is a certain type upfront. Just try to do whatever you need to do with the parameter. If it doesn't conform to the proper interface, it will throw a very clear exception and you will be able to find the problem very quickly. If someone passes a parameter of a type that was nominally unexpected but has the same interface as what you expected, then you've gained flexibility for free.
The most important thing, from a Java POV, is that it's perfectly ok to not make classes for everything. There are many situations where a procedural approach is simpler and shorter.
The next most important thing is that you will have to get over the notion that the type of an object controls what it may do; rather, the code controls what objects must be able to support at runtime (this is by virtue of duck-typing).
Oh, and use native lists and dicts (not customized descendants) as far as possible.
The way exceptions are treated in Python is different from
how they are treated in Java. While in Java the advice
is to use exceptions only for exceptional conditions this is not
so with Python.
In Python things like Iterator makes use of exception mechanism to signal that there are no more items.But such a design is not considered as good practice in Java.
As Alex Martelli puts in his book Python in a Nutshell
the exception mechanism with other languages (and applicable to Java)
is LBYL (Look Before You Leap) :
is to check in advance, before attempting an operation, for all circumstances that might make the operation invalid.
Where as with Python the approach is EAFP (it's easier to Ask for forgiveness than permission)
A corrollary to "Don't use classes for everything": callbacks.
The Java way for doing callbacks relies on passing objects that implement the callback interface (for example ActionListener with its actionPerformed() method). Nothing of this sort is necessary in Python, you can directly pass methods or even locally defined functions:
def handler():
print("click!")
button.onclick(handler)
Or even lambdas:
button.onclick(lambda: print("click!\n"))

Is it a good idea to use super() in Python?

Or should I just explicitly reference the superclasses whose methods I want to call?
It seems brittle to repeat the names of super classes when referencing their constructors, but this page http://fuhm.net/super-harmful/ makes some good arguments against using super().
The book Expert Python Programming has discussed the topic of "super pitfalls" in chapter 3. It is worth reading. Below is the book's conclusion:
Super usage has to be consistent: In a class hierarchy, super should be used everywhere or nowhere. Mixing super and classic calls is a confusing practice. People tend to avoid super, for their code to be more explicit.
Edit: Today I read this part of the book again. I'll copy some more sentences, since super usage is tricky:
Avoid multiple inheritance in your code.
Be consistent with its usage and don't mix new-style and
old-style.
Check the class hierarchy before calling its methods in
your subclass.
You can use super, but as the article says, there are drawbacks. As long as you know them, there is no problem with using the feature. It's like people saying "use composition, not inheritance" or "never use global variables". If the feature exists, there is a reason. Just be sure to understand the why and the what and use them wisely.
super() tries to solve for you the problem of multiple inheritance; it's hard to replicate its semantics and you certainly shouldn't create any new semantics unless you're completely sure.
For single inheritance, there's really no difference between
class X(Y):
def func(self):
Y.func(self)
and
class X(Y):
def func(self):
super().func()
so I guess that's just the question of taste.
I like super() more because it allows you to change the inherited class (for example when you're refactoring and add an intermediate class) without changing it on all the methods.
The problem people have with super is more a problem of multiple inheritance. So it is a little unfair to blame super. Without super multiple inheritance is even worse. Michele Simionato nicely wrapped this up in his blog article on super:
On the other hand, one may wonder if
all super warts aren't hints of some
serious problem underlying. It may
well be that the problem is not with
super, nor with cooperative methods:
the problem may be with multiple
inheritance itself.
So the main lesson is that you should try to avoid multiple inheritance.
In the interest of consistency I always use super, even if for single inheritance it does not really matter (apart from the small advantage of not having to know the parent class name). In Python 3+ super is more convenient, so there one should definitely use super.
Yes, you should use super() over other methods. This is now the standard object inheritance model in Python 3.
Just stick to keyword arguments in your __init__ methods and you shouldn't have too many problems. Additionally you can use **kwargs to support additional parameters that are not defined in levels of the inheritance chain.
I agree that it is brittle, but no less so than using the name of the inherited class.

Categories

Resources