I have currently this structure inside a file:
class Foo:
__init__(self):
pass
class FooType(object):
__init__(self, value):
_foo = value
__str__(self):
print ">>%s<<" % self._foo
class FooException(Exception):
pass
All the above classes are tightly related. My master class will have types, structures, enums all declared as a separated class with the prefix Foo. And as usual a custom exception should be declared for Foo. At the end of the day I will have a lot of related classes at the same level of other classes.
Is there a proper way to get a better structure? Perhaps a namespace can help, but I don't know how to use it.
In Python, the idiomatic way to do namespaces is to use different modules. You should name your file foo.py, and then import that. Then you use it with foo.Exception and foo.Type. If you need to have a more complex module that needs more than one file, you should make a folder called foo and put an __init__.py file along with your other components of the module. For further documentation on using modules, see the docs.
Another solution is to use nested classes to provide a namespace. The chosen answer to this question about nested classes recommends them for the purpose of namespacing.
You could organize your classes in modules, just like the Python standard library does.
Related
Assume I have some class with an inner class:
class A:
class B:
...
...
I would like to implement class B in a different file than the code for class A to increase readability.
How can this be accomplished?
EDIT: I don't want B to be accessible in any way other than A.B.
You could do this:
b_class.py:
class _B:
...
a_class.py:
import b_class
class A:
B = b_class._B
...
One of the principles of Pythonic OOP is "we're all consenting adults here". This means that you don't usually want to forcibly hide information. Leading underscores used to mean "don't use this", but they don't prevent things being accessed.
Although generally you don't need to nest classes like this. What do you want to do this for? There is probably a better solution.
You seem to be working on a C++ namespace or Java package declaration principle which is not the Python model. The implementation of namespaces in Python is based the file name. Objects (classes, functions, etc.) are named by the source file that contains them.
You want HMM and HMM.State? Fine, do as sweeneyrod suggested and use the has-a object composition principle:
import states
class HMM {
def __init__(self):
self.markov_stuff = stuff
self.state = states.State(more_stuff)
…
}
That's how it works, where states.py contains the definition of class State.
If you want to make it look more "integrated" you could use from states import State yielding
from states import State
class HMM {
def __init__(self):
self.markov_stuff = stuff
self.state = State(more_stuff)
…
}
That gives you separate files: markov.py and states.py with the syntax of one less level of . indirection.
If you desire something else, either you get into confusing (and unconventional) module dictionary munging which is not recommended. Different languages allow expression of intent in different ways, and this is the pythonic way.
Like in ruby we have module which can be included in any given class and then we can use the methods defined there, this saves us from inheritance which is considered as a coupling. In python is there any way to replicate same functionality? For eg I have two classes
class A:
pass
class B:
pass
I want to have both of the above two classes to have logging functionality which may be like:
def log(message):
print message
In Python, we generally just go ahead and use inheritance for this. It's essentially a mixin pattern.
You can use modules, though, if you really want to:
class Foo(object):
def __init__(self):
pass
from methods import baz, bar, quux
Here methods.py would be a module containing functions (or other attributes) named baz, bar, and quux, and possibly others. As shown, you can go ahead and define methods directly on the class as well.
Using a wildcard import gives you a warning in Python 2.7 and I believe it's disallowed entirely in Python 3.x:
class Foo(object):
from methods import *
So, as usual in Python, "explicit is better than implicit." That is, specify exactly which methods you want in your class instead of saying "give me all of them." It makes your code much clearer.
I am writing a framework to be used by people who know some Python. I have settled on some syntax, and it makes sense to me and them to use something like this, where Base is the Base class that implements the framework.
class A(Base):
#decorator1
#decorator2
#decorator3
def f(self):
pass
#decorator4
def f(self):
pass
#decorator5
def g(self)
pass
All my framework is implemented via Base's metaclass. This setup is appropriate for my use case, because all these user-defined classes have a rich inheritance graph. I expect the user to implement some of the methods, or just leave it with pass. Much of the information that the user is giving here is in the decorators. This allows me to avoid other solutions where the user would have to monkey-patch, give less structured dictionaries, and things like that.
My problem here is that f is defined twice by the user (with good reason), and this should be handled by my framework. Unfortunately, by the time this gets to the metaclass'__new__ method, the dictionary of attributes contains only one key f. My idea was to use yet another decorator, such as #duplicate for the user to signal this is happening, and the two f's to be wrapped differently so they don't overwrite each other. Can something like this work?
You should use a namespace to distinguish the different fs.
Heed the advice of the "Zen of Python":
Namespaces are one honking great idea -- let's do more of those!
Yes, you could, but only with an ugly hack, and only by storing the function with a new name.
You'd have to use the sys._getframe() function to retrieve the local namespace of the calling frame. That local namespace is the class-in-construction, and adding items to that namespace means they'll end up in the class dictionary passed to your metaclass.
The following retrieves that namespace:
callframe = sys._getframe(1)
namespace = callframe.f_locals
namespace is a dict-like object, just like locals(). You can now store something in that namespace (like __function_definitions__ or similar) to add extra references to the functions.
You might be thinking Java - methods overloading and arguments signature - but this is Python and you cannot do this. The second f() will override the first f() and you end up with only one f(). The namespace is a dictionary and you cannot have duplicated keys.
Python classes have no concept of public/private, so we are told to not touch something that starts with an underscore unless we created it. But does this not require complete knowledge of all classes from which we inherit, directly or indirectly? Witness:
class Base(object):
def __init__(self):
super(Base, self).__init__()
self._foo = 0
def foo(self):
return self._foo + 1
class Sub(Base):
def __init__(self):
super(Sub, self).__init__()
self._foo = None
Sub().foo()
Expectedly, a TypeError is raised when None + 1 is evaluated. So I have to know that _foo exists in the base class. To get around this, __foo can be used instead, which solves the problem by mangling the name. This seems to be, if not elegant, an acceptable solution. However, what happens if Base inherits from a class (in a separate package) called Sub? Now __foo in my Sub overrides __foo in the grandparent Sub.
This implies that I have to know the entire inheritance chain, including all "private" objects each uses. The fact that Python is dynamically-typed makes this even harder, since there are no declarations to search for. The worst part, however, is probably the fact Base might inherit from object right now, but in some future release, it switches to inheriting from Sub. Clearly if I know Sub is inherited from, I can rename my class, however annoying that is. But I can't see into the future.
Is this not a case where a true private data type would prevent a problem? How, in Python, can I be sure that I'm not accidentally stepping on somebody's toes if those toes might spring into existence at some point in the future?
EDIT: I've apparently not made clear the primary question. I'm familiar with name mangling and the difference between a single and a double underscore. The question is: how do I deal with the fact that I might clash with classes whose existence I don't know of right now? If my parent class (which is in a package I did not write) happens to start inheriting from a class with the same name as my class, even name mangling won't help. Am I wrong in seeing this as a (corner) case that true private members would solve, but that Python has trouble with?
EDIT: As requested, the following is a full example:
File parent.py:
class Sub(object):
def __init__(self):
self.__foo = 12
def foo(self):
return self.__foo + 1
class Base(Sub):
pass
File sub.py:
import parent
class Sub(parent.Base):
def __init__(self):
super(Sub, self).__init__()
self.__foo = None
Sub().foo()
The grandparent's foo is called, but my __foo is used.
Obviously you wouldn't write code like this yourself, but parent could easily be provided by a third party, the details of which could change at any time.
Use private names (instead of protected ones), starting with a double underscore:
class Sub(Base):
def __init__(self):
super(Sub, self).__init__()
self.__foo = None
# ^^
will not conflict with _foo or __foo in Base. This is because Python replaces the double underscore with a single underscore and the name of the class; the following two lines are equivalent:
class Sub(Base):
def x(self):
self.__foo = None # .. is the same as ..
self._Sub__foo = None
(In response to the edit:) The chance that two classes in a class hierarchy not only have the same name, but that they are both using the same property name, and are both using the private mangled (__) form is so minuscule that it can be safely ignored in practice (I for one haven't heard of a single case so far).
In theory, however, you are correct in that in order to formally verify correctness of a program, one most know the entire inheritance chain. Luckily, formal verification usually requires a fixed set of libraries in any case.
This is in the spirit of the Zen of Python, which includes
practicality beats purity.
Name mangling includes the class so your Base.__foo and Sub.__foo will have different names. This was the entire reason for adding the name mangling feature to Python in the first place. One will be _Base__foo, the other _Sub__foo.
Many people prefer to use composition (has-a) instead of inheritance (is-a) for some of these very reasons.
This implies that I have to know the entire inheritance chain. . .
Yes, you should know the entire inheritance chain, or the docs for the object you are directly sub-classing should tell you what you need to know.
Subclassing is an advanced feature, and should be treated with care.
A good example of docs specifying what should be overridden in a subclass is the threading class:
This class represents an activity that is run in a separate thread of control. There are two ways to specify the activity: by passing a callable object to the constructor, or by overriding the run() method in a subclass. No other methods (except for the constructor) should be overridden in a subclass. In other words, only override the __init__() and run() methods of this class.
How often do you modify base classes in inheritance chains to introduce inheritance from a class with the same name as a subclass further down the chain???
Less flippantly, yes, you have to know the code you are working with. You certainly have to know the public names being used, after all. Python being python, discovering the public names in use by your ancestor classes takes pretty much the same effort as discovering the private ones.
In years of Python programming, I have never found this to be much of an issue in practice. When you're naming instance variables, you should have a pretty good idea whether (a) a name is generic enough that it's likely to be used in other contexts and (b) the class you're writing is likely to be involved in an inheritance hierarchy with other unknown classes. In such cases, you think a bit more carefully about the names you're using; self.value isn't a great idea for an attribute name, and neither is something like Adaptor a great class name.
In contrast, I have run into difficulties with the overuse of double-underscore names a number of times. Python being Python, even "private" names tend to be accessed by code defined outside the class. You might think that it would always be bad practice to let an external function access "private" attributes, but what about things like getattr and hasattr? The invocation of them can be in the class's own code, so the class is still controlling all access to the private attributes, but they still don't work without you doing the name-mangling manually. If Python had actually-enforced private variables you couldn't use functions like those on them at all. These days I tend to reserve double-underscore names for cases when I'm writing something very generic like a decorator, metaclass, or mixin that needs to add a "secret attribute" to the instances of the (unknown) classes it's applied to.
And of course there's the standard dynamic language argument: the reality is that you have to test your code thoroughly to have much justification in making the claim "my software works". Such testing will be very unlikely to miss the bugs caused by accidentally clashing names. If you are not doing that testing, then many more uncaught bugs will be introduced by other means than by accidental name clashes.
In summation, the lack of private variables is just not that big a deal in idiomatic Python code in practice, and the addition of true private variables would cause more frequent problems in other ways IMHO.
Mangling happens with double underscores. Single underscores are more of a "please don't".
You don't need to know all the details of all parent classes (note that deep inheritance is usually best avoided), because you can still dir() and help() and any other form of introspection you can come up with.
As noted, you can use name mangling. However, you can stick with a single underscore (or none!) if you document your code adequately - you should not have so many private variables that this proves to be a problem. Just say if a method relies on a private variable, and add either the variable, or the name of the method to the class docstring to alert users.
Further, if you create unit tests, you should create tests that check invariants on members, and accordingly these should be able to show up such name clashes.
If you really want to have "private" variables, and for whatever reason name-mangling doesn't meet your needs, you can factor your private state into another object:
class Foo(object):
class Stateholder(object): pass
def __init__(self):
self._state = Stateholder()
self.state.private = 1
I'm developing a PyQT4 application, and it's getting pretty hard for me to navigate through all of the code at once. I know of the import foo statement, but I can't figure out how to make it import a chunk of code directly into my script, like the BASH source foo statement.
I'm trying to do this:
# File 'functions.py'
class foo(asd.fgh):
def __init__(self):
print 'foo'
Here is the second file.
# File 'main.py'
import functions
class foo(asd.fgh):
def qwerty(self):
print 'qwerty'
I want to include code or merge class decelerations from two separate files. In PHP, there is import_once('foo.php'), and as I mentioned previously, BASH has source 'foo.sh', but can I accomplish this with Python?
Thanks!
For some reason, my first thought was multiple inheritance. But why not try normal inheritance?
class foo(functions.foo):
# All of the methods that you want to add go here.
Is there some reason that this wont work?
Since you just want to merge class definitions, why don't you do:
# main.py
import functions
# All of the old stuff that was in main.foo is now in this class
class fooBase(asd.fgh):
def qwerty(self):
print 'qwerty'
# Now create a class that has methods and attributes of both classes
class foo(FooBase, functions.foo): # Methods from FooBase take precedence
pass
or
class foo(functions.foo, FooBase): # Methods from functions.foo take precedence
pass
This takes advantage of pythons capability for multiple inheritance to create a new class with methods from both sources.
You want execfile(). Although you really don't, since redefining a class, uh... redefines it.
monkey patching in python doesn't work in nearly the same way. This is normally considered poor form, but if you want to do it anyways, you can do this:
# File 'functions.py'
class foo(asd.fgh):
def __init__(self):
print 'foo'
the imported module remains unchanged. In the importing module, we do things quite differently.
# File 'main.py'
import functions
def qwerty(self):
print 'qwerty'
functions.foo.qwerty = qwerty
Note that there is no additional class definition, just a bare function. we then add the function as an attribute of the class.