I've been tasked with taking over a large, existing Python program.
The author created a MainClass(), along with many (technically) functions that include a 'self' first argument and are dynamically added to a ExternalClass() that gets imported/instantiated at runtime.
In this large file, it is very easy to confuse the methods of the MainClass() with the dynamic methods of ExternalClass(), especially since the both include a first argument of self and are separated only by a few blank lines.
I know I should refactor, but for now my first priority is to keep the program working while I learn more about it.
I have put a comment between the MainClass() methods and the dynamic methods.
What is the proper term for functions that are intended to be added to another class as methods at runtime?
I've been using the term "free methods", but that doesn't seem right.
class MainClass():
def __init__(self):...
def method_1(self):...
...
def method_n(self):...
#
# -- The following [term?] methods are added to ExternalClass() dynamically at runtime
#
def dynamic_method_1(self):...
def dynamic_method_n(self):...
I'm more of an engineer and less of a coder, but I know enough python and C++ to be dangerous.
I'm creating a python vector/matrix class as a helper class based upon numpy as well as cvxopt. The overall goal (which I've already obtained... the answer to this question will just make the class better) is to make dot products and other processes more unified and easier for numerical methods.
However, I'd like to make my helper class even more transparent. What I'd like to do is to redefine the cvxopt.matrix() init function based upon the current variable which was used. This is to say, if I have a custom matrix: "cstmat", I'd like the function "cvxopt.matrix(cstmat)" to be defined by my own methods instead of what is written in the cvxopt class.
In short, I'd like to "intercept" the other function call and use my own function.
The kicker, though, is that I don't want to take over cvxopt.matrix(any_other_type). I just want to redefine the function when it's called upon my own custom class. Is this possible?
Thanks,
Jon
You can do this, but it's not pretty.
You can do probably something along these lines:
cvxopt._orig_matrix = cvxopt.matrix
def my_matrix(*args, **kwargs):
if isinstance(arg[0], cstmat):
# do your stuff here
else:
cvxopt._orig_matrix(*args, **kwargs)
cvxopt.matrix = my_matrix
But you're probably better off finding a less weird way. And no guarantees that won't forget who "self" is.
Better would be to use inheritance! Kinda like this:
class Cstmat(cvsopt.matrix):
def __init__(self, ...):
pass
def matrix(self, arg):
if isinstance(arg, cstmat):
# do your stuff here
else:
cvsopt.matrix(arg)
When and how are static methods suppose to be used in python? We have already established using a class method as factory method to create an instance of an object should be avoided when possible. In other words, it is not best practice to use class methods as an alternate constructor (See Factory method for python object - best practice).
Lets say I have a class used to represent some entity data in a database. Imagine the data is a dict object containing field names and field values and one of the fields is an ID number that makes the data unique.
class Entity(object):
def __init__(self, data, db_connection):
self._data = data
self._db_connection
Here my __init__ method takes the entity data dict object. Lets say I only have an ID number and I want to create an Entity instance. First I will need to find the rest of the data, then create an instance of my Entity object. From my previous question, we established that using a class method as a factory method should probably be avoided when possible.
class Entity(object):
#classmethod
def from_id(cls, id_number, db_connection):
filters = [['id', 'is', id_number]]
data = db_connection.find(filters)
return cls(data, db_connection)
def __init__(self, data, db_connection):
self._data = data
self._db_connection
# Create entity
entity = Entity.from_id(id_number, db_connection)
Above is an example of what not to do or at least what not to do if there is an alternative. Now I am wondering if editing my class method so that it is more of a utility method and less of a factory method is a valid solution. In other words, does the following example comply with the best practice for using static methods.
class Entity(object):
#staticmethod
def data_from_id(id_number, db_connection):
filters = [['id', 'is', id_number]]
data = db_connection.find(filters)
return data
# Create entity
data = Entity.data_from_id(id_number, db_connection)
entity = Entity(data)
Or does it make more sense to use a standalone function to find the entity data from an ID number.
def find_data_from_id(id_number, db_connection):
filters = [['id', 'is', id_number]]
data = db_connection.find(filters)
return data
# Create entity.
data = find_data_from_id(id_number, db_connection)
entity = Entity(data, db_connection)
Note: I do not want to change my __init__ method. Previously people have suggested making my __init__ method to look something like this __init__(self, data=None, id_number=None) but there could be 101 different ways to find the entity data so I would prefer to keep that logic separate to some extent. Make sense?
When and how are static methods suppose to be used in python?
The glib answer is: Not very often.
The even glibber but not quite as useless answer is: When they make your code more readable.
First, let's take a detour to the docs:
Static methods in Python are similar to those found in Java or C++. Also see classmethod() for a variant that is useful for creating alternate class constructors.
So, when you need a static method in C++, you need a static method in Python, right?
Well, no.
In Java, there are no functions, just methods, so you end up creating pseudo-classes that are just bundles of static methods. The way to do the same thing in Python is to just use free functions.
That's pretty obvious. However, it's good Java style to look as hard as possible for an appropriate class to wedge a function into, so you can avoid writing those pseudo-classes, while doing the same thing is bad Python style—again, use free functions—and this is much less obvious.
C++ doesn't have the same limitation as Java, but many C++ styles are pretty similar anyway. (On the other hand, if you're a "Modern C++" programmer who's internalized the "free functions are part of a class's interface" idiom, your instincts for "where are static methods useful" are probably pretty decent for Python.)
But if you're coming at this from first principles, rather than from another language, there's a simpler way to look at things:
A #staticmethod is basically just a global function. If you have a function foo_module.bar() that would be more readable for some reason if it were spelled as foo_module.BazClass.bar(), make it a #staticmethod. If not, don't. That's really all there is to it. The only problem is building up your instincts for what's more readable to an idiomatic Python programmer.
And of course use a #classmethod when you need access to the class, but not the instance—alternate constructors are the paradigm case for that, as the docs imply. Although you often can simulate a #classmethod with a #staticmethod just by explicitly referencing the class (especially when you don't have much subclassing), you shouldn't.
Finally, getting to your specific question:
If the only reason clients ever need to look up data by ID is to construct an Entity, that sounds like an implementation detail you shouldn't be exposing, and it also makes client code more complex. Just use a constructor. If you don't want to modify your __init__ (and you're right that there are good reasons you might not want to), use a #classmethod as an alternate constructor: Entity.from_id(id_number, db_connection).
On the other hand, if that lookup is something that's inherently useful to clients in other cases that have nothing to do with Entity construction, it seems like this has nothing to do with the Entity class (or at least no more than anything else in the same module). So, just make it a free function.
The answer to the linked question specifically says this:
A #classmethod is the idiomatic way to do an "alternate constructor"—there are examples all over the stdlib—itertools.chain.from_iterable, datetime.datetime.fromordinal, etc.
So I don't know how you got the idea that using a classmethod is inherently bad. I actually like the idea of using a classmethod in your specific situation, as it makes following the code and using the api easy.
The alternative would be to use default constructor arguments like so:
class Entity(object):
def __init__(self, id, db_connection, data=None):
self.id = id
self.db_connection = db_connection
if data is None:
self.data = self.from_id(id, db_connection)
else:
self.data = data
def from_id(cls, id_number, db_connection):
filters = [['id', 'is', id_number]]
return db_connection.find(filters)
I prefer the classmethod version that you wrote originally however. Especially since data is fairly ambiguous.
Your first example makes the most sense to me: Entity.from_id is pretty succinct and clear.
It avoids the use of data in the next two examples, which does not describe what's being returned; the data is used to construct an Entity. If you wanted to be specific about the data being used to construct the Entity, then you could name your method something like Entity.with_data_for_id or the equivalent function entity_with_data_for_id.
Using a verb such as find can also be pretty confusing, as it doesn't give any indication of the return value — what is the function supposed to do when it's found the data? (Yes, I realize str has a find method; wouldn't it be better named index_of? But then there's also index...) It reminds me of the classic:
I always try to think what a name would indicate to someone with (a) no knowledge of the system, and (b) knowledge of other parts of the system — not to say I'm always successful!
Here is a decent use case for #staticmethod.
I have been working on a game as a side project. Part of that game includes rolling dice based on stats, and the possibility of picking up items and effects that impact your character's stats (for better or worse).
When I roll the dice in my game, I need to basically say... take the base character stats and then add any inventory and effect stats into this grand netted figure.
You can't take these abstract objects and add them without instructing the program how. I'm not doing anything at the class level or instance level either. I didn't want to define the function in some global module. The last best option was to go with a static method for adding up stats together. It just makes the most sense this way.
class Stats:
attribs = ['strength', 'speed', 'intellect', 'tenacity']
def __init__(self,
strength=0,
speed=0,
intellect=0,
tenacity=0
):
self.strength = int(strength)
self.speed = int(speed)
self.intellect = int(intellect)
self.tenacity = int(tenacity)
# combine adds stats objects together and returns a single stats object
#staticmethod
def combine(*args: 'Stats'):
assert all(isinstance(arg, Stats) for arg in args)
return_stats = Stats()
for stat in Stats.attribs:
for _ in args:
setattr(return_stats, stat,
getattr(return_stats, stat) + getattr(_, stat))
return (return_stats)
Which would make the stat combination calls work like this
a = Stats(strength=3, intellect=3)
b = Stats(strength=1, intellect=-1)
c = Stats(tenacity=5)
print(Stats.combine(a, b, c).__dict__)
{'strength': 4, 'speed': 0, 'intellect': 2, 'tenacity': 5}
I have a class in my Python module that frobs widgets. It has several ways to do this, but I don't know ahead of time which ones, if any, will actually work, and I might figure out new ways to frob the widgets later. Also, frobbing widgets can be expensive. So I'd like to have an instance of my class be able to look at itself, find all of the methods that it has for frobbing widgets, and to start trying to frob widgets until it succeeds, at which point it should stop caring about the methods it hasn't yet tried.
class WidgetFrobber:
def simpleFrobAttempt(widget, data):
# fastest way, might not work
def cleverFrobAttempt(widget, data):
# fiddly, fast, and doesn't always work
def boringFrobAttempt(widget, data):
# works most of the time, often slow
def desperateFrobAttempt(widget, data):
# doesn't always work, pathetically slow
With that in place, I'd like to define a method of the class that looks for methods named ^[a-z]+FrobAttempt$, makes a list of them, and tries to frob widgets until the widget is successfully frobbed (at which point it should stop caring about other methods) or it runs out of possible methods. Since it's my code, I can make sure that the whateverFrobAttempt methods all have the same naming conventions and required arguments. It's best if there's some ordering to the list of methods, so that the ones with a better average speed get tried first, but it's acceptable if they're attempted in random-ish order.
Is there a sane way to do this such that when new weirdFrobAttempt methods are added, they'll be tried automatically, or am I better off just maintaining a hardcoded list of such methods and iterating over that ?
I like sven's idea of having a register. This is hacky, but if storing a global list of method names is ok then we could do something like this:
FROB_METHOD_REGISTER = []
def frob_method(fn):
def decorate_frob(fn):
FROB_METHOD_REGISTER.add(fn.__name__)
return fn
return decorate_frob
class WidgetFrobber:
def get_frob_methods(self):
return [getattr(self, method) for method in FROB_METHOD_REGISTER]
#frob_method
def simpleFrobAttempt(widget, data):
# fastest way, might not work
#frob_method
def cleverFrobAttempt(widget, data):
# fiddly, fast, and doesn't always work
#frob_method
def boringFrobAttempt(widget, data):
# works most of the time, often slow
#frob_method
def desperateFrobAttempt(widget, data):
# doesn't always work, pathetically slow
so then you can do
for method in my_widget_frobber.get_frob_methods():
#do your thing
Not sure if this is the best or most pythonic way, and I'd be tempted to make the decorator use some sort of priority queue/ranking schema if there are some methods that are strictly better than others.
You could use __dict__. for example
[fn for name, fn in WidgetFrobber.__dict__.iteritems() if "FrobAttempt" in name]
would give you the set of methods you are looking for, but I would not recommend using things like __dict__ if there is a way to build code that is more clear and doesn't access python internals. And there almost always should be.
I have inherited code in which there are standalone functions, one per country code. E.g.
def validate_fr(param):
pass
def validate_uk(param):
pass
My idea is to create a class to group them together and consolidate the code into one method. Unfortunately that breaks cohesion. Another option is to dispatch to instance methods ?
class Validator(object):
def validate(param, country_code):
# dispatch
Alas, python does not have a switch statement.
UPDATE: I am still not convinced why I should leave them as global functions in my module. Lumping them as class methods seems cleaner.
I would keep the functions at module level -- no need for a class if you don't want to instantiate it anyway. The switch statement can easily be simulated using a dicitonary:
def validate_fr(param):
pass
def validate_uk(param)
pass
validators = {"fr": validate_fr,
"uk": validate_uk}
def validate(country_code, param):
return validators[country_code](param)
Given the naming scheme, you could also do it without the dictionary:
def validate(country_code, param):
return gloabls()["validate_" + country_code](param)
You do not need a switch statement for this.
validators = {
'fr': Validator(...),
'uk': Validator(...),
...
}
...
validators['uk'](foo)
Classes are not meant to group functions together, modules are. Functions in a class should be either methods that operate on the object itself (changing it's state, emitting information about the state, etc.) or class methods that do the same, but for the class itself (classes in Python are also objects). There's not even a need for static methods in Python, since you can always have functions at module level. As they say: Flat is better than nested.
If you want to have a set of functions place them in separate module.