I am dealing with a scenario where I have a python class Foo. Foo, among other things does many big calculations, which I would not do unless required. So, when I define the getter method for one of those big calculations, how do I make sure that the method corresponding to the calculation (here bigcalculation()) has already run?
class Foo:
def __init__(self):
#initialize some stuff
def bigcalculation(self):
#perform calculation
self.big_calc_result=[big list of numbers];
def get_big_calc_result(self):
if hasattr(self,'big_calc_result')==False:
self.bigcalculations();
return sef.big_calc_result;
If its run once, I don't want it to run again. and I don't want caller to have to keep track of whether it has run once or not.
Now, I do it using hasattr() function as above, but I think this is a really ugly way to do it. Is there a more elegant pythonic way to do it?
An alternative I can think of, is to define, in my init() function, all the variables that I would ever use in the class, as empty list. Then check whether big_calc_result is an empty list or not to determine if self.bigcalculation() has already run. Is this a better approach?
related question:Python lets me define variables on the fly in a class. But is that bad programming practice?
Edit: In retrospect, I also found that using exceptions can also be another way of handling this situation. That might be a more pythonic way of doing things.
def get_big_calc_result(self):
try:
return self.big_calc_result;
except AttributeError:
self.bigcalculations();
return self.big_calc_result;
The answers to this question are useful:
Checking for member existence in Python
You can memoize the result and store it as a property:
class Foo(object):
def __init__(self):
self._big_calc_result = None
#property
def big_calc_result(self):
if self._big_calc_result is not None:
return self._big_calc_result
else:
self._big_calc_result = self.big_calc_run()
return self._big_calc_result
def big_calc_run(self):
time.sleep(10) # Takes a long time ...
Now, you just initialize the class and get the result:
f = Foo()
x = f.big_calc_result
y = f.bic_calc_result # Look mom, this happened really quick
Of course, you don't have to use a property if that is less intuitive and you can change things around here to suit the API you're trying to provide. The real meat is in caching the result in the variable prefixed with an underscore which is to say "This is an implementation detail -- if you mess with it, you deserve to have your code break at some point in the future".
Related
I'm trying to add flexibility to a python class, so that it notices when one of the init arguments is already an instance of that class. Skip "Initial situation" if you don't mind, how I got here.
Initial situation
I have this class:
class Pet:
def __init__(self, animal):
self._animal = animal
#property
def present(self):
return "This pet is a " + self._animal
...
and there are many functions which accept an instance of this class as an argument (def f(pet, ...)). Everything worked as expected.
I then wanted to add some flexibility to the usage of these functions: if the caller passes a Pet instance, everything keeps on working as before. In all other cases, a Pet instance is created. One way to achieve that, is like this:
def f(pet_or_animal, ...):
if isinstance(pet_or_animal, Pet): #Pet instance was passed
pet = pet_or_animal
else: #animal string was passed
pet = Pet(pet_or_animal)
...
This also works as expected, but these lines are repeated in every function. Not DRY, not good.
Goal
So, I'd like to extract the if/else from each of the functions, and integrate it into the Pet class itself. I tried changing its __init__ method to
class PetA: #I've changed the name to facilitate discussion here.
def __init__(self, pet_or_animal):
if isinstance(pet_or_animal, PetA):
self = pet_or_animal
else:
self._animal = pet_or_animal
...
and start each function with
def f(pet_or_animal, ...):
pet = PetA(pet_or_animal)
...
However, that is not working. If a Pet instance is passed, everything is good, but if a string is called, a Pet instance is not correctly created.
Current (ugly) solution
What is working, is to add a class method to the class, like so:
class PetB: #I've changed the name to facilitate discussion here.
#classmethod
def init(cls, pet_or_animal):
if isinstance(pet_or_animal, PetB):
return pet_or_animal
else:
return cls(pet_or_animal)
def __init__(self, animal):
self._animal = animal
...
and also change the functions to
def f(pet_or_animal, ...):
pet = PetB.init(pet_or_animal) #ugly
...
Questions
Does anyone know, how to change class PetA so, that it has the intended behavior? To be sure, here is the quick test:
pb1 = PetB.init('dog')
pb2 = PetB.init(pb1) #correctly initialized; points to same instance as pb1 (as desired)
pa1 = PetA('cat')
pa2 = PetA(pa1) #incorrectly initialized; pa1 != pa2
More generally, is this the right way to go about adding this flexibility? Another option I considered was writing a separate function to just do the checking, but this too is rather ugly and yet another thing to keep track of. I'd rather keep everything neat and wrapped in the class itself.
And one final remark: I realize that some people might find the added class method (petB) a more elegant solution. The reason I prefer to add to the __init__ method (petA) is that, in my real-world use, I already allow for many different types of initialization arguments. So, there is already a list of if/elif/elif/... statements that check, just which of the possibilities is used by the creator. I'd like to extend that by one more case, namely, if an initialized instance is passed.
Many thanks
I believe your current "ugly" solution is actually the correct approach.
This pushes the flexibility up as far as possible, since it is messy. Even though python allows for arbitrary types and values to float around, your users and yourself will thank you for keeping that constrained to the outermost levels.
I would think of it as (don't need to implement it this way)
class Pet:
#classmethod
def from_animal(cls, ...):
...
#classmethod
def from_pet(cls, ...):
...
#classmethod
def auto(cls, ...):
if is_pet(...):
return cls.from_pet(...)
def __init__(cls, internal_rep):
...
etc.
It is a code smell if you don't know whether your function is taking an object or an initializer. See if you can do processing as up-front as possible with user input and standardize everything beyond there.
You could use a function instead to get the same behaviour you want:
def make_pet_if_required(pet_or_animal):
if isinstance(pet_or_animal, PetA):
return pet_or_animal
else:
return Pet(pet_or_animal)
And then:
def f(pet_or_animal, ...):
pet = make_pet_if_required(pet_or_animal)
...
For more "beauty" you can try turning that function call into a decorator.
So I've used python as a functional language for a while but I'm trying to do thing "right" and use classes now... and falling down. I'm trying to write a classmethod that can instantiate multiple members of the class (use case is load rows from SQLAlchemy.) I'd like to just be able to call the classmethod and have it return a status code (success/failure) rather than returning a list of objects. Then to access the objects I'll iterate through the class. Here's my code so far (which fails to iterate when I use the classmethod, works fine when I use the normal constructor.) Am I way off-base/crazy here? What's the "pythonic" way to do this? Any help is appreciated and thank you.
class KeepRefs(object):
__refs__ = defaultdict(list)
def __init__(self):
self.__refs__[self.__class__].append(weakref.ref(self))
#classmethod
def get_instances(cls):
for inst_ref in cls.__refs__[cls]:
inst = inst_ref()
if inst is not None:
yield inst
class Credentials(KeepRefs):
def __init__(self,name, username, password):
super(Credentials, self).__init__()
self.name=name
self.username=username
self.password=password
#classmethod
def loadcreds(cls):
Credentials('customer1','bob','password')
return True
success = Credentials.loadcreds()
for i in Credentials.get_instances():
print (i.name)
In your own words - yes, you are off-base and crazy :)
Status-Codes are a thing of C, not languages with proper exception semantics as Python. Modifying global state is a sure recipe for disaster. So - don't do it. Return a list of objects. Throw an exception if something disastrous happens, and just return an empty list if there happen to be no objects. This allows the client code to just do
for item in Thingies.load_thingies():
... # this won't do anything if load_thingies gave us an empty list
without having to painstakingly check something before using it.
Functional languages have certain advantages, and you are going too far the other way in your exploration of the procedural style. Global variables and class variable have their place, but what will happen if you need to fire off two SQAlchemy queries and consume the results in parallels? The second query will stomp over the class attributes that the first one still needs, is what. Using an object attribute (instance attribute) solves the problem, since each result contains its own handle.
If your concern is to avoid pre-fetching the array of results, you are in luck because Python offers the perfect solution: Generators, which are basically lazy functions. They are so nicely integrated in Python, I bet you didn't know you've been using them with every for-loop you write.
I have a function foo that takes a parameter stuff
Stuff can be something in a database and I'd like to create a function that takes a stuff_id, get the stuff from the db, execute foo.
Here's my attempt to solve it:
1/ Create a second function with suffix from_stuff_id
def foo(stuff):
do something
def foo_from_stuff_id(stuff_id):
stuff = get_stuff(stuff_id)
foo(stuff)
2/ Modify the first function
def foo(stuff=None, stuff_id=None):
if stuff_id:
stuff = get_stuff(stuff_id)
do something
I don't like both ways.
What's the most pythonic way to do it ?
Assuming foo is the main component of your application, your first way. Each function should have a different purpose. The moment you combine multiple purposes into a single function, you can easily get lost in long streams of code.
If, however, some other function can also provide stuff, then go with the second.
The only thing I would add is make sure you add docstrings (PEP-257) to each function to explain in words the role of the function. If necessary, you can also add comments to your code.
I'm not a big fan of type overloading in Python, but this is one of the cases where I might go for it if there's really a need:
def foo(stuff):
if isinstance(stuff, int):
stuff = get_stuff(stuff)
...
With type annotations it would look like this:
def foo(stuff: Union[int, Stuff]):
if isinstance(stuff, int):
stuff = get_stuff(stuff)
...
It basically depends on how you've defined all these functions. If you're importing get_stuff from another module the second approach is more Pythonic, because from an OOP perspective you create functions for doing one particular purpose and in this case when you've already defined the get_stuff you don't need to call it within another function.
If get_stuff it's not defined in another module, then it depends on whether you are using classes or not. If you're using a class and you want to use all these modules together you can use a method for either accessing or connecting to the data base and use that method within other methods like foo.
Example:
from some module import get_stuff
MyClass:
def __init__(self, *args, **kwargs):
# ...
self.stuff_id = kwargs['stuff_id']
def foo(self):
stuff = get_stuff(self.stuff_id)
# do stuff
Or if the functionality of foo depends on the existence of stuff you can have a global stuff and simply check for its validation :
MyClass:
def __init__(self, *args, **kwargs):
# ...
_stuff_id = kwargs['stuff_id']
self.stuff = get_stuff(_stuff_id) # can return None
def foo(self):
if self.stuff:
# do stuff
else:
# do other stuff
Or another neat design pattern for such situations might be using a dispatcher function (or method in class) that delegates the execution to different functions based on the state of stuff.
def delegator(stff, stuff_id):
if stuff: # or other condition
foo(stuff)
else:
get_stuff(stuff_id)
Consider the following example:
class Company():
def hireEmployee():
def fireEmployee():
def promoteEmployee():
etc...
class EngineeringFirm(Company):
pass
class PaintingFirm(Company):
pass
Suppose the Company class has a lot more methods. What if I want to rename these methods from the superclass so I can get the following:
class EngineeringFirm(Company):
def hireEngineer():
...
class PaintingFirm(Company):
def hirePainter():
...
...and so on. While using 'Employee' in this scenario really wouldn't hurt a bit, this is really just to illustrate the idea. How would I go about it?
My idea was to use a classFactory function that would take the type of employee as argument and generate a Company class, while a metaclass would handle the renaming by iterating through the attribute dictionary and replacing 'Employee' with said type.
class EngineeringFirm(companyFactory('Engineer'))
...
The only problem is this: What if the methods inside of Company make calls to one another by the default 'Employee' names? This is where I'm stumped. I had the idea that the metaclass involved in renaming the methods could also get the source of each function (via the inspect module) and search if a known method attribute is found within and, if so, replace that part and create a new function via exec and assigning it back to the right attribute key.
...But that really seems kinda of hacky. I am open to alternatives and although I realize there may be design-related issues with the question (I am open to suggestions on that front as well) I would be interested in finding out if this problem has a more elegant solution.
Thanks!
Edit: another solution
For the sake of argument, I'll assume for a moment that the code above is really what I'm working with; I figured I could address some of the concerns in the comments with another solution I had in mind, one I'd already considered and put away for reasons I'll explain.
If the Firm classes inherited from Company and I wished to maintain a identical interface (as one usually would in a case like this to allow dynamic calls to hire() or promote(), etc) I could implement a __getattribute__ that accepts HirePainter() (by accessing the original Employee method) while still allowing any other interface to use the HireEmployee() if necessary.
I wonder, supposing it's alright to extend my question, if this is something that would be considered bad practice if, say, I planned to do this because I thought that the code inside PaintingFirm would benefit in readability? Again, I realize this example is horrid in that readability here really does not seem to benefit in any way whatsoever, but suppose it did?
(The only reason I didn't suggest this idea in the first place is that my __getattribute__ already handles quite a bit, and adding extra noise to it didn't feel that appealing. Still, I could work it in, but this is a question I had to ask in case there were more magical (but not hacky) solutions out there..)
For posterity's sake, I'm posting a solution of my own that I believe is a decent alternative. I don't suggest this as the answer because the truth is I did not mention in my question that I preferred not adding extra names, or to retain the ability to call these attributes as self.hireEngineer rather than ClassDict['HireEngineer']. Given that, I can't really say any of these answers don't answer the question.
Solution:
In hindsight, the problem was a lot simpler than I made it out to be. I guess I got hooked on the metaclassery just for the sake of it. If it's not already obvious, I'm really only just learning about metaclasses and for a moment it seemed like a good opportunity to try them out. Alas.
I believe the following solution respects the spirit of Liskov's principle (thank you, Ignacio) while giving the derived class the ability to reference the derived methods in its own way. The class namespace stays the same and other objects can call upon these methods with their real names if necessary.
# superclass...
def __getattribute__(self, attr):
# Early exit (AttributeError) if attribute not found.
obj = object.__getattribute__(self, attr)
# All the extra code...
def __getattr__(self, attr):
# Ex. self.type == 'Engineer'
# Replacing titled-cased and lower-cased
# versions just to be safe (ex. self.employeeNames)
attr = (attr
.replace(self.type, 'Employee')
.replace(self.type.lower(), 'employee')
)
if attr in self.attributes:
return self.__getattribute__(attr)
else:
raise AttributeError
I'll try to do a better job next time around when outlining the requirements. Thanks, guys.
You could try adding in a dictionary for each class.
class EngineeringFirm(Company):
ClassDict = {'HireEngineer':self.HireEmployee,
...
};
Whenever you want to call the function you would use
<EngineeringFirmInstanc>.ClassDict['HireEngineer'](<arguments>)
It's not particularly elegant, but it might get you close to what you are asking.
I tend to agree with the comments on the question: I suspect that what you're asking would add unnecessary complication to the code, making it harder to read & maintain just to implement a minor "cosmetic" feature of dubious benefit.
However, if you really want to do this, perhaps you could create methods that are synonyms of the existing methods, so you can call a method with its original name or with a "customized" name when it seems appropriate.
Here's one fairly straight-forward way to do that. I guess there's some sleek way to do it with class decorators, but I don't know how to use those. :)
#! /usr/bin/env python
''' Class synonym demo
From http://stackoverflow.com/q/27729681/4014959
Written by PM 2Ring 2015.01.01
'''
class Foo(object):
def __init__(self, data):
self.foo_set(data)
def foo_set(self, data):
self.data = data
def foo_add(self, n):
self.data += n
return self.data
def foo_mul(self, n):
self.data *= n
return self.data
def foo_mul_add(self, n, m):
self.foo_mul(n)
return self.foo_add(m)
def make_synonyms(cls, old, new):
class newclass(cls):
pass
d = cls.__dict__
for k in d:
if k.startswith(old):
newname = k.replace(old, new)
#print k, d[k], newname
setattr(newclass, newname, d[k])
return newclass
#--------------------------------------
Bar = make_synonyms(Foo, 'foo', 'bar')
a = Foo(5)
print a.data
print a.foo_add(10)
print a.foo_mul(4)
print a.foo_mul_add(2, 1)
print '-' * 20
a = Bar(6)
print a.data
print a.foo_add(10)
print a.foo_mul(4)
print a.foo_mul_add(2, 1)
print '-' * 20
a.bar_set(5)
print a.data
print a.bar_add(10)
print a.bar_mul(4)
print a.bar_mul_add(2, 1)
output
5
15
60
121
--------------------
6
16
64
129
--------------------
5
15
60
121
I am writing a small mocking class to do some tests.
But this class needs to support the idea of having nested attributes.
This example should provide some insight to the problem:
class Foo(object):
def __init__(self):
self.x = True
From the above class, we can have:
f = Foo()
f.x
I know I can add attributes falling back to __getattr__ to avoid an AttributeError, but what if I need something like this to be valid:
f = Foo()
f.x
f.x.y
f.x.y.z()
I know what to return if the object gets called as f.x.y.z() but I just need to find a way to get to z() that makes sense.
You can "mock anything" by returning, on each attribute access, another instance of the "mock anything" class (which must also be callable, if you want to have the .z() part work;-).
E.g.:
class MockAny(object):
# mock special methods by making them noops
def __init__(self, *a, **k): pass
# or returning fixed values
def __len__(self): return 0
# mock attributes:
def getattr(self, name):
return MockAny()
# make it callable, if you need to
def __call__(self, *a, **k):
return MockAny()
The alternative, of course, is to know what it is that you're mocking (by introspection, or by some form of "declarative description", or simply by coding mock for specific things;-) rather than take the catch-all approach; but, the latter is also feasible, as you see in the above (partial) example.
Personally, I'd recommend using an existing mocking framework such as pymox rather than reinventing this particular wheel (also, the source code for such frameworks can be more instructive than a reasonably terse response on SO, like this one;-).
If you are calling something like f.x.y.z() in your unit tests, the chances are you're trying to test too much. Each of these nested attributes should be covered by the unit tests for their particular classes.
Take another look at your Foo class and see if you can test its own behaviour in your unit tests.
Perhaps not the answer you were looking for, but hopefully one that will help in the long run.