I'm trying to find a better way to execute the following functions. I have a series of steps that need to be completed, and if any fail, I need to undo the previous step like so:
try:
A = createA()
except:
return None
try:
B = createB(A)
except:
deleteA(A)
return None
try:
C = createC(B)
except:
deleteB(B)
deleteA(A)
return None
try:
D = createD(C)
except:
deleteC(C)
deleteB(B)
deleteA(A)
return None
return D
I would prefer not to repeat myself if possible. How can I improve this? Is there a known pattern to follow?
One thing I have considered would be adding deleteB() to deleteC(), and deleteA() to deleteB(). Is that the best possible way to do it?
If you look some design patterns, check following:
Command pattern
It is probably what you are looking. Also, it commands can have "undo" action. Check the following question also, if it contains a similar problem that you have. best design pattern for undo feature
As the comments point out, this is what the context manager protocol is for. However, if you don't want to dig into what is a fairly advanced feature yet, then you can define lambda anonymous functions as you go, to remember what to tidy ...
try:
deleters = []
A = createA()
deleters.append( lambda: deleteA(A) )
B = createB( A)
deleters.append( lambda: deleteB(B) )
C = createC( B)
deleters.append( lambda: deleteC(C) )
D = createD( C)
return D
except:
for d in reversed( deleters):
d()
return None
It depends on what exactly "undo" means. E.g. if A, B, C, D etc are arbitrary user commands and there may be a large number of them in any order, then likely you need to write some sort of abstraction around do/undo.
If alternatively A, B, C, D etc are resources that need tidying up (database connections, files, etc), then context managers may be more appropriate.
Or maybe A, B, C D are some other sort of thing altogether.
A brief bit of code using context managers:
class A:
def __enter__(self):
set things up
return thing
def __exit__(self, type, value, traceback):
tear things down
class B, C and D are similar
def doSomethingThatNeedsToUseD():
try:
with A() as a:
with B() as b:
with C() as c:
with D() as d:
d.doSomething()
except:
print("error")
What you are looking for is called memento pattern. It is one of the GoF design patterns:
Without violating encapsulation, capture and externalize an object's internal state so that the object can be restored to this state later.
The one way how it could be implemented using python could be find here.
Related
I have a rather easy problem but wish to find an elegant solution that does not come up to mind. Let's say I have a function that takes some arguments and performs calculations:
def f(a, b, c):
# preprocessing of some sort
d_discarded = ... # initialized and maybe some computations as well
for i in range(1000):
d_discarded = ...
final_value_update = ...
return final_value_update
Up to a user request, I would like to iteratively store and return also the value of d_discarded, but only up to a user request, so not necessarily. How could I envision an efficient way to do so?
A naive solution would be adding if statements and an additional argument like:
def f(a, b, c, keep_d = False):
# preprocessing of some sort
d_discarded = ... # initialized and maybe some computations as well
if keep_d:
l_discarded = []
l_discarded.append(d_discarded)
for i in range(1000):
d_discarded = ...
final_value_update = ...
if keep_d:
l_discarded.append(d_discarded)
if keep_d:
return final_value_update, l_discarded
return final_value_update
But this is not efficient, nor elegant, as it calls 1002 times passing an if statement. I surely can do this, but wish to learn a more clever process.
Any consideration is appreciated. I can understand the problem is rather broad but I chose to leave it as it is because it is indeed suitable for any setting.
I have code that looks like this:
if(func_cliche_start(line)):
a=func_cliche_start(line)
#... do stuff with 'a' and line here
elif(func_test_start(line)):
a=func_test_start(line)
#... do stuff with a and line here
elif(func_macro_start(line)):
a=func_macro_start(line)
#... do stuff with a and line here
...
Each of the func_blah_start functions either return None or a string (based on the input line). I don't like the redundant call to func_blah_start as it seems like a waste (func_blah_start is "pure", so we can assume no side effects). Is there a better idiom for this type of thing, or is there a better way to do it?
Perhaps I'm wrong, (my C is rusty), but I thought that you could do something this in C:
int a;
if(a=myfunc(input)){ /*do something with a and input here*/ }
is there a python equivalent?
Why don't you assign the function func_cliche_start to variable a before the if statement?
a = func_cliche_start(line)
if a:
pass # do stuff with 'a' and line here
The if statement will fail if func_cliche_start(line) returns None.
You can create a wrapper function to make this work.
def assign(value, lst):
lst[0] = value
return value
a = [None]
if assign(func_cliche_start(line), a):
#... do stuff with 'a[0]' and line here
elif assign(func_test_start(line), a):
#...
You can just loop thru your processing functions that would be easier and less lines :), if you want to do something different in each case, wrap that in a function and call that e.g.
for func, proc in [(func_cliche_start, cliche_proc), (func_test_start, test_proc), (func_macro_start, macro_proc)]:
a = func(line)
if a:
proc(a, line)
break;
I think you should put those blocks of code in functions. That way you can use a dispatcher-style approach. If you need to modify a lot of local state, use a class and methods. (If not, just use functions; but I'll assume the class case here.) So something like this:
from itertools import dropwhile
class LineHandler(object):
def __init__(self, state):
self.state = state
def handle_cliche_start(self, line):
# modify state
def handle_test_start(self, line):
# modify state
def handle_macro_start(self, line):
# modify state
line_handler = LineHandler(initial_state)
handlers = [line_handler.handle_cliche_start,
line_handler.handle_test_start,
line_handler.handle_macro_start]
tests = [func_cliche_start,
func_test_start,
func_macro_start]
handlers_tests = zip(handlers, tests)
for line in lines:
handler_iter = ((h, t(line)) for h, t in handlers_tests)
handler_filter = ((h, l) for h, l in handler_iter if l is not None)
handler, line = next(handler_filter, (None, None))
if handler:
handler(line)
This is a bit more complex than your original code, but I think it compartmentalizes things in a much more scalable way. It does require you to maintain separate parallel lists of functions, but the payoff is that you can add as many as you want without having to write long if statements -- or calling your function twice! There are probably more sophisticated ways of organizing the above too -- this is really just a roughed-out example of what you could do. For example, you might be able to create a sorted container full of (priority, test_func, handler_func) tuples and iterate over it.
In any case, I think you should consider refactoring this long list of if/elif clauses.
You could take a list of functions, make it a generator and return the first Truey one:
functions = [func_cliche_start, func_test_start, func_macro_start]
functions_gen = (f(line) for f in functions)
a = next((x for x in functions_gen if x), None)
Still seems a little strange, but much less repetition.
The accepted paradigm to deal with mutable default arguments is:
def func(self, a = None):
if a is None:
a = <some_initialisation>
self.a = a
As I might have to do this for several arguments, I would need to write very similar 3 lines over and over again. I find this un-pythonically a lot of text to read for a very very standard thing to do when initialising class objects or functions.
Isn't there an elegant one-liner to replace those 3 lines dealing with the potentially undefined argument and the standard required copying to the class instance variables?
If a "falsy" value (0, empty string, list, dict, etc.) is not a valid value for a, then you can cut down the initialization to one line:
a = a or <initialize_object>
Another way of doing the same thing is as follows:
def func(self,**kwargs):
self.a=kwargs.get('a',<a_initialization>)
...
This has the added bonus that the value of a passed to the function could be None and the initialization won't overwrite it. The disadvantage is that a user using the builtin help function won't be able to tell what keywords your function is looking for unless you spell it out explicitly in the docstring.
EDIT
One other comment. The user could call the above function with keywords which are not pulled out of the kwargs dictionary. In some cases, this is good (if you want to pass the keywords to another function for instance). In other cases, this is not what you want. If you want to raise an error if the user provides an unknown keyword, you can do the following:
def func(self,**kwargs):
self.a=kwargs.pop('a',"Default_a")
self.b=kwargs.pop('b',"Default_b")
if(kwargs):
raise ... #some appropriate exception...possibly using kwargs.keys() to say which keywords were not appropriate for this function.
You could do this
def func(self, a=None):
self.a = <some_initialisation> if a is None else a
But why the obsession with one liners? I would usually use the 3 line version even if it gets repeated all over the place because if makes your code very easy for experienced Python programmers to read
just a little solution I came up by using an extra function, can be improved of course:
defaultargs.py:
def doInit(var, default_value,condition):
if condition:
var = default_value
return var
def func(a=None, b=None, c=None):
a = doInit(a,5,(a is None or not isinstance(a,int)))
b = doInit(b,10.0,(a is None or not isinstance(a,float)))
c = doInit(c,"whatever",(a is None or not isinstance(c, str)))
print a
print b
print c
if __name__ == "__main__":
func(10)
func(None,12341.12)
func("foo",None,"whowho")
output:
10
10.0
whatever
5
10.0
whatever
5
10.0
whowho
I like your question. :)
Edit: If you dont care about the variables type, please dont use isinstance().
for the following code
a =func()
if a != None:
b.append(a)
a can be assigned to None, is there a way to avoid the if statement and only use one line of code?
original problem is the following
import xml.etree.ElementTree as etree
r = etree.parse(f).getroot()
b = etree.Element('register',{})
a = r.find('tag_name') # a may get None if did not find it
if a != None:
b.append(a)
ok, I used all the answers and got this, personally I think it's the most complex python I have ever wrote so far, lol
NS_MAP = {
'spirit' : 'http://www.spiritconsortium.org/XMLSchema/SPIRIT/1.4',
'app' : 'http://www.app.com/SPIRIT-app'
}
mp=etree.Element('MemoryProperty', {'version':'alpha'})
mpt=etree.ElementTree(mp)
def copy_tags(tp, op, p, tn, ns='spirit'):
c = p.find('{%s}%s'%(NS_MAP[ns],tn))
if c is not None:
(op == '<-') and tp.append(c)
return c
for reg in regs:
te = etree.Element('register',{})
copy_tags(te,'<-',reg,'name')
copy_tags(te,'<-',reg,'addressOffset')
copy_tags(te,'<-',reg,'access')
(lambda e, t: copy_tags(te,'<-',t,'usageConstraints',ns='app') if t is not None else None)(te, copy_tags(te,'|',reg,'vendorExtensions'))
mp.append(te)
mpt.write('map_gen.xml')
If you can call func() beforehand, and you want to combine the test and assignment statements into a single statement, then you can do this, with an if-else expression:
b += [a] if a is not None else []
If a is not None, then this will add [a] to b -- essentially the same operation as b.append(a)
If a is None, then this will add [] to b, which will leave b unchanged.
This won't work unless b is a list, or at least supports "+=" in-place addition. If it doesn't -- perhaps it's some custom object, then you should be able to do this:
(b.append(a) if a is not None else None)
This is an expression, evaluated for its side effects, and then thrown away. If a is None, then the b.append(a) call will never be executed. In either case, the value of the expression is None, but we don't care about it, so it gets ignored.
Now, if you want to combine the func() call with this, then you'll have to do something different in order to avoid calling func twice. If you can use the "+=" syntax, then you can do it like this:
b += filter(None, [func()])
filter(None, <list>) returns the list with all false elements (None included, but also 0 and []) removed. This statement, then, will add either [func()] or [] to b.
[Edited]
Finally, for the worst case scenario: If you can't call func() more than once, and you can't use b += <list>, and you need to accept 0, "", [], etc, and only exclude None, and you need it all on one line, here's the ugliest line of code yet:
(lambda l, a: l.append(a) if a is not None else None)(b, func())
This is essentially #ekhumoro's solution, compressed into one line. It defines an anonymous function, calls it, discards the value, and then discards the function, all for the sake of the side effect.
Now, this is a single line, but it's certainly not easier to read or understand than the original code. If I were you, I'd stick with the original, or go with #ekhumoro's idea of just defining a helper function and using that.
python 3.8 walrus operator
if a := func(): b.append(a)
You asked the wrong question here. The clue is in your reply to one of the comments where you say "I have 10+ tags, if I can get 3 line to 1 line, I will save 20+ lines".
So your problem actually is not that you have 3 lines of code but that you are needlessly repeating 3 lines of code over and over. You could use a function to extract the repeated lines, but it sounds like in this case you may actually want a loop:
THE_TAGS = ('tag1', 'tag2', 'and so on')
for tag in THE_TAGS:
a = r.find(tag) # a may get None if did not find it
if a != None:
b.append(a)
Or if you need to append to different lists:
def extract_tag(r, tag_name, to):
a = r.find(tag_name) # a may get None if did not find it
if a != None:
to.append(a)
extract_tag(r, 'tag1', b)
extract_tag(r, 'tag2', c)
Short answer: Not really.
Longer answer: If you really wanted to avoid this (perhaps because you want to implement this behavior --- appending only non-None values) from several different blocks of code) then you could create a class as a proxy around the underlying b object and hide the details in its append method.
class NonNoneAppender:
def __init__(self, obj):
if not hasattr(obj, 'append') or not callable(obj.append):
raise ValueError, "Object must have append method"
self.__obj = obj
def append(self, item):
if item is not None:
return self.__obj.append(item)
def __getattr__(self, attr):
return getattr( self.__obj, attr)
... and then you could do something like:
b = NonNoneAppender(b)
However, I'm not sure this would make any sense at all for your code.
Attacking your real problem, and doing it in two lines for clarity:
temp = [r.find(tag) for tag in list_of_tags]
b.extend(x for x in temp if x is not None)
Note: Element.extend is new in Python 2.7/3.2
Presumably you're not trying to remove just a single if statement from your code...
So the obvious answer is to use a function:
import xml.etree.ElementTree as etree
def append(parent, child):
if child is not None:
parent.append(child)
r = etree.parse(f).getroot()
b = etree.Element('register',{})
append(b, r.find('tag_name'))
You can just add everything and remove Nones at the end with b = [a for a in b if b is not None]. Or, in your particular use case, you can do b.extend(r.findall('tag_name')[:1]). This may be a bit slower, however, as it will go through the whole tree, rather than stopping at the first instance.
b+=list(set([r.find('tag_name')])-set([None]))
But it's very ugly. A little cleaner, but also a line longer:
b.append(r.find('tag_name'))
b.remove(None)
Still not very neat though. If I were you I'd just keep that if statement.
I would like to run a set of methods given some data. I was wondering how I can remove or chose to run different methods to be run. I would like to groups them within a larger method so I can call it; and it will go along the lines of test case.
In code: Now these are the methods that process the data. I may sometimes want to run all three or a subset thereof to collect information on this data set.
def one(self):
pass
def two(self):
pass
def three(self):
pass
I would like to be able to call of these methods with another call so I dont have to type out run this; run this. I am looking for elegant way to run a bunch of methods through one call so I can pick and choose which gets run.
Desired result
def run_methods(self, variables):
#runs all three or subset of
I hope I have been clear in my question. I am just looking for an elegant way to do this. Like in Java with reflection.
Please and thanks.
Send the methods you want to run as a parameter:
def runmethods(self, variables, methods):
for method in methods:
method(variables)
then call something like:
self.runmethods(variables, (method1, method2))
This is the nice thing of having functions as first-class objects in Python
For the question of the OP in the comment (different parameters for the functions), a dirty solution (sorry for that):
def rest(a, b):
print a - b
def sum(a, b):
print a + b
def run(adictio):
for method, (a, b) in adictio.iteritems():
method(a, b)
mydictio = {rest:(3, 2), sum:(4, 5)}
run(mydictio)
You could use other containers to send methods together with their variables but it is nice to see a function as the key of a dictionary
if your methods/functions use different numbers of parameters you can not use
for method, (a,b) in adictio.iteritems():
because it expects the same number of parameters for all methods. In this case you can use *args:
def rest(*args):
a, b = args
print a - b
def sum(*args):
a, b, c, d, e = args
print a + b + c + d + e
def run(adictio):
for method, params in adictio.iteritems():
method(*params)
mydictio = {rest:(3, 2), sum:(4, 5, 6, 7, 8)}
run(mydictio)
If you normally do all the functions but sometimes have exceptions, then it would be useful to have them done by default, but optionally disable them like this:
def doWalkDog():
pass
def doFeedKid():
pass
def doTakeOutTrash():
pass
def doChores(walkDog=True, feedKid=True, takeOutTrash=True):
if walkDog: doWalkDog()
if feedKid: doFeedKid()
if takeOutTrash: doTakeOutTrash()
# if the kid is at grandma's...
# we still walk the dog and take out the trash
doChores(feedKid=False)
To answer the question in the comment regarding passing arbitrary values:
def runmethods(self, methods):
for method, args in methods.iteritems():
method(*args[0], **args[1])
runmethods( {methodA: ([arg1, arg2], {'kwarg1:' 'one', 'kwarg2'})},
{methodB: ([arg1], {'kwarg1:' 'one'})}
)
But at this point, it's looking like more code than it's worth!