Develop Reference

Functional style with conditional returns in python

Suppose we have a function with the following structure:
def f():
# ...
# some computations
# ...
if something_is_wrong_with_previous_computations:
return None
# ...
# some other computations
# ...
if something_is_wrong_with_previous_computations2:
return some_variable
#...
return result
As I see it, using return statement in the middle of a function is not functional at all. If we were in some lispy language we would have the let (then computations could be written with let*) statement, which would help us deal with these situations with ease. Unfortunately, we don't have it here. What should we do?
Simulate let with creating lots of nested functions and calling them in place?
Use something like Maybe monad or another complex stuff like that?
Don't waste our time and write it imperatively?
Something else?

A return statement anywhere in a function is not functional.
In Python, you have no choice.
The Lisp code
(defun sgnum (x)
(cond
((< x 0) -1)
((zerop x) 0)
(t 1)))
turns into Python as
def sgnum(x):
if x < 0:
return -1
elif x == 0:
return 0
else:
return 1
In Lisp, we know we have deviated from functional coding when we use variable assignment, or the "program feature": an explicit progn construct, or an implicit equivalent, or any of is cousins like prog or prog1. A functional function in Lisp always has a body which is made up of a single expression (or possibly no expressions at all).
You can redefine what you mean by "functional coding" in Python. How about these
rules:
Every statement in a "functional function" must be a single statement; it cannot be followed by another statement. Thus, the whole body of a function is a single statement, and in it are embedded single statements.
No statement in the function may allow control to fall through it. Every statement must return. Thus return is not only considered "functional" but essential to achieving this goal.
A variable may be defined, but not redefined. Parallel, mutually exclusive control flows may assign the same variable different values, but no variable can be assigned more than once in the same control flow.
With these kinds of rules, you can get the program to have a control flow graph resembling that a program in the pure Lisp style: a control graph that is basically a tree of decisions with embedded calculations and variable binding, at the leaves of which are values to be returned.
Speaking of variable binding, we should probably have a fourth rule:
A statement may be preceded by a sequence of fresh variable assignments that contain no side effects. Such a sequence, together with the statement which follows it, counts as one statement.
Arguably also a fifth one:
No statement must be used which evaluates any contained expression or statement more than once.
Otherwise we permit loops, which are not functional. This is tricky because some looping constructs are relatively well behaved, like implicitly stepping a dummy variable over the elements of a list. The only way you can tell it's not functional is that a lexical closure captured in the loop will easily reveal there is only one variable being mutated and not a fresh variable being bound for each iteration.
According to these rules, sgnum is "functional": it contains just one if/elif/else statement, which does not allow control to fall through it: every branch returns:
This version of sgnum is not "functional" any more:
def sgnum(x):
if x < 0:
return -1
if x == 0:
return 0
return 1
It contains three statements in sequence. Whereas the following is "functional" even though it also consists of three statements:
def distance(x0, y0, x1, y1):
xd = x1 - x0
yd = y1 - y0
return math.sqrt(xd * xd + yd * yd)
These meet the rules. The first two statements bind fresh variables, meeting rule 3, so are permitted by 4 to precede a statement. The return statement meets rules 1 and 2. This is very similar to:
(defun distance (x0 y0 x1 y1)
(let ((xd (- x1 x0))
(yd (- y1 y0)))
(sqrt (+ (* xd xd) (* yd yd)))))
Lastly, note how our rules are at odds with the ancient programming advice of "have only one exit point in a function". That little tidbit you may find in some coding conventions is quite anti-functional. To achieve a single point of return in a nontrivial function requires imperative style control flows through multiple statements and/or variable assignments. From the functional point of view, it is a myopic, silly rule; but it makes sense in those contexts where it is recommended, because it can help improve very poorly structured imperative code.

Python: Should I avoid initialization of variables inside blocks?

Problem
I have a code like this
if condition:
a = f(x)
else:
a = g(y)
Initialization of a inside of the block looks bad for me. Can it be written better?
I cannot use ternary operator, because names of functions and/or lists of arguments are long.
Saying "long" I mean that the following expression
a = f(x) if condition else g(y)
will take more than 79 (sometimes even more than 119) symbols with real names instead of a, f, g, x, y and condition.
Usage of multiple slashes will make the code ugly and messy.
I don't want to initialize a with result of one of the functions by defaul, because both function are slow and I cannot allow such overhead
a = g(y)
if condition:
a = f(x)
I can initialize the variable with None, but is this solution pretty enough?
a = None
if condition:
a = f(x)
else:
a = g(y)
Let me explain my position: in C and C++ variables inside of a block have the block as their scope. In ES6 the let keyword was introduced — it allows to create variables with the same scoping rules as variables in C and C++. Variables defined with old var keyword have similar scoping rules as in Python.
That's why I think that initialization of variables should be made outside blocks if I want to use the variables outside these blocks.
Update
Here is more complicated example
for obj in gen:
# do something with the `obj`
if predicate(obj):
try:
result = f(obj)
except Exception as e:
log(e)
continue
else:
result = g(obj)
# do something useful with the `result`
else:
result = h(obj)
display(result)
I go through elements of some generator gen, process them and perform some actions on the result on each iteration.
Then I want to do something with the last result outside of the loop.
Is it pythonic enough to not assign a dummy value to the result beforehand?
Doesn't this make the code less readable?
Question
Is it good to initialize variables inside if/else/for/etc. in Python?

Python has no block scope... the scope is the whole function and it's perfectly pythonic to write
if <condition>:
a = f()
else:
a = g()
If you want to write in C++ then write in C++ using C++, don't write C++ using Python... it's a bad idea.

Ok, there are two points that need to be clarified here which are fundamental to python.
There is no variable declaration/initialization in python. An expression like a = f(x) is simply a scheme to name the object that is returned by f as a. That namea can be later used to name any other object no matter what its type is. See this answer.
A block in python is either the body of a module, a class or a function. Anything defined/named inside these objects are visible to later code until the end of a block. A loop or an if-else is not a block. So any name defined before outside the loop or if/else will be visible inside and vice versa. See this. global and nonlocal objects are a little different. There is no let in python since that is default behavior.
In your case the only concern is how you are using a further in the code. If you code expects the type of objects returned by f or g it should work fine unless there is an error. Because at least one of the if or the else should run in a normal operation so a will refer to some kind of an object (if the names were different in if and else that would be a problem). If you want to make sure that the subsequent code does not break you can use a try-except-else to catch any error generated by the functions and assign a default value to a in the except clause after appropriate reporting/logging of the error.
Hence to summarize and also to directly address your question, assigning names to objects inside an if-else statement or a loop is perfectly good practice provided:
The same name is used in both if and else clause so that the name is guaranteed to refer to an object at the end of the statement. Additional try-except-else error catching can take care of exceptions raised by the functions.
The names should not be too short, generic or something that does not make the intention of the code clear like a, res etc. A sensible name will lead to much better readability and prevent accidental use of the same name later for some other object thereby losing the original.

Let me clarify what I meant in my comments.
#this is not, strictly, needed, but it makes the
#exception handler more robust
a = b = None
try:
if condition:
a = f(x)
b = v(x)
else:
a = g(y)
b = v2(x)
return w(a, b)
except Exception, e:
logger.exception("exception:%s" % (e))
logger.exception(" the value of a was:%s" % (a))
logger.exception(" the value of b was:%s" % (b))
raise
This is pretty std code, you just want to wrap the whole thing in some logging code in case of exceptions. I re-raise the original exception, but could just as easily return a default value.
Problem is, unless the exception waits until return w(a, b) to happen, any access to a and b will throw its own NameError based on those variables not having been declared.
This has happened to me, a lot, with custom web unittesting code - I get a response from a get or post to an url and I run a bunch of tests against the response. If the original get/post that failed, the response doesn't exist, so any diagnostics like pretty printing that response's attributes will throw an exception, forcing you to clean things up before your exception handler is useful.
So, to guard against this, I initialize any variable referred to in the exception handler to None. Of course, if needed, you also have to guard against a being None, with something like logger("a.attr1:%s" % (getattr(a, "attr1","?")

How to reduce parameters needed in recursive function definition (for calculation of a*b)

I have a recursive function that calculates a*b however when I call it I don't want to have to give it a value: recursive_sum(5,5,0) where 0 is the saved sum.
def recursive_sum(a,b,saved):
if b == 1:
saved+=a
return saved
saved += a
b -= 1
b*recursive_sum(a,b,saved)
I understand this is a very simple example
When I add the line save=0 so I don't have to pass the pointless argument, obviously, the value is reset to 0 with each iteration.
My Question: is there a way to implement my function to work with the call recursive_sum(2,2) and not have to worry about passing 0 to saved?

This whole approach only really works for integers, but this might set you straight:
def recursive_sum(a,b):
if b == 1:
return a
return a + recursive_sum(a, b-1)
The idea here is that your return statement needs to call the next iteration of function call and then use the result it receives to calculate it's own return value. When you write a recursive function, you're really writing f(f(...f(x))). You have two components to implement:
The general (recursive) case - How can you use previous results to calculate the next result in sequence?
The base case - How do you determine that you don't need any additional recursion? In this case, you'll just return a value without calling the function again.
Try implementing a recursive factorial function. That's perhaps the easiest (except maybe a sum) recursive function to write.

Your code has two issues.
First, you've not included a default value for saved in the declaration of your function (though you mention trying to). I suspect when you tried it, you actually did something different (passing a keyword argument in the recursive call).
Second, you're not returning the result of the recursive call (you're instead multiplying the result by b, which doesn't make any sense and doesn't do anything useful, since you discard the result).
Here's a fixed version:
def recursive_sum(a,b,saved=0): # provide default value for saved here
saved+=a # minor improvement: moved this line here rather than repeating it twice below
if b == 1:
return saved
b -= 1
return recursive_sum(a,b,saved) # return result of recursive call (and don't multiply)
I suspect when you tried using a default value before, you also (or instead) put saved=0 in the recursive call at the end of the function (not only on the def line). That won't work correctly, as it passes 0 as a keyword argument, ignoring the value of saved you already have.
This version of the code is "tail recursive", since you return the result of the recursive call without doing anything else. This kind of recursion has some performance benefits in some other programming languages, but alas, not in Python. Unless you are choosing tail recursion for some specific reason (e.g. you're required use it for a homework assignment), the non-tail recursive code in RagingRoosevelt's answer is probably better than this version.

def recursive_sum(a,b):
def iterative_sum(x, saved):
if x == 0:
return saved
else:
return iterative_sum(x-1, saved+a)
return iterative_sum(b,0)

Modify a function in python without calling it

Suppose I have an arbitrary function f in Python, that takes parameters.
def f(x): return 2*x
Now suppose I want a function that takes a function and returns the same function, but flipped along the y-axis (if it were graphed).
The obvious way to do it is
def reverse_fn(f): return lambda x, funct=f: funct(-x)
However, stacking function-modifying functions like this ends up breaking max recursion depth after a while, since the result is just a function that called another function that calls more functions all the way down.
What is the best way to make function-modifying-functions in Python, that can be used over and over again without taking excessive call stack or nesting functions?

One approach is editing the bytecode of the function. This is a very advanced technique, and is also very fragile. So, don't use this for production code!
That said, there is a module out there which implements precisely the kind of editing you want. It's called bytecodehacks, first released on April 1, 2000 (yes, it was an April Fools' joke, but a completely functional one). A slightly later edition (from 2005) works fine on my install of Python 2.7.6; grab it from CVS and run setup.py as usual. (Don't use the April2000 version; it won't work on newer Pythons).
bytecodehacks basically implements a number of utility routines that make it possible to edit the bytecode of a section of code (a function, module, or even just a single block within a function). You can use it to implement macros, for example. For the purposes of modifying a function, the inline tool is probably the most useful.
Here's how you would implement reverse_fn using bytecodehacks:
from bytecodehacks.inline import inline
def reverse_fn(f):
def g(x):
# Note that we use a global name here, not `f`.
return _f(-x)
return inline(g, _f=f)
That's all! inline takes care of the dirty business of "inlining" the function f into the body of g. In effect, if f(x) was return 2*x, then the return from reverse_fn(f) would be a function equivalent to return 2*(-x) (which would not have any function calls in it).
Now, one limitation of bytecodehacks is that the variable renaming (in extend_and_rename in inline.py) is somewhat stupid. So, if you apply reverse_fn 1000 times in a row, you will get a huge slowdown as the local variable names will begin to explode in size. I'm not sure how to fix this, but if you do, it will substantially improve the performance for functions that are repeatedly inlined.

The default recursion limit of 1000 can be increased with sys.setrecursionlimit(), but even 1000 is extraordinarily deep recursion, and comes at a steep performance penalty if your wrappers tend to be this kind of trivial alteration you show in your example.
What you could do, if you're trying to build up complex functions procedurally from simple primitives, is to compose the compound functions as Python source text and pass them through eval() to get callable functions. This approach has the significant advantage that a function built up from 1000 primitives won't incur the cost of 1000 function calls and returns when executed.
Note that eval() should be used with caution; don't eval() untrusted sources.
eval() will be fairly expensive per function created, and without knowing a little more about what you're trying to do, it's hard to advise. You could also simply write a program that generates a big .py file full of the compound functions you want.

I don't think you can achieve this in any language that doesn't support Tail Call Optimization without using a trampoline. Another option is to extract the AST of the function under question and generate a "brand new" function that doesn't call the original function at all but implementing this is not trivial and requires good understanding of some of the more internal parts of Python.
A trampoline on the other hand is easy to implement but has the drawback that your functions cannot be simple Python functions anymore—every time they need to make a recursive call, they return that call as, say, a tuple in the form (some_fn, args, kwargs) (while normal return values would be wrapped in a 1-tuple), the trampoline would then make that call for you so that the stack doesn't grow.
def rec(fn, *args, **kwargs):
return (fn, args, kwargs)
def value(val):
return (val,)
def tailrec(fn, *args, **kwargs):
while True:
ret = fn(*args, **kwargs)
if ret is None:
return None
elif len(ret) == 1:
return ret[0]
else:
fn, args, kwargs = ret # no kwargs supported if using tuples
def greet_a_lot(n):
if n > 0:
print "hello: " + str(n)
return rec(greet_a_lot, n - 1)
else:
return value("done")
print tailrec(greet_a_lot, 10000)
Output:
hello: 100000
hello: 99999
...
hello: 3
hello: 2
hello: 1
done

explicitly passing functions in python

Out of curiosity is more desirable to explicitly pass functions to other functions, or let the function call functions from within. is this a case of Explicit is better than implicit?
for example (the following is only to illustrate what i mean)
def foo(x,y):
return 1 if x > y else 0
partialfun = functools.partial(foo, 1)
def bar(xs,ys):
return partialfun(sum(map(operator.mul,xs,ys)))
>>> bar([1,2,3], [4,5,6])
--or--
def foo(x,y):
return 1 if x > y else 0
partialfun = functools.partial(foo, 1)
def bar(fn,xs,ys):
return fn(sum(map(operator.mul,xs,ys)))
>>> bar(partialfun, [1,2,3], [4,5,6])

There's not really any difference between functions and anything else in this situation. You pass something as an argument if it's a parameter that might vary over different invocations of the function. If the function you are calling (bar in your example) is always calling the same other function, there's no reason to pass that as an argument. If you need to parameterize it so that you can use many different functions (i.e., bar might need to call many functions besides partialfun, and needs to know which one to call), then you need to pass it as an argument.

Generally, yes, but as always, it depends. What you are illustrating here is known as dependency injection. Generally, it is a good idea, as it allows separation of variability from the logic of a given function. This means, for example, that it will be extremely easy for you to test such code.
# To test the process performed in bar(), we can "inject" a function
# which simply returns its argument
def dummy(x):
return x
def bar(fn,xs,ys):
return fn(sum(map(operator.mul,xs,ys)))
>>> assert bar(dummy, [1,2,3], [4,5,6]) == 32

It depends very much on the context.
Basically, if the function is an argument to bar, then it's the responsibility of the caller to know how to implement that function. bar doesn't have to care. But consequently, bar's documentation has to describe what kind of function it needs.
Often this is very appropriate. The obvious example is the map builtin function. map implements the logic of applying a function to each item in a list, and giving back a list of results. map itself neither knows nor cares about what the items are, or what the function is doing to them. map's documentation has to describe that it needs a function of one argument, and each caller of map has to know how to implement or find a suitable function. But this arrangement is great; it allows you to pass a list of your custom objects, and a function which operates specifically on those objects, and map can go away and do its generic thing.
But often this arrangement is inappropriate. A function gives a name to a high level operation and hides the internal implementation details, so you can think of the operation as a unit. Allowing part of its operation to be passed in from outside as a function parameter exposes that it works in a way that uses that function's interface.
A more concrete (though somewhat contrived) example may help. Lets say I've implemented data types representing Person and Job, and I'm writing a function name_and_title for formatting someone's full name and job title into a string, for client code to insert into email signatures or on letterhead or whatever. It's obviously going to take a Person and Job. It could potentially take a function parameter to let the caller decide how to format the person's name: something like lambda firstname, lastname: lastname + ', ' + firstname. But to do this is to expose that I'm representing people's names with a separate first name and last name. If I want to change to supporting a middle name, then either name_and_title won't be able to include the middle name, or I have to change the type of the function it accepts. When I realise that some people have 4 or more names and decide to change to storing a list of names, then I definitely have to change the type of function name_and_title accepts.
So for your bar example, we can't say which is better, because it's an abstract example with no meaning. It depends on whether the call to partialfun is an implementation detail of whatever bar is supposed to be doing, or whether the call to partialfun is something that the caller knows about (and might want to do something else). If it's "part of" bar, then it shouldn't be a parameter. If it's "part of" the caller, then it should be a parameter.
It's worth noting that bar could have a huge number of function parameters. You call sum, map, and operator.mul, which could all be parameterised to make bar more flexible:
def bar(fn, xs,ys, g, h, i):
return fn(g(h(i,xs,ys))
And the way in which g is called on the output of h could be abstracted too:
def bar(fn, xs, ys, g, h, i, j):
return fn(j(g, h(i, xs, ys)))
And we can keep going on and on, until bar doesn't do anything at all, and everything is controlled by the functions passed in, and the caller might as well have just directly done what they want done rather than writing 100 functions to do it and passing those to bar to execute the functions.
So there really isn't a definite answer one way or the other that applies all the time. It depends on the particular code you're writing.