As an exercise, i implemented the map function using recursion in python as follows:
#map function that applies the function f on every element of list l and returns the new list
def map(l,f):
if l == []:
return []
else:
return [f(l[0])] + map(l[1:],f)
I am aware of the fact that python does not support tail recursion optimization, but how would i go about writing the same function in a tail recursive manner ?.
Please Help
Thank You
Tail recursion means you must be directly returning the result of a recursive call, with no further manipulation.
The obvious recursion in map is to compute the function on one element of the list, then use a recursive call to process the rest of the list. However, you need to combine the result of processing one element with the result of processing the rest of the list, which requires an operation after the recursive call.
A very common pattern for avoiding that is to move the combination inside the recursive call; you pass in the processed element as an argument, and make it part of map's responsibility to do the combining as well.
def map(l, f):
if l == []:
return []
else:
return map(l[1:], f, f(l[0]))
Now it's tail recursive! But it's also obviously wrong. In the tail recursive call, we're passing 3 arguments, but map only takes two arguments. And then there's the question of what do we do with the 3rd value. In the base case (when the list is empty), it's obvious: return a list containing the information passed in. In the recursive case, we're computing a new value, and we have this extra parameter passed in from the top, and we have the recursive call. The new value and the extra parameter need to be rolled up together to be passed into the recursive call, so that the recursive call can be tail recursive. All of which suggests the following:
def map(l, f):
return map_acc(l, f, [])
def map_acc(l, f, a):
if l == []:
return a
else:
b = a + [f(l[0])]
return map_acc(l[1:], f, b)
Which can be expressed more concisely and Pythonically as other answers have shown, without resorting to a separate helper function. But this shows a general way of turning non-tail-recursive functions into tail recursive functions.
In the above, a is called an accumulator. The general idea is to move the operations you normally do after a recursive call into the next recursive call, by wrapping up the work outer calls have done "so far" and passing that on in an accumulator.
If map can be thought of as meaning "call f on every element of l, and return a list of the results", map_acc can be thought of as meaning "call f on every element of l, returning a list of the results combined with a, a list of results already produced".
This will be an example of the implementation of the built-in function map in tail recursion:
def map(func, ls, res=None):
if res is None:
res = []
if not ls:
return res
res.append(func(ls[0]))
return map(func, ls[1:], res)
But it will not solve the problem of python not having support of TRE which mean that the call stack of each function call will be hold at all the time.
This seems to be tail recursive:
def map(l,f,x=[]):
if l == []:
return x
else:
return map(l[1:],f,x+[f(l[0])])
Or in more compact form:
def map(l,f,x=[]):
return l and map(l[1:],f,x+[f(l[0])]) or x
not really an answer, sorry, but another way to implement map is to write it in terms of a fold. if you try, you'll find that it only comes out "right" with foldr; using foldl gives you a reversed list. unfortunately, foldr isn't tail recursive, while foldl is. this suggests that there's something more "natural" about rev_map (map that returns a reversed list). unfortunately i am not well enough educated to take this any further (i suspect you might be able to generalise this to say that there is no solution that doesn't use an accumulator, but i personally don't see how to make the argument).
Related
I tried to create recursive function for generating Pascal's triangle as below.
numRows = 5
ans=[[1],[1,1]]
def pascal(arr,pre,idx):
if idx==numRows:
return ans
if len(arr)!=idx:
for i in range (0,len(pre)-1,1):
arr+=[pre[i]+pre[i+1]]
if len(arr)==idx:
arr+=[1]
ans.append(arr)
pascal([1],arr,idx+1)
a = pascal([1],ans[1],2)
return a
The output I got was an empty list [ ]. But if I add return when calling pascal as
return pascal([1],arr,idx+1)
the output was correct [[1],[1,1],[1,2,1],[1,3,3,1],[1,4,6,4,1]].
As I understand, a should have been assigned by return ans. Then why a failed to get an answer when calling pascal without return and why return is necessary in this case?
When you have recursion, you usually combine the returns in some way. Could be a sum, like fibonacci:
fibonacci(n+1) = fibonnaci(n)+fibonacci(n-1)
Or appending a line to a matrix, like your case. Anyway, if you don't have a return, you have no information to combine! Consider the fibonnaci case without return:
def fibonnaci(n):
if n<2:
return 1
fib_ans = fibonnaci(n-2)+fibonnaci(n-1)
In this case, if I called fibonnaci(0) or fibonnaci(1) the output would be 1, like you return ans if idx == numRows, but if I called fibonnaci(2), then the variable fib_ans would receive 2, which is the expected answer, but it would be available outside the scope of the funtion. Python "would add" return None to the end of my function, just below the fib_ans attribution. So, I need to return fib_ans
Well as far as i know, if you want to get a value back you need the "return" statement...
The point is if you don't have "return", you will be getting no values back...
Hopefully this helps..
For all algol languages that has a return keyword it exits the nearest function completely and the result of it is the result of the expression of the return argument. eg.
def test(v):
if v == 0:
return someFun(10)
...
If v is zero the result of the function is the value returned by someFun(10). The rest of the function denoted by ... is never executed unless v is nonzero.
If we write the same without return:
def test(v):
if v == 0:
someFun(10)
...
Now, when v is zero someFun(10) is still called, but the value returned by it is discarded and in order for it to have any true meaning someFun needs to do some side effects, like printing, storing values, updating object. In addition all the rest of the function denoted by ... is then continued once someFun(10) is done.
For Python and many other languages not having a return at all doesn't mean it doesn't return anything. In Python there is an invisible return None in the last line of every function/method.
When you do pascal([1],arr,idx+1), you are doing the recursive call but then discarding the value which it returns. If you want to return it to the caller, then you need to use explicitly return pascal(...).
In fact recursion is not necessary in this example anyway. You could easily refactor your code to use a simple for loop. For example:
def pascal(numRows):
ans = [[1]]
for _ in range(1, numRows):
pre = ans[-1]
arr = [1]
for i in range(0,len(pre)-1,1):
arr+=[pre[i]+pre[i+1]]
arr+=[1]
ans.append(arr)
return ans
print(pascal(5))
(I'm using the name _ here as the for loop variable according to convention because it is a dummy variable not used inside the loop, but you could use something else e.g. row if you prefer.)
The above is as close to the original code as possible, but you should also consider using arr.append(value) in place of arr += [value] -- this would be the normal way to append a single value to a list.
I recently studied a python recursion function and found that the recursion stops when it uses element in []. So I made a simple test function, found that there is even no print out. So how can I understand the element in []? Why does the function stop when referring to element in []?
b=1
def simple():
for i in []:
print('i am here')
return i+b
a = simple()
Python's in keyword has two purposes.
One use in as part of a for loop, which is written for element in iterable. This assigns each value from iterable to element on each pass through the loop body. This is how your example function is using in (though since the list you're looping over is empty, the loop never does anything).
The other way you can use in is as an operator. An expression like x in y tests if element x is present in container y. (There's also a negated version of the in operator, not in. The expression x not in y is exactly equivalent to not (x in y).) I suspect this is what your recursive code is doing. This would also not be useful to do with an empty list literal (since an empty list by definition doesn't contain anything), but I'm guessing the real recursive function is a bit more complicated.
As an example of both uses of in, here's a generator function that uses a set to filter out duplicate items from some other iterable. It has a for loop that has in, and it also uses in (well, technically not in) as an operator to test if the next value from the input iterator is contained in the seen set:
def unique(iterable):
seen = set()
for item in iterable: # "in" used by for loop
if item not in seen: # "in" used here as an operator
yield item
seen.add(item)
A recursive function calls itself n-number of times, then returns a terminating value on the last recursion that backs out of the recursive stacks.
Example:
compute the factorial of a number:
def fact(n):
# ex: 5 * 4 * 3 * 2 * 1
# n == 0 is your terminating recursion
if n == 0:
return 1
# else is your recursion call to fact(n-1)
else:
return n * fact(n-1)
In your example, there is no recursive call to simple() within the function, nor are there any element inside the empty list [] to step through, therefore your for loop never executed
Its concerned about mechanism of 'for loop'.
Superficially, the iterator you want to travese (which is "[]" in you example) has a length of 0, so the body of the loop (which include "print" an so on) will not be executed.
Hope it helps.
I need to Check that every number in numberList is positive and implement the below
function using recursion. I'm stuck. Just learning recursion and I'm completely lost as I am very new to programming. Help!
def isEveryNumberPositiveIn(numberList):
foundCounterexampleYet = False
for number in numberList:
if(number <= 0):
foundCounterexampleYet = True
return not(foundCounterexampleYet)
Your function is not recursive because it never calls itself; a recursive version would look like
def all_positive(lst):
if lst:
return lst[0] > 0 and all_positive(lst[1:])
# ^
# this is the recursive bit -
# the function calls itself
else:
return True
# this keeps the function from looping forever -
# when it runs out of list items, it stops calling itself
This is a bad example to choose for a recursive function because (a) there is a simple non-recursive solution and (b) passing it a large list (ie over 1000 items) will overflow the call stack and crash your program. Instead, try:
def all_positive(lst):
return all(i > 0 for i in lst)
Your indentation is incorrect, but your thinking is correct, though the algorithm is not recursive. You could make it a bit more efficient though, by jumping out of the loop when a negative number is detected:
def isEveryNumberPositiveIn(numberList):
foundCounterexampleYet = False
for number in numberList:
if number <= 0:
foundCounterexampleYet = True
break
return not foundCounterexampleYet
then for example:
a = [1,-2,3,4,45]
print(isEveryNumberPositiveIn(a))
returns False
By the way, those parentheses forif and not are unnecessary.
With this sort of recursive problem, here is how you should think about it:
There should be a "basis case", which answers the question trivially.
There should be a part that does something that brings you closer to a solution.
In this case, the "basis case" will be an empty list. If the list is empty, then return True.
The part that brings you closer to a solution: shorten the list. Once the list get shortened all the way to a zero-length (empty) list, you have reached the basis case.
In pseudocode:
define function all_positive(lst)
# basis case
if lst is zero-length:
return True
if the first item in the list is not positive:
return False
# the actual recursive call
return all_positive(lst[with_first_value_removed]
Try to convert the above pseudocode into Python code and get it working. When you are ready to peek at my answer, it's below.
def all_positive(lst):
"""
Recursive function to find out if all members of lst are positive.
Because it is recursive, it must only be used with short lists.
"""
# basis case
if len(lst) == 0:
return True
if lst[0] <= 0:
return False
# recursive call
return all_positive(lst[1:])
There's several ways you can write this. One way would be to use lst.pop() to remove one element from the list. You could combine that with the if statement and it would be kind of elegant. Then the list would already be shortened and you could just do the recursive call with the list.
if lst.pop() <= 0:
return False
return all_positive(lst)
There is one problem though: this destroys the list! Unless the caller knows that it destroys the list, and the caller makes a copy of the list, this is destructive. It's just plain dangerous. It's safer to do it the way I wrote it above, where you use "list slicing" to make a copy of the list that leaves off the first item.
Usually in a language like Python, we want the safer program, so we make copies of things rather than destructively changing them ("mutating" them, as we say).
Here's one more version of all_positive() that makes a single copy of the list and then destroys that copy as it works. It relies on a helper function; the helper is destructive. We don't expect the user to call the helper function directly so it has a name that starts with an underscore.
def _all_positive_helper(lst):
"""
Recursive function that returns True if all values in a list are positive.
Don't call this directly as it destroys its argument; call all_positive() instead.
"""
if len(lst) == 0:
return True
if lst.pop() <= 0:
return False
return _all_positive_helper(lst)
def all_positive(lst):
"""
Return True if all members of lst are positive; False otherwise.
"""
# use "list slicing" to make a copy of the list
lst_copy = lst[:]
# the copy will be destroyed by the helper but we don't care!
return _all_positive_helper(lst_copy)
It's actually possible in Python to use a default argument to implement the above all in one function.
def all_positive(lst, _lst_copy=None):
"""
Return True if all members of lst are positive; False otherwise.
"""
if _lst_copy is None:
return all_positive(lst, lst[:])
if len(_lst_copy) == 0:
return True
if _lst_copy.pop() <= 0:
return False
return all_positive(lst, _lst_copy)
Recursion doesn't really help you with this. A better use for recursion would be, for example, visiting every node in a binary tree.
Is it always possible to convert a recursion into a tail recursive one?
I am having a hard time converting the following Python function into a tail-recursive one.
def BreakWords(glob):
"""Break a string of characters, glob, into a list of words.
Args:
glob: A string of characters to be broken into words if possible.
Returns:
List of words if glob can be broken down. List can be empty if glob is ''.
None if no such break is possible.
"""
# Base case.
if len(glob) == 0:
return []
# Find a partition.
for i in xrange(1, len(glob) + 1):
left = glob[:i]
if IsWord(left):
right = glob[i:]
remaining_words = BreakWords(right)
if remaining_words is not None:
return [left] + remaining_words
return None
I'n not sure if is always the case, but most of recursive functions can be implemented as tail recursives. Besides Tail Recursion is different from Tail Recursion optimization.
Differences Tail Recursion and "Regular" ones
There are two elements that must be present in a recursive function:
The recursive call
A place to keep count of the return values.
A "regular" recursive function keeps (2) in the stack frame.
The return values in regular recursive function are composed of two types of values:
Other return values
Result of the owns function computation
Let's see a example:
def factorial(n):
if n == 1 return 1
return n * factorial(n-1)
The frame f(5) "stores" the result of it's own computation (5) and the value of f(4), for example. If i call factorial(5), just before the stack calls begin to colapse, i have:
[Stack_f(5): return 5 * [Stack_f(4): 4 * [Stack_f(3): 3 * ... [1[1]]
Notice that each stack stores, besides the values i mentioned, the whole scope of the function. So, the memory usage for a recursive function f is O(x), where x is the number of recursive calls i have to made. So, if i needb 1kb of RAM to calculate factorial(1) or factorial(2), i need ~100k to calculate factorial(100), and so on.
A Tail Recursive function put (2) in it's arguments.
In a Tail Recursion, i pass the result of the partial calculations in each recursive frame to the next one using parameters. Let's see our factorial example, Tail Recursive:
def factorial(n):
def tail_helper(n, acc):
if n == 1 or n == 2: return acc
return tail_helper(n-1, acc + n)
return tail_helper(n,0)
Let's look at it's frames in factorial(4):
[Stack f(4, 5): Stack f(3, 20): [Stack f(2,60): [Stack f(1, 120): 120]]]]
See the differences? In "regular" recursive calls the return functions recursively compose the final value. In Tail Recursion they only reference the base case (last one evaluated). We call accumulator the argument that keeps track of the older values.
Recursion Templates
The regular recursive function go as follows:
def regular(n)
base_case
computation
return (result of computation) combined with (regular(n towards base case))
To transform it in a Tail recursion we:
Introduce a helper function that carries the accumulator
run the helper function inside the main function, with the accumulator set to the base case.
Look:
def tail(n):
def helper(n, accumulator):
if n == base case:
return accumulator
computation
accumulator = computation combined with accumulator
return helper(n towards base case, accumulator)
helper(n, base case)
Your example:
I did something like this:
def BreakWords(glob):
def helper(word, glob, acc_1, acc_2):
if len(word) == 0 and len(glob) == 0:
if not acc_1:
return None
return acc
if len(word) == 0:
word = glob.pop[0]
acc_2 = 0
if IsWord(word.substring[:acc_2]):
acc_1.append(word[:acc_2])
return helper(word[acc_2 + 1:], glob, acc_1, acc_2 + 1)
return helper(word[acc_2 + 1:], glob, acc_1, acc_2 + 1)
return helper("", glob, [], 0)
In order to eliminate the for statement you made, i did my recursive helper function with 2 accumulators. One to store the results, and one to store the position i'm currently trying.
Tail Call optimization
Since no state is being stored on the Non-Border-Cases of the Tail Call stacks, they aren't so important. Some languages/interpreters then substitute the old stack with the new one. So, with no stack frames constraining the number of calls, the Tail Calls behave just like a for-loop.
But unfortunately for you Python isn't one of these cases. You'll get a RunTimeError when the stack gets bigger than 1000. Mr. Guido
thinks that the clarity lost to debugging purposes due to Tail Call Optimization (caused by the frames thrown awy) is more important than the feature. That's a shame. Python has so many cool functional stuff, and tail recursion would be great on top of it :/
I have a structure with an x amount of lists in lists, and each list an x amount of tuples. I don't know beforehand how many nested lists there are, or how many tuples in each list.
I want to make dictionaries out of all the tuples and because I don't know the depth of the lists I want to use recursion. What I did was
def tupleToDict(listOfList, dictList):
itemDict = getItems(list) # a function that makes a dictionary out of all the tuples in list
dictList.append(itemDict)
for nestedList in listOfList:
getAllNestedItems(nestedList, dictList)
return dictList
this works, but I end up with a huge list at the end. I would rather return the itemDict at every round of recursion. However, I don't know how to (if it is possible) return a value without stopping the recursion.
You're looking for yield:
def tupleToDict(listOfList):
yield getItems(listofList)
for nestedList in listOfList:
for el in getAllNestedItems(nestedList):
yield el
In Python 3.3+, you can replace the last two lines with a yield from.
You may want to rewrite your function to be iterative:
def tupleToDict(listOfList):
q = [listOfList]
while q:
l = q.pop()
yield getItems(l)
for nestedList in listOfList:
q += getAllNestedItems(nestedList)
Who are you going to return it to? I mean if your thread is busy running the recursive algorithm, who gets the "interim results" to process?
Best bet is to tweak your algorithm to include some processing before it recurses again.
I'm not sure what you're trying to do, but you could try to make a recursive generator by using the yield statement to return the dict at the desired intervals. Either that or shove copies of it into a global list?
You got two possible solutions:
The generator approach: a function with a yield statement, which may be a hassle to implement in a recursive function. (Look at phihags proposal for an example)
The callback approach: You call a helper-function/method from inside the recursion and can monitor the progress through a second outer function.
Here a non-recursive recursion example: ;-)
def callback(data):
print "from the depths of recursion: {0}".format(data)
def recursion(arg, callbackfunc):
arg += 1
callbackfunc(arg)
if arg <10:
recursion(arg, callbackfunc)
return arg
print recursion(1, callback)