Develop Reference

Recursively generating a list of lists in a triangular format given a height and value

I recently started looking into recursion to clean up my code and "up my game" as it were. As such, I'm trying to do things which could normally be accomplished rather simply with loops, etc., but practicing them with recursive algorithms instead.
Currently, I am attempting to generate a two-dimensional array which should theoretically resemble a sort of right-triangle in an NxN formation given some height n and the value which will get returned into the 2D-array.
As an example, say I call: my_function(3, 'a');, n = 3 and value = 'a'
My output returned should be: [['a'], ['a', 'a'], ['a', 'a', 'a']]
[['a'],
['a', 'a'],
['a', 'a', 'a']]
Wherein n determines both how many lists will be within the outermost list, as well as how many elements should successively appear within those inner-lists in ascending order.
As it stands, my code currently looks as follows:
def my_function(n, value):
base_val = [value]
if n == 0:
return [base_val]
else:
return [base_val] + [my_function(n-1, value)]
Unfortunately, using my above example n = 3 and value = 'a', this currently outputs: [['a'], [['a'], [['a'], [['a']]]]]
Now, this doesn't have to get formatted or printed the way I showed above in a literal right-triangle formation (that was just a visualization of what I want to accomplish).
I will answer any clarifying questions you need, of course!

return [base_val]
Okay, for n == 0 we get [[value]]. Solid. Er, sort of. That's the result with one row in it, right? So, our condition for the base case should be n == 1 instead.
Now, let's try the recursive case:
return [base_val] + [my_function(n-1, value)]
We had [[value]], and we want to end up with [[value], [value, value]]. Similarly, when we have [[value], [value, value]], we want to produce [[value], [value, value], [value, value, value]] from it. And so on.
The plan is that we get one row at the moment, and all the rest of the rows by recursing, yes?
Which rows will we get by recursing? Answer: the ones at the beginning, because those are the ones that still look like a triangle in isolation.
Therefore, which row do we produce locally? Answer: the one at the end.
Therefore, how do we order the results? Answer: we need to get the result from the recursive call, and add a row to the end of it.
Do we need to wrap the result of the recursive call? Answer: No. It is already a list of lists. We're just going to add one more list to the end of it.
How do we produce the last row? Answer: we need to repeat the value, n times, in a list. Well, that's easy enough.
Do we need to wrap the local row? Answer: Yes, because we want to append it as a single item to the recursive result - not concatenate all its elements.
Okay, let's re-examine the base case. Can we properly handle n == 0? Yes, and it makes perfect sense as a request, so we should handle it. What does our triangle look like with no rows in it? Well, it's still a list of rows, but it doesn't have any rows in it. So that's just []. And we can still append the first row to that, and proceed recursively. Great.
Let's put it all together:
if n == 0:
return []
else:
return my_function(n-1, value) + [[value] * n]
Looks like base_val isn't really useful any more. Oh well.
We can condense that a little further, with a ternary expression:
return [] if n == 0 else (my_function(n-1, value) + [[value] * n])

You have a couple logic errors: off-by-1 with n, growing the wrong side (critically, the non-base implementation should not use a base-sized array), growing by an array of the wrong size. A fixed version:
#!/usr/bin/env python3
def my_function(n, value):
if n <= 0:
return []
return my_function(n-1, value) + [[value]*n]
def main():
print(my_function(3, 'a'))
if __name__ == '__main__':
main()
Since you're returning mutable, you can get some more efficiency by using .append rather than +, which would make it no longer functional. Also note that the inner mutable objects don't get copied (but since the recursion is internal this doesn't really matter in this case).
It would be possible to write a tail-recursive version of this instead, by adding a parameter.
But python is a weird language for using unnecessary recursion.

The easiest way for me to think about recursive algorithms is in terms of the base case and how to build on that.
The base case (case where no recursion is necessary) is when n = 1 (or n = 0, but I'm going to ignore that case). A 1x1 "triangle" is just a 1x1 list: [[a]].
So how do we build on that? Well, if n = 2, we can assume we already have that base case value (from calling f(1)) of [[a]]. So we need to add [a, a] to that list.
We can generalize this as:
f(1) = [[a]]
f(n > 1) = f(n - 1) + [[a] * n]
, or, in Python:
def my_function(n, value):
if n == 1:
return [[value]]
else:
return my_function(n - 1, value) + [[value] * n]

While the other answers proposed another algorithm for solving your Problem, it could have been solved by correcting your solution:
Using a helper function such as:
def indent(x, lst):
new_lst = []
for val in lst:
new_lst += [x] + val
return new_lst
You can implement the return in the original function as:
return [base_val] + indent(value, [my_function(n-1, value)])
The other solutions are more elegant though so feel free to accept them.
Here is an image explaining this solution.
The red part is your current function call and the green one the previous function call.
As you can see, we also need to add the yellow part in order to complete the triangle.
These are the other solutions.
In these solutions you only need to add a new row, so that it's more elegant overall.

Python Coin Change Dynamic Programming

I am currently trying to implement dynamic programming in Python, but I don't know how to setup the backtracking portion so that it does not repeat permutations.
For example, an input would be (6, [1,5]) and the expected output should be 2 because there are 2 possible ways to arrange 1 and 5 so that their sum is equivalent to 6. Those combinations are {1,1,1,1,1,1} and {1,5} but the way my program currently works, it accounts for the combinations displayed above and the combination {5,1}. This causes the output to be 3 which is not what I wanted. So my question is "How do I prevent from repeating permutations?". My current code is shown below.
import collections as c
class DynamicProgram(object):
def __init__(self):
self.fib_memo = {}
# nested dictionary, collections.defaultdict works better than a regular nested dictionary
self.coin_change_memo = c.defaultdict(dict)
self.__dict__.update({x:k for x, k in locals().items() if x != 'self'})
def coin_change(self, n, coin_array):
# check cache
if n in self.coin_change_memo:
if len(coin_array) in self.coin_change_memo[n]:
return [n][len(coin_array)]
# base cases
if n < 0: return 0
elif n == 1 or n == 0: return 1
result = 0
i = 0
# backtracking (the backbone of how this function works)
while i <= n and i < len(coin_array):
result += self.coin_change(n-coin_array[i], coin_array)
i += 1
# append to cache
self.coin_change_memo[n][len(coin_array)] = result
# return result
return result

One of the way of avoiding permutation is to use the numbers in "non-decreasing" order. By doing so you will never add answer for [5 1] because it is not in "non-decreasing" order.And [1 5] will be added as it is in "non-decreasing" order.
So the change in your code will be if you fix to use the ith number in sorted order than you will never ever use the number which is strictly lower than this.
The code change will be as described in Suparshva's answer with initial list of numbers sorted.

Quick fix would be:
result += self.coin_change(n-coin_array[i], coin_array[i:]) # notice coin_array[i:] instead of coin_array
But you want to avoid this as each time you will be creating a new list.
Better fix would be:
Simply add a parameter lastUsedCoinIndex in the function. Then always use coins with index >= lastUsedCoinIndex from coin array. This will ensure that the solutions are distinct.
Also you will have to make changes in your memo state. You are presently storing sum n and size of array(size of array is not changing in your provided implementation unlike the quick fix I provided, so its of no use there!!) together as a state for memo. Now you will have n and lastUsedCoinIndex, together determining a memo state.
EDIT:
Your function would look like:
def coin_change(self,coin_array,n,lastUsedCoinIndex):
Here, the only variables changing will be n and lastUsedCoinIndex. So you can also modify your constructor such that it takes coin_array as input and then you will access the coin_array initialized by constructor through self.coin_array. Then the function would become simply:
def coin_change(self,n,lastUsedCoinIndex):

python list generation/saving bug

I am trying to make program that prints all the possible combinations for a to zzz. I tried to add a save state feature, and it works fine but there is this bug.
Let's say I interrupted the program when it printed something like e. When I execute the program again, it works fine until z but after z instead of printing aa it prints ba and continues from ba. This happens right after it prints zz too. it prints baa instead of aaa. How can I fix this?
Here is what I did so far:
import pickle,os,time
alphabet="abcdefghijklmnopqrstuvwxyz"
try:
if os.path.isfile("save.pickle")==True:
with open("save.pickle","rb") as f:
tryn=pickle.load(f)
for i in range(3):
a=[x for x in alphabet]
for j in range(i):
a=[x+i for x in alphabet for i in a]
b=a[tryn:]
for k in b:
print(k)
time.sleep(0.01)
tryn+=1
else:
tryn=0
for i in range(3):
a=[x for x in alphabet]
for j in range(i):
a=[x+i for x in alphabet for i in a]
for k in a:
print(k)
tryn+=1
time.sleep(0.01)
except KeyboardInterrupt:
with open("save.pickle","wb") as f:
pickle.dump(tryn,f)

If you're using python2, or python3 as the tag suggests, this exists in the standard library already. See itertools, product py2, and product py3, for a simple way to solve this problem.

for i in range(3):
a=[x for x in alphabet]
for j in range(i):
a=[x+i for x in alphabet for i in a]
b=a[tryn:]
Here's your bug. You skip the first tryn strings of every length, rather than just the first tryn strings. This would be easier to recognize in the output if it weren't for the following:
for k in b:
print(k)
time.sleep(0.01)
tryn+=1
You modify tryn, the number of things you're skipping. When you print out length-2 strings, you skip a number of them equal to the number of length-1 strings. When you print out length-3 strings, you skip a number of them equal to the number of length-2 strings. If tryn were bigger than the number of length-1 strings, you would skip even more.

your problem is almost certainly here:
a=[x for x in alphabet]
for j in range(i):
a=[x+i for x in alphabet for i in a]
Perhaps you shouldn't assign the in-loop value to a, but instead use a different name? Otherwise, you are changing what you use every time through the loop....
Edit: More detail. So, technically user2357112's answer is more correct, but I'm amending mine. The initial answer was just from a quick reading, so the other answer is close to the original intent. But, the original version is inefficient (for more reasons than not using product :), since you are generating the inner loops more than once. So let's walk through why this is a bad idea, as an educational exercise:
Initial algorithm:
for i in range(n):
assign a to alphabet
for j in range(i):
i times, we rewrite a to be all combinations of the current set against the alphabet.
Note that for this algorithm, to generate the length(n) product, we have to generate all previous products length(n-1), length(n-2), ..., length(1). But you aren't saving those.
You'd be better off doing something like this:
sum_list = alphabet[:]
#get a copy
product_list = alphabet[:]
#Are we starting at 0, or 1? In any case, skip the first, since we preloaded it
for i in range(1, n):
# Your existing list comprehension was equivalent here, and could still be used
# it MIGHT be faster to do '%s%s'%(x,y) instead of x+y... but maybe not
# with these short strings
# This comprehension takes the result of the last iteration, and makes the next iteration
product_list = [x+y for x,y in product(product_list, alphabet)]
# So product list is JUST the list for range (n) - i.e. if we are on loop 2, this
# is aaa...zzz. But you want all lengths together. So, as you go, add these
# sublists to a main list.
sum_list.extend(product_list)
Overall, you are doing a lot less work.
Couple other things:
You're using i as a loop variable, then re-using it in the loop comprehension. This is conflicting, and probably not working the way you'd expect.
If this is to learn how to write save/restore type apps... it's not a good one. Note that the restore function is re-calculating every value to be able to get back where it left off - if you could rewrite this algorithm to write more information out to the file (such as the current value of product_list) and make it more generator-like, then it will actually work more like a real-world example.

Here is how I would suggest solving this problem in Python. I didn't implement the save state feature; this sequence is not a really long one and your computer should be able to produce this sequence pretty fast, so I don't think it is worth the effort to try to make it cleanly interruptable.
import itertools as it
def seq(alphabet, length):
for c in range(1, length+1):
for p in it.product(alphabet, repeat=c):
yield ''.join(p)
alphabet="abcdefghijklmnopqrstuvwxyz"
for x in seq(alphabet, 3):
print(x)
If you really wanted to, you could make a one-liner using itertools. I think this is too hard to read and understand; I prefer the above version. But this does work and will be somewhat faster, due to the use of itertools.chain and itertools.imap() rather than a Python for loops.
import itertools as it
def seq(alphabet, length):
return it.imap(''.join, it.chain.from_iterable(it.product(alphabet, repeat=c) for c in range(1, length+1)))
alphabet="abcdefghijklmnopqrstuvwxyz"
for x in seq(alphabet, 3):
print(x)
In Python 3.x you could just use map() rather than itertools.imap().

Why doesnt this sort function work for Python

Please tell me why this sort function for Python isnt working :)
def sort(list):
if len(list)==0:
return list
elif len(list)==1:
return list
else:
for b in range(1,len(list)):
if list[b-1]>list[b]:
print (list[b-1])
hold = list[b-1]
list[b-1]=list[b]
list[b] = hold
a = [1,2,13,131,1,3,4]
print (sort(a))

It looks like you're attempting to implement a neighbor-sort algorithm. You need to repeat the loop N times. Since you only loop through the array once, you end up with the largest element being in its place (i.e., in the last index), but the rest is left unsorted.
You could debug your algorithm on your own, using pdb.
Or, you could use python's built-in sorting.

Lets take a look at you code. Sort is a built in Python function (at least I believe it is the same for both 2.7 and 3.X) So when you are making your own functions try to stay away from name that function with inbuilt functions unless you are going to override them (Which is a whole different topic.) This idea also applies to the parameter that you used. list is a type in the python language AKA you will not be able to use that variable name. Now for some work on your code after you change all the variables and etc...
When you are going through your function you only will swap is the 2 selected elements are next to each other when needed. This will not work with all list combinations. You have to be able to check that the current i that you are at is in the correct place. So if the end element is the lowest in the List then you have to have it swap all the way to the front of the list. There are many ways of sorting (ie. Quick sort, MergeSort,Bubble Sort) and this isnt the best way... :) Here is some help:
def sortThis(L):
if (len(L) == 0 or len(L) == 1):
return list
else:
for i in range(len(L)):
value = L[i]
j = i - 1
while (j >= 0) and (L[j] > value):
L[j+1] = L[j]
j -= 1
L[j+1] = value
a = [1,2,13,131,1,3,4]
sortThis(a)
print a
Take a look at this for more sorting Fun: QuickSort MergeSort

If it works, it would be the best sorting algotithm in the world (O(n)). Your algorithm only puts the greatest element at the end of the list. you have to apply recursively your function to list[:-1].
You should not use python reserved words

Python lists, dictionary optimization

I have been attending a couple of hackathons. I am beginning to understand that writing code is not enough. The code has to be optimized. That brings me to my question. Here are two questions that I faced.
def pairsum(numbers, k)
"""Write a function that returns two values in numbers whose sum is K"""
for i, j in numbers:
if i != j:
if i+j == k
return i, j
I wrote this function. And I was kind of stuck with optimization.
Next problem.
string = "ksjdkajsdkajksjdalsdjaksda"
def dedup(string):
""" write a function to remove duplicates in the variable string"""
output = []
for i in string:
if i not in output:
output.append(i)
These are two very simple programs that I wrote. But I am stuck with optimization after this. More on this, when we optimize code, how does the complexity reduce? Any pointers will help. Thanks in advance.

Knowing the most efficient Python idioms and also designing code that can reduce iterations and bail out early with an answer is a major part of optimization. Here are a few examples:
List list comprehensions and generators are usually fastest:
With a straightforward nested approach, a generator is faster than a for loop:
def pairsum(numbers, k):
"""Returns two unique values in numbers whose sum is k"""
return next((i, j) for i in numbers for j in numbers if i+j == k and i != j)
This is probably faster on average since it only goes though one iteration at most and does not check if a possible result is in numbers unless k-i != i:
def pairsum(numbers, k):
"""Returns two unique values in numbers whose sum is k"""
return next((k-i, i) for i in numbers if k-i != i and k-i in numbers)
Ouput:
>>> pairsum([1,2,3,4,5,6], 8)
(6, 2)
Note: I assumed numbers was a flat list since the doc string did not mention tuples and it makes the problem more difficult which is what I would expect in a competition.
For the second problem, if you are to create your own function as opposed to just using ''.join(set(s)) you were close:
def dedup(s):
"""Returns a string with duplicate characters removed from string s"""
output = ''
for c in s:
if c not in output:
output += c
return output
Tip: Do not use string as a name
You can also do:
def dedup(s):
for c in s:
s = c + s.replace(c, '')
return s
or a much faster recursive version:
def dedup(s, out=''):
s0, s = s[0], s.replace(s[0], '')
return dedup(s, n + s0) if s else out + s0
but not as fast as set for strings without lots of duplicates:
def dedup(s):
return ''.join(set(s))
Note: set() will not preserve the order of the remaining characters while the other approaches will preserve the order based on first occurrence.

Your first program is a little vague. I assume numbers is a list of tuples or something? Like [(1,2), (3,4), (5,6)]? If so, your program is pretty good, from a complexity standpoint - it's O(n). Perhaps you want a little more Pythonic solution? The neatest way to clean this up would be to join your conditions:
if i != j and i + j == k:
But this simply increases readability. I think it may also add an additional boolean operation, so it might not be an optimization.
I am not sure if you intended for your program to return the first pair of numbers which sum to k, but if you wanted all pairs which meet this requirement, you could write a comprehension:
def pairsum(numbers, k):
return list(((i, j) for i, j in numbers if i != j and i + j == k))
In that example, I used a generator comprehension instead of a list comprehension so as to conserve resources - generators are functions which act like iterators, meaning that they can save memory by only giving you data when you need it. This is called lazy iteration.
You can also use a filter, which is a function which returns only the elements from a set for which a predicate returns True. (That is, the elements which meet a certain requirement.)
import itertools
def pairsum(numbers, k):
return list(itertools.ifilter(lambda t: t[0] != t[1] and t[0] + t[1] == k, ((i, j) for i, j in numbers)))
But this is less readable in my opinion.
Your second program can be optimized using a set. If you recall from any discrete mathematics you may have learned in grade school or university, a set is a collection of unique elements - in other words, a set has no duplicate elements.
def dedup(mystring):
return set(mystring)
The algorithm to find the unique elements of a collection is generally going to be O(n^2) in time if it is O(1) in space - if you allow yourself to allocate more memory, you can use a Binary Search Tree to reduce the time complexity to O(n log n), which is likely how Python sets are implemented.
Your solution took O(n^2) time but also O(n) space, because you created a new list which could, if the input was already a string with only unique elements, take up the same amount of space - and, for every character in the string, you iterated over the output. That's essentially O(n^2) (although I think it's actually O(n*m), but whatever). I hope you see why this is. Read the Binary Search Tree article to see how it improves your code. I don't want to re-implement one again... freshman year was so grueling!

The key to optimization is basically to figure out a way to make the code do less work, in terms of the total number of primitive steps that needs to be performed. Code that employs control structures like nested loops quickly contributes to the number of primitive steps needed. Optimization is therefore often about replacing loops iterating over the a full list with something more clever.
I had to change the unoptimized pairsum() method sligtly to make it usable:
def pairsum(numbers, k):
"""
Write a function that returns two values in numbers whose sum is K
"""
for i in numbers:
for j in numbers:
if i != j:
if i+j == k:
return i,j
Here we see two loops, one nested inside the other. When describing the time complexity of a method like this, we often say that it is O(n²). Since when the length of the numbers array passed in grows proportional to n, then the number of primitive steps grows proportional to n². Specifically, the i+j == k conditional is evaluated exactly len(number)**2 times.
The clever thing we can do here is to presort the array at the cost of O(n log(n)) which allows us to hone in on the right answer by evaluating each element of the sorted array at most one time.
def fast_pairsum(numbers, k):
sortedints = sorted(numbers)
low = 0
high = len(numbers) - 1
i = sortedints[0]
j = sortedints[-1]
while low < high:
diff = i + j - k
if diff > 0:
# Too high, let's lower
high -= 1
j = sortedints[high]
elif diff < 0:
# Too low, let's increase.
low += 1
i = sortedints[low]
else:
# Just right
return i, j
raise Exception('No solution')
These kinds of optimization only begin to really matter when the size of the problem becomes large. On my machine the break-even point between pairsum() and fast_pairsum() is with a numbers array containing 13 integers. For smaller arrays pairsum() is faster, and for larger arrays fast_pairsum() is faster. As the size grows fast_pairsum() becomes drastically faster than the unoptimized pairsum().
The clever thing to do for dedup() is to avoid having to linearly scan through the output list to find out if you've already seen a character. This can be done by storing information about which characters you've seen in a set, which has O(log(n)) look-up cost, rather than the O(n) look-up cost of a regular list.
With the outer loop, the total cost becomes O(n log(n)) rather than O(n²).
def fast_dedup(string):
# if we didn't care about the order of the characters in the
# returned string we could simply do
# return set(string)
seen = set()
output = []
seen_add = seen.add
output_append = output.append
for i in string:
if i not in seen:
seen_add(i)
output_append(i)
return output
On my machine the break-even point between dedup() and fast_dedup() is with a string of length 30.
The fast_dedup() method also shows another simple optimization trick: Moving as much of the code out of the loop bodies as possible. Since looking up the add() and append() members in the seen and output objects takes time, it is cheaper to do it once outside the loop bodies and store references to the members in variables that is used repeatedly inside the loop bodies.

To properly optimize Python, one needs to find a good algorithm for the problem and a Python idiom close to that algorithm. Your pairsum example is a good case. First, your implementation appears wrong — numbers is most likely a sequence of numbers, not a sequence of pairs of numbers. Thus a naive implementation would look like this:
def pairsum(numbers, k)
"""Write a function that returns two values in numbers whose sum is K"""
for i in numbers:
for j in numbers:
if i != j and i + j != k:
return i, j
This will perform n^2 iterations, n being the length of numbers. For small ns this is not a problem, but once n gets into hundreds, the nested loops will become visibly slow, and once n gets into thousands, they will become unusable.
An optimization would be to recognize the difference between the inner and the outer loops: the outer loop traverses over numbers exactly once, and is unavoidable. The inner loop, however, is only used to verify that the other number (which has to be k - i) is actually present. This is a mere lookup, which can be made extremely fast by using a dict, or even better, a set:
def pairsum(numbers, k)
"""Write a function that returns two values in numbers whose sum is K"""
numset = set(numbers)
for i in numbers:
if k - i in numset:
return i, k - i
This is not only faster by a constant because we're using a built-in operation (set lookup) instead of a Python-coded loop. It actually does less work because set has a smarter algorithm of doing the lookup, it performs it in constant time.
Optimizing dedup in the analogous fashion is left as an excercise for the reader.

Your string one, order preserving is most easily and should be fairly efficient written as:
from collections import OrderedDict
new_string = ''.join(OrderedDict.fromkeys(old_string))