Related
I came across the following problem.
You are given the root of a binary tree with n nodes.
Each node is uniquely assigned a value from 1 to n.
You are also given an integer startValue representing
the value of the start node s,
and a different integer destValue representing
the value of the destination node t.
Find the shortest path starting from node s and ending at node t.
Generate step-by-step directions of such path as a string consisting of only the
uppercase letters 'L', 'R', and 'U'. Each letter indicates a specific direction:
'L' means to go from a node to its left child node.
'R' means to go from a node to its right child node.
'U' means to go from a node to its parent node.
Return the step-by-step directions of the shortest path from node s to node t
Example 1:
Input: root = [5,1,2,3,null,6,4], startValue = 3, destValue = 6
Output: "UURL"
Explanation: The shortest path is: 3 → 1 → 5 → 2 → 6.
Example 2:
Input: root = [2,1], startValue = 2, destValue = 1
Output: "L"
Explanation: The shortest path is: 2 → 1.
I created the solution by finding the least common ancestor and then doing a depth-first-search to find the elements, Like this:-
# Definition for a binary tree node.
# class TreeNode(object):
# def __init__(self, val=0, left=None, right=None):
# self.val = val
# self.left = left
# self.right = right
class Solution(object):
def getDirections(self, root, startValue, destValue):
"""
:type root: Optional[TreeNode]
:type startValue: int
:type destValue: int
:rtype: str
"""
def lca(root):
if root == None or root.val == startValue or root.val == destValue:
return root
left = lca(root.left)
right = lca(root.right)
if left and right:
return root
return left or right
def dfs(root, value, path):
if root == None:
return ""
if root.val == value:
return path
return dfs(root.left, value, path + "L") + dfs(root.right, value, path + "R")
root = lca(root)
return "U"*len(dfs(root, startValue, "")) + dfs(root, destValue, "")
The solution runs good, however for a very large input it throws "Memory Limit Exceeded" error, can anyone tell me how I can optimise the solution, or what might I be doing that could be getting me into it ?
The reason you're getting a memory limit exceeded is the arguments to the dfs function. Your 'path' variable is a string that can be as large as the height of the tree (which can be the size of the whole tree if it's unbalanced).
Normally that wouldn't be a problem, but path + "L" creates a new string for every recursive call of the function. Besides being very slow, this means that your memory usage is O(n^2), where n is the number of nodes in the tree.
For example, if your final path is "L" * 1000, your call stack for dfs will look like this:
Depth 0: dfs(root, path = "")
Depth 1: dfs(root.left, path = "L")
Depth 2: dfs(root.left.left, path = "LL")
...
Depth 999: path = "L"*999
Depth 1000: path = "L"*1000
Despite all those variables being called path, they are all completely different strings, for a total memory usage of ~(1000*1000)/2 = 500,000 characters at one time. With one million nodes, this is half a trillion characters.
Now, this doesn't happen just because strings are immutable; in fact, even if you were using lists (which are mutable), you'd still have this problem, as path + ["L"] would still be forced to create a copy of path.
To solve this, you need to have exactly one variable for the path stored outside of the dfs function, and only append to it from the recursive dfs function. This will ensure you only ever use O(n) space.
def dfs(root, value, path):
if root is None:
return False
if root.val == value:
return True
if dfs(root.left, value, path):
path.append("L")
return True
elif dfs(root.right, value, path):
path.append("R")
return True
return False
root = lca(root)
start_to_root = []
dfs(root, startValue, start_to_root)
dest_to_root = []
dfs(root, destValue, dest_to_root)
return "U" * len(start_to_root) + ''.join(reversed(dest_to_root))
Most of the questions I've searched for regarding binary trees shows the implementation of binary search trees, but not binary trees. The terms of a complete binary tree are:
Either an empty tree or it has 1 node with 2 children, where each
child is another Binary Tree.
All levels are full (except for possibly the last level)
All leaves on the bottom-most level are
as far left as possible.
I've come up with a concept but it doesn't seem to running through the recursion properly -- Does anyone know what I'm doing wrong?
class Node():
def __init__(self, key):
self.key = key
self.left = None
self.right = None
def add(self, key):
if self.key:
if self.left is None:
self.left = Node(key)
else:
self.left.add(key)
if self.right is None:
self.right = Node(key)
else:
self.right.add(key)
else:
self.key = key
return (self.key)
The problem in your code is that you are adding the same value multiple times. You add the node, and then still recurse deeper, where you do the same.
The deeper problem is that you don't really know where to insert the node before you have reached the bottom level of the tree, and have detected where that level is incomplete. Finding the correct insertion point may need a traversal through the whole tree... which is defeating the speed gain you would expect to get from using binary trees in the first place.
I provide here three solutions, starting with the most efficient:
1. Using a list as tree implementation
For complete trees there is a special consideration to make: if you number the nodes by level, starting with 0 for the root, and within each level from left to right, you notice that the number of a node's parent is (k-1)/2 when its own number is k. In the other direction: if a node with number k has children, then its left child has number k*2+1, and the right child has a number that is one greater.
Because the tree is complete, there will never be gaps in this numbering, and so you could store the nodes in a list, and use the indexes of that list for the node numbering. Adding a node to the tree now simply means you append it to that list. Instead of a Node object, you just have the tree list, and the index in that list is your node reference.
Here is an implementation:
class CompleteTree(list):
def add(self, key):
self.append(key)
return len(self) - 1
def left(self, i):
return i * 2 + 1 if i * 2 + 1 < len(self) else -1
def right(self, i):
return i * 2 + 2 if i * 2 + 2 < len(self) else -1
#staticmethod
def parent(i):
return (i - 1) // 2
def swapwithparent(self, i):
if i > 0:
p = self.parent(i)
self[p], self[i] = self[i], self[p]
def inorder(self, i=0):
left = self.left(i)
right = self.right(i)
if left >= 0:
yield from self.inorder(left)
yield i
if right >= 0:
yield from self.inorder(right)
#staticmethod
def depth(i):
return (i + 1).bit_length() - 1
Here is a demo that creates your example tree, and then prints the keys visited in an in-order traversal, indented by their depth in the tree:
tree = CompleteTree()
tree.add(1)
tree.add(2)
tree.add(3)
tree.add(4)
tree.add(5)
for node in tree.inorder():
print(" " * tree.depth(node), tree[node])
Of course, this means you have to reference nodes a bit different from when you would use a real Node class, but the efficiency gain pays off.
2. Using an extra property
If you know how many nodes there are in a (sub)tree, then from the bit representation of that number, you can know where exactly the next node should be added.
For instance, in your example tree you have 5 nodes. Imagine you want to add a 6 to that tree. The root node would tell you that you currently have 5 and so you need to update it to 6. In binary that is 110. Ignoring the left-most 1-bit, the rest of the bits tell you whether to go left or right. In this case, you should go right (1) and then finally left (0), creating the node in that direction. You can do this iteratively or recursively.
Here is an implementation with recursion:
class Node():
def __init__(self, key):
self.key = key
self.left = None
self.right = None
self.count = 1
def add(self, key):
self.count += 1
if self.left is None:
self.left = Node(key)
elif self.right is None:
self.right = Node(key)
# extract from the count the second-most significant bit:
elif self.count & (1 << (self.count.bit_length() - 2)):
self.right.add(key)
else:
self.left.add(key)
def inorder(self):
if self.left:
yield from self.left.inorder()
yield self
if self.right:
yield from self.right.inorder()
tree = Node(1)
tree.add(2)
tree.add(3)
tree.add(4)
tree.add(5)
for node in tree.inorder():
print(node.key)
3. Without extra property
If no property can be added to Node objects, then a more extensive search is needed to find the right insertion point:
class Node():
def __init__(self, key):
self.key = key
self.left = None
self.right = None
def newparent(self):
# Finds the node that should serve as parent for a new node
# It returns a tuple:
# if parent found: [-1, parent for new node]
# if not found: [height, left-most leaf]
# In the latter case, the subtree is perfect, and its left-most
# leaf is the node to be used, unless self is a right child
# and its sibling has the insertion point.
if self.right:
right = self.right.newparent()
if right[0] == -1: # found inbalance
return right
left = self.left.newparent()
if left[0] == -1: # found inbalance
return left
if left[0] != right[0]:
return [-1, right[1]] # found inbalance
# temporary result in perfect subtree
return [left[0]+1, left[1]]
elif self.left:
return [-1, self] # found inbalance
# temporary result for leaf
return [0, self]
def add(self, key):
_, parent = self.newparent()
if not parent.left:
parent.left = Node(key)
else:
parent.right = Node(key)
def __repr__(self):
s = ""
if self.left:
s += str(self.left).replace("\n", "\n ")
s += "\n" + str(self.key)
if self.right:
s += str(self.right).replace("\n", "\n ")
return s
tree = Node(1)
tree.add(2)
tree.add(3)
tree.add(4)
tree.add(5)
print(tree)
This searches recursively the tree from right to left, to find the candidate parent of the node to be added.
For large trees, this can be improved a bit, by doing a binary-search among paths from root to leaf, based on the length of those paths. But it will still not be as efficient as the first two solutions.
You can use the sklearn Decision trees, as they are able to be set up as binary decision trees as well. link to the documentation here.
You really need to augment your tree in some way. Since this is not a binary search tree, the only real information you have about each node is whether or not it has a left and right child. Unfortunately, this isn't helpful in navigating a complete binary tree. Imagine a complete binary tree with 10 levels. Until the 9th level, every single node has both a left child and a right child, so you have no way of knowing which path to take down to the leaves. So the question is, what information do you add to each node? I would add the count of nodes in that tree.
Maintaining the count is easy, since every time you descend down a subtree you know to add one to the count at that node. What you want to recognize is the leftmost imperfect subtree. Every perfect binary tree has n = 2^k - 1, where k is the number of levels and n is the number of nodes. There are quick and easy ways to check if a number is 1 less than a power of two (see the first answer to this question), and in fact in a complete binary tree every node has at most one child that isn't the root of a perfect binary tree. Follow a simple rule to add nodes:
If the left child is None, set root.left = Node(key) and return
Else if the right child is None, set root.right = Node(key) and return
If one of the children of the current node is the root of an imperfect subtree, make that node the current node (descend down that subtree)
Else if the sizes are unequal, make the node with the smaller subtree the current node.
Else, make the left child the current node.
By augmenting each node with the size of the subtree rooted there, you have all the information you need at every node to build a recursive solution.
So far I have done this. I am stuck on recursion. I have no idea how to move forward, joining and reversing etc.
def callrecursion(s):
a=s.index('(')
z=len(s) - string[::-1].index(')') -1
newStr=s[a+1:z]
# Something is missing here i cant figure it out
print(newStr)
return newStr
def reverseParentheses(s):
if '(' in s:
return reverseParentheses(callrecursion(s))
print('wabba labba dub dub')
else:
return s
string='a(bcdefghijkl(mno)p)q'
reverseParentheses(string)
EXPECTED OUTPUT : "apmnolkjihgfedcbq"
def reverseParentheses(s):
if '(' in s:
posopen=s.find('(')
s=s[:posopen]+reverseParentheses(s[posopen+1:])
posclose=s.find(')',posopen+1)
s=s[:posopen]+s[posopen:posclose][::-1]+s[posclose+1:]
return s
string='a(bcdefghijkl(mno)p)q'
print(string)
print(reverseParentheses(string))
print('apmnolkjihgfedcbq') # your test
string='a(bc)(ef)g'
print(string)
print(reverseParentheses(string))
The idea is to go 'inward' as long as possible (where 'inward' does not even mean 'nesting', it goes as long as there are any opening parentheses), so the innermost pairs are flipped first, and then the rest as the recursion returns. This way 'parallel' parentheses seem to work too, while simple pairing of "first opening parentheses" with "last closing ones" do not handle them well. Or at least that is what I think.
Btw: recursion is just a convoluted replacement for rfind here:
def reverseParentheses(s):
while '(' in s:
posopen=s.rfind('(')
posclose=s.find(')',posopen+1)
s=s[:posopen]+s[posopen+1:posclose][::-1]+s[posclose+1:]
return s;
(... TBH: now I tried, and the recursive magic dies on empty parentheses () placed in the string, while this one works)
I've come up with tho following logic (assuming the parentheses are properly nested).
The base case is the absence of parentheses in s, so it is returned unchanged.
Otherwise we locate indices of leftmost and rightmost opening and closing parentheses
(taking care of possible string reversal, so ')' might appear opening and '(' -- as closing).
Having obtained beg and end the remaining job is quite simple: one has to pass the reversed substring contained between beg and end to the subsequent recursive call.
def reverseParentheses(s):
if s.find('(') == -1:
return s
if s.find('(') < s.find(')'):
beg, end = s.find('('), s.rfind(')')
else:
beg, end = s.find(')'), s.rfind('(')
return s[:beg] + reverseParentheses(s[beg + 1:end][::-1]) + s[end + 1:]
Assuming that number of opening and closing brackets always match, this might be the one of the simplest method to reverse words in parenthesis:
def reverse_parentheses(st: str) -> str:
while True:
split1 = st.split('(')
split2 = split1[-1].split(')')[0]
st = st.replace(f'({split2})', f'{split2[::-1]}')
if '(' not in st and ')' not in st:
return st
# s = "(abcd)"
# s = "(ed(et)el)"
# s = "(ed(et(oc))el)"
# s = "(u(love)i)"
s= "((ng)ipm(ca))"
reversed = reverse_parentheses(s)
print(reversed)
You have a few issues in your code, and much of the logic missing. This adapts your code and produces the desired output:
def callrecursion(s):
a=s.index('(')
# 's' not 'string'
z=len(s) - s[::-1].index(')') -1
newStr=s[a+1:z][::-1]
# Need to consider swapped parentheses
newStr=newStr.replace('(', "$") # Placeholder for other swap
newStr=newStr.replace(')', "(")
newStr=newStr.replace('$', ")")
#Need to recombine initial and trailing portions of original string
newStr = s[:a] + newStr + s[z+1:]
return newStr
def reverseParentheses(s):
if '(' in s:
return reverseParentheses(callrecursion(s))
print('wabba labba dub dub')
else:
return s
string='a(bcdefghijkl(mno)p)q'
print(reverseParentheses(string))
>>>apmnolkjihgfedcbq
While the existing O(n^2) solutions were sufficient here, this problem is solvable in O(n) time, and the solution is pretty fun.
The idea is to build a k-ary tree to represent our string, and traverse it with DFS. Each 'level' of the tree represents one layer of nested parentheses. There is one node for each set of parentheses, and one node for each letter, so there are only O(n) nodes in the tree.
For example, the tree-nodes at the top level are either:
A letter that is not contained in parentheses
A tree-node representing a pair of parentheses at the outermost layer of our string, which may have child tree-nodes
To get the effect of reversals, we can traverse the tree in a depth-first way recursively. Besides knowing our current node, we just need to know if we're in 'reverse mode': a boolean to tell us whether to visit our node's children from left to right, or right to left.
Every time we go down a level in our tree, whether we're in 'reverse mode' or not is flipped.
Python code:
class TreeNode:
def __init__(self, parent=None):
self.parent = parent
self.children = []
def reverseParentheses(s: str) -> str:
root_node = TreeNode()
curr_node = root_node
# Build the tree
for let in s:
# Go down a level-- new child
if let == '(':
new_child = TreeNode(parent=curr_node)
curr_node.children.append(new_child)
curr_node = new_child
# Go back to our parent
elif let == ')':
curr_node = curr_node.parent
else:
curr_node.children.append(let)
answer = []
def dfs(node, is_reversed: bool):
nonlocal answer
num_children = len(node.children)
if is_reversed:
range_start, range_end, range_step = num_children-1, -1, -1
else:
range_start, range_end, range_step = 0, num_children, 1
for i in range(range_start, range_end, range_step):
if isinstance(node.children[i], str):
answer.append(node.children[i])
else:
dfs(node.children[i], not is_reversed)
dfs(root_node, False)
return ''.join(answer)
Here is the correct version for your callrecursion function:
def callrecursion(text):
print(text)
a = text.find('(') + 1
z = text.rfind(')') + 1
newStr = text[:a - 1] + text[a:z-1][::-1].replace('(', ']').replace(')', '[').replace(']', ')').replace('[', '(') + text[z:]
return newStr
You probably have to take into account if the parethesis is the first/last character.
I'm working on a project to implement infix-to-postfix transformations in python.
The implementation of the code works as long as the expression is fully bracketed. It cannot handle expressions where humans would implicitly assume an order to the calculation.
For instance, I can use a fully-bracketed expression such as:
((3+15)*2)+(6-3)
And get the right result.
However, humans might normally write:
(3+15)*2+(6-3)
Where the first outer bracket is assumed.
Are there any algorithms that could correctly add brackets. If not, is there a best-practice solution for how to handle this sort of problem?
Update:
Here is the implementation of the parse tree function:
class BinaryTree:
def __init__(self, root):
self.key = root
self.left_child = None
self.right_child = None
def insert_left(self, new_node):
if self.left_child == None:
self.left_child = BinaryTree(new_node)
else:
t = BinaryTree(new_node)
t.left_child = self.left_child
self.left_child = t
def insert_right(self, new_node):
if self.right_child == None:
self.right_child = BinaryTree(new_node)
else:
t = BinaryTree(new_node)
t.right_child = self.right_child
self.right_child = t
def get_right_child(self):
return self.right_child
def get_left_child(self):
return self.left_child
def set_root_val(self, obj):
self.key = obj
def get_root_val(self):
return self.key
def build_parse_tree(fp_exp):
fp_list = re.findall('[+-/*//()]|\d+', fp_exp)
p_stack = Stack()
e_tree = BinaryTree('')
p_stack.push(e_tree)
current_tree = e_tree
for i in fp_list:
if i == '(':
current_tree.insert_left('')
p_stack.push(current_tree)
current_tree = current_tree.get_left_child()
elif i not in ['+', '-', '*', '/', ')']:
current_tree.set_root_val(int(i))
parent = p_stack.pop()
current_tree = parent
elif i in ['+', '-', '*', '/']:
current_tree.set_root_val(i)
current_tree.insert_right('')
p_stack.push(current_tree)
current_tree = current_tree.get_right_child()
elif i == ')':
current_tree = p_stack.pop()
else:
raise ValueError
return e_tree
def postorder(tree):
if tree != None:
postorder(tree.get_left_child())
postorder(tree.get_right_child())
print (tree.get_root_val())
The output from the second expression postorder:
3
6
15
2
3
-
+
The one with the first (correct) is:
3
15
+
6
2
3
-
+
Disclaimer: Sorry for that huge response; I was curious and just wrote down what I did during testing around a bit for your question. Find the entire code here: https://gist.github.com/jbndlr/3657fa890539d29c9e4b0311dc60835d
By the way, this is just test code and not meant to be used in production, as it may still be flawed.
Response
Your sequence parsing and tree setup with pushes of empty strings seems a bit odd, but I cannot accurately point to your error. Your parsing somehow swallows the * operator, probably because its left element is a closing bracket.
While I was playing around with this a bit, I tried to reproduce and came up with a solution that correctly parses simple equations and can generate the required parentheses. Even though no longer required, if the tree is already parsed correctly, you can use this to generate fully bracketed equations, or extend it by your own needs.
Preparation: The Imports
from __future__ import print_function
import re
Step 1: Tokenizing the Input
This function takes a string as an expression and generates a list of tuples representing your tokens. It also already classifies them as kind of simple (string-represented) types for later processing.
def tokenize(expression):
'''Generate tokens from a string following fixed rules.
'''
scanner = re.Scanner([
(r'[0-9]\.[0-9]+', lambda _, t: ('FLOAT', t)),
(r'[0-9]+', lambda _, t: ('INTEGER', t)),
(r'[a-z_]+', lambda _, t: ('IDENTIFIER', t)),
(r'\(', lambda _, t: ('P_OPEN', t)),
(r'\)', lambda _, t: ('P_CLOSE', t)),
(r'[+\-*/]', lambda _, t: ('OPERATOR', t)),
(r'\s+', None),
])
tokens, _ = scanner.scan(expression)
return tokens
This approach is by far not complete, but it is sufficient for demonstrating building binary parse trees. Note that the order of rules is important; it makes no difference here, as I do not catch single dots, but putting INTEGER before FLOAT could mess things up later.
Step 2: Parse the Hierarchy
The next function takes a list of tokens as generated in Step 1 and resolves all parts that are put into brackets as individual sub-lists. The result is a nested list where each previously bracketed part is shifted to a deeper level.
def parse(tokens, in_parens=False):
'''Parse a list of tokens that may contain brackets into a token hierarchy
where all brackets are removed and replaced by list nesting.
'''
cur = []
i = 0
while i < len(tokens):
t = tokens[i]
if t[0] == 'P_OPEN':
# If we hit an opening bracket, we memorize its position and search
# for the corresponding closing one by counting the stacked
# brackets.
pos_open = i
pos_close = None
par_stack = 0
for j, p in enumerate(tokens[i:]):
if p[0] == 'P_OPEN':
# Deeper nesting, increase.
par_stack += 1
elif p[0] == 'P_CLOSE':
# Level closed, decrease.
par_stack -= 1
if par_stack == 0:
# If we hit level 0, we found the corresponding closing
# bracket for the opening one.
pos_close = i + j
break
if pos_close is None:
# If we did not find a corresponding closing bracket, there
# must be some syntax error.
raise Exception('Syntax error; missing closing bracket.')
# For the bracketed subset we just found, we invoke a recursive
# parsing for its contents and append the result to our hierarchy.
elem = parse(tokens[i + 1:j], in_parens=True)
cur.append(elem)
i = j
elif t[0] == 'P_CLOSE':
if not in_parens:
# If we hit a closing bracket but are not searching for one, we
# found too many closing brackets, which is a syntax error.
raise Exception('Syntax error; too many closing brackets.')
return cur
else:
cur.append(t)
i += 1
return cur
This makes sure that we do not miss the explicit grouping given by parentheses in the expression. At the same time, as we count parenthesis levels, we can spot syntax errors that result from wrong bracket counts.
Step 3: Build a Tree
In order to proceed, we need to build an actual binary tree from our hierarchy. The hierarchy we got from Step 2 still has all un-bracketed chained operators on the same level, so we do not know yet about the order in which the operators need to be executed. This is what is solved now.
When creating a new Node from a hierarchy (i.e. a nested list of tokens), we search for a pivot element that we can use as the operator of the currently built Node. We choose the weakest binding operator, because we build the tree top-down, but it will be evaluated bottom-up. Hence, the operation that shall be performed last is the one we want to have in the upmost Node of our tree.
class Node(object):
def __init__(self, hierarchy, parent=None):
if len(hierarchy) == 1 and type(hierarchy[0]) is list:
hierarchy = hierarchy[0] # Bracketed descent
# Find position of operator that has the weakest binding priority and
# use it as pivot element to split the sequence at. The weakest binding
# is executed last, so it's the topmost node in the tree (which is
# evaluated bottom-up).
pivot = self._weakest_binding_position(hierarchy)
if pivot is not None:
self.left = Node(hierarchy[:pivot], parent=self)
self.op = hierarchy[pivot][1]
self.right = Node(hierarchy[pivot + 1:], parent=self)
else:
# There is no pivot element if there is no operator in our
# hierarchy. If so, we hit an atomic item and this node will be
# a leaf node.
self.value = hierarchy[0]
def _binding_order(self, operator):
'''Resolve operator to its binding order.'''
if operator in '+-':
return 1
elif operator in '*/':
return 2
raise Exception('Parsing error; operator binding cannot be assessed.')
def _weakest_binding_position(self, tokens):
'''Return position of operator with weakest binding (maintains LTR).'''
ops = sorted([
(i, self._binding_order(t[1]))
for i, t in enumerate(tokens)
if t[0] == 'OPERATOR'
], key=lambda e: e[1], reverse=True)
if len(ops) == 0:
if len(tokens) != 1:
raise Exception('Parsing error; found sequence w/o operator.')
return None
return ops[-1][0]
def isleaf(self):
if hasattr(self, 'value'):
return True
return False
def __str__(self):
if self.isleaf():
return str(self.value[1])
else:
return '({:s} {:s} {:s})'.format(self.left, self.op, self.right)
If you want to see how the tree is set up, just print(self) at the end of Node.__init__(). This will give you a bottom-up print of all nodes.
I added some parentheses in the Node.__str__() method to actually make a fully-bracketed expression from the input. You can verify with some samples like so:
if __name__ == '__main__':
expressions = [
'(3+15)*2+6-3',
'(a+15)*2+6/3'
]
for expr in expressions:
root = Node(parse(tokenize(expr)))
print(root)
... yields
>>> ((((3 + 15) * 2) + 6) - 3)
>>> (((a + 15) * 2) + (6 / 3))
So, if you want to print (or return) this in postfix notation now, you can just switch the operator and operands by changing this row in the Node.__str__() method:
<<<<<<<<
return '({:s} {:s} {:s})'.format(self.left, self.op, self.right)
======
return '({:s} {:s} {:s})'.format(self.left, self.right, self.op)
>>>>>>>>
If you want your postfix notation to be returned for further processing instead of just obtaining it as a string, just write another method like so (warning: pseudo-code):
def postfix(self):
if self.isleaf():
return self.value
else:
return (self.left.postfix(), self.right.postfix(), self.op)
and then invoke it from your root node:
pf = root.postfix()
Step 4: Evaluation
Finally, you can put a method into the Node class to evaluate the expression. This method checks whether or not we have a leaf node, and, if so, returns its value using the correct type. Otherwise, it evaluates its left and right child and applies the desired operator and then passes the result upwards the tree.
def eval(self, variables={}):
if self.isleaf():
ttype, value = self.value
if ttype == 'FLOAT':
return float(value)
elif ttype == 'INTEGER':
return int(value)
elif ttype == 'IDENTIFIER':
if value in variables.keys():
return variables[value]
else:
raise Exception('Unbound variable: {:s}'.format(value))
else:
raise Exception('Unknown type: {:s}'.format(ttype))
else:
left = self.left.eval(variables=variables)
right = self.right.eval(variables=variables)
if self.op == '+':
return left + right
elif self.op == '-':
return left - right
elif self.op == '*':
return left * right
elif self.op == '/':
return left / right
else:
raise Exception('Unknown operator: {:s}'.format(self.op))
Some special thing here is, that you can also use variables (like a in my example in Step 3), but you have to map them to actual (un-typed) values on evaluation:
if __name__ == '__main__':
expression = '(a+15)*2+6/3'
tokens = tokenize(expression)
hierarchy = parse(tokens)
root = Node(hierarchy)
print(root)
print(root.eval({'a': 7}))
... yields:
>>> (((a + 15) * 2) + (6 / 3))
>>> 46
Final Thoughts
As already stated, this is far from perfect. I even noticed, that it somehow fails to parse an expression, where a single operator connects two bracketed parts like (1-2)/(0+5) -- but I leave this to whoever wants to have a look at it ;)
Hope it helps somehow; and sorry for this huge response. I was just curious and had a little bit of spare time.
I'm having an issue with a search algorithm over a Huffman tree: for a given probability distribution I need the Huffman tree to be identical regardless of permutations of the input data.
Here is a picture of what's happening vs what I want:
Basically I want to know if it's possible to preserve the relative order of the items from the list to the tree. If not, why is that so?
For reference, I'm using the Huffman tree to generate sub groups according to a division of probability, so that I can run the search() procedure below. Notice that the data in the merge() sub-routine is combined, along with the weight. The codewords themselves aren't as important as the tree (which should preserve the relative order).
For example if I generate the following Huffman codes:
probabilities = [0.30, 0.25, 0.20, 0.15, 0.10]
items = ['a','b','c','d','e']
items = zip(items, probabilities)
t = encode(items)
d,l = hi.search(t)
print(d)
Using the following Class:
class Node(object):
left = None
right = None
weight = None
data = None
code = None
def __init__(self, w,d):
self.weight = w
self.data = d
def set_children(self, ln, rn):
self.left = ln
self.right = rn
def __repr__(self):
return "[%s,%s,(%s),(%s)]" %(self.data,self.code,self.left,self.right)
def __cmp__(self, a):
return cmp(self.weight, a.weight)
def merge(self, other):
total_freq = self.weight + other.weight
new_data = self.data + other.data
return Node(total_freq,new_data)
def index(self, node):
return node.weight
def encode(symbfreq):
pdb.set_trace()
tree = [Node(sym,wt) for wt,sym in symbfreq]
heapify(tree)
while len(tree)>1:
lo, hi = heappop(tree), heappop(tree)
n = lo.merge(hi)
n.set_children(lo, hi)
heappush(tree, n)
tree = tree[0]
def assign_code(node, code):
if node is not None:
node.code = code
if isinstance(node, Node):
assign_code(node.left, code+'0')
assign_code(node.right, code+'1')
assign_code(tree, '')
return tree
I get:
'a'->11
'b'->01
'c'->00
'd'->101
'e'->100
However, an assumption I've made in the search algorithm is that more probable items get pushed toward the left: that is I need 'a' to have the '00' codeword - and this should always be the case regardless of any permutation of the 'abcde' sequence. An example output is:
codewords = {'a':'00', 'b':'01', 'c':'10', 'd':'110', 'e':111'}
(N.b even though the codeword for 'c' is a suffix for 'd' this is ok).
For completeness, here is the search algorithm:
def search(tree):
print(tree)
pdb.set_trace()
current = tree.left
other = tree.right
loops = 0
while current:
loops+=1
print(current)
if current.data != 0 and current is not None and other is not None:
previous = current
current = current.left
other = previous.right
else:
previous = other
current = other.left
other = other.right
return previous, loops
It works by searching for the 'leftmost' 1 in a group of 0s and 1s - the Huffman tree has to put more probable items on the left. For example if I use the probabilities above and the input:
items = [1,0,1,0,0]
Then the index of the item returned by the algorithm is 2 - which isn't what should be returned (0 should, as it's leftmost).
The usual practice is to use Huffman's algorithm only to generate the code lengths. Then a canonical process is used to generate the codes from the lengths. The tree is discarded. Codes are assigned in order from shorter codes to longer codes, and within a code, the symbols are sorted. This gives the codes you are expecting, a = 00, b = 01, etc. This is called a Canonical Huffman code.
The main reason this is done is to make the transmission of the Huffman code more compact. Instead of sending the code for each symbol along with the compressed data, you only need to send the code length for each symbol. Then the codes can be reconstructed on the other end for decompression.
A Huffman tree is not normally used for decoding either. With a canonical code, simple comparisons to determine the length of the next code, and an index using the code value will take you directly to the symbol. Or a table-driven approach can avoid the search for the length.
As for your tree, there are arbitrary choices being made when there are equal frequencies. In particular, on the second step the first node pulled is c with probability 0.2, and the second node pulled is b with probability 0.25. However it would have been equally valid to pull, instead of b, the node that was made in the first step, (e,d), whose probability is also 0.25. In fact that is what you'd prefer for your desired end state. Alas, you have relinquished the control of that arbitrary choice to the heapq library.
(Note: since you are using floating point values, 0.1 + 0.15 is not necessarily exactly equal to 0.25. Though it turns out it is. As another example, 0.1 + 0.2 is not equal to 0.3. You would be better off using integers for the frequencies if you want to see what happens when sums of frequencies are equal to other frequencies or sums of frequencies. E.g. 6,5,4,3,2.)
Some of the wrong ordering can be fixed by fixing some mistakes: change lo.merge(high) to hi.merge(lo), and reverse the order of the bits to: assign_code(node.left, code+'1') followed by assign_code(node.right, code+'0'). Then at least a gets assigned 00 and d is before e and b is before c. The ordering is then adebc.
Now that I think about it, even if you pick (e,d) over b, e.g by setting the probability of b to 0.251, you still don't get the complete order that you're after. No matter what, the probability of (e,d) (0.25) is greater than the probability of c (0.2). So even in that case, the final ordering would be (with the fixes above) abdec instead of your desired abcde. So it is not possible to get what you want assuming a consistent tree ordering and bit assignment with respect to the probabilities of groups of symbols. E.g., assuming that for each branch the stuff on the left has a greater or equal probability than the stuff on the right, and 0 is always assigned to left and 1 is always assigned to right. You would need to do something different.
The different thing that comes to mind is what I said at the start of the answer. Use the Huffman algorithm just to get the code lengths. Then you can assign the codes to the symbols in whatever order you like, and build a new tree. That would be much easier than trying to come up with some sort of scheme to coerce the original tree to be what you want, and proving that that works in all cases.
I'll flesh out what Mark Adler said with working code. Everything he said is right :-) The high points:
You must not use floating-point weights, or any other scheme that loses information about weights. Use integers. Simple and correct. If, e.g., you have 3-digit floating probabilities, convert each to an integer via int(round(the_probability * 1000)), then maybe fiddle them to ensure the sum is exactly 1000.
heapq heaps are not "stable": nothing is defined about which item is popped if multiple items have the same minimal weight.
So you can't get what you want while building the tree.
A small variation of "canonical Huffman codes" appears to be what you do want. Constructing a tree for that is a long-winded process, but each step is straightforward enough. The first tree built is thrown away: the only information taken from it is the lengths of the codes assigned to each symbol.
Running:
syms = ['a','b','c','d','e']
weights = [30, 25, 20, 15, 10]
t = encode(syms, weights)
print t
prints this (formatted for readability):
[abcde,,
([ab,0,
([a,00,(None),(None)]),
([b,01,(None),(None)])]),
([cde,1,
([c,10,(None),(None)]),
([de,11,
([d,110,(None),(None)]),
([e,111,(None),(None)])])])]
Best I understand, that's exactly what you want. Complain if it isn't ;-)
EDIT: there was a bug in the assignment of canonical codes, which didn't show up unless weights were very different; fixed it.
class Node(object):
def __init__(self, data=None, weight=None,
left=None, right=None,
code=None):
self.data = data
self.weight = weight
self.left = left
self.right = right
self.code = code
def is_symbol(self):
return self.left is self.right is None
def __repr__(self):
return "[%s,%s,(%s),(%s)]" % (self.data,
self.code,
self.left,
self.right)
def __cmp__(self, a):
return cmp(self.weight, a.weight)
def encode(syms, weights):
from heapq import heapify, heappush, heappop
tree = [Node(data=s, weight=w)
for s, w in zip(syms, weights)]
sym2node = {s.data: s for s in tree}
heapify(tree)
while len(tree) > 1:
a, b = heappop(tree), heappop(tree)
heappush(tree, Node(weight=a.weight + b.weight,
left=a, right=b))
# Compute code lengths for the canonical coding.
sym2len = {}
def assign_codelen(node, codelen):
if node is not None:
if node.is_symbol():
sym2len[node.data] = codelen
else:
assign_codelen(node.left, codelen + 1)
assign_codelen(node.right, codelen + 1)
assign_codelen(tree[0], 0)
# Create canonical codes, but with a twist: instead
# of ordering symbols alphabetically, order them by
# their position in the `syms` list.
# Construct a list of (codelen, index, symbol) triples.
# `index` breaks ties so that symbols with the same
# code length retain their original ordering.
triples = [(sym2len[name], i, name)
for i, name in enumerate(syms)]
code = oldcodelen = 0
for codelen, _, name in sorted(triples):
if codelen > oldcodelen:
code <<= (codelen - oldcodelen)
sym2node[name].code = format(code, "0%db" % codelen)
code += 1
oldcodelen = codelen
# Create a tree corresponding to the new codes.
tree = Node(code="")
dir2attr = {"0": "left", "1": "right"}
for snode in sym2node.values():
scode = snode.code
codesofar = ""
parent = tree
# Walk the tree creating any needed interior nodes.
for d in scode:
assert parent is not None
codesofar += d
attr = dir2attr[d]
child = getattr(parent, attr)
if codesofar == scode:
# We're at the leaf position.
assert child is None
setattr(parent, attr, snode)
elif child is not None:
assert child.code == codesofar
else:
child = Node(code=codesofar)
setattr(parent, attr, child)
parent = child
# Finally, paste the `data` attributes together up
# the tree. Why? Don't know ;-)
def paste(node):
if node is None:
return ""
elif node.is_symbol():
return node.data
else:
result = paste(node.left) + paste(node.right)
node.data = result
return result
paste(tree)
return tree
Duplicate symbols
Could I swap the sym2node dict to an ordereddict to deal with
repeated 'a'/'b's etc?
No, and for two reasons:
No mapping type supports duplicate keys; and,
The concept of "duplicate symbols" makes no sense for Huffman encoding.
So, if you're determined ;-) to pursue this, first you have to ensure that symbols are unique. Just add this line at the start of the function:
syms = list(enumerate(syms))
For example, if the syms passed in is:
['a', 'b', 'a']
that will change to:
[(0, 'a'), (1, 'b'), (2, 'a')]
All symbols are now 2-tuples, and are obviously unique since each starts with a unique integer. The only thing the algorithm cares about is that symbols can be used as dict keys; it couldn't care less whether they're strings, tuples, or any other hashable type that supports equality testing.
So nothing in the algorithm needs to change. But before the end, we'll want to restore the original symbols. Just insert this before the paste() function:
def restore_syms(node):
if node is None:
return
elif node.is_symbol():
node.data = node.data[1]
else:
restore_syms(node.left)
restore_syms(node.right)
restore_syms(tree)
That simply walks the tree and strips the leading integers off the symbols' .data members. Or, perhaps simpler, just iterate over sym2node.values(), and transform the .data member of each.