Related
Dear experienced friends, I am looking for an algorithm (Python) that outputs the width of a tree at each level. Here are the input and expected outputs.
(I have updated the problem with a more complex edge list. The original question with sorted edge list can be elegantly solved by #Samwise answer.)
Input (Edge List: source-->target)
[[11,1],[11,2],
[10,11],[10,22],[10,33],
[33,3],[33,4],[33,5],[33,6]]
The tree graph looks like this:
10
/ | \
11 22 33
/ \ / | \ \
1 2 3 4 5 6
Expected Output (Width of each level/height)
[1,3,6] # according to the width of level 0,1,2
I have looked through the web. It seems this topic related to BFS and Level Order Traversal. However, most solutions are based on the binary tree. How can solve the problem when the tree is not binary (e.g. the above case)?
(I'm new to the algorithm, and any references would be really appreciated. Thank you!)
Build a dictionary of the "level" of each node, and then count the number of nodes at each level:
>>> from collections import Counter
>>> def tree_width(edges):
... levels = {} # {node: level}
... for [p, c] in edges:
... levels[c] = levels.setdefault(p, 0) + 1
... widths = Counter(levels.values()) # {level: width}
... return [widths[level] for level in sorted(widths)]
...
>>> tree_width([[0,1],[0,2],[0,3],
... [1,4],[1,5],
... [3,6],[3,7],[3,8],[3,9]])
[1, 3, 6]
This might not be the most efficient, but it requires only two scans over the edge list, so it's optimal up to a constant factor. It places no requirement on the order of the edges in the edge list, but does insist that each edge be (source, dest). Also, doesn't check that the edge list describes a connected tree (or a tree at all; if the edge list is cyclic, the program will never terminate).
from collections import defauiltdict
# Turn the edge list into a (non-binary) tree, represented as a
# dictionary whose keys are the source nodes with the list of children
# as its value.
def edge_list_to_tree(edges):
'''Given a list of (source, dest) pairs, constructs a tree.
Returns a tuple (tree, root) where root is the root node
and tree is a dict which maps each node to a list of its children.
(Leaves are not present as keys in the dictionary.)
'''
tree = defaultdict(list)
sources = set() # nodes used as sources
dests = set() # nodes used as destinations
for source, dest in edges:
tree[source].append(dest)
sources.add(source)
dests.add(dest)
roots = sources - dests # Source nodes which are not destinations
assert(len(roots) == 1) # There is only one in a tree
tree.default_factory = None # Defang the defaultdict
return tree, roots.pop()
# A simple breadth-first-search, keeping the count of nodes at each level.
def level_widths(tree, root):
'''Does a BFS of tree starting at root counting nodes at each level.
Returns a list of counts.
'''
widths = [] # Widths of the levels
fringe = [root] # List of nodes at current level
while fringe:
widths.append(len(fringe))
kids = [] # List of nodes at next level
for parent in fringe:
if parent in tree:
for kid in tree[parent]:
kids.append(kid)
fringe = kids # For next iteration, use this level's kids
return widths
# Put the two pieces together.
def tree_width(edges):
return level_widths(*edge_list_to_tree(edges))
Possible solution that is based on Width-First-Traversal
In Width-First-Traversal we add the node to the array, but in this solution we put the array in an object together with its level and then add it to the array.
function levelWidth(root) {
const counter = [];
const traverseBF = fn => {
const arr = [{n: root, l:0}];
const pushToArr = l => n => arr.push({n, l});
while (arr.length) {
const node = arr.shift();
node.n.children.forEach(pushToArr(node.l+1));
fn(node);
}
};
traverseBF(node => {
counter[node.l] = (+counter[node.l] || 0) + 1;
});
return counter;
}
I was given a question during an interview and although my answer was accepted at the end they wanted a faster approach and I went blank..
Question :
Given an undirected graph, can you see if it's a tree? If so, return true and false otherwise.
A tree:
A - B
|
C - D
not a tree:
A
/ \
B - C
/
D
You'll be given two parameters: n for number of nodes, and a multidimensional array of edges like such: [[1, 2], [2, 3]], each pair representing the vertices connected by the edge.
Note:Expected space complexity : O(|V|)
The array edges can be empty
Here is My code: 105ms
def is_graph_tree(n, edges):
nodes = [None] * (n + 1)
for i in range(1, n+1):
nodes[i] = i
for i in range(len(edges)):
start_edge = edges[i][0]
dest_edge = edges[i][1]
if nodes[start_edge] != start_edge:
start_edge = nodes[start_edge]
if nodes[dest_edge] != dest_edge:
dest_edge = nodes[dest_edge]
if start_edge == dest_edge:
return False
nodes[start_edge] = dest_edge
return len(edges) <= n - 1
Here's one approach using a disjoint-set-union / union-find data structure:
def is_graph_tree(n, edges):
parent = list(range(n+1))
size = [1] * (n + 1)
for x, y in edges:
# find x (path splitting)
while parent[x] != x:
x, parent[x] = parent[x], parent[parent[x]]
# find y
while parent[y] != y:
y, parent[y] = parent[y], parent[parent[y]]
if x == y:
# Already connected
return False
# Union (by size)
if size[x] < size[y]:
x, y = y, x
parent[y] = x
size[x] += size[y]
return True
assert not is_graph_tree(4, [(1, 2), (2, 3), (3, 4), (4, 2)])
assert is_graph_tree(6, [(1, 2), (2, 3), (3, 4), (3, 5), (1, 6)])
The runtime is O(V + E*InverseAckermannFunction(V)), which better than O(V + E * log(log V)), so it's basically O(V + E).
Tim Roberts has posted a candidate solution, but this will work in the case of disconnected subtrees:
import queue
def is_graph_tree(n, edges):
# A tree with n nodes has n - 1 edges.
if len(edges) != n - 1:
return False
# Construct graph.
graph = [[] for _ in range(n)]
for first_vertex, second_vertex in edges:
graph[first_vertex].append(second_vertex)
graph[second_vertex].append(first_vertex)
# BFS to find edges that create cycles.
# The graph is undirected, so we can root the tree wherever we want.
visited = set()
q = queue.Queue()
q.put((0, None))
while not q.empty():
current_node, previous_node = q.get()
if current_node in visited:
return False
visited.add(current_node)
for neighbor in graph[current_node]:
if neighbor != previous_node:
q.put((neighbor, current_node))
# Only return true if the graph has only one connected component.
return len(visited) == n
This runs in O(n + len(edges)) time.
You could approach this from the perspective of tree leaves. Every leaf node in a tree will have exactly one edge connected to it. So, if you count the number of edges for each nodes, you can get the list of leaves (i.e. the ones with only one edge).
Then, take the linked node from these leaves and reduce their edge count by one (as if you were removing all the leaves from the tree. That will give you a new set of leaves corresponding to the parents of the original leaves. Repeat the process until you have no more leaves.
[EDIT] checking that the number of edges is N-1 eliminiates the need to do the multi-root check because there will be another discrepancy (e.g. double link, missing node) in the graph if there are multiple 'roots' or a disconnected subtree
If the graph is a tree, this process should eliminate all nodes from the node counts (i.e. they will all be flagged as leaves at some point).
Using the Counter class (from collections) will make this relatively easy to implement:
from collections import Counter
def isTree(N,E):
if N==1 and not E: return True # root only is a tree
if len(E) != N-1: return False # a tree has N-1 edges
counts = Counter(n for ab in E for n in ab) # edge counts per node
if len(counts) != N : return False # unlinked nodes
while True:
leaves = {n for n,c in counts.items() if c==1} # new leaves
if not leaves:break
for a,b in E: # subtract leaf counts
if counts[a]>1 and b in leaves: counts[a] -= 1
if counts[b]>1 and a in leaves: counts[b] -= 1
for n in leaves: counts[n] = -1 # flag leaves in counts
return all(c==-1 for c in counts.values()) # all must become leaves
output:
G = [[1,2],[1,3],[4,5],[4,6]]
print(isTree(6,G)) # False (disconnected sub-tree)
G = [[1,2],[1,3],[1,4],[2,3],[5,6]]
print(isTree(6,G)) # False (doubly linked node 3)
G = [[1,2],[2,6],[3,4],[5,1],[2,3]]
print(isTree(6,G)) # True
G = [[1,2],[2,3]]
print(isTree(3,G)) # True
G = [[1,2],[2,3],[3,4]]
print(isTree(4,G)) # True
G = [[1,2],[1,3],[2,5],[2,4]]
print(isTree(6,G)) # False (missing node)
Space complexity is O(N) because the counts dictionary has one entry per node(vertex) with an integer as value. Time complexity will be O(ExL) where E is the number of edges and L is the number of levels in the tree. The worts case time is O(E^2) for a tree where all parents have only one child node. However, since the initial condition is for E to be less than V, the worst case will actually be O(V^2)
Note that this algorithm makes no assumption on edge order or numerical relationships between node numbers. The root (last node to be made a leaf) found by this algorithm is not necessarily the only possible root given that, unless the nodes have an implicit cardinality relationship (or edges have an order), there could be ambiguous scenarios:
[1,2],[2,3],[2,4] could be:
1 2 3
|_2 OR |_1 OR |_2
|_3 |_3 |_1
|_4 |_4 |_4
If a cardinality relationship between node numbers or an order of edges can be relied upon, the algorithm could potentially be made more time efficient (because we could easily determine which node is the root and start from there).
[EDIT2] Alternative method using groups.
When the number of edges is N-1, if the graph is a tree, all nodes should be reachable from any other node. This means that, if we form groups of reachable nodes for each node and merge them together based on the edges, we should end up with a single group after going through all the edges.
Here is the modified function based on that approach:
def isTree(N,E):
if N==1 and not E: return True # root only is a tree
if len(E) != N-1: return False # a tree has N-1 edges
groups = {n:[n] for ab in E for n in ab} # each node in its own group
if len(groups) != N : return False # unlinked nodes
for a,b in E:
groups[a].extend(groups[b]) # merge groups
for n in groups[b]: groups[n] = groups[a] # update nodes' groups
return len(set(map(id,groups.values()))) == 1 # only one group when done
Given that we start out with fewer edges than nodes and that group merging will consume at most 2x a group size (so also < N), the space complexity will remain O(V). The time complexity will also be O(V^2) at for the worts case scenarios
You don't even need to know how many edges there are:
def is_graph_tree(n, edges):
seen = set()
for a,b in edges:
b = max(a,b)
if b in seen:
return False
seen.add(b)
return True
a = [[1,2],[2,3],[3,4]]
print(is_graph_tree(0,a))
b = [[1,2],[1,3],[2,3],[2,4]]
print(is_graph_tree(0,b))
Now, this WON'T catch the case of disconnected subtrees, but that wasn't in the problem description...
Problem
I have a list of line segments:
exampleLineSegments = [(1,2),(2,3),(3,4),(4,5),(5,6),(4,7),(8,7)]
These segments include the indices of the corresponding point in a separate array.
From this sublist, one can see that there is a branching point (4). So three different branches are emerging from this branching point.
(In other, more specific problems, there might be / are multiple branching points for n branches.)
Target
My target is to get a dictionary including information about the existing branches, so e.g.:
result = { branch_1: [1,2,3,4],
branch_2: [4,5,6],
branch_3: [4,7,8]}
Current state of work/problems
Currently, I am identifying the branch points first by setting up a dictionary for each point and checking for each entry if there are more than 2 neighbor points found. This means that there is a branching point.
Afterwards I am crawling through all points emerging from these branch points, checking for successors etc.
In these functions, there are a some for loops and generally an intensive "crawling". This is not the cleanest solution and if the number of points increasing, the performance is not so good either.
Question
What is the best / fastest / most performant way to achieve the target in this case?
I think you can achieve it by following steps:
use a neighbors dict to store the graph
find all branch points, which neighbors count > 2
start from every branch point, and use dfs to find all the paths
from collections import defaultdict
def find_branch_paths(exampleLineSegments):
# use dict to store the graph
neighbors = defaultdict(list)
for p1, p2 in exampleLineSegments:
neighbors[p1].append(p2)
neighbors[p2].append(p1)
# find all branch points
branch_points = [k for k, v in neighbors.items() if len(v) > 2]
res = []
def dfs(cur, prev, path):
# reach the leaf
if len(neighbors[cur]) == 1:
res.append(path)
return
for neighbor in neighbors[cur]:
if neighbor != prev:
dfs(neighbor, cur, path + [neighbor])
# start from all the branch points
for branch_point in branch_points:
dfs(branch_point, None, [branch_point])
return res
update an iteration version, for big data, which may cause a recursion depth problem:
def find_branch_paths(exampleLineSegments):
# use dict to store the graph
neighbors = defaultdict(list)
for p1, p2 in exampleLineSegments:
neighbors[p1].append(p2)
neighbors[p2].append(p1)
# find all branch points
branch_points = [k for k, v in neighbors.items() if len(v) > 2]
res = []
# iteration way to dfs
stack = [(bp, None, [bp]) for bp in branch_points]
while stack:
cur, prev, path = stack.pop()
if len(neighbors[cur]) == 1 or (prev and cur in branch_points):
res.append(path)
continue
for neighbor in neighbors[cur]:
if neighbor != prev:
stack.append((neighbor, cur, path + [neighbor]))
return res
test and output:
print(find_branch_paths([(1, 2), (2, 3), (3, 4), (4, 5), (5, 6), (4, 7), (8, 7)]))
# output:
# [[4, 3, 2, 1], [4, 5, 6], [4, 7, 8]]
Hope that helps you, and comment if you have further questions. : )
UPDATE: if there are many branch points, the path will grow exponentially. So if you only want distinct segments, you can end the path when encounter another branch point.
change this line
if len(neighbors[cur]) == 1:
to
if len(neighbors[cur]) == 1 or (prev and cur in branch_points):
I am working a problem of sorting edges, where edges are stored in a tuple form (node_i, node_j) like below
>> edgeLst
>> [('123','234'),
('123','456'),
('123','789'),
('456','765'),
('456','789')
('234','765')]
Note that edges are unique, and if you see ('123', '234'), you won't see ('234', '123') (the graph is undirected). And there might be a loop in the graph. Since the graph is very large, can anyone show me the efficient way to sort edges in BFS and DFS with a given start node, e.g., '123'?
Demo output:
>> edgeSorting(input_lst=edgeLst, by='BFS', start_node='123')
>> [('123','234'),
('123','456'),
('123','789'),
('234','765')
('456','765'),
('456','789')]
Here is how you could do it for BFS and DFS:
from collections import defaultdict
def sorted_edges(input_lst, by, start_node):
# Key the edges by the vertices
vertices = defaultdict(lambda: [])
for u, v in input_lst:
vertices[u].append(v)
vertices[v].append(u)
if by == 'DFS':
# Sort the lists
for lst in vertices:
lst = sorted(lst)
# Perform DFS
visited = set()
def recurse(a):
for b in vertices[a]:
if not (a, b) in visited:
yield ((a,b))
# Make sure this edge is not visited anymore in either direction
visited.add((a,b))
visited.add((b,a))
for edge in recurse(b):
yield edge
for edge in recurse(start_node):
yield edge
else: #BFS
# Collect the edges
visited = set()
queue = [start_node]
while len(queue):
for a in queue: # Process BFS level by order of id
level = []
for b in vertices[a]:
if not (a, b) in visited:
yield ((a,b))
# Add to next level to process
level.append(b)
# Make sure this edge is not visited anymore in either direction
visited.add((a,b))
visited.add((b,a))
queue = sorted(level)
edgeLst = [('123','234'),
('123','456'),
('123','789'),
('456','765'),
('456','789'),
('234','765')]
print (list(sorted_edges(edgeLst, 'DFS', '123')))
In Python 3, you can simplify:
for edge in recurse(b):
yield edge
to:
yield from recurse(b)
... and same for start_node.
This is a follow-up question to Combinatorics in Python
I have a tree or directed acyclic graph if you will with a structure as:
Where r are root nodes, p are parent nodes, c are child nodes and b are hypothetical branches. The root nodes are not directly linked to the parent nodes, it is only a reference.
I am intressted in finding all the combinations of branches under the constraints:
A child can be shared by any number of parent nodes given that these parent nodes do not share root node.
A valid combination should not be a subset of another combination
In this example only two valid combinations are possible under the constraints:
combo[0] = [b[0], b[1], b[2], b[3]]
combo[1] = [b[0], b[1], b[2], b[4]]
The data structure is such as b is a list of branch objects, which have properties r, c and p, e.g.:
b[3].r = 1
b[3].p = 3
b[3].c = 2
This problem can be solved in Python easily and elegantly, because there is a module called "itertools".
Lets say we have objects of type HypotheticalBranch, which have attributes r, p and c. Just as you described it in your post:
class HypotheticalBranch(object):
def __init__(self, r, p, c):
self.r=r
self.p=p
self.c=c
def __repr__(self):
return "HypotheticalBranch(%d,%d,%d)" % (self.r,self.p,self.c)
Your set of hypothetical branches is thus
b=[ HypotheticalBranch(0,0,0),
HypotheticalBranch(0,1,1),
HypotheticalBranch(1,2,1),
HypotheticalBranch(1,3,2),
HypotheticalBranch(1,4,2) ]
The magical function that returns a list of all possible branch combos could be written like so:
import collections, itertools
def get_combos(branches):
rc=collections.defaultdict(list)
for b in branches:
rc[b.r,b.c].append(b)
return itertools.product(*rc.values())
To be precise, this function returns an iterator. Get the list by iterating over it. These four lines of code will print out all possible combos:
for combo in get_combos(b):
print "Combo:"
for branch in combo:
print " %r" % (branch,)
The output of this programme is:
Combo:
HypotheticalBranch(0,1,1)
HypotheticalBranch(1,3,2)
HypotheticalBranch(0,0,0)
HypotheticalBranch(1,2,1)
Combo:
HypotheticalBranch(0,1,1)
HypotheticalBranch(1,4,2)
HypotheticalBranch(0,0,0)
HypotheticalBranch(1,2,1)
...which is just what you wanted.
So what does the script do? It creates a list of all hypothetical branches for each combination (root node, child node). And then it yields the product of these lists, i.e. all possible combinations of one item from each of the lists.
I hope I got what you actually wanted.
You second constraint means you want maximal combinations, i.e. all the combinations with the length equal to the largest combination.
I would approach this by first traversing the "b" structure and creating a structure, named "c", to store all branches coming to each child node and categorized by the root node that comes to it.
Then to construct combinations for output, for each child you can include one entry from each root set that is not empty. The order (execution time) of the algorithm will be the order of the output, which is the best you can get.
For example, your "c" structure, will look like:
c[i][j] = [b_k0, ...]
--> means c_i has b_k0, ... as branches that connect to root r_j)
For the example you provided:
c[0][0] = [0]
c[0][1] = []
c[1][0] = [1]
c[1][1] = [2]
c[2][0] = []
c[2][1] = [3, 4]
It should be fairly easy to code it using this approach. You just need to iterate over all branches "b" and fill the data structure for "c". Then write a small recursive function that goes through all items inside "c".
Here is the code (I entered your sample data at the top for testing sake):
class Branch:
def __init__(self, r, p, c):
self.r = r
self.p = p
self.c = c
b = [
Branch(0, 0, 0),
Branch(0, 1, 1),
Branch(1, 2, 1),
Branch(1, 3, 2),
Branch(1, 4, 2)
]
total_b = 5 # Number of branches
total_c = 3 # Number of child nodes
total_r = 2 # Number of roots
c = []
for i in range(total_c):
c.append([])
for j in range(total_r):
c[i].append([])
for k in range(total_b):
c[b[k].c][b[k].r].append(k)
combos = []
def list_combos(n_c, n_r, curr):
if n_c == total_c:
combos.append(curr)
elif n_r == total_r:
list_combos(n_c+1, 0, curr)
elif c[n_c][n_r]:
for k in c[n_c][n_r]:
list_combos(n_c, n_r+1, curr + [b[k]])
else:
list_combos(n_c, n_r+1, curr)
list_combos(0, 0, [])
print combos
There are really two problems here: firstly, you need to work out the algorithm that you will use to solve this problem and secondly, you need to implement it (in Python).
Algorithm
I shall assume you want a maximal collection of branches; that is, once to which you can't add any more branches. If you don't, you can consider all subsets of a maximal collection.
Therefore, for a child node we want to take as many branches as possible, subject to the constraint that no two parent nodes share a root. In other words, from each child you may have at most one edge in the neighbourhood of each root node. This seems to suggest that you want to iterate first over the children, then over the (neighbourhoods of the) root nodes, and finally over the edges between these. This concept gives the following pseudocode:
for each child node:
for each root node:
remember each permissible edge
find all combinations of permissible edges
Code
>>> import networkx as nx
>>> import itertools
>>>
>>> G = nx.DiGraph()
>>> G.add_nodes_from(["r0", "r1", "p0", "p1", "p2", "p3", "p4", "c0", "c1", "c2"])
>>> G.add_edges_from([("r0", "p0"), ("r0", "p1"), ("r1", "p2"), ("r1", "p3"),
... ("r1", "p4"), ("p0", "c0"), ("p1", "c1"), ("p2", "c1"),
... ("p3", "c2"), ("p4", "c2")])
>>>
>>> combs = set()
>>> leaves = [node for node in G if not G.out_degree(node)]
>>> roots = [node for node in G if not G.in_degree(node)]
>>> for leaf in leaves:
... for root in roots:
... possibilities = tuple(edge for edge in G.in_edges_iter(leaf)
... if G.has_edge(root, edge[0]))
... if possibilities: combs.add(possibilities)
...
>>> combs
set([(('p1', 'c1'),),
(('p2', 'c1'),),
(('p3', 'c2'), ('p4', 'c2')),
(('p0', 'c0'),)])
>>> print list(itertools.product(*combs))
[(('p1', 'c1'), ('p2', 'c1'), ('p3', 'c2'), ('p0', 'c0')),
(('p1', 'c1'), ('p2', 'c1'), ('p4', 'c2'), ('p0', 'c0'))]
The above seems to work, although I haven't tested it.
For each child c, with hypothetical parents p(c), with roots r(p(c)), choose exactly one parent p from p(c) for each root r in r(p(c)) (such that r is the root of p) and include b in the combination where b connects p to c (assuming there is only one such b, meaning it's not a multigraph). The number of combinations will be the product of the numbers of parents by which each child is hypothetically connected to each root. In other words, the size of the set of combinations will be equal to the product of the hypothetical connections of all child-root pairs. In your example all such child-root pairs have only one path, except r1-c2, which has two paths, thus the size of the set of combinations is two.
This satisfies the constraint of no combination being a subset of another because by choosing exactly one parent for each root of each child, we maximize the number connections. Subsequently adding any edge b would cause its root to be connected to its child twice, which is not allowed. And since we are choosing exactly one, all combinations will be exactly the same length.
Implementing this choice recursively will yield the desired combinations.