I feel like there is a redundency between:
def _determine_nodes_with_no_predecessors(nodes:List[Node])->List[str]:
"""
Given the list of nodes returns the identifiers of nodes that doesn't have any predecessor.
"""
nodes_ids_without_prdecessors = []
for node in nodes:
if not node.predecessors:
nodes_ids_without_prdecessors.append(node.id_)
return nodes_ids_without_prdecessors
And this one:
def _determine_nodes_with_no_successors(nodes:List[Node])->List[str]:
"""
Given the list of nodes returns the identifiers of nodes that doesn't have any successor.
"""
nodes_ids_without_successors = []
for node in nodes:
if not node.successors:
nodes_ids_without_successors.append(node.id_)
return nodes_ids_without_successors
How to write less code in this case? Is it possible to write only one function? I thought about adding a bool as an argument, something like START = True and then write in if-else statement, but I don't know if it's clean.
If you wanted to generalize this, you could pass in a accessor function. There are multiple ways to do this, but a simple lambda would suffice:
def _determine_nodes_with_no_using(nodes: List[Node], accessor: Callable[[Node], List[Node]])-> List[str]:
"""
Given the list of nodes returns the identifiers of nodes that doesn't have any successor.
"""
nodes_ids_without = []
for node in nodes:
if not accessor(node):
nodes_ids_without.append(node.id_)
return nodes_ids_without
Then
_determine_nodes_with_no_using(nodes, lambda node: node.predecessors)
_determine_nodes_with_no_using(nodes, lambda node: node.successors)
You could also use the operator module:
from operator import attrgetter
_determine_nodes_with_no_using(nodes, attrgetter('successors'))
_determine_nodes_with_no_using(nodes, attrgetter('predecessors'))
Related
I'm looking to build an extremely restricted lambda processor submitted by users. I'm using the following code to do so:
import ast, operator
from math import *
#make a list of safe functions
safe_list = ["math", "lambda"]
# use the list to filter the local namespace
safe_dict = dict([(k, locals().get(k, None)) for k in safe_list])
all_args = []
user_submitted_fxn_example = "d>15"
node = ast.parse(user_submitted_fxn_example, mode="eval")
for elem in ast.walk(node):
if isinstance(elem, ast.Name):
all_args.append(str(elem.id))
lambda_string = f"lambda {','.join(all_args)}: {fxn}"
a = eval(f"lambda {','.join(all_args)}: {fxn}", {"__builtins__": None}, safe_dict)
print(a(11)) # => Correctly reports 'False'
According to this, the right way to build the list of variables to put in the lambad is with ast.Name but when I do the following, I get both sin and x as a Name.
user_submitted_fxn_example = "sin(d)>0.5"
What am I doing wrong?
ast.Call nodes represent function calls. They have 2 key attributes: func and args. The ast.Name node in func represents the method name. args is a list of ast nodes. You might want to do a depth-first search for each node in args, and then look for ast.Name nodes. The reason for doing a DFS is that you could have an input like sin(f(x)), where f(x) would be yet another ast.Call node which contains ast.Name in the args field.
I recommend that you play around with some examples here.
Hi! I am new to Python and I am struggling for many hours so far with some problem regarding Shortest Path Algorithm implementation in Python.
I am expected to solve some task about finding shortest paths (graph problem) among given persons, and at the end find a common person who connects all of them.
I've made something like this so far:
import itertools
class centralperson():
def initialization(self,N,data,a,b,c):
self.data = data
self.N = N
self.a = a
self.b = b
self.c = c
self.list_of_values = [self.a,self.b,self.c]
self.list_of_paths = []
self.common_person = []
def makeGraph(self):
# Create dict with empty list for each key
graph = {key: [] for key in range(self.N)}
self.graph = graph
for key in self.graph:
for x in self.data:
if key in x:
x = x.copy()
x.remove(key)
self.graph[key].extend(x)
def find_path(self,start, end):
path = []
path = path + [start]
if start == end:
return path
if start not in self.graph.keys():
raise ValueError('No such key in graph!')
for node in self.graph[start]:
if node not in path:
newpath = self.find_path(self.graph, node, end, path)
if newpath: return newpath
return self.list_of_paths.append(path)
def findPaths(self):
for pair in itertools.combinations(self.list_of_values, r=5):
self.find_path(*pair)
def commonperson(self):
list_of_lens = {}
commonalities = set(self.list_of_paths[0]) & set(self.list_of_paths[1]) & set(self.list_of_paths[2])
for i in self.list_of_values:
list_of_lens[i] = (len(self.graph[i]))
if len(commonalities)>1 or len(commonalities)<1:
for k,v in list_of_lens.items():
if v==1 and self.graph[k][0] in commonalities:
output = self.graph[k]
self.common_person.append(output)
else:
output = list(commonalities)
self.common_person.append(output)
return
def printo(self):
#return(self.common_person[0])
return(self.list_of_paths,self.list_of_values)
Description of each function and inputs:
N -> number of unique nodes
a,b,c -> some arbitrarily chosen nodes to find common one among them
initialization -> just initialize our global variables used in other methods, and store the list of outputs
makeGraph -> makes an Adjacency List out of an input.
find_path -> find path between two given nodes (backtracking recursion)
findPaths -> it was expected to call find_path here for every combination of A,B,C i.e A->B, A->C, B->C
commonperson -> expected to find common person from the output of list_of_paths list
printo -> print this common person
Generally It works (I'think) when I'am running each function separately. But, when I try to make a huge class of it, it doesn't work :(
I think the problem is with this recursive function find_path. It is supposed to find a path between two person given, and append the result path to the list with all paths. Yet, as I have 3 different persons, and find_path is a recursive function with only 2 parameters.
Hence, I need to find all paths that connects them (3 paths) and append it to a bigger list list_of_paths. I've created a def findPaths to make use of itertools.combinations and in for loop cal function find_path for every combination of start and end argument of this function, but it seems not to work...
Can you help me with this? Also I don't know how to run all the def functions at once, because honestly I wouldn't like to run all instances of the class separately... My desired version is to:
Provide Input to a class i.e : N,data,a,b,c where N is number of unique nodes, data is just list of list with networks assigned, and A,B,C are my persons.
Get Output: which is a common person for all this 3 persons, (I planned to store it in common_person list.
The code inside you class should be indented, i.e.:
class centralperson:
def __init__(self, ...):
...
def makeGraph(self, ...):
...
instead of
class centralperson:
def __init__(self, ...):
...
def makeGraph(self, ...):
...
Try googling for 'python class examples'. I hope this helps!
It might also be useful to experiment with simpler classes before working on this problem.
itertools.combinations(self.list_of_values, r=5)
returns an empty list, since self.list_of_values only has 3 elements, from which you cannot pick 5.
Perhaps you meant:
itertools.combinations(self.list_of_values, r=2)
I have a string with Python code in it that I could evaluate as Python with literal_eval if it only had instances of OrderedDict replaced with {}.
I am trying to use ast.parse and ast.NodeTransformer to do the replacement, but when I catch the node with nodetype == 'Name' and node.id == 'OrderedDict', I can't find the list that is the argument in the node object so that I can replace it with a Dict node.
Is this even the right approach?
Some code:
from ast import NodeTransformer, parse
py_str = "[OrderedDict([('a', 1)])]"
class Transformer(NodeTransformer):
def generic_visit(self, node):
nodetype = type(node).__name__
if nodetype == 'Name' and node.id == 'OrderedDict':
pass # ???
return NodeTransformer.generic_visit(self, node)
t = Transformer()
tree = parse(py_str)
t.visit(tree)
The idea is to replace all OrderedDict nodes, represented as ast.Call having specific attributes (which can be seen from ordered_dict_conditions below), with ast.Dict nodes whose key / value arguments are extracted from the ast.Call arguments.
import ast
class Transformer(ast.NodeTransformer):
def generic_visit(self, node):
# Need to call super() in any case to visit child nodes of the current one.
super().generic_visit(node)
ordered_dict_conditions = (
isinstance(node, ast.Call)
and isinstance(node.func, ast.Name)
and node.func.id == 'OrderedDict'
and len(node.args) == 1
and isinstance(node.args[0], ast.List)
)
if ordered_dict_conditions:
return ast.Dict(
[x.elts[0] for x in node.args[0].elts],
[x.elts[1] for x in node.args[0].elts]
)
return node
def transform_eval(py_str):
return ast.literal_eval(Transformer().visit(ast.parse(py_str, mode='eval')).body)
print(transform_eval("[OrderedDict([('a', 1)]), {'k': 'v'}]")) # [{'a': 1}, {'k': 'v'}]
print(transform_eval("OrderedDict([('a', OrderedDict([('b', 1)]))])")) # {'a': {'b': 1}}
Notes
Because we want to replace the innermost node first, we place a call to super() at the beginning of the function.
Whenever an OrderedDict node is encountered, the following things are used:
node.args is a list containing the arguments to the OrderedDict(...) call.
This call has a single argument, namely a list containing key-value pairs as tuples, which is accessible by node.args[0] (ast.List) and node.args[0].elts are the tuples wrapped in a list.
So node.args[0].elts[i] are the different ast.Tuples (for i in range(len(node.args[0].elts))) whose elements are accessible again via the .elts attribute.
Finally node.args[0].elts[i].elts[0] are the keys and node.args[0].elts[i].elts[1] are the values which are used in the OrderedDict call.
The latter keys and values are then used to create a fresh ast.Dict instance which is then used to replace the current node (which was ast.Call).
You could use the ast.NodeVisitor class to observe the OrderedDict tree in order to build the {} tree manually from the encountered nodes, using the parsed nodes from an empty dict as a basis.
import ast
from collections import deque
class Builder(ast.NodeVisitor):
def __init__(self):
super().__init__()
self._tree = ast.parse('[{}]')
self._list_node = self._tree.body[0].value
self._dict_node = self._list_node.elts[0]
self._new_item = False
def visit_Tuple(self, node):
self._new_item = True
self.generic_visit(node)
def visit_Str(self, node):
if self._new_item:
self._dict_node.keys.append(node)
self.generic_visit(node)
def visit_Num(self, node):
if self._new_item:
self._dict_node.values.append(node)
self._new_item = False
self.generic_visit(node)
def literal_eval(self):
return ast.literal_eval(self._list_node)
builder = Builder()
builder.visit(ast.parse("[OrderedDict([('a', 1)])]"))
print(builder.literal_eval())
Note that this only works for the simple structure of your example which uses str as keys and int as values. However extensions to more complex structures should be possible in a similar fashion.
Instead of using ast for parsing and transforming the expression you could also use a regular expression for doing that. For example:
>>> re.sub(
... r"OrderedDict\(\[((\(('[a-z]+'), (\d+)\)),?\s*)+\]\)",
... r'{\3: \4}',
... "[OrderedDict([('a', 1)])]"
... )
"[{'a': 1}]"
The above expression is based on the example string of the OP and considers single quoted strings as keys and positive integers as values, but of course it can be extended to more complex cases.
I'm implementing a disjoint set system in Python, but I've hit a wall. I'm using a tree implementation for the system and am implementing Find(), Merge() and Create() functions for the system.
I am implementing a rank system and path compression for efficiency.
The catch is that these functions must take the set of disjoint sets as a parameter, making traversing hard.
class Node(object):
def __init__(self, value):
self.parent = self
self.value = value
self.rank = 0
def Create(values):
l = [Node(value) for value in values]
return l
The Create function takes in a list of values and returns a list of singular Nodes containing the appropriate data.
I'm thinking the Merge function would look similar to this,
def Merge(set, value1, value2):
value1Root = Find(set, value1)
value2Root = Find(set, value2)
if value1Root == value2Root:
return
if value1Root.rank < value2Root.rank:
value1Root.parent = value2Root
elif value1Root.rank > value2Root.rank:
value2Root.parent = value1Root
else:
value2Root.parent = value1Root
value1Root.rank += 1
but I'm not sure how to implement the Find() function since it is required to take the list of Nodes and a value (not just a node) as the parameters. Find(set, value) would be the prototype.
I understand how to implement path compression when a Node is taken as a parameter for Find(x), but this method is throwing me off.
Any help would be greatly appreciated. Thank you.
Edited for clarification.
The implementation of this data structure becomes simpler when you realize that the operations union and find can also be implemented as methods of a disjoint set forest class, rather than on the individual disjoint sets.
If you can read C++, then have a look at my take on the data structure; it hides the actual sets from the outside world, representing them only as numeric indices in the API. In Python, it would be something like
class DisjSets(object):
def __init__(self, n):
self._parent = range(n)
self._rank = [0] * n
def find(self, i):
if self._parent[i] == i:
return i
else:
self._parent[i] = self.find(self._parent[i])
return self._parent[i]
def union(self, i, j):
root_i = self.find(i)
root_j = self.find(j)
if root_i != root_j:
if self._rank[root_i] < self._rank[root_j]:
self._parent[root_i] = root_j
elif self._rank[root_i] > self._rank[root_j]:
self._parent[root_j] = root_i
else:
self._parent[root_i] = root_j
self._rank[root_j] += 1
(Not tested.)
If you choose not to follow this path, the client of your code will indeed have to have knowledge of Nodes and Find must take a Node argument.
Clearly merge function should be applied to pair of nodes.
So find function should take single node parameter and look like this:
def find(node):
if node.parent != node:
node.parent = find(node.parent)
return node.parent
Also wikipedia has pseudocode that is easily translatable to python.
Find is always done on an item. Find(item) is defined as returning the set to which the item belongs. Merger as such must not take nodes, merge always takes two items/sets. Merge or union (item1, item2) must first find(item1) and find(item2) which will return the sets to which each of these belong. After that the smaller set represented by an up-tree must be added to the taller. When a find is issued, always retrace the path and compress it.
A tested implementation with path compression is here.
I've been trying to think of a way to traverse a hierarchical structure, like a linked list, using a list expression, but haven't come up with anything that seems to work.
Basically, I want to convert this code:
p = self.parent
names = []
while p:
names.append(p.name)
p = p.parent
print ".".join(names)
into a one-liner like:
print ".".join( [o.name for o in <???>] )
I'm not sure how to do the traversal in the ??? part, though, in a generic way (if its even possible). I have several structures with similar .parent type attributes, and don't want to have write a yielding function for each.
Edit:
I can't use the __iter__ methods of the object itself because its already used for iterating over the values contained within the object itself. Most other answers, except for liori's, hardcode the attribute name, which is what I want to avoid.
Here's my adaptation based upon liori's answer:
import operator
def walk(attr, start):
if callable(attr):
getter = attr
else:
getter = operator.attrgetter(attr)
o = getter(start)
while o:
yield o
o = getter(o)
The closest thing I can think of is to create a parent generator:
# Generate a node's parents, heading towards ancestors
def gen_parents(node):
node = node.parent
while node:
yield node
node = node.parent
# Now you can do this
parents = [x.name for x in gen_parents(node)]
print '.'.join(parents)
If you want your solution to be general, use a general techique. This is a fixed-point like generator:
def fixedpoint(f, start, stop):
while start != stop:
yield start
start = f(start)
It will return a generator yielding start, f(start), f(f(start)), f(f(f(start))), ..., as long as neither of these values are equal to stop.
Usage:
print ".".join(x.name for x in fixedpoint(lambda p:p.parent, self, None))
My personal helpers library has similar fixedpoint-like function for years... it is pretty useful for quick hacks.
List comprehension works with objects that are iterators (have the next() method). You need to define an iterator for your structure in order to be able to iterate it this way.
Your LinkedList needs to be iterable for it to work properly.
Here's a good resource on it. (PDF warning) It is very in depth on both iterators and generators.
Once you do that, you'll be able to just do this:
print ".".join( [o.name for o in self] )