I'm trying to use the ast module in Python to parse input code, but am struggling with a lot of the syntax of how to do so. For instance, I have the following code as a testing environment:
import ast
class NodeVisitor(ast.NodeVisitor):
def visit_Call(self, node):
for each in node.args:
print(ast.literal_eval(each))
self.generic_visit(node)
line = "circuit = QubitCircuit(3, True)"
tree = ast.parse(line)
print("VISITOR")
visitor = NodeVisitor()
visitor.visit(tree)
Output:
VISITOR
3
True
In this instance, and please correct me if I'm wrong, the visit_Call will be used if it's a function call? So I can get each argument, however there's no guarantee it will work like this as there are different arguments available to be provided. I understand that node.args is providing my arguments, but I'm not sure how to do things with them?
I guess what I'm asking is how do I check what the arguments are and do different things with them? I'd like to check, perhaps, that the first argument is an Int, and if so, run processInt(parameter) as an example.
The value each in your loop in the method will be assigned to the AST node for each of the arguments in each function call you visit. There are lots of different types of AST nodes, so by checking which kind you have, you may be able to learn things about the argument being passed in.
Note however that the AST is about syntax, not values. So if the function call was foo(bar), it's just going to tell you that the argument is a variable named bar, not what the value of that variable is (which it does not know). If the function call was foo(bar(baz)), it's going to show you that the argument is another function call. If you only need to handle calls with literals as their arguments, then you're probably going to be OK, you'll just look instances of AST.Num and similar.
If you want to check if the first argument is a number and process it if it is, you can do something like:
def visit_Call(self, node):
first_arg = node.args[0]
if isinstance(first_arg, ast.Num):
processInt(first_arg.n)
else:
pass # Do you want to do something on a bad argument? Raise an exception maybe?
Related
My software supports python to automate tasks (Maya). When I undo or redo in this software it prints the last command, unfortunately for Python this is the memory address of the function rather than something actually useful. So the user sees the output Undo: <functools.partial object at 0x000002235DEDDF48> instead of something actually useful like Undo: Set Key on something at frame x
There appears to be no option to make Maya print a useful result from within it's own functionality, so now I want to ask if there's some obscure way cheese it with python to have that instance call itself something useful in a way the software will print while hopefully not interfering with the functionality. I'll try anything at this point!
def testFunc():
pass
test = partial(testFunc)
test results in <functools.partial object at 0x000002235DEA95E8>
If anyone can think of a more accurate title please edit / suggest.
Thanks to kindall giving me a lead in the comments I was able to find an answer. Subclassing partial and defining __repr__() is the key.
By grabbing the *args on __init__() and storing it as self.result we can use it on __repr__() to return the last argument given to *args as the result given by Maya when using Undo/Redo.
class rpartial(partial):
def __init__(self, *args):
self.result = args[-1]
def __repr__(self):
return self.result
rpartial(function, arg1, arg2, undoredo)
The string given to rpartial on the last line for undoredo is what will be printed by Maya when using Undo/Redo.
I'd be v. gratefull to learn something, usefull, as for now, I've been moving blindly.
so the problem lays in python's ast.NodeTransformer. I was wondering if one is able to add an function to existing class using this way, and not getting mad.
this is how I proceded so far.
import ast, inspect, cla # cla is a name of class to which we want to add a new function
klass = inspect.getsource(cla)
tree = ast.parse(klass)
st = '''def function(): return 1'''
Foo = ast.parse(st)
class AddFunc(ast.NodeTransformer):
def visit_ClassDef(self, node):
return node, node.body + Foo.body
self.generic_visit(node)
inst = AddFunc()
stuff = i.visit(tree)
# now the trouble begins, a compiling..
co = compile(stuff, filename='<ast>', mode='exec')
# i get TypeError: required "lineno" missing from stmt
I have tried (unsuccesfully as you could probably guess) to handle this by using
ast library helper functions ast.fix_missing_locations(),
and ast.copy_locaction(), but in most cases I've ended up guessing or
facing AttributeError by tuple which is inside AddFunc class.
Have anybody some idea, how to manage this?
kolko's answer is correct, in this instance, you need to return an AST node. Foo in this case is a module, and its body should be the statements constructing the function, which can simply be spliced into the class definition.
It helps to understand that ast.NodeTransformer is very particular:
it calls the visit method by the specific class name, ignoring hierarchy
it evaluates the return value with isinstance(AST) and isinstance(list), anything else is ignored!
You may also get this error or the closely related "lineno missing from expr" even when you're using ast.fix_missing_locations to add location data.
If this happens, it's because your AST has an element that's not allowed in that place. Use astunparse.dump to carefully examine the structure. Within a module body, there can only be statements. Within a function def, there can only be statements. Then there are specific places that you can have an expression.
To solve this, write out the python you're expecting to generate, parse it, dump it and check it against what you're generating.
I found answer in twitter:
https://twitter.com/mitsuhiko/status/91169383254200320
mitsuhiko: "TypeError: required field "lineno" missing from stmt" — no, what you actually mean is "tuple is not a statement'. #helpfulpython
You return here tuple, but u must return ast
class AddFunc(ast.NodeTransformer):
def visit_ClassDef(self, node):
node.body += Foo.body
return node
I want to envoke a method in my code in a supercass, to do some subclass- specific processing before continuing on. I come to python recently from C#... there, I'd probably use an interface. Here's the gist of it (as I picture it, but it's not working):
class superClass:
def do_specific_stuff(self): #To be implemented entirely by the subclass,
#but called from the superclass
pass
def do_general_stuff1(self):
#do misc
def do_general_stuff2(self):
#do more misc
def main_general_stuff(self):
do_general_stuff1()
do_specific_stuff()
do_general_stuff2()
I have a rather complicated implementation of this; this example is exactly what I need and far less painful to understand for a first- time viewer. Calling do_specific_stuff() at the moment gives me the error
'global name 'do_specific_stuff' is not defined.
When I add 'self' as in self.do_specific_stuff I get the error
'TypeError: do_specific_stuff() takes 0 positional arguments but 1 was given.' Any takers? Thanks in advance...
It needs to be
def main_general_stuff(self):
self.do_general_stuff1()
self.do_specific_stuff()
...
The problem is that you are missing the explicit reference to self: Python thinks you mean a global function without it. Note that there is no implicit this like in Java: You need to specify it.
I am using a block like this:
def served(fn) :
def wrapper(*args, **kwargs):
p = xmlrpclib.ServerProxy(SERVER, allow_none=True )
return (p.__getattr__(fn.__name__)(*args, **kwargs)) # do the function call
return functools.update_wrapper(wrapper,fn)
#served
def remote_function(a, b):
pass
to wrap a series of XML-RPC calls into a python module. The "served" decorator gets called on stub functions to expose operations on a remote server.
I'm creating stubs like this with the intention of being able to inspect them later for information about the function, specifically its arguments.
As listed, the code above does not transfer argument information from the original function to the wrapper. If I inspect with inspect.getargspec( remote_function ) then I get essentially an empty list, instead of args=['a','b'] that I was expecting.
I'm guessing I need to give additional direction to the functools.update_wrapper() call via the optional assigned parameter, but I'm not sure exactly what to add to that tuple to get the effect I want.
The name and the docstring are correctly transferred to the new function object, but can someone advise me on how to transfer argument definitions?
Thanks.
Previous questions here and here suggest that the decorator module can do this.
I am writing a script at the moment that will grab certain information from HTML using dom4j.
Since Python/Jython does not have a native switch statement I decided to use a whole bunch of if statements that call the appropriate method, like below:
if type == 'extractTitle':
extractTitle(dom)
if type == 'extractMetaTags':
extractMetaTags(dom)
I will be adding more depending on what information I want to extract from the HTML and thought about taking the dictionary approach which I found elsewhere on this site, example below:
{
'extractTitle': extractTitle,
'extractMetaTags': extractMetaTags
}[type](dom)
I know that each time I run the script the dictionary will be built, but at the same time if I were to use the if statements the script would have to check through all of them until it hits the correct one. What I am really wondering, which one performs better or is generally better practice to use?
Update: #Brian - Thanks for the great reply. I have a question, if any of the extract methods require more than one object, e.g.
handle_extractTag(self, dom, anotherObject)
# Do something
How would you make the appropriate changes to the handle method to implemented this? Hope you know what I mean :)
Cheers
To avoid specifying the tag and handler in the dict, you could just use a handler class with methods named to match the type. Eg
class MyHandler(object):
def handle_extractTitle(self, dom):
# do something
def handle_extractMetaTags(self, dom):
# do something
def handle(self, type, dom):
func = getattr(self, 'handle_%s' % type, None)
if func is None:
raise Exception("No handler for type %r" % type)
return func(dom)
Usage:
handler = MyHandler()
handler.handle('extractTitle', dom)
Update:
When you have multiple arguments, just change the handle function to take those arguments and pass them through to the function. If you want to make it more generic (so you don't have to change both the handler functions and the handle method when you change the argument signature), you can use the *args and **kwargs syntax to pass through all received arguments. The handle method then becomes:
def handle(self, type, *args, **kwargs):
func = getattr(self, 'handle_%s' % type, None)
if func is None:
raise Exception("No handler for type %r" % type)
return func(*args, **kwargs)
With your code you're running your functions all get called.
handlers = {
'extractTitle': extractTitle,
'extractMetaTags': extractMetaTags
}
handlers[type](dom)
Would work like your original if code.
It depends on how many if statements we're talking about; if it's a very small number, then it will be more efficient than using a dictionary.
However, as always, I strongly advice you to do whatever makes your code look cleaner until experience and profiling tell you that a specific block of code needs to be optimized.
Your use of the dictionary is not quite correct. In your implementation, all methods will be called and all the useless one discarded. What is usually done is more something like:
switch_dict = {'extractTitle': extractTitle,
'extractMetaTags': extractMetaTags}
switch_dict[type](dom)
And that way is facter and more extensible if you have a large (or variable) number of items.
The efficiency question is barely relevant. The dictionary lookup is done with a simple hashing technique, the if-statements have to be evaluated one at a time. Dictionaries tend to be quicker.
I suggest that you actually have polymorphic objects that do extractions from the DOM.
It's not clear how type gets set, but it sure looks like it might be a family of related objects, not a simple string.
class ExtractTitle( object ):
def process( dom ):
return something
class ExtractMetaTags( object ):
def process( dom ):
return something
Instead of setting type="extractTitle", you'd do this.
type= ExtractTitle() # or ExtractMetaTags() or ExtractWhatever()
type.process( dom )
Then, you wouldn't be building this particular dictionary or if-statement.