I have a list like this cs_id["CS_A1","CS_b7",...] in a function. At the end of the function the list ist filled with 80 values. How can I use this list (and values) in another function? Here I want to use the list cs_id[] from function unzip in function changecs. (By the way, the second function isn't ready yet.)
Update
I still dont get it....dont know why.
Here is my full code...maybe someone can help.
maker.py
#!/usr/bin/python
import getopt
import sys
import functions as func
ifile = ''
ofile = ''
instances = 0
def main(argv):
try:
opts, args = getopt.getopt(argv, "hi:o:n:d", ["help", "ifile=", "ofile=", "NumberOfInstances="])
except getopt.GetoptError:
func.usage()
sys.exit(2)
for opt, arg in opts:
if opt in ("-h", "--help"):
func.usage()
sys.exit()
elif opt in '-d':
global _debug
_debug = 1
elif opt in ("-i", "--ifile"):
global ifile
ifile = arg
elif opt in ("-o", "--ofile"):
global ofile
ofile = arg
elif opt in ("-n", "--NumberOfInstances"):
global instances
instances = int(arg)
func.unzip(ifile, instances)
func.changecs()
if __name__ == "__main__":
main(sys.argv[1:])
functions.py
import os
import zipfile
import sys
import string
import random
# printing usage of warmaker.py
def usage():
print "How to use warmaker.py"
print 'Usage: ' + sys.argv[0] + ' -i <inputfile> -o <outputfile> -n <NumberOfInstances>'
# creating random IDs for CS instance e.g. CS_AE, CS_3B etc.
def id_generator(size=2, chars=string.ascii_uppercase + string.digits):
return ''.join(random.choice(chars) for _ in range(size))
# unzip the reference warfile and build n instances
def unzip(ifile, instances,):
newinstance = ifile
cs_id = []
for i in xrange(instances):
cs_id.append('CS_' + id_generator())
i += 1
print 'Searching for reference file ' + newinstance
if os.path.isfile(newinstance): # check if file exists
print 'Found ' + newinstance
else:
print newinstance + ' not fonund. Try again.'
sys.exit()
print 'Building ' + str(instances) + ' instances... '
for c in xrange(instances):
extract = zipfile.ZipFile(newinstance)
extract.extractall(cs_id[c])
extract.close()
print cs_id[c] + ' done'
c += 1
return cs_id
#def create_war_file():
def changecs(cs_id):
n = 0
for item in cs_id:
cspath = cs_id[n] + '/KGSAdmin_CS/conf/contentserver/contentserver-conf.txt'
if os.path.isfile(cspath):
print 'contentserver-conf.txt found'
else:
print 'File not found. Try again.'
sys.exit()
n += 1
#f = open(cspath)
#row = f.read()
Two ways.
1/ Return the list in unzip
def unzip(ifile, instances):
# No need for this global
# global cs_id
cs_id = []
# Do stuff
# [...]
# Return the list
return cs_id
In this case you can call unzip and get the complete list as return value:
def changecs(instances):
# The following line is equivalent to
# cs_id = unzip(ifile, instances)
# for c in cs_id:
for c in unzip(ifile, instances):
cspath = cs_id + '/abc/myfile.txt'
2/ Pass it as a parameter and modify it in unzip.
def unzip(ifile, instances, cs_id):
# Do stuff
# [...]
In this case you can pass unzip the empty list and let it modify it in place:
def changecs(instances):
cs_id = []
unzip(ifile, instances, cs_id):
for c in cs_id:
cspath = cs_id + '/abc/myfile.txt'
I prefer the first approach. No need to provide unzip with an empty list. The second approach is more suited if you have to call unzip on an existing non-empty list.
Edit:
Since your edit, unzip returns cs_id and changecs uses it as an input.
def unzip(ifile, instances,):
[...]
return cs_id
def changecs(cs_id):
[....]
But you call them like this:
func.unzip(ifile, instances)
func.changecs() # This should trigger an Exception since changecs expects a positional argument
You should call them like this:
variable = func.unzip(ifile, instances)
func.changecs(variable)
or just
func.changecs(func.unzip(ifile, instances))
One possibility is to initialize the list outside of the function, then call it in both, i guess.
you can call the function and return it to the other function. Not sure where you'd want to use it but as an example this is the same concept
def foo():
l = [1,3,2,5,4,6,5]
l.append(10)
return l
def bar():
l = foo()
print (l)
return terminates the function and in your case you'd want to put it at the end of the function.
Make unzip return the value. Call unzip from the main block. Then pass the return value from unzip into changecs. Simple.
def unzip(...):
...
return cs_id
def changecs(cs_id, ...):
... do stuff with cs_id ...
if __name__ == "__main__":
... main block ... # This can be replaced with any kind of driver code
Alternatively, you can call unzip directly from within changecs (if that flow is permissible).
The code is all but there. At the bottom of def unzip() the array cs_id is being returned using the return command. The returned array can be stored in a variable from the function which runs unzip(). In this case, it's our main python function.
After you have the cs_id array in a variable in your main function, pass it to changecs() like: func.changecs(cs_id)
Edit last two lines of def main(argv) to:
cs_id = func.unzip(ifile, instances)
func.changecs(cs_id)
In the above code, unzip(ifile, instances) returns our array cs_id. We pass this into changecs(cs_id) as an parameter.
Related
This question already has answers here:
Getting the name of a variable as a string
(32 answers)
Closed 4 months ago.
Is it possible to get the original variable name of a variable passed to a function? E.g.
foobar = "foo"
def func(var):
print var.origname
So that:
func(foobar)
Returns:
>>foobar
EDIT:
All I was trying to do was make a function like:
def log(soup):
f = open(varname+'.html', 'w')
print >>f, soup.prettify()
f.close()
.. and have the function generate the filename from the name of the variable passed to it.
I suppose if it's not possible I'll just have to pass the variable and the variable's name as a string each time.
EDIT: To make it clear, I don't recommend using this AT ALL, it will break, it's a mess, it won't help you in any way, but it's doable for entertainment/education purposes.
You can hack around with the inspect module, I don't recommend that, but you can do it...
import inspect
def foo(a, f, b):
frame = inspect.currentframe()
frame = inspect.getouterframes(frame)[1]
string = inspect.getframeinfo(frame[0]).code_context[0].strip()
args = string[string.find('(') + 1:-1].split(',')
names = []
for i in args:
if i.find('=') != -1:
names.append(i.split('=')[1].strip())
else:
names.append(i)
print names
def main():
e = 1
c = 2
foo(e, 1000, b = c)
main()
Output:
['e', '1000', 'c']
To add to Michael Mrozek's answer, you can extract the exact parameters versus the full code by:
import re
import traceback
def func(var):
stack = traceback.extract_stack()
filename, lineno, function_name, code = stack[-2]
vars_name = re.compile(r'\((.*?)\).*$').search(code).groups()[0]
print vars_name
return
foobar = "foo"
func(foobar)
# PRINTS: foobar
Looks like Ivo beat me to inspect, but here's another implementation:
import inspect
def varName(var):
lcls = inspect.stack()[2][0].f_locals
for name in lcls:
if id(var) == id(lcls[name]):
return name
return None
def foo(x=None):
lcl='not me'
return varName(x)
def bar():
lcl = 'hi'
return foo(lcl)
bar()
# 'lcl'
Of course, it can be fooled:
def baz():
lcl = 'hi'
x='hi'
return foo(lcl)
baz()
# 'x'
Moral: don't do it.
Another way you can try if you know what the calling code will look like is to use traceback:
def func(var):
stack = traceback.extract_stack()
filename, lineno, function_name, code = stack[-2]
code will contain the line of code that was used to call func (in your example, it would be the string func(foobar)). You can parse that to pull out the argument
You can't. It's evaluated before being passed to the function. All you can do is pass it as a string.
#Ivo Wetzel's answer works in the case of function call are made in one line, like
e = 1 + 7
c = 3
foo(e, 100, b=c)
In case that function call is not in one line, like:
e = 1 + 7
c = 3
foo(e,
1000,
b = c)
below code works:
import inspect, ast
def foo(a, f, b):
frame = inspect.currentframe()
frame = inspect.getouterframes(frame)[1]
string = inspect.findsource(frame[0])[0]
nodes = ast.parse(''.join(string))
i_expr = -1
for (i, node) in enumerate(nodes.body):
if hasattr(node, 'value') and isinstance(node.value, ast.Call)
and hasattr(node.value.func, 'id') and node.value.func.id == 'foo' # Here goes name of the function:
i_expr = i
break
i_expr_next = min(i_expr + 1, len(nodes.body)-1)
lineno_start = nodes.body[i_expr].lineno
lineno_end = nodes.body[i_expr_next].lineno if i_expr_next != i_expr else len(string)
str_func_call = ''.join([i.strip() for i in string[lineno_start - 1: lineno_end]])
params = str_func_call[str_func_call.find('(') + 1:-1].split(',')
print(params)
You will get:
[u'e', u'1000', u'b = c']
But still, this might break.
You can use python-varname package
from varname import nameof
s = 'Hey!'
print (nameof(s))
Output:
s
Package below:
https://github.com/pwwang/python-varname
For posterity, here's some code I wrote for this task, in general I think there is a missing module in Python to give everyone nice and robust inspection of the caller environment. Similar to what rlang eval framework provides for R.
import re, inspect, ast
#Convoluted frame stack walk and source scrape to get what the calling statement to a function looked like.
#Specifically return the name of the variable passed as parameter found at position pos in the parameter list.
def _caller_param_name(pos):
#The parameter name to return
param = None
#Get the frame object for this function call
thisframe = inspect.currentframe()
try:
#Get the parent calling frames details
frames = inspect.getouterframes(thisframe)
#Function this function was just called from that we wish to find the calling parameter name for
function = frames[1][3]
#Get all the details of where the calling statement was
frame,filename,line_number,function_name,source,source_index = frames[2]
#Read in the source file in the parent calling frame upto where the call was made
with open(filename) as source_file:
head=[source_file.next() for x in xrange(line_number)]
source_file.close()
#Build all lines of the calling statement, this deals with when a function is called with parameters listed on each line
lines = []
#Compile a regex for matching the start of the function being called
regex = re.compile(r'\.?\s*%s\s*\(' % (function))
#Work backwards from the parent calling frame line number until we see the start of the calling statement (usually the same line!!!)
for line in reversed(head):
lines.append(line.strip())
if re.search(regex, line):
break
#Put the lines we have groked back into sourcefile order rather than reverse order
lines.reverse()
#Join all the lines that were part of the calling statement
call = "".join(lines)
#Grab the parameter list from the calling statement for the function we were called from
match = re.search('\.?\s*%s\s*\((.*)\)' % (function), call)
paramlist = match.group(1)
#If the function was called with no parameters raise an exception
if paramlist == "":
raise LookupError("Function called with no parameters.")
#Use the Python abstract syntax tree parser to create a parsed form of the function parameter list 'Name' nodes are variable names
parameter = ast.parse(paramlist).body[0].value
#If there were multiple parameters get the positional requested
if type(parameter).__name__ == 'Tuple':
#If we asked for a parameter outside of what was passed complain
if pos >= len(parameter.elts):
raise LookupError("The function call did not have a parameter at postion %s" % pos)
parameter = parameter.elts[pos]
#If there was only a single parameter and another was requested raise an exception
elif pos != 0:
raise LookupError("There was only a single calling parameter found. Parameter indices start at 0.")
#If the parameter was the name of a variable we can use it otherwise pass back None
if type(parameter).__name__ == 'Name':
param = parameter.id
finally:
#Remove the frame reference to prevent cyclic references screwing the garbage collector
del thisframe
#Return the parameter name we found
return param
If you want a Key Value Pair relationship, maybe using a Dictionary would be better?
...or if you're trying to create some auto-documentation from your code, perhaps something like Doxygen (http://www.doxygen.nl/) could do the job for you?
I wondered how IceCream solves this problem. So I looked into the source code and came up with the following (slightly simplified) solution. It might not be 100% bullet-proof (e.g. I dropped get_text_with_indentation and I assume exactly one function argument), but it works well for different test cases. It does not need to parse source code itself, so it should be more robust and simpler than previous solutions.
#!/usr/bin/env python3
import inspect
from executing import Source
def func(var):
callFrame = inspect.currentframe().f_back
callNode = Source.executing(callFrame).node
source = Source.for_frame(callFrame)
expression = source.asttokens().get_text(callNode.args[0])
print(expression, '=', var)
i = 1
f = 2.0
dct = {'key': 'value'}
obj = type('', (), {'value': 42})
func(i)
func(f)
func(s)
func(dct['key'])
func(obj.value)
Output:
i = 1
f = 2.0
s = string
dct['key'] = value
obj.value = 42
Update: If you want to move the "magic" into a separate function, you simply have to go one frame further back with an additional f_back.
def get_name_of_argument():
callFrame = inspect.currentframe().f_back.f_back
callNode = Source.executing(callFrame).node
source = Source.for_frame(callFrame)
return source.asttokens().get_text(callNode.args[0])
def func(var):
print(get_name_of_argument(), '=', var)
If you want to get the caller params as in #Matt Oates answer answer without using the source file (ie from Jupyter Notebook), this code (combined from #Aeon answer) will do the trick (at least in some simple cases):
def get_caller_params():
# get the frame object for this function call
thisframe = inspect.currentframe()
# get the parent calling frames details
frames = inspect.getouterframes(thisframe)
# frame 0 is the frame of this function
# frame 1 is the frame of the caller function (the one we want to inspect)
# frame 2 is the frame of the code that calls the caller
caller_function_name = frames[1][3]
code_that_calls_caller = inspect.findsource(frames[2][0])[0]
# parse code to get nodes of abstract syntact tree of the call
nodes = ast.parse(''.join(code_that_calls_caller))
# find the node that calls the function
i_expr = -1
for (i, node) in enumerate(nodes.body):
if _node_is_our_function_call(node, caller_function_name):
i_expr = i
break
# line with the call start
idx_start = nodes.body[i_expr].lineno - 1
# line with the end of the call
if i_expr < len(nodes.body) - 1:
# next expression marks the end of the call
idx_end = nodes.body[i_expr + 1].lineno - 1
else:
# end of the source marks the end of the call
idx_end = len(code_that_calls_caller)
call_lines = code_that_calls_caller[idx_start:idx_end]
str_func_call = ''.join([line.strip() for line in call_lines])
str_call_params = str_func_call[str_func_call.find('(') + 1:-1]
params = [p.strip() for p in str_call_params.split(',')]
return params
def _node_is_our_function_call(node, our_function_name):
node_is_call = hasattr(node, 'value') and isinstance(node.value, ast.Call)
if not node_is_call:
return False
function_name_correct = hasattr(node.value.func, 'id') and node.value.func.id == our_function_name
return function_name_correct
You can then run it as this:
def test(*par_values):
par_names = get_caller_params()
for name, val in zip(par_names, par_values):
print(name, val)
a = 1
b = 2
string = 'text'
test(a, b,
string
)
to get the desired output:
a 1
b 2
string text
Since you can have multiple variables with the same content, instead of passing the variable (content), it might be safer (and will be simpler) to pass it's name in a string and get the variable content from the locals dictionary in the callers stack frame. :
def displayvar(name):
import sys
return name+" = "+repr(sys._getframe(1).f_locals[name])
If it just so happens that the variable is a callable (function), it will have a __name__ property.
E.g. a wrapper to log the execution time of a function:
def time_it(func, *args, **kwargs):
start = perf_counter()
result = func(*args, **kwargs)
duration = perf_counter() - start
print(f'{func.__name__} ran in {duration * 1000}ms')
return result
Not sure if this makes sense at all, but here's an example:
Let's say I have a script. In this script, I create a list
list = [1,2,3,4]
Maybe I just don't have the technical vocabulary to find what I'm looking for, but is there any way I could set some logging up so that any time I created a variable I could store information in a log file? Given the above example, maybe I'd want to see how many elements are in the list?
I understand that I could simply write a function and call that over and over again, but let's say I might want to know information about a ton of different data types, not just lists. It wouldn't be clean to call a function repeatedly.
this is hackery but what the heck
class _LoggeryType(type):
def __setattr__(cls,attr,value):
print("SET VAR: {0} = {1}".format(attr,value))
globals().update({attr:value})
# Python3
class Loggery(metaclass=_LoggeryType):
pass
# python2
class Loggery:
__metaclass__=_LoggeryType
Loggery.x = 5
print("OK set X={0}".format(x))
note i wouldn't really recommend using this
One method would be to use the powerful sys.settrace. I've written up a small (but somewhat incomplete) example:
tracer.py:
import inspect
import sys
import os
import logging
logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger('tracing-logger')
FILES_TO_TRACE = [os.path.basename(__file__), 'tracee.py']
print(FILES_TO_TRACE)
def new_var(name, value, context):
logger.debug(f"New {context} variable called {name} = {value}")
# do some analysis here, for example
if type(value) == list:
logger.debug(f"\tNumber of elements: {len(value)}")
def changed_var(name, value, context):
logger.debug(f"{context} variable called {name} of was changed to: {value}")
def make_tracing_func():
current_locals = {}
current_globals = {}
first_line_executed = False
def tracing_func(frame, event, arg):
nonlocal first_line_executed
frame_info = inspect.getframeinfo(frame)
filename = os.path.basename(frame_info.filename)
line_num = frame_info.lineno
if event == 'line':
# check for difference with locals
for var_name in frame.f_code.co_varnames:
if var_name in frame.f_locals:
var_value = frame.f_locals[var_name]
if var_name not in current_locals:
current_locals[var_name] = var_value
new_var(var_name, var_value, 'local')
elif current_locals[var_name] != var_value:
current_locals[var_name] = var_value
changed_var(var_name, var_value, 'local')
for var_name, var_value in frame.f_globals.items():
if var_name not in current_globals:
current_globals[var_name] = var_value
if first_line_executed:
new_var(var_name, var_value, 'global')
elif current_globals[var_name] != var_value:
current_globals[var_name] = var_value
changed_var(var_name, var_value, 'global')
first_line_executed = True
return tracing_func
elif event == 'call':
if os.path.basename(filename) in FILES_TO_TRACE:
return make_tracing_func()
return None
return tracing_func
sys.settrace(make_tracing_func())
import tracee
tracee.py
my_list = [1, 2, 3, 4]
a = 3
print("tracee: I have a list!", my_list)
c = a + sum(my_list)
print("tracee: A number:", c)
c = 12
print("tracee: I changed it:", c)
Output:
DEBUG:tracing-logger:New global variable called my_list = [1, 2, 3, 4]
DEBUG:tracing-logger: Number of elements: 4
DEBUG:tracing-logger:New global variable called a = 3
tracee: I have a list! [1, 2, 3, 4]
DEBUG:tracing-logger:New global variable called c = 13
tracee: A number: 13
DEBUG:tracing-logger:global variable called c was changed to: 12
tracee: I changed it: 12
There are some additional cases you may want to handle (duplicated changes to globals due to function calls, closure variables, etc.). You can also use linecache to find the contents of the lines, or use the line_num variable in the logging.
I have a python function that runs other functions.
def main():
func1(a,b)
func2(*args,*kwargs)
func3()
Now I want to apply exceptions on main function. If there was an exception in any of the functions inside main, the function should not stop but continue executing next line. In other words, I want the below functionality
def main():
try:
func1()
except:
pass
try:
func2()
except:
pass
try:
func3()
except:
pass
So is there any way to loop through each statement inside main function and apply exceptions on each line.
for line in main_function:
try:
line
except:
pass
I just don't want to write exceptions inside the main function.
Note : How to prevent try catching every possible line in python? this question comes close to solving this problem, but I can't figure out how to loop through lines in a function.
If you have any other way to do this other than looping, that would help too.
What you want is on option that exists in some languages where an exception handler can choose to proceed on next exception. This used to lead to poor code and AFAIK has never been implemented in Python. The rationale behind is that you must explicitely say how you want to process an exception and where you want to continue.
In your case, assuming that you have a function called main that only calls other function and is generated automatically, my advice would be to post process it between its generation and its execution. The inspect module can even allow to do it at run time:
def filter_exc(func):
src = inspect.getsource(func)
lines = src.split('\n')
out = lines[0] + "\n"
for line in lines[1:]:
m = re.match('(\s*)(.*)', line)
lead, text = m.groups()
# ignore comments and empty lines
if not (text.startswith('#') or text.strip() == ""):
out += lead + "try:\n"
out += lead + " " + text + "\n"
out += lead + "except:\n" + lead + " pass\n"
return out
You can then use the evil exec (the input in only the source from your function):
exec(filter_exc(main)) # replaces main with the filtered version
main() # will ignore exceptions
After your comment, you want a more robust solution that can cope with multi line statements and comments. In that case, you need to actually parse the source and modify the parsed tree. ast module to the rescue:
class ExceptFilter(ast.NodeTransformer):
def visit_Expr(self, node):
self.generic_visit(node)
if isinstance(node.value, ast.Call): # filter all function calls
# print(node.value.func.id)
# use a dummy try block
n = ast.parse("""try:
f()
except:
pass""").body[0]
n.body[0] = node # make the try call the real function
return n # and use it
return node # keep other nodes unchanged
With that example code:
def func1():
print('foo')
def func2():
raise Exception("Test")
def func3(x):
print("f3", x)
def main():
func1()
# this is a comment
a = 1
if a == 1: # this is a multi line statement
func2()
func3("bar")
we get:
>>> node = ast.parse(inspect.getsource(main))
>>> exec(compile(ExceptFilter().visit(node), "", mode="exec"))
>>> main()
foo
f3 bar
In that case, the unparsed node(*) write as:
def main():
try:
func1()
except:
pass
a = 1
if (a == 1):
try:
func2()
except:
pass
try:
func3('bar')
except:
pass
Alternatively it is also possible to wrap every top level expression:
>>> node = ast.parse(inspect.getsource(main))
>>> for i in range(len(node.body[0].body)): # process top level expressions
n = ast.parse("""try:
f()
except:
pass""").body[0]
n.body[0] = node.body[0].body[i]
node.body[0].body[i] = n
>>> exec(compile(node, "", mode="exec"))
>>> main()
foo
f3 bar
Here the unparsed tree writes:
def main():
try:
func1()
except:
pass
try:
a = 1
except:
pass
try:
if (a == 1):
func2()
except:
pass
try:
func3('bar')
except:
pass
BEWARE: there is an interesting corner case if you use exec(compile(... in a function. By default exec(code) is exec(code, globals(), locals()). At top level, local and global dictionary is the same dictionary, so the top level function is correctly replaced. But if you do the same in a function, you only create a local function with the same name that can only be called from the function (it will go out of scope when the function will return) as locals()['main'](). So you must either alter the global function by passing explicitely the global dictionary:
exec(compile(ExceptFilter().visit(node), "", mode="exec"), globals(), globals())
or return the modified function without altering the original one:
def myfun():
# print(main)
node = ast.parse(inspect.getsource(main))
exec(compile(ExceptFilter().visit(node), "", mode="exec"))
# print(main, locals()['main'], globals()['main'])
return locals()['main']
>>> m2 = myfun()
>>> m2()
foo
f3 bar
(*) Python 3.6 contains an unparser in Tools/parser, but a simpler to use version exists in pypi
You could use a callback, like this:
def main(list_of_funcs):
for func in list_of_funcs:
try:
func()
except Exception as e:
print(e)
if __name__ == "__main__":
main([func1, func2, func3])
How could I inspect the values of function arguments inside the sys.settrace call? It seems that I have possibility to output pretty much everything (lines, filenames, stacktraces, return values, etc) except arguments. Is there a workaround that would allow me to track function argument values as well?
You can use the combination of Code Objects and Frame Objects.
See for the descriptions of these in the Python Data-Model Reference.
import sys
def fn(frame, msg, arg):
if msg != 'call': return
# Filter as appropriate
if frame.f_code.co_filename.startswith("/usr"): return
print("Called", frame.f_code.co_name)
for i in range(frame.f_code.co_argcount):
name = frame.f_code.co_varnames[i]
print(" Argument", name, "is", frame.f_locals[name])
sys.settrace(fn)
def hai(a, b, c):
print(a, b, c)
hai("Hallo", "Welt", "!")
The crucial thing to realize is that
we can see all local variables in the frame as f_locals.
We can extract the names of the variables in the parameter list from f_code.co_varnames.
I turned Marcs answer into a script which can be used for inspecting other scripts:
print_func_calls.py:
#!/usr/bin/env python
import sys
# opt-out file names which start with one of these prefixes
FILENAME_FILTER = {"/usr", "<"}
# opt-in file names again which match one of these prefixes
FILENAME_UNFILTER = {"/lib/python/some-important-module"}
# opt-out function names
FN_NAME_FILTER = {"<module>", "__new__", "__setattr__", "<lambda>"}
def to_str(exp):
"""Turn an argument value into a string without dying on exceptions"""
try:
return repr(exp)[:100]
except Exception as exc:
return "n/a"
def fn(frame, msg, arg):
if msg != 'call':
return
filename, fn_name = frame.f_code.co_filename, frame.f_code.co_name
if (not all(not filename.startswith(p) for p in FILENAME_FILTER) and
all(not filename.startswith(p) for p in FILENAME_UNFILTER) or
fn_name in FN_NAME_FILTER):
return
argstr = ", ".join("%s=%s" % (
frame.f_code.co_varnames[i], to_str(frame.f_locals[frame.f_code.co_varnames[i]]))
for i in range(frame.f_code.co_argcount))
print(">>> %s::\033[37m%s\033[0m(%s)" % (filename, fn_name, argstr))
sys.settrace(fn)
sys.argv = sys.argv[1:]
exec(open(sys.argv[0]).read())
Use it like this:
print_func_calls.py my-script.py arg1..argN
I've been using the following code to trace the execution of my programs:
import sys
import linecache
import random
def traceit(frame, event, arg):
if event == "line":
lineno = frame.f_lineno
filename = frame.f_globals["__file__"]
if filename == "<stdin>":
filename = "traceit.py"
if (filename.endswith(".pyc") or
filename.endswith(".pyo")):
filename = filename[:-1]
name = frame.f_globals["__name__"]
line = linecache.getline(filename, lineno)
print "%s:%s:%s: %s" % (name, lineno,frame.f_code.co_name , line.rstrip())
return traceit
def main():
print "In main"
for i in range(5):
print i, random.randrange(0, 10)
print "Done."
sys.settrace(traceit)
main()
Using this code, or something like it, is it possible to report the values of certain function arguments? In other words, the above code tells me "which" functions were called and I would like to know "what" the corresponding values of the input variables for those function calls.
Thanks in advance.
The traceit function that you posted can be used to print information as each line of code is executed. If all you need is the function name and arguments when certain functions are called, I would suggest using this trace decorator instead:
import functools
def trace(f):
'''This decorator shows how the function was called'''
#functools.wraps(f)
def wrapper(*arg,**kw):
arg_str=','.join(['%r'%a for a in arg]+['%s=%s'%(key,kw[key]) for key in kw])
print "%s(%s)" % (f.__name__, arg_str)
return f(*arg, **kw)
return wrapper
You could use it as follows:
#trace
def foo(*args,**kws):
pass
foo(1)
# foo(1)
foo(y=1)
# foo(y=1)
foo(1,2,3)
# foo(1,2,3)
Edit: Here is an example using trace and traceit in conjunction:
Below, trace is used in 2 different ways. The normal way is to decorate functions you define:
#trace
def foo(i):
....
But you can also "monkey-patch" any function whether you defined it or not like this:
random.randrange=trace(random.randrange)
So here's the example:
import sys
import linecache
import random
import functools
def trace(f):
'''This decorator shows how the function was called'''
#functools.wraps(f)
def wrapper(*arg,**kw):
arg_str=','.join(['%r'%a for a in arg]+['%s=%s'%(key,kw[key]) for key in kw])
print "%s(%s)" % (f.__name__, arg_str)
return f(*arg, **kw)
return wrapper
def traceit(frame, event, arg):
if event == "line":
lineno = frame.f_lineno
filename = frame.f_globals["__file__"]
if filename == "<stdin>":
filename = "traceit.py"
if (filename.endswith(".pyc") or
filename.endswith(".pyo")):
filename = filename[:-1]
name = frame.f_globals["__name__"]
line = linecache.getline(filename, lineno)
print "%s:%s:%s: %s" % (name, lineno,frame.f_code.co_name , line.rstrip())
return traceit
random.randrange=trace(random.randrange)
#trace
def foo(i):
print i, random.randrange(0, 10)
def main():
print "In main"
for i in range(5):
foo(i)
print "Done."
sys.settrace(traceit)
main()
frame.f_locals will give you the values of the local variables, and I guess you could keep track of the last frame you've seen and if frame.f_back is not the lastframe dump frame.f_locals.
I'd predict though that you're pretty quickly going be snowed under with too much data doing this.
Here's your code modified to do this:
import sys
import linecache
import random
class Tracer(object):
def __init__(self):
self.lastframe = None
def traceit(self, frame, event, arg):
if event == "line":
lineno = frame.f_lineno
filename = frame.f_globals["__file__"]
if filename == "<stdin>":
filename = "traceit.py"
if (filename.endswith(".pyc") or
filename.endswith(".pyo")):
filename = filename[:-1]
name = frame.f_globals["__name__"]
line = linecache.getline(filename, lineno)
if frame.f_back is self.lastframe:
print "%s:%s:%s: %s" % (name, lineno,frame.f_code.co_name , line.rstrip())
else:
print "%s:%s:%s(%s)" % (name, lineno,frame.f_code.co_name , str.join(', ', ("%s=%r" % item for item in frame.f_locals.iteritems())))
print "%s:%s:%s: %s" % (name, lineno,frame.f_code.co_name , line.rstrip())
#print frame.f_locals
self.lastframe = frame.f_back
return self.traceit
def main():
print "In main"
for i in range(5):
print i, random.randrange(0, 10)
print "Done."
sys.settrace(Tracer().traceit)
main()
web.py had a method called "upvars" that did something similar, taking in the variables from the calling frame. Note the comment:
def upvars(level=2):
"""Guido van Rossum sez: don't use this function."""
return dictadd(
sys._getframe(level).f_globals,
sys._getframe(level).f_locals)
What is much more useful to me in tracing than dumping ALL the state of variables at the time of execution is to do an eval of each and every line of code, ie:
for modname in modnames: | for init in ., init, encoding
|
if not modname or '.' in modname: | if not init or '.' in init
continue | continue
|
try: |
ie: where real line of code is on the left and each line of code is on the right. I've implemented this in perl and it is a LIFESAVER there. I'm in the process of trying to implement this in python but I'm not as familiar with the language so it will take a bit of time.
Anyways, if anybody has ideas how to implement this, I'd love to hear them. As far as I can tell, it comes down to this function
interpolate_frame(frame, string)
where is the frame passed to the trace function, and the string is the line of code to be interpolated with variables in the current frame. Then, the above code becomes:
print "%s:%s:%s: %s|%s" % (name, lineno,frame.f_code.co_name,
padded(line.rstrip(),10),padded(interpolate_frame(frame, line.rstrip()),100)
I'm going to try to hack this up, but again, if anybody has ideas on this I'm welcome to hear them.