I'm working on a script that takes a few minutes to run, and would like to provide some output to the user about its progress. Unfortunately i'm exceedingly lazy. what I'd like to do is write a function without the logging, and then apply a decorator to it which steps through the function and prints each line before executing that line. Basically what I'm looking for is a loggingdecorator such that:
>>> #loggingdecorator
... def myfunction():
... foo()
... bar()
... baz()
>>> myfunction()
Starting myfunction
foo() ... [OK]
bar() ... [OK]
baz() ... [OK]
myfunction Done!
Here's what I've tried so far:
import sys
def logging_tracer(frame, event, arg):
def local_tracer(local_frame, event, arg):
if frame is local_frame:
print frame.f_code.co_name, event, arg
print frame.f_code.co_name, event, arg
return local_tracer
def loggingdecorator(func):
def _wrapper():
old_trace_function = sys.gettrace()
sys.settrace(logging_tracer)
try:
result = func()
except:
raise
else:
return result
finally:
sys.settrace(old_trace_function)
return _wrapper
Unfortunately, this prints a bit too much; it follows function calls and prints them out, line by line as well (well, this doesn't actually print the source line, existing answers using inspect, combined with stuff on the frame object in the trace function would do it), but I'm a bit stumped as to how the logging_tracer unless the function in question is actually decorated.
So here's what I came up with. #Corley Brigman's comment got me started in the right direction. This is a bit hackey, sys.gettrace/settrace are rather promenently documented as "CPython implementation details", and so this solution should not be expected to work in other implementations. That said, it seems to work out pretty well. The tracing functionality in cpython does not provide any notification of "line finished executing", so the [ok] from my question doesn't really make any sense.
Fixing the recursive tracing issue is just a simple matter of keeping a cache of the functions that were decorated, and then only produce output if the frame being traced is from a function that was decorated.
import inspect
import sys
def logging_tracer(frame, event, arg):
lines, firstline = inspect.getsourcelines(frame)
def local_tracer(local_frame, event, arg):
if event == 'line' and frame is local_frame:
print event, frame.f_lineno,'\t', lines[frame.f_lineno - firstline]
#print event, lines[frame.f_lineno - firstline]
#print frame.f_code.co_name, frame.f_lineno, event, arg
if frame.f_code in LOG_THESE_FUNCTIONS:
print event, frame.f_lineno,'\t', lines[frame.f_lineno - firstline + (event == 'call')]
#print frame.f_code.co_name, event, arg
return local_tracer
else:
return None
LOG_THESE_FUNCTIONS = set()
def loggingdecorator(func):
LOG_THESE_FUNCTIONS.add(func.func_code)
def _wrapper():
old_trace_function = sys.gettrace()
sys.settrace(logging_tracer)
try:
result = func()
except:
raise
else:
return result
finally:
sys.settrace(old_trace_function)
return _wrapper
Here's an ugly example that works if there's only one indentation level:
import inspect
def myDec(func):
temp = list(inspect.getsourcelines(func)[0])
temp.append(' print("{} Done!")\n'.format(func.__name__))
for index in range(len(temp)-2, 0, -1):
temp.insert(index+1, " print('''{}...[OK]''')\n".format(temp[index].strip().rstrip("\n")))
temp.insert(1, ' print("Starting {}")\n'.format(func.__name__))
temp = "".join(temp)
exec(temp)
return locals()[func.__name__]
def foo():
a = 4+5
list_int = range(100)
foo = myDec(foo)
foo()
Really ugly though...
You can do something like this:
>>> class loggingdecorator:
... def __init__(self, func):
... self.func = func
... def __call__(self):
... print "Entering", self.func.__name__
... self.func()
... print "Exited", self.func.__name__
...
Then with each of your functions:
>>> #loggingdecorator
... def func1():
... print "Hello from func1(), how are you doing?"
...
>>> #loggingdecorator
... def func2():
... print "Hello from func2(), Whaddup Dawg...?"
...
>>> func1()
Entering func1
Hello from func1(), how are you doing?
Exited func1
>>> func2()
Entering func2
Hello from func2(), Whaddup Dawg...?
Exited func2
Found the example from this page
Related
I'm trying to build a decorator-based dispatcher, such as you find used by Flask or Pyramid. I got something that works, but ran into a bit of a catch-22. The following code works, but only because foo() gets executed and sets the .mq_path-attribute. When starting the application and building a list of the dispatchable functions no attributes are thus set yet. I want to execute foo() driven by events.
I could "manually" prepare a list of functions ahead and updated as I add functions, but I enjoy the way Flask works, by just adding a decorator to a function that handles an URL (or in this case a MQ path).
list_of_paths = []
path_dispatcher = {}
def handle_mq(path):
def decorator(fn):
def decorated(*args,**kwargs):
decorated.mq_path = path
print "Hello from the handle_mq() decorator, your path is: {0}".format(path)
return fn(*args,**kwargs)
return decorated
return decorator
#handle_mq('/some/path')
def foo():
print "foo!"
foo() # <- this code only works if I first call the decorated function
for k, v in globals().items():
if hasattr(v, 'mq_path'):
list_of_paths.append(v.mq_path)
path_dispatcher[v.mq_path] = v
print list_of_paths
print path_dispatcher
path_dispatcher['/some/path']()
So basically the question is, how to gather a list of the decorated functions before they are first executed?
I'm on Python 2.7.
I found the answer!
list_of_paths = []
path_dispatcher = {}
def handle_mq(path):
def decorator(fn):
list_of_paths.append(path)
path_dispatcher[path] = fn
def decorated(*args,**kwargs):
print "Hello from handl_mq decorator, your path is: {0}".format(path)
return fn(*args,**kwargs)
return decorated
return decorator
#handle_mq('/some/path')
def foo():
print "foo!"
print list_of_paths
print path_dispatcher
path_dispatcher['/some/path']()
Python: How to get the caller's method name in the called method?
Assume I have 2 methods:
def method1(self):
...
a = A.method2()
def method2(self):
...
If I don't want to do any change for method1, how to get the name of the caller (in this example, the name is method1) in method2?
inspect.getframeinfo and other related functions in inspect can help:
>>> import inspect
>>> def f1(): f2()
...
>>> def f2():
... curframe = inspect.currentframe()
... calframe = inspect.getouterframes(curframe, 2)
... print('caller name:', calframe[1][3])
...
>>> f1()
caller name: f1
this introspection is intended to help debugging and development; it's not advisable to rely on it for production-functionality purposes.
Shorter version:
import inspect
def f1(): f2()
def f2():
print 'caller name:', inspect.stack()[1][3]
f1()
(with thanks to #Alex, and Stefaan Lippen)
This seems to work just fine:
import sys
print sys._getframe().f_back.f_code.co_name
I would use inspect.currentframe().f_back.f_code.co_name. Its use hasn't been covered in any of the prior answers which are mainly of one of three types:
Some prior answers use inspect.stack but it's known to be too slow.
Some prior answers use sys._getframe which is an internal private function given its leading underscore, and so its use is implicitly discouraged.
One prior answer uses inspect.getouterframes(inspect.currentframe(), 2)[1][3] but it's entirely unclear what [1][3] is accessing.
import inspect
from types import FrameType
from typing import cast
def demo_the_caller_name() -> str:
"""Return the calling function's name."""
# Ref: https://stackoverflow.com/a/57712700/
return cast(FrameType, cast(FrameType, inspect.currentframe()).f_back).f_code.co_name
if __name__ == '__main__':
def _test_caller_name() -> None:
assert demo_the_caller_name() == '_test_caller_name'
_test_caller_name()
Note that cast(FrameType, frame) is used to satisfy mypy.
Acknowlegement: comment by 1313e for an answer.
I've come up with a slightly longer version that tries to build a full method name including module and class.
https://gist.github.com/2151727 (rev 9cccbf)
# Public Domain, i.e. feel free to copy/paste
# Considered a hack in Python 2
import inspect
def caller_name(skip=2):
"""Get a name of a caller in the format module.class.method
`skip` specifies how many levels of stack to skip while getting caller
name. skip=1 means "who calls me", skip=2 "who calls my caller" etc.
An empty string is returned if skipped levels exceed stack height
"""
stack = inspect.stack()
start = 0 + skip
if len(stack) < start + 1:
return ''
parentframe = stack[start][0]
name = []
module = inspect.getmodule(parentframe)
# `modname` can be None when frame is executed directly in console
# TODO(techtonik): consider using __main__
if module:
name.append(module.__name__)
# detect classname
if 'self' in parentframe.f_locals:
# I don't know any way to detect call from the object method
# XXX: there seems to be no way to detect static method call - it will
# be just a function call
name.append(parentframe.f_locals['self'].__class__.__name__)
codename = parentframe.f_code.co_name
if codename != '<module>': # top level usually
name.append( codename ) # function or a method
## Avoid circular refs and frame leaks
# https://docs.python.org/2.7/library/inspect.html#the-interpreter-stack
del parentframe, stack
return ".".join(name)
Bit of an amalgamation of the stuff above. But here's my crack at it.
def print_caller_name(stack_size=3):
def wrapper(fn):
def inner(*args, **kwargs):
import inspect
stack = inspect.stack()
modules = [(index, inspect.getmodule(stack[index][0]))
for index in reversed(range(1, stack_size))]
module_name_lengths = [len(module.__name__)
for _, module in modules]
s = '{index:>5} : {module:^%i} : {name}' % (max(module_name_lengths) + 4)
callers = ['',
s.format(index='level', module='module', name='name'),
'-' * 50]
for index, module in modules:
callers.append(s.format(index=index,
module=module.__name__,
name=stack[index][3]))
callers.append(s.format(index=0,
module=fn.__module__,
name=fn.__name__))
callers.append('')
print('\n'.join(callers))
fn(*args, **kwargs)
return inner
return wrapper
Use:
#print_caller_name(4)
def foo():
return 'foobar'
def bar():
return foo()
def baz():
return bar()
def fizz():
return baz()
fizz()
output is
level : module : name
--------------------------------------------------
3 : None : fizz
2 : None : baz
1 : None : bar
0 : __main__ : foo
You can use decorators, and do not have to use stacktrace
If you want to decorate a method inside a class
import functools
# outside ur class
def printOuterFunctionName(func):
#functools.wraps(func)
def wrapper(self):
print(f'Function Name is: {func.__name__}')
func(self)
return wrapper
class A:
#printOuterFunctionName
def foo():
pass
you may remove functools, self if it is procedural
An alternative to sys._getframe() is used by Python's Logging library to find caller information. Here's the idea:
raise an Exception
immediately catch it in an Except clause
use sys.exc_info to get Traceback frame (tb_frame).
from tb_frame get last caller's frame using f_back.
from last caller's frame get the code object that was being executed in that frame.
In our sample code it would be method1 (not method2) being executed.
From code object obtained, get the object's name -- this is caller method's name in our sample.
Here's the sample code to solve example in the question:
def method1():
method2()
def method2():
try:
raise Exception
except Exception:
frame = sys.exc_info()[2].tb_frame.f_back
print("method2 invoked by: ", frame.f_code.co_name)
# Invoking method1
method1()
Output:
method2 invoked by: method1
Frame has all sorts of details, including line number, file name, argument counts, argument type and so on. The solution works across classes and modules too.
Code:
#!/usr/bin/env python
import inspect
called=lambda: inspect.stack()[1][3]
def caller1():
print "inside: ",called()
def caller2():
print "inside: ",called()
if __name__=='__main__':
caller1()
caller2()
Output:
shahid#shahid-VirtualBox:~/Documents$ python test_func.py
inside: caller1
inside: caller2
shahid#shahid-VirtualBox:~/Documents$
I found a way if you're going across classes and want the class the method belongs to AND the method. It takes a bit of extraction work but it makes its point. This works in Python 2.7.13.
import inspect, os
class ClassOne:
def method1(self):
classtwoObj.method2()
class ClassTwo:
def method2(self):
curframe = inspect.currentframe()
calframe = inspect.getouterframes(curframe, 4)
print '\nI was called from', calframe[1][3], \
'in', calframe[1][4][0][6: -2]
# create objects to access class methods
classoneObj = ClassOne()
classtwoObj = ClassTwo()
# start the program
os.system('cls')
classoneObj.method1()
Hey mate I once made 3 methods without plugins for my app and maybe that can help you, It worked for me so maybe gonna work for you too.
def method_1(a=""):
if a == "method_2":
print("method_2")
if a == "method_3":
print("method_3")
def method_2():
method_1("method_2")
def method_3():
method_1("method_3")
method_2()
I am trying to learn decorators by following the 'python decorators in 12 steps'. http://simeonfranklin.com/blog/2012/jul/1/python-decorators-in-12-steps/
I tried replicating one of the In one of the code examples:
def outer(some_func):
def inner():
print ("before some_func")
ret = some_func()
return ret + 1
return inner
def foo():
return 1
def main():
decorated = outer(foo)
decorated()
if __name__ == "__main__": main()
This results in :
before some_func
In the example under Decorators!
>>> def outer(some_func):
... def inner():
... print ("before some_func")
... ret = some_func() # 1
... return ret + 1
... return inner
>>> def foo():
... return 1
>>> decorated = outer(foo) # 2
>>> decorated()
Returns:
before some_func
2
The main difference is that in the example, they are using Python running directly from cmd and I am using the Sublime text with python build as well as using a main() function. In my mind, I feel like these are doing the exact same thing. Is there something different between running code in the cmd versus with sublime that I am not getting?
The interactive Python interpreter automatically prints the results of a line if it is not assigned to a variable. This is useful for debugging. For example, if you call foo(), it will automatically print 1. The call below would not result in any extra prints in the interactive interpreter.
>>> result = decorated()
For example i have this piece of code:
def example():
a = 'goodbye'
if True:
print a
return 1
else:
print a
return 0
I would like to know if there is any possible solution to write once "print a" and execute it before each "return" statement automaticaly. So that if I add more return statements I wouldn't need to add anything, but "print a" would execute. Result would look like something:
def example():
a = "goodbye"
""" some code to implement print a """
if True:
return 1
else:
return 0
Each time there is return statement it still would print a.
I tried to google, but don't know how word query, since all results are about returning multiple values.
UPDATE: My question was answered, thanks to all of you.
Although wrapping functions are correct answer, but I have chosen answer by GingerPlusPlus who suggested to use try...finally for simplicity.
try .. finally:
def example():
try:
if True:
return 1
else:
return 0
finally:
print 'goodbye'
>>> example()
goodbye
1
A finally clause is always executed before leaving the try statement, whether an exception has occurred or not. Docs
You can use a context. Initialize it with the value you want to print. Then print when context exit, i.e. upon return.
class PrinterOnContextExit():
def __init__( self, a ): self.a = a
def __enter__( self ): pass
def __exit__( self, exc_type, exc_value, traceback ): print( self.a )
def example():
a = 'goodbye'
with PrinterOnContextExit( a ):
if True:
return 1
else:
return 0
Note that you cannot print the returned value this way. If you ever wanted to print the returned value, then you should use a decorator.
class PrintOnReturn():
def __init__( self, a ): self.a = a
def __call__( self, func ): return lambda *args, **kwargs: self.callFunc( func, *args, **kwargs )
def callFunc( self, func, *args, **kwargs ): r = func( *args, **kwargs ); print( self.a, r ); return r
#PrintOnReturn( "hello" )
def example():
if True:
return 1
else:
return 0
This will print whatever string you passed to the decorator, followed by the value returned from the decorated function. Here hello 1.
Code:
def example():
a = 'goodbye'
print a
if True:
return 1
else:
return 0
If you print a before if else then it will print value every time you call the function.
You could also use a decorator, if it suits your case:
>>> def decorator(text):
... def wrapped(func):
... def inner(*args, **kwargs):
... result = func(*args, **kwargs)
... print text
... return result
... return inner
... return wrapped
...
>>> #decorator('goodbye')
... def example():
... return True
...
>>> example()
goodbye
>>> True
Decorator will allow you to print any text after the decorated function was called. Or before.
Create a value returnval
returnval = 0 #default value
testval = 0 # Code to set up if
# code to set various values of testval
if testval == 0:
returnval = 1
elif testval == 5:
returnval = 2
else:
returnval = 10
print a
return returnval
Def example():
a = 'goodbye'
if True:
return 1,str(a)
else:
return 0,str(a)
print example()
Thats the only way...I dont think there is a way to avoid typing what you want to be printed...sorry mate! expect if you type a function type the thinks you
An easier alternative as i've also posted here for a similar topic:
def master_example():
a = []
def example():
a.append('goodbye')
if True:
return 1
else:
return 0
example()
print a[0]
I'm actually trying doing this in Java, but I'm in the process of teaching myself python and it made me wonder if there was an easy/clever way to do this with wrappers or something.
I want to know how many times a specific method was called inside another method. For example:
def foo(z):
#do something
return result
def bar(x,y):
#complicated algorithm/logic involving foo
return foobar
So for each call to bar with various parameters, I'd like to know how many times foo was called, perhaps with output like this:
>>> print bar('xyz',3)
foo was called 15 times
[results here]
>>> print bar('stuv',6)
foo was called 23 times
[other results here]
edit: I realize I could just slap a counter inside bar and dump it when I return, but it would be cool if there was some magic you could do with wrappers to accomplish the same thing. It would also mean I could reuse the same wrappers somewhere else without having to modify any code inside the method.
Sounds like almost the textbook example for decorators!
def counted(fn):
def wrapper(*args, **kwargs):
wrapper.called += 1
return fn(*args, **kwargs)
wrapper.called = 0
wrapper.__name__ = fn.__name__
return wrapper
#counted
def foo():
return
>>> foo()
>>> foo.called
1
You could even use another decorator to automate the recording of how many times a function is called inside another function:
def counting(other):
def decorator(fn):
def wrapper(*args, **kwargs):
other.called = 0
try:
return fn(*args, **kwargs)
finally:
print '%s was called %i times' % (other.__name__, other.called)
wrapper.__name__ = fn.__name__
return wrapper
return decorator
#counting(foo)
def bar():
foo()
foo()
>>> bar()
foo was called 2 times
If foo or bar can end up calling themselves, though, you'd need a more complicated solution involving stacks to cope with the recursion. Then you're heading towards a full-on profiler...
Possibly this wrapped decorator stuff, which tends to be used for magic, isn't the ideal place to be looking if you're still ‘teaching yourself Python’!
This defines a decorator to do it:
def count_calls(fn):
def _counting(*args, **kwargs):
_counting.calls += 1
return fn(*args, **kwargs)
_counting.calls = 0
return _counting
#count_calls
def foo(x):
return x
def bar(y):
foo(y)
foo(y)
bar(1)
print foo.calls
After your response - here's a way with a decorator factory...
import inspect
def make_decorators():
# Mutable shared storage...
caller_L = []
callee_L = []
called_count = [0]
def caller_decorator(caller):
caller_L.append(caller)
def counting_caller(*args, **kwargs):
# Returning result here separate from the count report in case
# the result needs to be used...
result = caller(*args, **kwargs)
print callee_L[0].__name__, \
'was called', called_count[0], 'times'
called_count[0] = 0
return result
return counting_caller
def callee_decorator(callee):
callee_L.append(callee)
def counting_callee(*args, **kwargs):
# Next two lines are an alternative to
# sys._getframe(1).f_code.co_name mentioned by Ned...
current_frame = inspect.currentframe()
caller_name = inspect.getouterframes(current_frame)[1][3]
if caller_name == caller_L[0].__name__:
called_count[0] += 1
return callee(*args, **kwargs)
return counting_callee
return caller_decorator, callee_decorator
caller_decorator, callee_decorator = make_decorators()
#callee_decorator
def foo(z):
#do something
return ' foo result'
#caller_decorator
def bar(x,y):
# complicated algorithm/logic simulation...
for i in xrange(x+y):
foo(i)
foobar = 'some result other than the call count that you might use'
return foobar
bar(1,1)
bar(1,2)
bar(2,2)
And here's the output (tested with Python 2.5.2):
foo was called 2 times
foo was called 3 times
foo was called 4 times