Develop Reference

Is it possible to programmatically search Python's middle scopes? [duplicate]

I want to get information about the callers of a specific function in python. For example:
class SomeClass():
def __init__(self, x):
self.x = x
def caller(self):
return special_func(self.x)
def special_func(x):
print "My caller is the 'caller' function in an 'SomeClass' class."
Is it possible with python?

Yes, the sys._getframe() function let's you retrieve frames from the current execution stack, which you can then inspect with the methods and documentation found in the inspect module; you'll be looking for specific locals in the f_locals attribute, as well as for the f_code information:
import sys
def special_func(x):
callingframe = sys._getframe(1)
print 'My caller is the %r function in a %r class' % (
callingframe.f_code.co_name,
callingframe.f_locals['self'].__class__.__name__)
Note that you'll need to take some care to detect what kind of information you find in each frame.
sys._getframe() returns a frame object, you can chain through the whole stack by following the f_back reference on each. Or you can use the inspect.stack() function to produce a lists of frames with additional information.

An example:
def f1(a):
import inspect
print 'I am f1 and was called by', inspect.currentframe().f_back.f_code.co_name
return a
def f2(a):
return f1(a)
Will retrieve the "immediate" caller.
>>> f2(1)
I am f1 and was called by f2
And if wasn't called from another you get (in IDLE):
>>> f1(1)
I am f1 and was called by <module>

Thanks to Jon Clements answer I was able to make a function that returns an ordered list of all callers:
def f1():
names = []
frame = inspect.currentframe()
## Keep moving to next outer frame
while True:
try:
frame = frame.f_back
name = frame.f_code.co_name
names.append(name)
except:
break
return names
and when called in a chain:
def f2():
return f1()
def f3():
return f2()
def f4():
return f3()
print f4()
looks like this:
['f2', 'f3', 'f4', '<module>']
In my case I filter out anything at '<module>' and after, and then take the last item to be the name of the originating caller.
Or modify the original loop to bail at the first appearance of any name starting with '<':
frame = frame.f_back
name = frame.f_code.co_name
if name[0] == '<':
break
names.append(name)

How can I tell what function called my function? [duplicate]

Python: How to get the caller's method name in the called method?
Assume I have 2 methods:
def method1(self):
...
a = A.method2()
def method2(self):
...
If I don't want to do any change for method1, how to get the name of the caller (in this example, the name is method1) in method2?

inspect.getframeinfo and other related functions in inspect can help:
>>> import inspect
>>> def f1(): f2()
...
>>> def f2():
... curframe = inspect.currentframe()
... calframe = inspect.getouterframes(curframe, 2)
... print('caller name:', calframe[1][3])
...
>>> f1()
caller name: f1
this introspection is intended to help debugging and development; it's not advisable to rely on it for production-functionality purposes.

Shorter version:
import inspect
def f1(): f2()
def f2():
print 'caller name:', inspect.stack()[1][3]
f1()
(with thanks to #Alex, and Stefaan Lippen)

This seems to work just fine:
import sys
print sys._getframe().f_back.f_code.co_name

I would use inspect.currentframe().f_back.f_code.co_name. Its use hasn't been covered in any of the prior answers which are mainly of one of three types:
Some prior answers use inspect.stack but it's known to be too slow.
Some prior answers use sys._getframe which is an internal private function given its leading underscore, and so its use is implicitly discouraged.
One prior answer uses inspect.getouterframes(inspect.currentframe(), 2)[1][3] but it's entirely unclear what [1][3] is accessing.
import inspect
from types import FrameType
from typing import cast
def demo_the_caller_name() -> str:
"""Return the calling function's name."""
# Ref: https://stackoverflow.com/a/57712700/
return cast(FrameType, cast(FrameType, inspect.currentframe()).f_back).f_code.co_name
if __name__ == '__main__':
def _test_caller_name() -> None:
assert demo_the_caller_name() == '_test_caller_name'
_test_caller_name()
Note that cast(FrameType, frame) is used to satisfy mypy.
Acknowlegement: comment by 1313e for an answer.

I've come up with a slightly longer version that tries to build a full method name including module and class.
https://gist.github.com/2151727 (rev 9cccbf)
# Public Domain, i.e. feel free to copy/paste
# Considered a hack in Python 2
import inspect
def caller_name(skip=2):
"""Get a name of a caller in the format module.class.method
`skip` specifies how many levels of stack to skip while getting caller
name. skip=1 means "who calls me", skip=2 "who calls my caller" etc.
An empty string is returned if skipped levels exceed stack height
"""
stack = inspect.stack()
start = 0 + skip
if len(stack) < start + 1:
return ''
parentframe = stack[start][0]
name = []
module = inspect.getmodule(parentframe)
# `modname` can be None when frame is executed directly in console
# TODO(techtonik): consider using __main__
if module:
name.append(module.__name__)
# detect classname
if 'self' in parentframe.f_locals:
# I don't know any way to detect call from the object method
# XXX: there seems to be no way to detect static method call - it will
# be just a function call
name.append(parentframe.f_locals['self'].__class__.__name__)
codename = parentframe.f_code.co_name
if codename != '<module>': # top level usually
name.append( codename ) # function or a method
## Avoid circular refs and frame leaks
# https://docs.python.org/2.7/library/inspect.html#the-interpreter-stack
del parentframe, stack
return ".".join(name)

Bit of an amalgamation of the stuff above. But here's my crack at it.
def print_caller_name(stack_size=3):
def wrapper(fn):
def inner(*args, **kwargs):
import inspect
stack = inspect.stack()
modules = [(index, inspect.getmodule(stack[index][0]))
for index in reversed(range(1, stack_size))]
module_name_lengths = [len(module.__name__)
for _, module in modules]
s = '{index:>5} : {module:^%i} : {name}' % (max(module_name_lengths) + 4)
callers = ['',
s.format(index='level', module='module', name='name'),
'-' * 50]
for index, module in modules:
callers.append(s.format(index=index,
module=module.__name__,
name=stack[index][3]))
callers.append(s.format(index=0,
module=fn.__module__,
name=fn.__name__))
callers.append('')
print('\n'.join(callers))
fn(*args, **kwargs)
return inner
return wrapper
Use:
#print_caller_name(4)
def foo():
return 'foobar'
def bar():
return foo()
def baz():
return bar()
def fizz():
return baz()
fizz()
output is
level : module : name
--------------------------------------------------
3 : None : fizz
2 : None : baz
1 : None : bar
0 : __main__ : foo

You can use decorators, and do not have to use stacktrace
If you want to decorate a method inside a class
import functools
# outside ur class
def printOuterFunctionName(func):
#functools.wraps(func)
def wrapper(self):
print(f'Function Name is: {func.__name__}')
func(self)
return wrapper
class A:
#printOuterFunctionName
def foo():
pass
you may remove functools, self if it is procedural

An alternative to sys._getframe() is used by Python's Logging library to find caller information. Here's the idea:
raise an Exception
immediately catch it in an Except clause
use sys.exc_info to get Traceback frame (tb_frame).
from tb_frame get last caller's frame using f_back.
from last caller's frame get the code object that was being executed in that frame.
In our sample code it would be method1 (not method2) being executed.
From code object obtained, get the object's name -- this is caller method's name in our sample.
Here's the sample code to solve example in the question:
def method1():
method2()
def method2():
try:
raise Exception
except Exception:
frame = sys.exc_info()[2].tb_frame.f_back
print("method2 invoked by: ", frame.f_code.co_name)
# Invoking method1
method1()
Output:
method2 invoked by: method1
Frame has all sorts of details, including line number, file name, argument counts, argument type and so on. The solution works across classes and modules too.

Code:
#!/usr/bin/env python
import inspect
called=lambda: inspect.stack()[1][3]
def caller1():
print "inside: ",called()
def caller2():
print "inside: ",called()
if __name__=='__main__':
caller1()
caller2()
Output:
shahid#shahid-VirtualBox:~/Documents$ python test_func.py
inside: caller1
inside: caller2
shahid#shahid-VirtualBox:~/Documents$

I found a way if you're going across classes and want the class the method belongs to AND the method. It takes a bit of extraction work but it makes its point. This works in Python 2.7.13.
import inspect, os
class ClassOne:
def method1(self):
classtwoObj.method2()
class ClassTwo:
def method2(self):
curframe = inspect.currentframe()
calframe = inspect.getouterframes(curframe, 4)
print '\nI was called from', calframe[1][3], \
'in', calframe[1][4][0][6: -2]
# create objects to access class methods
classoneObj = ClassOne()
classtwoObj = ClassTwo()
# start the program
os.system('cls')
classoneObj.method1()

Hey mate I once made 3 methods without plugins for my app and maybe that can help you, It worked for me so maybe gonna work for you too.
def method_1(a=""):
if a == "method_2":
print("method_2")
if a == "method_3":
print("method_3")
def method_2():
method_1("method_2")
def method_3():
method_1("method_3")
method_2()

getting the module and line that a function was called from

Is there a way to programatically get the line number and name of a function?
For example, I want to pass a list of strings to a function :
s = [calling_module, calling_function, line_number]
report(s)
Currently I just put it all in manually :
s = ["module abc", "func()", "line 22", "notes"]
report(s)
But I would like to know if there is an automatic way for python to fill in the module name (I think __name__ does that), function name and line number for me. Is there a way?

Use inspect module functions. For example,
import inspect
def b():
f = inspect.currentframe()
current = inspect.getframeinfo(f)
caller = inspect.getframeinfo(f.f_back)
#caller = inspect.getframeinfo(inspect.getouterframes(f)[1][0])
print(__name__, current.filename, current.function, current.lineno, caller.function)
def a():
b()
a()

You may want something along the lines of traceback.extract_stack():
>>> def test():
... print "In Function"
... print traceback.extract_stack()
...
>>>
>>> test()
In Function
[('<stdin>', 1, '<module>', None), ('<stdin>', 3, 'test', None)]
Though the results would need to be parsed.

from inspect import currentframe, getframeinfo, getmodulename
def report():
f = getframeinfo(currentframe().f_back)
print getmodulename(f.filename), f.lineno, f.function
Note that using
__name__
will return the name of the module containing report, while the code above will show the name of the module that called report.

How to watch for a variable change in python without dunder setattr or pdb

There is large python project where one attribute of one class just have wrong value in some place.
It should be sqlalchemy.orm.attributes.InstrumentedAttribute, but when I run tests it is constant value, let's say string.
There is some way to run python program in debug mode, and run some check (if variable changed type) after each step throught line of code automatically?
P.S. I know how to log changes of attribute of class instance with help of inspect and property decorator. Possibly here I can use this method with metaclasses...
But sometimes I need more general and powerfull solution...
Thank you.
P.P.S. I need something like there: https://stackoverflow.com/a/7669165/816449, but may be with more explanation of what is going on in that code.

Well, here is a sort of slow approach. It can be modified for watching for local variable change (just by name). Here is how it works: we do sys.settrace and analyse the value of obj.attr each step. The tricky part is that we receive 'line' events (that some line was executed) before line is executed. So, when we notice that obj.attr has changed, we are already on the next line and we can't get the previous line frame (because frames aren't copied for each line, they are modified ). So on each line event I save traceback.format_stack to watcher.prev_st and if on the next call of trace_command value has changed, we print the saved stack trace to file. Saving traceback on each line is quite an expensive operation, so you'd have to set include keyword to a list of your projects directories (or just the root of your project) in order not to watch how other libraries are doing their stuff and waste cpu.
watcher.py
import traceback
class Watcher(object):
def __init__(self, obj=None, attr=None, log_file='log.txt', include=[], enabled=False):
"""
Debugger that watches for changes in object attributes
obj - object to be watched
attr - string, name of attribute
log_file - string, where to write output
include - list of strings, debug files only in these directories.
Set it to path of your project otherwise it will take long time
to run on big libraries import and usage.
"""
self.log_file=log_file
with open(self.log_file, 'wb'): pass
self.prev_st = None
self.include = [incl.replace('\\','/') for incl in include]
if obj:
self.value = getattr(obj, attr)
self.obj = obj
self.attr = attr
self.enabled = enabled # Important, must be last line on __init__.
def __call__(self, *args, **kwargs):
kwargs['enabled'] = True
self.__init__(*args, **kwargs)
def check_condition(self):
tmp = getattr(self.obj, self.attr)
result = tmp != self.value
self.value = tmp
return result
def trace_command(self, frame, event, arg):
if event!='line' or not self.enabled:
return self.trace_command
if self.check_condition():
if self.prev_st:
with open(self.log_file, 'ab') as f:
print >>f, "Value of",self.obj,".",self.attr,"changed!"
print >>f,"###### Line:"
print >>f,''.join(self.prev_st)
if self.include:
fname = frame.f_code.co_filename.replace('\\','/')
to_include = False
for incl in self.include:
if fname.startswith(incl):
to_include = True
break
if not to_include:
return self.trace_command
self.prev_st = traceback.format_stack(frame)
return self.trace_command
import sys
watcher = Watcher()
sys.settrace(watcher.trace_command)
testwatcher.py
from watcher import watcher
import numpy as np
import urllib2
class X(object):
def __init__(self, foo):
self.foo = foo
class Y(object):
def __init__(self, x):
self.xoo = x
def boom(self):
self.xoo.foo = "xoo foo!"
def main():
x = X(50)
watcher(x, 'foo', log_file='log.txt', include =['C:/Users/j/PycharmProjects/hello'])
x.foo = 500
x.goo = 300
y = Y(x)
y.boom()
arr = np.arange(0,100,0.1)
arr = arr**2
for i in xrange(3):
print 'a'
x.foo = i
for i in xrange(1):
i = i+1
main()

There's a very simple way to do this: use watchpoints.
Basically you only need to do
from watchpoints import watch
watch(your_object.attr)
That's it. Whenever the attribute is changed, it will print out the line that changed it and how it's changed. Super easy to use.
It also has more advanced features, for example, you can call pdb when the variable is changed, or use your own callback functions instead of print it to stdout.

A simpler way to watch for an object's attribute change (which can also be a module-level variable or anything accessible with getattr) would be to leverage hunter library, a flexible code tracing toolkit. To detect state changes we need a predicate which can look like the following:
import traceback
class MutationWatcher:
def __init__(self, target, attrs):
self.target = target
self.state = {k: getattr(target, k) for k in attrs}
def __call__(self, event):
result = False
for k, v in self.state.items():
current_value = getattr(self.target, k)
if v != current_value:
result = True
self.state[k] = current_value
print('Value of attribute {} has chaned from {!r} to {!r}'.format(
k, v, current_value))
if result:
traceback.print_stack(event.frame)
return result
Then given a sample code:
class TargetThatChangesWeirdly:
attr_name = 1
def some_nested_function_that_does_the_nasty_mutation(obj):
obj.attr_name = 2
def some_public_api(obj):
some_nested_function_that_does_the_nasty_mutation(obj)
We can instrument it with hunter like:
# or any other entry point that calls the public API of interest
if __name__ == '__main__':
obj = TargetThatChangesWeirdly()
import hunter
watcher = MutationWatcher(obj, ['attr_name'])
hunter.trace(watcher, stdlib=False, action=hunter.CodePrinter)
some_public_api(obj)
Running the module produces:
Value of attribute attr_name has chaned from 1 to 2
File "test.py", line 44, in <module>
some_public_api(obj)
File "test.py", line 10, in some_public_api
some_nested_function_that_does_the_nasty_mutation(obj)
File "test.py", line 6, in some_nested_function_that_does_the_nasty_mutation
obj.attr_name = 2
test.py:6 return obj.attr_name = 2
... return value: None
You can also use other actions that hunter supports. For instance, Debugger which breaks into pdb (debugger on an attribute change).

Try using __setattr__ to override the function that is called when an attribute assignment is attempted. Documentation for __setattr__

You can use the python debugger module (part of the standard library)
To use, just import pdb at the top of your source file:
import pdb
and then set a trace wherever you want to start inspecting the code:
pdb.set_trace()
You can then step through the code with n, and investigate the current state by running python commands.

def __setattr__(self, name, value):
if name=="xxx":
util.output_stack('xxxxx')
super(XXX, self).__setattr__(name, value)
This sample code helped me.

Get function callers' information in python

I want to get information about the callers of a specific function in python. For example:
class SomeClass():
def __init__(self, x):
self.x = x
def caller(self):
return special_func(self.x)
def special_func(x):
print "My caller is the 'caller' function in an 'SomeClass' class."
Is it possible with python?

Yes, the sys._getframe() function let's you retrieve frames from the current execution stack, which you can then inspect with the methods and documentation found in the inspect module; you'll be looking for specific locals in the f_locals attribute, as well as for the f_code information:
import sys
def special_func(x):
callingframe = sys._getframe(1)
print 'My caller is the %r function in a %r class' % (
callingframe.f_code.co_name,
callingframe.f_locals['self'].__class__.__name__)
Note that you'll need to take some care to detect what kind of information you find in each frame.
sys._getframe() returns a frame object, you can chain through the whole stack by following the f_back reference on each. Or you can use the inspect.stack() function to produce a lists of frames with additional information.

An example:
def f1(a):
import inspect
print 'I am f1 and was called by', inspect.currentframe().f_back.f_code.co_name
return a
def f2(a):
return f1(a)
Will retrieve the "immediate" caller.
>>> f2(1)
I am f1 and was called by f2
And if wasn't called from another you get (in IDLE):
>>> f1(1)
I am f1 and was called by <module>

Thanks to Jon Clements answer I was able to make a function that returns an ordered list of all callers:
def f1():
names = []
frame = inspect.currentframe()
## Keep moving to next outer frame
while True:
try:
frame = frame.f_back
name = frame.f_code.co_name
names.append(name)
except:
break
return names
and when called in a chain:
def f2():
return f1()
def f3():
return f2()
def f4():
return f3()
print f4()
looks like this:
['f2', 'f3', 'f4', '<module>']
In my case I filter out anything at '<module>' and after, and then take the last item to be the name of the originating caller.
Or modify the original loop to bail at the first appearance of any name starting with '<':
frame = frame.f_back
name = frame.f_code.co_name
if name[0] == '<':
break
names.append(name)