I'm trying to get all function parameters and their values from within the current function in Python and almost have it working except for one thing: I don't know how to call it from inside the function without explicitly giving it the name of the object.
For example: the code below works for the case of the function being a class method ("my_func_in_class", but I don't know what to pass into the inspect.signature() call for the "my_func" function that is not part of a class without mentioning the function by name.
This isn't critical to solving my problem at hand but I'd still like to know how to make it work
import inspect
import sys
def my_func(a, b, c=None):
this_func = inspect.currentframe()
func_name = inspect.getframeinfo(this_func).function
print(f"Function Name {func_name}")
(sig, local_vars) = inspect.signature(my_func), locals()
args = {}
for a in sig.parameters.keys():
args[a] = local_vars[a]
return args
class MyClass(object):
def __init__(self):
pass
def my_func_in_class(self, a, b, c=None):
this_func = inspect.currentframe()
func_name = inspect.getframeinfo(this_func).function
print(f"Function Name {func_name}")
(sig, local_vars) = inspect.signature(getattr(self, func_name)), locals()
args = {}
for a in sig.parameters.keys():
args[a] = local_vars[a]
return args
if __name__ == "__main__":
class1 = MyClass()
args1 = my_func(1, 2)
args2 = class1.my_func_in_class(10, 20, c=30)
print(f"args1:")
for (k, v) in args1.items():
print(f"{k}: {v}")
print(f"\nargs2")
for (k, v) in args2.items():
print(f"{k}: {v}")
python ./get_func_args.py
Function Name my_func
Function Name my_func_in_class
args1:
a: 1
b: 2
c: None
args2
a: 10
b: 20
c: 30
But:
def my_func(a, b, c=None):
this_func = inspect.currentframe()
func_name = inspect.getframeinfo(this_func).function
print(f"Function Name {func_name}")
(sig, local_vars) = inspect.signature(this_func), locals()
returns the error:
Function Name my_func
Traceback (most recent call last):
File "./get_func_args.py", line 41, in <module>
args1 = my_func(1, 2)
File "./get_func_args.py", line 12, in my_func
(sig, local_vars) = inspect.signature(this_func), locals()
File "/home/mlissa2/cmm/python/miniconda3/lib/python3.6/inspect.py", line 3065, in signature
return Signature.from_callable(obj, follow_wrapped=follow_wrapped)
File "/home/mlissa2/cmm/python/miniconda3/lib/python3.6/inspect.py", line 2815, in from_callable
follow_wrapper_chains=follow_wrapped)
File "/home/mlissa2/cmm/python/miniconda3/lib/python3.6/inspect.py", line 2193, in _signature_from_callable
raise TypeError('{!r} is not a callable object'.format(obj))
TypeError: <frame object at 0x7ff7dcc92048> is not a callable object
So I'm very close but don't have the final step in place.
If your only concern is to get arguments information from the signature of the function along with their values, then you don't need to use inspect.signature(). You can use the following approach.
def my_func(a, b, c=None):
# get arg-values without reference to callable
cframe = inspect.currentframe()
args_info = inspect.getargvalues(cframe)
# Collecting args-values of my_func in a dictionary
this_func_argvals = {arg: args_info.locals.get(arg) for arg in args_info.args}
return this_func_argvals
That's it.
However, if you must insist on getting the signature too for whatnot than use the following approach.
The reason you were able to get my_func_in_class reference in your class method was because your class provided access to its namespace, so getting attribute from that namespace was easy. In order to get reference of my_func we need to access namespace of the immediate outer stack of my_func. From there we can get my_func reference. Here's how to do that:
def some_fun(a, b, c=None):
# Get Stacks. We only need FrameInfo for current frame and immediate outer frame
stacks = inspect.stack()
# Get my_func's name (str) from current frame
this_func_name = stacks[0].function
# Get immediate outer frame
outer_frame = stacks[1].frame
# From outer frame's locals, get reference to my_fun using its name as key
this_func_reference = outer_frame.f_locals.get(this_func_name)
# Voila!
sigs = inspect.signature(this_func_reference)
Related
In the following example I'm trying to pass arguments to a function that itself has been passed as a kwarg. I have not been successful in passing arguments to the function 'func' from within the class 'TestClass' in the following example:
import sys, threading; from threading import Thread
def func(kwargs):
print('IN:', sys._getframe(0).f_code.co_name)
for key, value in kwargs.items() :
print ('KEY:', key, ', VAL:', value, sep='')
class TestClass(Thread):
def __init__(self, name = sys._getframe(0).f_code.co_name, kwargs = None):
Thread.__init__(self)
self.name = name
self.kwargs = kwargs
print('IN:', self.name)
def run(self):
func = self.kwargs['func']
func_kwargs_inner = {'arg_1': 'INNER-1', 'arg_2': 'INNER-2'}
func() # how to pass func_kwargs_inner to func?
def main():
func_kwargs = {'arg_1': 'OUTER-1', 'arg_2': 'OUTER-2'} # these get passed
# func_kwargs = {} # func_kwargs never gets populated
kwargs = {'func': (lambda: func(func_kwargs))}
test = TestClass(name='my-test', kwargs=kwargs)
test.start()
print('PROGRAM END')
if __name__ == '__main__':
main()
If I try to pass 'func_kwargs_inner' to 'func()', I get syntax errors; if I leave the argument list empty - as in the example - the result is:
IN: my-test
IN: func
KEY:arg_1, VAL:OUTER-1
KEY:arg_2, VAL:OUTER-2
PROGRAM END
whereas the required output once I find a way to pass the arguments correctly is:
IN: my-test
IN: func
KEY:arg_1, VAL:INNER-1
KEY:arg_2, VAL:INNER-2
PROGRAM END
How do I pass 'func_kwargs_inner' to 'func()'?
It seems that if you do the obvious thing, then it will work, and that your code at present explicitly avoids passing the arguments that you want. Specifically, in your TestClass.run you are not passing any arguments to func but instead relies on function arguments that are hard-coded into the lambda expression. So change your line:
func() # how to pass func_kwargs_inner to func?
to pass the arguments:
func(func_kwargs_inner)
Then in main, instead of that lambda expression:
kwargs = {'func': (lambda: func(func_kwargs))}
simply pass the function object itself:
kwargs = {'func': func}
Then you get the expected output:
IN: my-test
IN: func
PROGRAM END
KEY:arg_1, VAL:INNER-1
KEY:arg_2, VAL:INNER-2
I am trying to get a list of functions from a module using inspect.getmembers in the order of source code.
Below is the code
def get_functions_from_module(app_module):
list_of_functions = dict(inspect.getmembers(app_module,
inspect.isfunction))
return list_of_functions.values
The current code will not return the list of function objects in order of the source code and I'm wondering if it would be possible to order it.
Thank you!
I think I came up with a solution.
def get_line_number_of_function(func):
return func.__code__.co_firstlineno
def get_functions_from_module(app_module):
list_of_functions = dict(inspect.getmembers(app_module,
inspect.isfunction))
return sorted(list_of_functions.values(), key=lambda x:
get_line_number_of_function(x))
You can sort by line numbers using inspect.findsource. Docstring from the source code of that function:
def findsource(object):
"""Return the entire source file and starting line number for an object.
The argument may be a module, class, method, function, traceback, frame,
or code object. The source code is returned as a list of all the lines
in the file and the line number indexes a line in that list. An OSError
is raised if the source code cannot be retrieved."""
Here's an example in Python 2.7:
import ab.bc.de.t1 as t1
import inspect
def get_functions_from_module(app_module):
list_of_functions = inspect.getmembers(app_module, inspect.isfunction)
return list_of_functions
fns = get_functions_from_module(t1)
def sort_by_line_no(fn):
fn_name, fn_obj = fn
source, line_no = inspect.findsource(fn_obj)
return line_no
for name, fn in sorted(fns, key=sort_by_line_no):
print name, fn
My ab.bc.de.t1 is defined as follows:
class B(object):
def a():
print 'test'
def c():
print 'c'
def a():
print 'a'
def b():
print 'b'
And the output I get when I try retrieving sorted functions is below:
c <function c at 0x00000000362517B8>
a <function a at 0x0000000036251438>
b <function b at 0x0000000036251668>
>>>
I have a very long function func which takes a browser handle and performs a bunch of requests and reads a bunch of responses in a specific order:
def func(browser):
# make sure we are logged in otherwise log in
# make request to /search and check that the page has loaded
# fill form in /search and submit it
# read table of response and return the result as list of objects
Each operation require a large amount of code due to the complexity of the DOM and they tend to grow really fast.
What would be the best way to refactor this function into smaller components so that the following properties still hold:
the execution flow of the operations and/or their preconditions is guaranteed just like in the current version
the preconditions are not checked with asserts against the state, as this is a very costly operation
func can be called multiple times on the browser
?
Just wrap the three helper methods in a class, and track which methods are allowed to run in an instance.
class Helper(object):
def __init__(self):
self.a = True
self.b = False
self.c = False
def funcA(self):
if not self.A:
raise Error("Cannot run funcA now")
# do stuff here
self.a = False
self.b = True
return whatever
def funcB(self):
if not self.B:
raise Error("Cannot run funcB now")
# do stuff here
self.b = False
self.c = True
return whatever
def funcC(self):
if not self.C:
raise Error("Cannot run funcC now")
# do stuff here
self.c = False
self.a = True
return whatever
def func(...):
h = Helper()
h.funcA()
h.funcB()
h.funcC()
# etc
The only way to call a method is if its flag is true, and each method clears its own flag and sets the next method's flag before exiting. As long as you don't touch h.a et al. directly, this ensures that each method can only be called in the proper order.
Alternately, you can use a single flag that is a reference to the function currently allowed to run.
class Helper(object):
def __init__(self):
self.allowed = self.funcA
def funcA(self):
if self.allowed is not self.funcA:
raise Error("Cannot run funcA now")
# do stuff
self.allowed = self.funcB
return whatever
# etc
Here's the solution I came up with. I used a decorator (closely related to the one in this blog post) which only allows for a function to be called once.
def call_only_once(func):
def new_func(*args, **kwargs):
if not new_func._called:
try:
return func(*args, **kwargs)
finally:
new_func._called = True
else:
raise Exception("Already called this once.")
new_func._called = False
return new_func
#call_only_once
def stateA():
print 'Calling stateA only this time'
#call_only_once
def stateB():
print 'Calling stateB only this time'
#call_only_once
def stateC():
print 'Calling stateC only this time'
def state():
stateA()
stateB()
stateC()
if __name__ == "__main__":
state()
You'll see that if you re-call any of the functions, the function will throw an Exception stating that the functions have already been called.
The problem with this is that if you ever need to call state() again, you're hosed. Unless you implement these functions as private functions, I don't think you can do exactly what you want due to the nature of Python's scoping rules.
Edit
You can also remove the else in the decorator and your function will always return None.
Here a snippet I used once for my state machine
class StateMachine(object):
def __init__(self):
self.handlers = {}
self.start_state = None
self.end_states = []
def add_state(self, name, handler, end_state=0):
name = name.upper()
self.handlers[name] = handler
if end_state:
self.end_states.append(name)
def set_start(self, name):
# startup state
self.start_state = name
def run(self, **kw):
"""
Run
:param kw:
:return:
"""
# the first .run call call the first handler with kw keywords
# each registered handler should returns the following handler and the needed kw
try:
handler = self.handlers[self.start_state]
except:
raise InitializationError("must call .set_start() before .run()")
while True:
(new_state, kw) = handler(**kw)
if isinstance(new_state, str):
if new_state in self.end_states:
print("reached ", new_state)
break
else:
handler = self.handlers[new_state]
elif hasattr(new_state, "__call__"):
handler = new_state
else:
return
The use
class MyParser(StateMachine):
def __init__(self):
super().__init__()
# define handlers
# we can define many handler as we want
self.handlers["begin_parse"] = self.begin_parse
# define the startup handler
self.set_start("begin_parse")
def end(self, **kw):
logging.info("End of parsing ")
# no callable handler => end
return None, None
def second(self, **kw):
logging.info("second ")
# do something
# if condition is reach the call `self.end` handler
if ...:
return self.end, {}
def begin_parse(self, **kw):
logging.info("start of parsing ")
# long process until the condition is reach then call the `self.second` handler with kw new keywords
while True:
kw = {}
if ...:
return self.second, kw
# elif other cond:
# return self.other_handler, kw
# elif other cond 2:
# return self.other_handler 2, kw
else:
return self.end, kw
# start the state machine
MyParser().run()
will print
INFO:root:start of parsing
INFO:root:second
INFO:root:End of parsing
You could use local functions in your func function. Ok, they are still declared inside one single global function, but Python is nice enough to still give you access to them for tests.
Here is one example of one function declaring and executing 3 (supposedly heavy) subfunctions. It takes one optional parameter test that when set to TEST prevent actual execution but instead gives external access to individual sub-functions and to a local variable:
def func(test=None):
glob = []
def partA():
glob.append('A')
def partB():
glob.append('B')
def partC():
glob.append('C')
if (test == 'TEST'):
global testA, testB, testC, testCR
testA, testB, testC, testCR = partA, partB, partC, glob
return None
partA()
partB()
partC()
return glob
When you call func, the 3 parts are executed in sequence. But if you first call func('TEST'), you can then access the local glob variable as testCR, and the 3 subfunctions as testA, testB and testC. This way you can still test individually the 3 parts with well defined input and control their output.
I would insist on the suggestion given by #user3159253 in his comment on the original question:
If the sole purpose is readability I would split the func into three "private" > or "protected" ones (i.e. _func1 or __func1) and a private or protected property > which keeps the state shared between the functions.
This makes a lot of sense to me and seems more usual amongst object oriented programming than the other options. Consider this example as an alternative:
Your class (teste.py):
class Test:
def __init__(self):
self.__environment = {} # Protected information to be shared
self.public_stuff = 'public info' # Accessible to outside callers
def func(self):
print "Main function"
self.__func_a()
self.__func_b()
self.__func_c()
print self.__environment
def __func_a(self):
self.__environment['function a says'] = 'hi'
def __func_b(self):
self.__environment['function b says'] = 'hello'
def __func_c(self):
self.__environment['function c says'] = 'hey'
Other file:
from teste import Test
t = Test()
t.func()
This will output:
Main function says hey guys
{'function a says': 'hi', 'function b says': 'hello', 'function c says': 'hey'}
If you try to call one of the protected functions, an error occurs:
Traceback (most recent call last):
File "C:/Users/Lucas/PycharmProjects/testes/other.py", line 6, in <module>
t.__func_a()
AttributeError: Test instance has no attribute '__func_a'
Same thing if you try to access the protected environment variable:
Traceback (most recent call last):
File "C:/Users/Lucas/PycharmProjects/testes/other.py", line 5, in <module>
print t.__environment
AttributeError: Test instance has no attribute '__environment'
In my view this is the most elegant, simple and readable way to solve your problem, let me know if it fits your needs :)
I have created a class that can take a function with a set of arguments. I would like to run the passed function every time the event handler signals.
I am attaching my code below which runs when I pass a fun2 which has no arguments but not with fun1. Any suggestions that I can make to the code below work with fun1 and fun2? If I omit the return statement from fun1, I get an error that 'str' object is not callable.
>>> TimerTest.main()
function 1. this function does task1
my function from init from function1
my function in start of runTimerTraceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Program Files (x86)\IronPython 2.7\TimerTest.py", line 57, in main
File "C:\Program Files (x86)\IronPython 2.7\TimerTest.py", line 25, in runTime
r
TypeError: str is not callable
import System
from System.Timers import (Timer, ElapsedEventArgs)
class timerTest:
def __init__ (self, interval,autoreset, fun):
self.Timer = Timer()
self.Timer.Interval= interval
self.Timer.AutoReset = autoreset
self.Timer.Enabled = True
self.myfunction = fun
def runTimer(self):
print 'my function in start of runTimer', self.myfunction ()
self.Timer.Start()
def OnTimedEvent (s, e):
print "The Elapsed event was raised at " , e.SignalTime
print 'printing myfunction...', self.myfunction()
self.myfunction()
self.Timer.Elapsed += OnTimedEvent
def stopTimer(self):
self.Timer.Stop()
self.Timer.Dispose= True
def fun1(a,b):
print 'function 1. this function does task1'
return 'from function1'
def fun2():
print 'Function 2. This function does something'
print 'Test 1...2...3...'
return 'From function 2'
def main():
a = timerTest(1000, True, fun1(10,20))
a.runTimer()
b= timerTest(3000,True,fun2)
b.runTimer()
if __name__ == '__main__':
main()
I am learning Python and I apologize if my questions are basic.
To change the interval, I stop the timer using a stopTimer method I added to the timerTest class:
def stopTimer(self):
self.Timer.Stop()
I take the new user input to call the runTimer method which I have revised per Paolo Moretti's suggestions:
def runTimer(self, interval,autoreset,fun,arg1, arg2, etc.):
self.Timer.Interval= interval
self.Timer.AutoReset = autoreset
myfunction = fun
my_args = args
self.Timer.Start()
def OnTimedEvent (s, e):
print "The Elapsed event was raised at " , e.SignalTime
myfunction(*my_args)
self.Timer.Elapsed += OnTimedEvent
Whenever a command button is pressed, the following method is called:
requestTimer.runTimer((self.intervalnumericUpDown.Value* 1000),True, function, *args)
I do not understand why stopping the timer and sending the request causes the runTimer method to be executed multiple times and it seems dependent on how many times I change the interval. I have tried a couple of methods: Close and Dispose with no success.
A second question on slightly different subject.
I have been looking at other .NET classes with Timer classes. A second question is on how I would translate the following VB sample code into Python. Is "callback As TimerCallback" equivalent to myfunction(*my_args)?
Public Sub New ( _
callback As TimerCallback, _
state As Object, _
dueTime As Integer, _
period As Integer _
)
per .NET documentation:
callback
Type: System.Threading.TimerCallback
A TimerCallback delegate representing a method to be executed.
I can partially get the timer event to fire if I define a function with no arguments such as:
def fun2(stateinfo):
# function code
which works with:
self.Timer = Timer(fun2, self.autoEvent, self.dueTime,self.period)
The function call fails if I replace fun2 with a more generic function call myfunction(*my_args)
You can also use * syntax for calling a function with an arbitrary argument list:
class TimerTest:
def __init__(self, interval, autoreset, fun, *args):
# ...
self.my_function = fun
self.my_args = args
# ...
def run_timer(self):
# ...
def on_timed_event(s, e):
# ...
self.my_function(*self.my_args)
# ...
Usage:
>>> t1 = TimerTest(1000, True, fun1, 10, 20)
>>> t2 = TimerTest(1000, True, fun2)
And check out the PEP8 style guide as well. Python's preferred coding conventions are different than many other common languages.
Question 1
Every time you use the addition assignment operator (+=) you are attaching a new event handler to the event. For example this code:
timer = Timer()
def on_timed_event(s, e):
print "Hello form my event handler"
timer.Elapsed += on_timed_event
timer.Elapsed += on_timed_event
timer.Start()
will print the "Hello form my event handler"phrase twice.
For more information you can check out the MSDN documentation, in particular Subscribe to and Unsubscribe from Events .
So, you should probably move the event subscription to the __init__ method, and only start the timer in your run_timer method:
def run_timer(self):
self.Timer.Start()
You could also add a new method (or use a property) for changing the interval:
def set_interval(self, interval):
self.Timer.Interval = interval
Question 2
You are right about TimerCallback: it's a delegate representing a method to be executed.
For example, this Timer constructor:
public Timer(
TimerCallback callback
)
is expecting a void function with a single parameter of type Object.
public delegate void TimerCallback(
Object state
)
When you are invoking a function using the * syntax you are doing something completely different. It's probably easier if I'll show you an example:
def foo(a, b, *args):
print a
print b
print args
>>> foo(1, 2, 3, 4, 5)
1
2
(3, 4, 5)
>>> args = (1, 2, 3)
>>> foo(1, 2, *args)
1
2
(1, 2, 3)
Basically in the second case you are invoking a function with additional arguments unpacked from a tuple.
So If you want to pass a function with a different signature to a constructor which accepts a TimerCallback delegate you have to create a new function, like #Lasse is suggesting.
def my_func(state, a, b):
pass
You can do this either using the lambda keyword:
t1 = Timer(lambda state: my_func(state, 1, 2))
or by declaring a new function:
def time_proc(state):
my_func(state, 1, 2)
t2 = Timer(time_proc)
If the function takes no parameters, simply pass it without calling it:
b = timerTest(3000, True, fun2)
If it takes parameters, you need to convert it to a function that doesn't take parameters. What you're doing is calling it, and then you pass the result, which in this case is a string. Instead do this:
a = timerTest(1000, True, lambda: fun1(10, 20))
Sorry if I did not explain myself clearly.
I would like to create a wrapper to call pre-defined functions with different number of inputs. Of course, I can create an individual wrapper for each function, but I am wondering if there is a way to create a generic wrapper for all cases.
The functions that should be called are named 'fun1' and 'fun2' with different number of inputs. I need to create a wrapper 'fun_wrap(func_name, uncertain amount of inputs)', which only needs the function name to be called and its associated amount of inputs.
One more thing, I need to change the input names by adding '_in' and make them global variables first. Below is my broken code. Thanks for any suggestions!
def fun1(a,b):
return a+b
def fun2(a,b,c):
return a*b/c
def set_globals(**kwargs):
for argname in kwargs:
globals()['%s_in' % argname] = kwargs[argname]
def fun_wrap(func_name, uncertain amount of inputs):
ffunc_name(set_globals(uncertain amount of inputs))
In this way, if I can call final_fun with arguments like:
fun_wrap(fun1,a,b)
fun_wrap(fun2,a,b)
UPDATE
I tried to use *arg, but failed...
def fun1(a,b):
return a+b
def fun2(a,b,c):
return a*b/c
def set_globals(**kwargs):
for argname in kwargs:
globals()['%s_in' % argname] = kwargs[argname]
def fun_wrap(func_name, *arg):
func_name(set_globals(*arg))
fun_wrap(fun2,a=1,b=2,c=3)
got error:
Traceback (most recent call last):
File "D:\Dropbox\AppPest\rice\try.py", line 19, in <module>
fun_wrap(fun2,a=1,b=2,c=3)
TypeError: fun_wrap() got an unexpected keyword argument 'a'
def fun1(a,b):
return a + b
def fun2(a,b,c):
return a * b / c
def set_globals(**kwargs):
for argname in kwargs:
globals()['%s_in' % argname] = kwargs[argname]
def fun_wrap(func, **kwargs):
set_globals(**kwargs) # made the call to set_globals before calling your function
return func(**kwargs) # return the value returned by the functions called