I have an unknown number of functions in my python script (well, it is known, but not constant) that start with site_...
I was wondering if there's a way to go through all of these functions in some main function that calls for them.
something like:
foreach function_that_has_site_ as coolfunc
if coolfunc(blabla,yada) == true:
return coolfunc(blabla,yada)
so it would go through them all until it gets something that's true.
thanks!
The inspect module, already mentioned in other answers, is especially handy because you get to easily filter the names and values of objects you care about. inspect.getmembers takes two arguments: the object whose members you're exploring, and a predicate (a function returning bool) which will accept (return True for) only the objects you care about.
To get "the object that is this module" you need the following well-known idiom:
import sys
this_module = sys.modules[__name__]
In your predicate, you want to select only objects which are functions and have names that start with site_:
import inspect
def function_that_has_site(f):
return inspect.isfunction(f) and f.__name__.startswith('site_')
With these two items in hand, your loop becomes:
for n, coolfunc in inspect.getmembers(this_module, function_that_has_site):
result = coolfunc(blabla, yada)
if result: return result
I have also split the loop body so that each function is called only once (which both saves time and is a safer approach, avoiding possible side effects)... as well as rewording it in Python;-)
Have you tried using the inspect module?
http://docs.python.org/library/inspect.html
The following will return the methods:
inspect.getmembers
Then you could invoke with:
methodobjToInvoke = getattr(classObj, methodName)
methodobj("arguments")
This method goes through all properties of the current module and executes all functions it finds with a name starting with site_:
import sys
import types
for elm in dir():
f = getattr(sys.modules[__name__], elm)
if isinstance(f, types.FunctionType) and f.__name__[:5] == "site_":
f()
The function-type check is unnecessary if only functions are have names starting with site_.
def run():
for f_name, f in globals().iteritems():
if not f_name.startswith('site_'):
continue
x = f()
if x:
return x
It's best to use a decorator to enumerate the functions you care about:
_funcs = []
def enumfunc(func):
_funcs.append(func)
return func
#enumfunc
def a():
print 'foo'
#enumfunc
def b():
print 'bar'
#enumfunc
def c():
print 'baz'
if __name__ == '__main__':
for f in _funcs:
f()
Try dir(), globals() or locals(). Or inspect module (as mentioned above).
def site_foo():
pass
def site_bar():
pass
for name, f in globals().items():
if name.startswith("site_"):
print name, f()
Related
I'm developing a bot on Python (2.7, 3.4). I defined a about 30+ dynamic functions which to be used based on bot commands. While development, since not all functions are done, I have to define for them an empty functions (if I not define then code won't run) like this:
def c_about():
return
def c_events():
return
def c_currentlocation():
return
etc. many dummy functions.
Question:
it is somehow possible in Python to define same function but with multiple names?
Something like this:
def c_about(), c_events(), c_currentlocation():
return
Yes, it's perfectly possible since defined functions are stored in variables like everything else.
def foo():
pass
baz = bar = foo
There is still some metadata relating to the original function (help(bar) will still mention foo), but it doesn't affect functionality.
Another option is to use lambdas for one-liners:
foo = bar = baz = lambda: None
Functions do not intern in Python (i.e., automatically share multiple references to the same immutable object), but can share the same name:
>>> def a(): pass
...
>>> a
<function a at 0x101c892a8>
>>> def b(): pass
...
>>> b
<function b at 0x101c89320>
>>> c=a
>>> c
<function a at 0x101c892a8> # note the physical address is the same as 'a'
So clearly you can do:
>>> c=d=e=f=g=a
>>> e
<function a at 0x101c892a8>
For the case of functions not yet defined, you can use a try/catch block by catching either a NameError:
def default():
print "default called"
try:
not_defined()
except NameError:
default()
Or use a dict of funcs and catch the KeyError:
funcs={"default": default}
try:
funcs['not_defined']()
except KeyError:
funcs['default']()
Or, you can do funcs.get(not_defined, default)() if you prefer that syntax with a dict of funcs.
Yes, it is possible, and it is even possible to store them in lists using loops. For instance:
l = []
for i in range(10):
l.append(lambda: None)
And you can reference any of them through indices like l[index]()
For example:
c_events = 0
c_about = 1
l[c_events]()
I am trying to produce a better answer to the frequently-asked question "How do I do function-local static variables in Python?" (1, 2, 3, ...) "Better" means completely encapsulated in a decorator, that can be used in any context where a function definition may appear. In particular, it must DTRT when applied to methods and nested functions; it must play nice with other decorators applied to the same function (in any order); it must accept arbitrary initializers for the static variables, and it must not modify the formal parameter list of the decorated function. Basically, if this were to be proposed for inclusion in the standard library, nobody should be able to object on quality-of-implementation grounds.
Ideal surface syntax would be
#static_vars(a=0, b=[])
def test():
b.append(a)
a += 1
sys.stdout.write(repr(b) + "\n")
I would also accept
#static_vars(a=0, b=[])
def test():
static.b.append(static.a)
static.a += 1
sys.stdout.write(repr(static.b) + "\n")
or similar, as long as the namespace for the static variables is not the name of the function! (I intend to use this in functions that may have very long names.)
A slightly more motivated example involves precompiled regular expressions that are only relevant to one function:
#static_vars(encode_re = re.compile(
br'[\x00-\x20\x7F-\xFF]|'
br'%(?!(?:[0-9A-Fa-f]{2}|u[0-9A-Fa-f]{4}))')
def encode_nonascii_and_percents(segment):
segment = segment.encode("utf-8", "surrogateescape")
return encode_re.sub(
lambda m: "%{:02X}".format(ord(m.group(0))).encode("ascii"),
segment).decode("ascii")
Now, I already have a mostly-working implementation. The decorator rewrites each function definition as if it had read like so (using the first example):
def _wrap_test_():
a = 0
b = 1
def test():
nonlocal a, b
b.append(a)
a += 1
sys.stdout.write(repr(b) + "\n")
test = _wrap_test_()
del _wrap_test_
It seems that the only way to accomplish this is to munge the AST. I have code that works for simple cases (see below) but I strongly suspect it is wrong in more complicated cases. For instance, I think it will break if applied to a method definition, and of course it also breaks in any situation where inspect.getsource() fails.
So the question is, first, what should I do to make it work in more cases, and second, is there a better way to define a decorator with the same black-box effects?
Note 1: I only care about Python 3.
Note 2: Please assume that I have read all of the proposed solutions in all of the linked questions and found all of them inadequate.
#! /usr/bin/python3
import ast
import functools
import inspect
import textwrap
def function_skeleton(name, args):
"""Return the AST of a function definition for a function named NAME,
which takes keyword-only args ARGS, and does nothing. Its
.body field is guaranteed to be an empty array.
"""
fn = ast.parse("def foo(*, {}): pass".format(",".join(args)))
# The return value of ast.parse, as used here, is a Module object.
# We want the function definition that should be the Module's
# sole descendant.
assert isinstance(fn, ast.Module)
assert len(fn.body) == 1
assert isinstance(fn.body[0], ast.FunctionDef)
fn = fn.body[0]
# Remove the 'pass' statement.
assert len(fn.body) == 1
assert isinstance(fn.body[0], ast.Pass)
fn.body.clear()
fn.name = name
return fn
class static_vars:
"""Decorator which provides functions with static variables.
Usage:
#static_vars(foo=1, bar=2, ...)
def fun():
foo += 1
return foo + bar
The variables are implemented as upvalues defined by a wrapper
function.
Uses introspection to recompile the decorated function with its
context changed, and therefore may not work in all cases.
"""
def __init__(self, **variables):
self._variables = variables
def __call__(self, func):
if func.__name__ in self._variables:
raise ValueError(
"function name {} may not be the same as a "
"static variable name".format(func.__name__))
fname = inspect.getsourcefile(func)
lines, first_lineno = inspect.getsourcelines(func)
mod = ast.parse(textwrap.dedent("".join(lines)), filename=fname)
# The return value of ast.parse, as used here, is a Module
# object. Save that Module for use later and extract the
# function definition that should be its sole descendant.
assert isinstance(mod, ast.Module)
assert len(mod.body) == 1
assert isinstance(mod.body[0], ast.FunctionDef)
inner_fn = mod.body[0]
mod.body.clear()
# Don't apply decorators twice.
inner_fn.decorator_list.clear()
# Fix up line numbers. (Why the hell doesn't ast.parse take a
# starting-line-number argument?)
ast.increment_lineno(inner_fn, first_lineno - inner_fn.lineno)
# Inject a 'nonlocal' statement declaring the static variables.
svars = sorted(self._variables.keys())
inner_fn.body.insert(0, ast.Nonlocal(svars))
# Synthesize the wrapper function, which will take the static
# variableas as arguments.
outer_fn_name = ("_static_vars_wrapper_" +
inner_fn.name + "_" +
hex(id(self))[2:])
outer_fn = function_skeleton(outer_fn_name, svars)
outer_fn.body.append(inner_fn)
outer_fn.body.append(
ast.Return(value=ast.Name(id=inner_fn.name, ctx=ast.Load())))
mod.body.append(outer_fn)
ast.fix_missing_locations(mod)
# The new function definition must be evaluated in the same context
# as the original one. FIXME: supply locals if appropriate.
context = func.__globals__
exec(compile(mod, filename="<static-vars>", mode="exec"),
context)
# extract the function we just defined
outer_fn = context[outer_fn_name]
del context[outer_fn_name]
# and call it, supplying the static vars' initial values; this
# returns the adjusted inner function
adjusted_fn = outer_fn(**self._variables)
functools.update_wrapper(adjusted_fn, func)
return adjusted_fn
if __name__ == "__main__":
import sys
#static_vars(a=0, b=[])
def test():
b.append(a)
a += 1
sys.stdout.write(repr(b) + "\n")
test()
test()
test()
test()
Isn't this what classes are for?
import sys
class test_class:
a=0
b=[]
def test(self):
test_class.b.append(test_class.a)
test_class.a += 1
sys.stdout.write(repr(test_class.b) + "\n")
t = test_class()
t.test()
t.test()
[0]
[0, 1]
Here is a version of your regexp encoder:
import re
class encode:
encode_re = re.compile(
br'[\x00-\x20\x7F-\xFF]|'
br'%(?!(?:[0-9A-Fa-f]{2}|u[0-9A-Fa-f]{4}))')
def encode_nonascii_and_percents(self, segment):
segment = segment.encode("utf-8", "surrogateescape")
return encode.encode_re.sub(
lambda m: "%{:02X}".format(ord(m.group(0))).encode("ascii"),
segment).decode("ascii")
e = encode()
print(e.encode_nonascii_and_percents('foo bar'))
foo%20bar
There is always the singleton class.
Is there a simple, elegant way to define Singletons in Python?
I saw this particular piece of code:
def g(x,y):
return x+y
def g(x,y):
return x*y
x,y=6,7
print (g(x,y))
The output is obviously(but not to me) is 42. Can somebody please explain this behavior? This is method overriding I suppose, but I'm still not getting the flow here.
When you define a function, and you redefine it, it will use the last one you defined, even the parameter is different:
def g(x,y):
return x+y
def g(x,y):
return x*y
x,y=6,7
print (g(x,y))
def hello():
return 'hello'
def hello():
return 'bye'
print hello()
def withone(word):
return word
def withone():
return 1==1
print withone('ok')
Output:
42
bye
TypeError: withone() takes no arguments (1 given)
And function name in Python is more like simple variable:
def hello():
return 'hello'
iamhello = hello # bind to the old one
def hello():
return 'bye'
print hello() # here is the new guy
print iamhello()
OutPut:
bye
hello
The devil is in the order of function definitions.
This is not technically method overriding as that requires class inheritance, instead it's a result of how python declares and references functions.
When declaring a function, python stores a reference to that function in a variable named after the function definition. (e.g. variable would be "foo" for "def foo():")
By declaring the function twice, the value of that variable gets overwritten by the second definition.
A Python script is parsed from top till bottom.
So anytime the same name of a variable or function or class occurs, it overwrites any definitions that where associated with this name before.
def g(x,z):
print('first')
def g():
print('second')
g = 3
print g
print g()
So look at this example which will result in the printout of '3' and then in an Exception: 'TypeError: 'int' object is not callable'
The name g is at first a function with two parameters, then it gets redefined to be a function with no parameters, then it gets redefined to be an int.
Which cannot be called obviously :)
Everything in python is treated as object, whether it be a function name or class name. So, when we define a function using 'def', the memory allocation is done for that method. Then python points the name that we assign to the function, to this allocated memory location. So if we define a method :-
def demo():
print 'hi'
the memory is allocated for the method, and the name 'demo' is pointed to its memory location as follows :-
Now as described by zoosuck in his second example, when you assign the function name to another variable :-
demo2 = demo # bind to the old one
then in that case, the assigned memory location to demo, is assigned to demo2 as well. So both demo and demo2 points to same location 12506.
print id(demo) # will print 12506
print id(demo2) # will print 12506
Now if we modify the above piece of code and in the next line, define a new method with same name demo:-
def demo():
print 'hi'
demo2 = demo # bind to the old one
demo() # Will print hi
def demo():
print "hello"
demo() # Will print hello
demo2() # Will print hi
then a completely new memory location 12534 is allocated for this new method, and now demo will point to this new location 12534 instead of pointing to the old one i.e. to 12506. But demo2 is still pointing to the location 12506.
I hope this will give you a clear idea of what is going on and how the method name is over-written.
Order matters, if names are same,last function you defined is processing. In your case it's;
def g(x,y):
return x*y
g is just a variable. The fact that the object it refers to is a function doesn't make it special in Python, so you can assign and reassign it as you want. In this case, the second assignment (which is what a function definition is) simply replaces the object stored there with a different one.
Functions and methods are normal objects like any others. So in
def g(x, y):
return x + y
def g(x, y):
return x * y
the second object g will override(replace) the first one, just like object a does below:
a = 1
a = 2
The number, type or order of parameters does not make any difference, because Python does not support function/method override and does not allow two functions/methods to have the same name.
If you are familiar with lambda function, also often called anonymous\inline functions, this might clear things up a bit
These two code blocks are essentially equal
def g(x,y):
return x+y
def g(x,y):
return x*y
g = lambda x,y: x+y
g = lambda x,y: x*y
I know how to override string class with:
class UserString:
def __str__(self):
return 'Overridden String'
if __name__ == '__main__':
print UserString()
But how can i use this class instead of built-in str class without defining implicitly UserString class?. To be clear
I want this:
>>> print "Boo boo!"
Overridden String
It is not possible. You have not overridden string class.
You cannot override classes. You can override methods. What you have done is defined a class and only overridden its str() method.
But you can do something like this...
def overriden_print(x):
print "Overriden in the past!"
from __future__ import print_function # import after the definition of overriden_print
print = overriden_print
print("Hello")
Output:
Overriden in the past!
It's impossible to do what you want without hacking the python executable itself... after all, str is a built-in type, and the interpreter, when passed 'string' type immediates, will always create built-in strings.
However... it is possible, using delegation, to do something like this. This is slightly modified from another stackoverflow recipe (which sadly, I did not include a link to in my code...), so if this is your code, please feel free to claim it :)
def returnthisclassfrom(specials):
specialnames = ['__%s__' % s for s in specials.split()]
def wrapit(cls, method):
return lambda *a: cls(method(*a))
def dowrap(cls):
for n in specialnames:
method = getattr(cls, n)
setattr(cls, n, wrapit(cls, method))
return cls
return dowrap
Then you use it like this:
#returnthisclassfrom('add mul mod')
class UserString(str):
pass
In [11]: first = UserString('first')
In [12]: print first
first
In [13]: type(first)
Out[13]: __main__.UserString
In [14]: second = first + 'second'
In [15]: print second
firstsecond
In [16]: type(second)
Out[16]: __main__.UserString
One downside of this is that str has no radd support, so 'string1' + UserString('string2') will give a string, whereas UserString('string1') + 'string2' gives a UserString. Not sure if there is a way around that.
Maybe not helpful, but hopefully it puts you on the right track.
The globals function returns a dictionary that contains the functions in a module, and the dir function returns a list that contains the names of the functions in a module, but they are in alphabetical order or are in a dictionary.
Is there a way to get the names of all the functions in a module in the order they appear in a file?
Here is my solution. Read in the source file as a string, filter out lines that begin with def, strip leading whitespace and extract the substring between the first space and first left paren.
def extract_fns(filename):
with open(filename) as f:
lines = f.readlines()
return [line.split(' ', 1)[1].split('(')[0] for line in lines
if line.strip().startswith('def')]
When I had a need like this, I used a decorator.
def remember_order(func, counter=[0]):
func._order = counter[0]
counter[0] += 1
return func
#remember_order
def foo():
print "foo"
#remember_order
def bar():
print "bar"
#remember_order
def baz():
print "baz"
Yes, you have to decorate each function individually. Explicit is better than implicit, as they say, and because you are doing something unnatural it's good to call it out as plainly as possible.
Now you want to get all the decorated functions in the order they were defined?
import sys
module = sys.modules[__name__] # get reference to current module
# you could also use a reference to some other module where the functions are
# get the functions in the order defined
funcs = sorted((func for func in
(getattr(module, name) for name in dir(module))
if callable(func) and hasattr(func, "_order")),
key = lambda func: func._order)
# call them in that order
for func in funcs:
func()
But it'd be easier to just give them names in alphabetical order...
My solution was to parse the ast. As it turns out elements in the ast tree are in the order they are parsed. Hence functions are in the order they appear in the file.
import inspect
import ast
def get_function_names_in_order_of_appearence(module_name):
this_source = inspect.getsource(module_name)
tree = ast.parse(this_source)
functions = []
for elem in tree.body:
if type(elem) is ast.FunctionDef:
this_func_name = elem.name
functions.append(this_func_name)
return functions
Assuming the module that is to be analyzed is:
# fancy_module
def z_func(x):
return x+1
def a_func(a, b):
pass
def b_func(mystr, **kw):
pass
Will give:
func_list = get_function_names_in_order_of_appearence(fancy_module)
for func in func_list:
print(func)
# -------
# output:
z_func
a_func
b_func
The advantage is you can add other stuff from the ast tree, like the signatures of the functions.