Develop Reference

Set print flush=True to default?

I am aware that you can flush after a print statement by setting flush=True like so:
print("Hello World!", flush=True)
However, for cases where you are doing many prints, it is cumbersome to manually set each print to flush=True. Is there a way to set the default to flush=True for Python 3.x? I am thinking of something similar to the print options numpy gives using numpy.set_printoptions.

You can use partial:
from functools import partial
print_flushed = partial(print, flush=True)
print_flushed("Hello world!")
From the documentation:
The partial() is used for partial function application which “freezes” some portion of a function’s arguments and/or keywords resulting in a new object with a simplified signature.

You can use something like this:
def decorator(func):
printer = func
def wrapped(*args):
printer(*args, end=' *\n', flush=True)
return wrapped
print = decorator(print)
for i in range(5):
print(i)
0 *
1 *
2 *
3 *
4 *

Get call string for a Python function, as typed on the command line

I would like to store the exact string that was typed for a function call, from the function itself using introspection (I cannot/don't want to hack the command line interpreter -- so, for example getting history from readline or whatever is not what I am looking for).
Let's say, if user typed:
>>> myfunc('a', 'b', 1, mykwarg='hello')
I would like to get the call string (i.e. myfunc('a', 'b', 1, mykwarg='hello'))
from the code inside myfunc.
I can craft something like this:
def myfunc(a,b,c,mykwarg=None):
frame = inspect.currentframe()
sig = inspect.signature(myfunc)
args = []
for param in sig.parameters.values():
if param.name in frame.f_locals:
args.append(f"{param.name}={str(frame.f_locals[param.name])}")
cmd = f"{frame.f_code.co_name}({','.join(args)})"
print(cmd)
I get:
>>> myfunc('a', 'b', 1, mykwarg='hello')
myfunc(a=a,b=b,c=1,mykwarg=hello)
Which is not exactly what user typed. Also, I hope there is something more
robust and less 'hackish' to try...
Use case: I want to be able to associate a command call from my library with its result. I do not want to hard-code the command call storage for each function, I would prefer to use a decorator or something like that. This is probably much easier to do from the REPL, but I would like to not depend on it (like, if user calls the function from its own program, it should still be able to associate the command call with the result).

Finally I answer my own question, hopefully it can help someone else one day.
I decided to try to go the dis way, ie. "disassembling" the Python code object of
the outer frame calling my function, to see how it has been called to be able to
reconstruct the command line:
import dis
import inspect
import io
import ast
import re
def get_function_call_string():
in_func = False
args=[]
kwargs=[]
func_name = inspect.stack()[1][3]
frame = inspect.currentframe().f_back.f_back
dis_output = io.StringIO()
dis.dis(frame.f_code, file=dis_output)
for line in dis_output.getvalue().split('\n'):
instr = re.findall(r'^.*\s([A-Z_]+)\s*', line)[0]
if instr.startswith("CALL_FUNCTION") and in_func:
break
elif instr.startswith('LOAD_'):
name = re.findall(r'^.*\s\((.+)\)$', line)[0]
if in_func:
if name.startswith('('):
kwargs = ast.literal_eval(name)
else:
args.append(name)
elif name == func_name:
in_func = True
kwargs = [f"{a}={args.pop()}" for a in kwargs]
return f"{func_name}({', '.join(args)}{', ' if kwargs else ''}{', '.join(kwargs)})"
Example:
>>> def f(a,b,arg="toto"):
print(get_function_call_string())
>>> f(1,"test","example")
f(1, 'test', 'example')
>>> f(1, "test", arg="hello")
(1, 'test', arg='hello')
It is not a complete answer, since it cannot handle some argument types like
dict or list... I maybe will continue on this or change my mind and do differently.

Dynamic generation of Python function with given parameter names

I would like to create functions in my Python3 code, making use of data supplied during run-time. I'm stuck on how I can write a function along the lines of
def foo(varname1, varname2):
return varname1 + varname2
where the strings varname1 and varname2 that give the parameter names are specified as arguments to some constructor function, e.g.:
def makeNewFooFunc(varname1, varname2):
# do magic
return foo
fooFunc = makeNewFooFunc('first', 'second')
print(fooFunc(first=1, second=2))
# should return 3
what would be the #do magic step? Is this a job for eval, or is there an alternative?

You don't need to write a function like that.
just use **kwargs
def foo_func(**kwargs):
return sum(kwargs.values())
foo_func(any_name=1, any_name_2=2)
but if you still need to do what you want, you can try
def make_new_func(var_name_1, var_name_2):
def foo(**kwargs):
# make sure in kwargs only expected parameters
assert set(kwargs) == {var_name_1, var_name_2}
return kwargs[var_name_1] + kwargs[var_name_2]
return foo
foo_func = make_new_func('a', 'b')
foo_func(a=1, b=2)

Get functions called in a Python expression

I have a database that holds the name of Python functions and a string for their code. I want the user to be able to enter a Python code and see the result. The problem is that I need to know the names of the functions they call in order to retrieve the code from the database. For instance, if they enter cubic_fit(1, 2, get_data()), I need a way to get the function names cubic_fit and get_data. Is there a good way to do this?

The built-in function compile will do that for you exactly:
>>> compile("cubic_fit(1, 2, get_data())", '<string>', 'eval').co_names
('cubic_fit', 'get_data')
And it is safe to run. No code is actually being executed just compiled.

A quick example to you started. Note that you'll be expecting valid python semantics for this to work.
You can extend this to also parse your arguments...
import token, tokenize, StringIO
def extract_names(src):
rawstr = StringIO.StringIO(unicode(src))
tokens = tokenize.generate_tokens(rawstr.readline)
for i, item in enumerate(tokens):
toktype, toktext, (srow,scol), (erow,ecol), line = item
if token.tok_name[toktype] == 'NAME':
print 'name:', toktext
extract_names("cubic_fit(1, 2, get_data())")
# --> output:
# name: cubic_fit
# name: get_data

If you just want the names, then the compile() and co_names method will work best.
You also might take advantage of the capability of eval() to use any mapping object as its locals parameter. You could create a mapping object to look up and compile the objects from your database as needed by eval().
Example:
class LookitUp(object):
def __init__(self):
# simulate some data
self.d = { "foo": "def foo(a):\n return a + 2"}
def __getitem__(self,key):
localdict = {}
c = compile(self.d.get(key,""),"<string>","exec")
eval(c,globals(),localdict)
return localdict[key]
d = LookitUp()
def bar(a):
return a - 1
print "foo from database :",eval("foo(3)",globals(), d)
print "bar from globals():",eval("bar(3)",globals(), d)
print "foo(bar(3)) :",eval("foo(bar(3))",globals(), d)
Result:
foo from database : 5
bar from globals(): 2
foo(bar(3)) : 4
You may need to modify based on what your source in the database looks like, but it's a place to start.

Function Decorators

I like being able to measure performance of the python functions I code, so very often I do something similar to this...
import time
def some_function(arg1, arg2, ..., argN, verbose = True) :
t = time.clock() # works best in Windows
# t = time.time() # apparently works better in Linux
# Function code goes here
t = time.clock() - t
if verbose :
print "some_function executed in",t,"sec."
return return_val
Yes, I know you are supposed to measure performance with timeit, but this works just fine for my needs, and allows me to turn this information on and off for debugging very smoothly.
That code of course was from before I knew about function decorators... Not that I know much about them now, but I think I could write a decorator that did the following, using the **kwds dictionary:
some_function(arg1, arg2, ..., argN) # Does not time function
some_function(arg1, arg2, ..., argN, verbose = True) # Times function
I would nevertheless like to duplicate the prior working of my functions, so that the working would be something more like:
some_function(arg1, arg2, ..., argN) # Does not time function
some_function(arg1, arg2, ..., argN, False) # Does not time function
some_function(arg1, arg2, ..., argN, True) # Times function
I guess this would require the decorator to count the number of arguments, know how many the original function will take, strip any in excess, pass the right number of them to the function... I'm uncertain though on how to tell python to do this... Is it possible? Is there a better way of achieving the same?

Though inspect may get you a bit on the way, what you want is in general not possible:
def f(*args):
pass
Now how many arguments does f take? Since *args and **kwargs allow for an arbitrary number of arguments, there is no way to determine the number of arguments a function requires. In fact there are cases where the function really handles as many as there are thrown at it!
Edit: if you're willing to put up with verbose as a special keyword argument, you can do this:
import time
def timed(f):
def dec(*args, **kwargs):
verbose = kwargs.pop('verbose', False)
t = time.clock()
ret = f(*args, **kwargs)
if verbose:
print("%s executed in %ds" % (f.__name__, time.clock() - t))
return ret
return dec
#timed
def add(a, b):
return a + b
print(add(2, 2, verbose=True))
(Thanks Alex Martelli for the kwargs.pop tip!)

+1 on Stephan202's answer, however (putting this in a separate answer since comments don't format code well!), the following bit of code in that answer:
verbose = False
if 'verbose' in kwargs:
verbose = True
del kwargs['verbose']
can be expressed much more clearly and concisely as:
verbose = kwargs.pop('verbose', False)

it might be difficult but you can do something on these lines. Code below tries to remove any extra arguments and prints them out.
def mydeco(func):
def wrap(*args, **kwargs):
"""
we want to eat any extra argument, so just count args and kwargs
and if more(>func.func_code.co_argcount) first take it out from kwargs
based on func.func_code.co_varnames, else last one from args
"""
extraArgs = []
newKwargs = {}
for name, value in kwargs.iteritems():
if name in func.func_code.co_varnames:
newKwargs[name] = value
else:
extraArgs.append(kwargs[name])
diff = len(args) + len(newKwargs) - func.func_code.co_argcount
if diff:
extraArgs.extend(args[-diff:])
args = args[:-diff]
func(*args, **newKwargs)
print "%s has extra args=%s"%(func.func_name, extraArgs)
return wrap
#mydeco
def func1(a, b, c=3):
pass
func1(1,b=2,c=3, d="x")
func1(1,2,3,"y")
output is
func1 has extra args=['x']
func1 has extra args=['y']