I want to make a function that does the following:
def func(*args):
for arg in args:
arg+=1
a = 5
b = 6
c = 7
func(a,b,c)
print("%i,%i,%i"%(a,b,c))
I want it to return:
6,7,8
How would I do this?
You can’t! Ta-da.
Python does not support pass-by-reference in any form. Return values instead:
def func(*args):
return [arg + 1 for arg in args]
a = 5
b = 6
c = 7
a, b, c = func(a, b, c)
print("%i,%i,%i" % (a, b, c))
You can't do this, because when you pass in a variable, it takes it in as its value, not as its variable.
Instead, return the value:
def func(*args):
args = list(args)
for i in range(len(args)):
args[i]+=1
return args
a = 5
b = 6
c = 7
a, b, c = func(a,b,c)
print("%i,%i,%i"%(a,b,c))
Which outputs:
>>> print("%i,%i,%i"%(a,b,c))
6,7,8
>>>
You can't, at least not with integer values. Integers are immutable, so you can't change their values, and a function doesn't have access to the namespace of its caller, so you can't rebind the variables (i.e., assign a new value to the variable a outside the function). See this question and various others about what you can and cannot do to affect variables in functions.
If your variables are mutable types like lists, you can achieve a similar effect by mutating the list's value:
def func(*args):
for arg in args:
arg[0] += 1
a = [5]
b = [6]
c = [7]
func(a,b,c)
print("%i,%i,%i"%(a,b,c))
However, you should think about why you want to do this. It may be better to simply return the values and assign them outside the function.
3 posts to tell "You can't" But "Impossible n'est pas français".
Python is the lingua franca of programming languages.
So it's possible:
#!/usr/bin/env python
def func(args):
for i in range(len(args)):
args[i] += 1
abc = [5, 6, 7]
func(abc)
print("%i,%i,%i" % tuple(abc))
actually prints
6,7,8
You can't do it easily because Python doesn't pass immutable objects such as integers by reference. However if you pass the function the names of objects in the current scope, you can achieve your goal like this:
import sys
def func(*args):
namespace = sys._getframe(1).f_globals # caller's globals
for arg in args:
namespace[arg] += 1
a = 5
b = 6
c = 7
func('a','b','c') # note variable *names* passed to function
print("%i,%i,%i" % (a,b,c)) # -> 6,7,8
Related
In an unpacking assignment statement, can the assigned object inspect the number of variables it is being assigned to?
class MyObject:
def __iter__(self):
n = some_diabolical_hack()
print(f"yielding {n} vals")
return iter(["potato"]*n)
Something like:
>>> x, y = MyObject()
yielding 2 vals
>>> a, b, c = MyObject()
yielding 3 vals
In the more general case, can it introspect the "shape" of the target_list being used in an assignment?
>>> first, *blob, d[k], (x, y), L[3:7], obj.attr, last = MyObject()
unpacking to <_ast.Tuple object at 0xcafef00d>
Example potential use case: an improved MagicMock() which doesn't need to be pre-configured with a fixed iteration length when being used to patch out some object on the right hand side of an assignment statement.
You could use the traceback module:
import traceback
def diabolically_invoke_traceback():
call = traceback.extract_stack()[-2]
print call[3]
unpackers = call[3].split('=')[0].split(',')
print len (unpackers)
return range(len(unpackers))
In [63]: a, b, c = diabolically_invoke_traceback()
a, b, c = diabolically_invoke_traceback()
3
In [64]: a
Out[64]: 0
In [65]: b
Out[65]: 1
In [66]: c
Out[66]: 2
(Disclaimer: I don't recommend using diabolical techniques in production-quality code. Everything in this answer might not work on a different computer from mine, or a different Python version from mine, or on a non-CPython distribution, and it might not work tomorrow morning.)
Perhaps you could do this by inspecting the calling frame's bytecode. If I'm reading the bytecode guide correctly, multiple assignment is handled by the instructions UNPACK_SEQUENCE or UNPACK_EX, depending on whether the target list has a starred name. Both of these instructions provide information about the shape of the target list in their arguments.
You could write your diabolical function to climb the frame hierarchy until it finds the calling frame, and inspect the bytecode instruction that occurs after the FUNCTION_CALL that represents the right-hand-side of the assignment. (this is assuming that your call to MyObject() is the only thing on the right side of the statement). Then you can extract the target list size from the instruction's argument and return it.
import inspect
import dis
import itertools
def diabolically_retrieve_target_list_size():
#one f_back takes us to `get_diabolically_sized_list`'s frame. A second one takes us to the frame of the caller of `get_diabolically_sized_list`.
frame = inspect.currentframe().f_back.f_back
#explicitly delete frame when we're done with it to avoid reference cycles.
try:
#get the bytecode instruction that immediately follows the CALL_FUNCTION that is executing right now
bytecode_idx = frame.f_lasti // 2
unresolved_bytecodes = itertools.islice(dis.get_instructions(frame.f_code), bytecode_idx+1, bytecode_idx+3)
next_bytecode = next(unresolved_bytecodes)
if next_bytecode.opname == "UNPACK_SEQUENCE": #simple multiple assignment, like `a,b,c = ...`
return next_bytecode.arg
elif next_bytecode.opname == "EXTENDED_ARG": #multiple assignment with splat, like `a, *b, c = ...`
next_bytecode = next(unresolved_bytecodes)
if next_bytecode.opname != "UNPACK_EX":
raise Exception(f"Expected UNPACK_EX after EXTENDED_ARG, got {next_bytecode.opname} instead")
args_before_star = next_bytecode.arg % 256
args_after_star = next_bytecode.arg >> 8
return args_before_star + args_after_star
elif next_bytecode.opname in ("STORE_FAST", "STORE_NAME"): #single assignment, like `a = ...`
return 1
else:
raise Exception(f"Unrecognized bytecode: {frame.f_lasti} {next_bytecode.opname}")
finally:
del frame
def get_diabolically_sized_list():
count = diabolically_retrieve_target_list_size()
return list(range(count))
a,b,c = get_diabolically_sized_list()
print(a,b,c)
d,e,f,g,h,i = get_diabolically_sized_list()
print(d,e,f,g,h,i)
j, *k, l = get_diabolically_sized_list()
print(j,k,l)
x = get_diabolically_sized_list()
print(x)
Result:
0 1 2
0 1 2 3 4 5
0 [] 1
[0]
I am not sure the proper way to phrase this question. I would like to assign/bind some arithmetic (with references/pointers to other "subVariables") to a variable and have the value of the variable update if any of the contributing "subVariables" are updated.
>>> a = 1
>>> b = 2
>>> c = a + b
>>> c
3
>>> a = 2
>>> c
3
In the ideal situation c would have a value of 4 at the end of this code sample.
--
Additional information: I am generating the arithmetic randomly and would like to be able to nest these variables (ex: d = a + c would be the same as d = a + (a + b) where d would reflect any changes that happen to a or b)
What you want isn't possible with immutable built-in types like int. Because a and b are bound to immutable types (int), even if some hypothetical class preserved references to its inputs, those inputs never change (a can be rebound to a new value, but a would no longer have anything to do with the value it was previously bound to, and the class that aliased the old value of a would remain unchanged; it doesn't preserve a direct tie to a itself).
The only way this could possibly work is if a and b were of a mutable type where the contents could be updated or reassigned, and they had an overload of __add__/__radd__ that produced yet another class that stored references to instances of said mutable type and/or instances of itself (to allow the nested case). Either way, implementing such a suite of classes is way beyond the scope of a simple answer; I'd strongly recommend finding a different solution to your problem.
Make c as a function which returns a+b value
Simple numbers in Python are immutable references. You can't do it that directly.
You could create objects with that kind of behavior in various ways. Here's an approximation using simple function calls.
>>> a = 1
>>> b = 2
>>> def c():
return a + b
>>> c()
3
>>> a = 2
>>> c()
4
You can avoid the () at the cost of a dot by using __getattribute__
>>> class CallsAttrs:
def __getattribute__(self, attr):
return object.__getattribute__(self, attr)()
>>> Q = CallsAttrs()
>>> a = 1
>>> b = 2
>>> Q.c = lambda: a + b
>>> Q.c
3
>>> a = 2
>>> Q.c
4
And, of course, the lambdas can get the variables from Q too.
>>> Q.a = lambda: 1
>>> Q.b = lambda: 2
>>> Q.c = lambda: Q.a + Q.b
>>> Q.c
3
>>> Q.a = lambda: 40
>>> Q.c
42
You could also override the globals dict to work this way,
>>> class ThunkDict(dict):
def __getitem__(self, key):
return super().__getitem__(key)()
>>> exec("""
a = lambda: 1
b = lambda: 2
c = lambda: a + b
print(c)
a = lambda: -10
print(c)
""", ThunkDict())
3
-8
but it's not as interactive if you have to use exec.
This question already has answers here:
What do lambda function closures capture?
(7 answers)
Closed 6 months ago.
If I do
def f():
a = 1
g = lambda: a
a = 2
return g
print(f()())
The value that is printed is 2, because a is mutated after g is constructed.
How do I get g to capture the value of a statically so that later modifications are ignored?
For simple cases, such as when the code is short and we don't have many variables to capture, we can create a temporary lambda and call it:
def f():
a = 1
g = (lambda a: lambda: a)(a)
a = 2
return g
The issue here is that the code can quickly become harder to read.
Alternatively, we can capture the variable as an optional argument:
def f():
a = 1
g = lambda a=a: a
a = 2
return g
The issue here is, of course, that we might not want the caller to be able to specify this parameter.
(And the code can be a little less readable, too.)
A fully general solution might be the following, except that it does not capture globals:
def bind(*args, **kwargs):
# Use '*args' so that callers aren't limited in what names they can specify
func = args[0]
include_by_default = args[1] if len(args) > 1 else None
# if include_by_default == False, variables are NOT bound by default
if include_by_default == None: include_by_default = not kwargs
fc = func.__code__
fv = fc.co_freevars
q = func.__closure__
if q:
ql = []
i = 0
for qi in q:
fvi = fv[i]
ql.append((lambda v: (lambda: v).__closure__[0])(
kwargs.get(fvi, qi.cell_contents))
if include_by_default or fvi in kwargs
else qi)
i += 1
q = q.__class__(ql)
del ql
return func.__class__(fc, func.__globals__, func.__name__, func.__defaults__, q)
The reason I do not attempt to capture globals here is that the semantics can get confusing -- if an inner function says global x; x = 1, it certainly does want the global x to be modified, so suppressing this change would quickly make the code very counterintuitive.
However, barring that, we would be able to simply use it as follows:
def f():
a = 1
g = bind(lambda: a)
a = 2
return g
print(f()())
And voilà, a is instantly captured. And if we want to only capture some variables, we can do:
def f():
a = 1
b = 2
g = bind(lambda: a + b, b=5) # capture 'b' as 5; leave 'a' free
a = 2
b = 3
return g
print(f()())
In python3.8 the CellType class was added to types, which means you can create a function with a custom closure. That allows us to write a function that converts functions with closures that reference a parent frame to closures with static values:
from types import FunctionType, CellType
def capture(f):
""" Returns a copy of the given function with its closure values and globals shallow-copied """
closure = tuple(CellType(cell.cell_contents) for cell in f.__closure__)
return FunctionType(f.__code__, f.__globals__.copy(), f.__name__, f.__defaults__, closure)
print([f() for f in [ lambda: i for i in range(5)]]) # Outputs [4, 4, 4, 4, 4]
print([f() for f in [capture(lambda: i) for i in range(5)]]) # Outputs [0, 1, 2, 3, 4]
The capture function could be tweaked in a few ways; The current implementation captures all globals and free vars, and does so shallowly. You could decide to deepcopy the captured values, to capture only the free vars, to capture only specific variable names according to an argument, etc.
For example:
def func(a):
# how to get the name "x"
x = 1
func(x)
If I use inspect module I can get the stack frame object:
import inspect
def func(a):
print inspect.stack()
out:
[
(<frame object at 0x7fb973c7a988>, 'ts.py', 9, 'func', [' stack = inspect.stack()\n'], 0)
(<frame object at 0x7fb973d74c20>, 'ts.py', 18, '<module>', ['func(x)\n'], 0)
]
or use inspect.currentframe() I can get the current frame.
But I can only get the name "a" by inspect the function object.
Use inspect.stack I can get the call stack:"['func(x)\n']",
How can I get the name of actual parameters("x" here) when call the func by parsing the "['func(x)\n']"?
If I call func(x) then I get "x"
if I call func(y) then I get "y"
Edit:
A example:
def func(a):
# Add 1 to acttual parameter
...
x = 1
func(x)
print x # x expected to 2
y = 2
func(y)
print y # y expected to 3
Looking at your comment explanations, the reason you are trying to do this is:
Because I want to impelment Reference Parameters like in C++
You can't do that. In python, everything is passed by value, but that value is a reference to the object you pass. Now, if that object is mutable (like a list), you can modify it, but if it's immutable, a new object is created and set. In your code example, x is an int which is immutable, so if you want to change the value of x it, just return its new value from the function you are invoking:
def func(a):
b = 5 * a
# ... some math
return b
x = 1
x = func(x)
So what if you want to return multiple values? Use a tuple!
def func(a, b):
a, b = 5 * b, 6 * a
# ...
return b, a
a, b = 2, 3
a, b = func(a, b)
That is feasible, up to a point - but nonetheless,useless in any kind of "real code".
You can use it for backyard magic tricks:
Just retrieve the calling stack_frame like you are doing,
then loop linearly through the variables available in the calling frame for
one that references the same object you got as a parameter. The variables are in the
f_locals and f_globals dictionaries which are attributes of the frame object.
Of course, the parameter passed may not be in a variable name to start with:
it might be a constant in the caller, or it might be inside a dict or a list.
Either way, as #Nasser put it in the answer: it is not the way Python works. Objects exist in memory, and variable names, in each scope, just point to those objects. The names themselves are meaningless otherwise.
import inspect
from itertools import chain
def magic(x):
# note that in Python2, 'currentframe' could get a parameter
# to indicate which frame you want on the stack. Not so in Python3
fr = inspect.currentframe().f_back
for var_name, value in chain(fr.f_locals.items(), fr.f_globals.items()):
if value is x:
print ("The variable name in the caller context is {}".format(var_name))
def caler():
y = "My constant"
magic(y)
use lists as references
def func(a):
a[0] = a[0] + 1
x = [1]
func(x)
print x[0] # x expected to 2
y = [2]
func(y)
print y[0] # y expected to 3
This is my final solution. Use ast to parse the function statement and get the args.:
import inspect
import re
import ast
def func(a,b,c):
stack = inspect.stack()
stack_pre = stack[1]
function_call_string = ';'.join( stack_pre[4] ).strip('\n')
ret = re.search(r'(%s[ ]*\(.*\))' % func.func_name, function_call_string)
striped_function_call_string = ret.group()
parser = ast.parse(striped_function_call_string)
frame = inspect.currentframe()
for arg in parser.body[0].value.args:
iter_frame = frame.f_back
while iter_frame:
if arg.id in iter_frame.f_locals:
iter_frame.f_locals[arg.id] += 1
break
iter_frame = iter_frame.f_back
x=1;y=2;z=3;func(x,y,z)
print x,y,z
Out:
2 3 4
If I write this:
c = []
def cf(n):
c = range (5)
print c
if any((i>3) for i in c) is True:
print 'hello'
cf(1)
print c
Then I get:
[1, 2, 3, 4]
hello
[]
I'm really new to programming, so please explain it really simply, but how do I stop Python from forgetting what c is after the function has ended? I thought I could fix it by defining c before the function, but obviously that c is different to the one created just for the function loop.
In my example, I could obviously just write:
c = range (5)
def cf(n)
But the program I'm trying to write is more like this:
b = [blah]
c = []
def cf(n):
c = [transformation of b]
if (blah) is True:
'loop' cf
else:
cf(1)
g = [transformation of c that produces errors if c is empty or if c = b]
So I can't define c outside the function.
In python you can read global variables in functions, but you cant assigned to them by default. the reason is that whenever python finds c = it will create a local variable. Thus to assign to global one, you need explicitly specify that you are assigning to global variable.
So this will work, e.g.:
c = [1,2,3]
def cf():
print(c) # it prints [1,2,3], it reads global c
However, this does not as you would expect:
c = [1,2,3]
def cf():
c = 1 # c is local here.
print(c) # it prints 1
cf()
print(c) # it prints [1,2,3], as its value not changed inside cf()
So to make c be same, you need:
c = [1,2,3]
def cf():
global c
c = 1 # c is global here. it overwrites [1,2,3]
print(c) # prints 1
cf()
print(c) # prints 1. c value was changed inside cf()
To summarise a few of these answers, you have 3 basic options:
Declare the variable as global at the top of your function
Return the local instance of the variable at the end of your function
Pass the variable as an argument to your function
You can also pass the array c into the function after declaring it. As the array is a function argument the c passed in will be modified as long as we don't use an = statement. This can be achieved like this:
def cf(n, c):
c.extend(range(5))
print c
if any((i>3) for i in c) is True:
print 'hello'
if __name__ == '__main__':
c = []
cf(1, c)
print c
For an explanation of this see this
This is preferable to introducing global variables into your code (which is generally considered bad practice). ref
Try this
c = []
def cf(n):
global c
c = range (5)
print c
if any((i>3) for i in c) is True:
print 'hello'
cf(1)
print c
If you want your function to modify c then make it explicit, i.e. your function should return the new value of c. This way you avoid unwanted side effects:
def cf(n, b):
"""Given b loops n times ...
Returns
------
c: The modified value
"""
c = [transformation of b]
...
return c # <<<<------- This
c = cf(1)