I have a function in a file neural_network.py that defines a loss function:
def loss(a, b):
...
debug = tf.Print(a, [a], message = 'debug: ')
debug.eval(session = ???)
return tf.add(a, b)
To explain, somewhere in this function I want to print a tensor. However, I don't have any session declared in this function; my sessions are declared in another file called forecaster.py. Therefore, when I try to put tf.Print() in loss(), I can't because I don't know which session to eval with. Is there a way to solve this problem, either by using tf.Print() or other debug methods? Thanks!
tf.Print works as an identity function which returns the same tensor that you passed as the first parameter, having a side effect of printing the list of tensors specified as the second parameter.
So you should use as following:
def loss(a, b):
...
a = tf.Print(a, [a], message = 'debug: ')
return tf.add(a, b)
a will be printed each time tensor tf.add(a, b) is evaluated.
Related
I'm creating a wrapper for a function with functools.wraps. My wrapper has the effect of overriding a default parameter (and it doesn't do anything else):
def add(*, a=1, b=2):
"Add numbers"
return a + b
#functools.wraps(add)
def my_add(**kwargs):
kwargs.setdefault('b', 3)
return add(**kwargs)
This my_add definition behaves the same as
#functools.wraps(add)
def my_add(*, a=1, b=3):
return add(a=a, b=b)
except that I didn't have to manually type out the parameter list.
However, when I run help(my_add), I see the help string for add, which has the wrong function name and the wrong default argument for the parameter b:
add(*, a=1, b=2)
Add numbers
How can I override the function name and the default argument in this help() output?
(Or, is there a different way to define my_add, using for example some magic function my_add = magic(add, func_name='my_add', kwarg_defaults={'b': 3}) that will do what I want?)
Let me try and explain what happens.
When you call the help functions, this is going to request information about your function using the inspect module. Therefore you have to change the function signature, in order to change the default argument.
Now this is not something that is advised, or often preferred, but who cares about that right? The provided solution is considered hacky and probably won't work for all versions of Python. Therefore you might want to reconsider how important the help function is... Any way let's start with some explanation on how it was done, followed by the code and test case.
Copying functions
Now the first thing we will do is copy the entire function, this is because I only want to change the signature of the new function and not the original function. This decouples the new my_add signature (and default values) from the original add function.
See:
How to create a copy of a python function
How can I make a deepcopy of a function in Python?
For ideas of how to do this (I will show my version in a bit).
Copying / updating signature
The next step is to get a copy of the function signature, for that this post was very useful. Except for the part where we have to adjust the signature parameters to match the new keyword default arguments.
For that we have to change the value of a mappingproxy, which we can see when running the debugger on the return value of inspect.signature(g). Now so far this can only be done by changing the private variables (the values with leading underscores _private). Therefore this solution will be considered hacky and is not guaranteed to withstand possible updates. That said, let's see the solution!
Full code
import inspect
import types
import functools
def update_func(f, func_name='', update_kwargs: dict = None):
"""Based on http://stackoverflow.com/a/6528148/190597 (Glenn Maynard)"""
g = types.FunctionType(
code=f.__code__,
globals=f.__globals__.copy(),
name=f.__name__,
argdefs=f.__defaults__,
closure=f.__closure__
)
g = functools.update_wrapper(g, f)
g.__signature__ = inspect.signature(g)
g.__kwdefaults__ = f.__kwdefaults__.copy()
# Adjust your arguments
for key, value in (update_kwargs or {}).items():
g.__kwdefaults__[key] = value
g.__signature__.parameters[key]._default = value
g.__name__ = func_name or g.__name__
return g
def add(*, a=1, b=2):
"Add numbers"
return a + b
my_add = update_func(add, func_name="my_add", update_kwargs=dict(b=3))
Example
if __name__ == '__main__':
a = 2
print("*" * 50, f"\nMy add\n", )
help(my_add)
print("*" * 50, f"\nOriginal add\n", )
help(add)
print("*" * 50, f"\nResults:"
f"\n\tMy add : a = {a}, return = {my_add(a=a)}"
f"\n\tOriginal add: a = {a}, return = {add(a=a)}")
Output
**************************************************
My add
Help on function my_add in module __main__:
my_add(*, a=1, b=3)
Add numbers
**************************************************
Original add
Help on function add in module __main__:
add(*, a=1, b=2)
Add numbers
**************************************************
Results:
My add : a = 2, return = 5
Original add: a = 2, return = 4
Usages
f: is the function that you want to update
func_name: is optionally the new name of the function (if empty, keeps the old name)
update_kwargs: is a dictionary containing the key and value of the default arguments that you want to update.
Notes
The solution is using copy variables to make full copies of dictionaries, such that there is no impact on the original add function.
The _default value is a private variable, and can be changed in future releases of python.
Very new to python and I'm learning about defining and calling functions.
When def function4(x): and then define its output as a variable m, it prints the first two lines of the function before I even call it. Then when I call the function it only displays the return value. I was under the impression that anything indented under def function4(x): would not be executed unless function4(x) was specifically called?
Example:
def function4(x):
print(x)
print("still in this function")
return 3*x
m = function4(5)
print("BREAK")
print(m)
Output:
5
still in this function
BREAK
15
Process finished with exit code 0
Thanks for your time!
You're right, the function will not execute until you call it. You are calling it, though, right here:
m = function4(5)
So your print statements are executing in exactly the right place. You are setting m to the value returned by function4(5).
print does not call anything. It simply prints the string representation of whatever you give it to the console:
# a simple function to demonstrate
def f(x):
print("I am ", x)
return x
# I have not called f yet
print('Hello! ')
# I have printed the *function* f, but I still have not called it
# note the lack of parentheses
print('Here is a function: ', f)
print('We will call it now!')
# *Now* I am calling the function, as noted by the parentheses
x = f(1)
print('I have returned a value to x: ', x)
Which will do the following:
Hello!
Here is a function: <function f at 0x7fa958141840>
We will call it now!
I am 1
I have returned a value to x: 1
First of all I strongly recommend you that you use http://pythontutor.com/javascript.html#mode=edit if you want to know how your code is working, is very useful if you are new on Python.
Then, as for your code, you are calling the function when you declare the m variable, so that's the reason why the two print statements appear first.
The return value will only appear if you print the function, that's why the number 15 appears after all because you've printed it when you wrote print(m).
Hope this can help you, Goobye then and goodluck!.
I just tried to get an understanding of the TensorFlow naming behavior and I still need some clarification.
I worked on a project where I went into trouble with the naming of tensor, because they were predefined in a function, which was called later.
So I have the following example here:
import tensorflow as tf
def foo():
with tf.variable_scope("foo", reuse=True):
a = tf.placeholder(tf.float32,name="a")
b = tf.placeholder(tf.float32,name="b")
return a,b
##
a,b = foo()
print(a)
print(b)
I get the output:
Tensor("foo/a:0", dtype=float32)
Tensor("foo/b:0", dtype=float32)
When I call it again though, I get the output:
Tensor("foo_1/a:0", dtype=float32)
Tensor("foo_1/b:0", dtype=float32)
Why is this the case? I set reuse on true, so I expect the tensors to be in the same variable_scope "foo" again or that the program throws an error like "tensors already defined".
So, I tried a workaround with tf.get_variable:
def foo():
with tf.variable_scope("foo", reuse=True):
a = tf.get_variable("v", [1])
return a
##
a1 = foo()
print(a1)
graph = tf.get_default_graph()
#call tensors by name in tensorflow to avoid confusion with the naming
graph.get_tensor_by_name("foo/v:0")
Here, i get always the same output:
<tf.Variable 'foo/v:0' shape=(1,) dtype=float32_ref>
Unfortunately, I cannot work with variables because you cannot define a dynamic shape for them. You need placeholders to define variable shapes.
Can someone explain me why the program continues to create new variable_scopes for placeholders but not when I call tf.get_variable() ?
Thank you!
You can force the reuse of a scope by adding a '/' after the name i.e.: tf.variable_scope("foo/", reuse=True):
However that won't solve your problem.
In the case of variables, calling tf.Variable will always create a new variable, whereas calling tf.get_variable will reuse it if it already exists.
But with Placeholders there is no tf.get_placeholder.
What you can do is define your placeholders outside of foo, only once, and get them by name using tf.get_default_graph().get_tensor_by_name(name) or directly using the python variable whenever you need them.
example with get_tensor_by_name:
import tensorflow as tf
with tf.name_scope("scope"):
tf.placeholder(tf.float32,name="a")
tf.placeholder(tf.float32,name="b")
def foo():
a = tf.get_default_graph().get_tensor_by_name("scope/a:0")
b = tf.get_default_graph().get_tensor_by_name("scope/b:0")
return a,b
a,b = foo()
print(a)
print(b)
Note that placeholders, unlike variables, do not maintain a state that can be reused or not. They are merely a "pointer" to a tensor which will be fed later. They should not be part of your model, but an input to it, so you should not be creating them several times anyway.
Just for the case that you ran the code in a Jupyter notebook twice make sure that the notebook's kernel is not reusing the variables.
If you, instead, call the function foo() multiple times within the scope of the default graph you always get the same result:
def foo():
with tf.variable_scope("foo", reuse=True):
a = tf.placeholder(tf.float32,name="a")
b = tf.placeholder(tf.float32,name="b")
return a,b
with tf.Graph().as_default():
a,b = foo()
print(a) # gives variable name 'foo/a'
print(b) # gives variable name 'foo/b'
But that is different from the use case, where the function is called repeatedly to create more placeholders.
c,d = foo()
In this case the answer above of f4 is the recommended solution.
I want to print a tensor in my program to see its internal values once it gets evaluated. The problem, however, is that the tensor being declared inside a function. To understand my problem better, here is some example code to better explain what it is I want to do:
a = tf.Variable([[2,3,4], [5,6,7]])
b = tf.Variable([[1,2,2], [3,3,3]])
def divide(a,b):
with tf.variable_scope('tfdiv', reuse=True):
c = tf.divide(a,b, name='c')
# Cannot print(c) here, as this will only yield tf info on c
return c
d = divide(a,b)
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
sess.run(d)
sess.run(tf.get_variable('tfdiv/c:0').eval(session=sess))
Previously, I have been able to do a print(c.eval(session=sess)), but as c is a local variable inside a function now, that does not work. As can be seen in the code above, I have tried to use tensorflow's variable scope in order to access the variable and then evaluate it. Unfortunately, this results in the error message:
ValueError: Shape of a new variable (tfdiv/c:0) must be fully defined, but
instead was <unknown>.
I tried to use the reuse=True flag, but I still get the same error. Any thoughts on how I can solve this problem? Best would be if there is a print(c) equivalent that can be put into the divide function, as written in the code above.
This will achieve what you want to do:
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
print(sess.run(d))
Alternatively, you could replace the last line with:
print(sess.run(tf.get_default_graph().get_tensor_by_name('tfdiv/c:0')))
It is important to understand the difference between Python side code and TensorFlow side code. In python, you only setup the graph: d = divide(a, b) creates something like:
You set up a node (square) that will divide the data in the nodes a and b. It doesn't divide them right away! Note that in black you have the python variable names, and in gray you have the TensorFlow node names 1. a and b also have some default names, if you didn't specify them. The gray "c" you specified with name='c'. And local variable c and global variable d (Python) both refer to that same operation (node).
This is why if you say print(d) merely prints the info about that node. Once you setup the graph, doing sess.run(d) runs all the nodes required by the node in d on TensorFlow side. Then it retrieves the result and makes it available on python side as a numpy array.
You can use tf.Print(input, data) to print tensors on TF side. Note this is an operation (a node in the graph) that does nothing to the input tensor, it merely passes it through, while also printing everything in data.
In your case, you can use Print on tensorflow side it like this:
def divide(a,b):
with tf.variable_scope('tfdiv', reuse=True):
c = tf.divide(a,b, name='c')
cp = tf.Print(c, [c], message='Value of c: ', name='P')
return cp
This effectively adds another node in the graph (named P on TF side):
Now the value of operation c will be printed every time it will be evaluated. Note it will also be printed every time one of its dependencies will be evaluated, for example if you later do e = d + 1, when you evaluate e, it needs d, which refers to the printing node (returned from the function divide).
Finally, note that if you do this in a Jupyter notebook, the print will appear in the terminal of the notebook server. The details of this are not important for now :).
1 the :0 is added by default so that you can retrieve any tensor with by using name_of_op:0. Distinction between name of operation (tfdiv/c) and name of tensor(tfdiv/c:0).
I'm new to Python and have a problem-specific question about how to access non-returned variables defined within a function.
I am using an analysis package in Python that requires a user-defined function as input. It expects a function defined in a particular way that returns a single output. However, for diagnostic purposes, I would like to obtain multiple outputs from this function after it is called (e.g. to produce diagnostic plots at a particular stage of the analysis). However, I can't modify the function to return multiple outputs (unless I modify every instance of it being called by the analysis code--not practical); this would result in an error.
For instance, in the following example,the user defined function returns f1+f2, but for diagnostic purposes, say I would like to know what f1 and f2 are individually:
def my2dfunction(x,y,theta):
'''x and y are 1-d arrays of len(x)==len(y)
theta is an array of 5 model parameters '''
f1=theta[0]+theta[1]*x+theta[2]*x**2
f2=theta[3]*y+theta[4]*y**2
return f1+f2
Researching on this site I've come up with 2 possible solutions:
Create a global variable containing the values for f1 and f2 from the latest function call, which I can access at any time:
live_diagnostic_info={'f1':0, 'f2':0}
def my2dfunction(x,y,theta):
f1=theta[0]+theta[1]*x+theta[2]*x**2
f2=theta[3]*y+theta[4]*y**2
global live_diagnostic_info={'f1':f1, 'f2':f2}
return f1+f2
Define a second function identical to the first with multiple return values to call in only the instances where I need diagnostic information.
def my2dfunction_extra(x,y,theta):
f1=theta[0]+theta[1]*x+theta[2]*x**2
f2=theta[3]*y+theta[4]*y**2
return f1+f2,f1,f2
I think both would work for my purposes, but I'm wondering if there is another way to pass non-returned variables from a function in Python. (e.g. I do a lot of coding in IDL, where extra information can be passed through keywords, without modifying the return statement, and wonder what the Python equivalent would be if it exists).
You can write a decorator, so you don't need to copy-paste anything for your second solution:
def return_first(f):
return lambda *args, **kwargs: f(*args, **kwargs)[0]
return_first takes a function, and returns the same function, but with only first returned value.
def my2dfunction_extra(x,y,theta):
f1=theta[0]+theta[1]*x+theta[2]*x**2
f2=theta[3]*y+theta[4]*y**2
return f1+f2,f1,f2
my2dfunction = return_first(my2dfunction_extra)
Add an optional argument to the function
def my2dfunction(x,y,theta, local_debug=None):
'''x and y are 1-d arrays of len(x)==len(y)
theta is an array of 5 model parameters '''
f1=theta[0]+theta[1]*x+theta[2]*x**2
f2=theta[3]*y+theta[4]*y**2
if local_debug is not None:
local_debug["f1"] = f1
local_debug["f2"] = f2
return f1+f2
then call it with a dictionary
local_data = {}
my2dfunction(x,y,theta, local_data)
on return you have the information in the dict. Old client code will not be affected either in the return or in the supplied input values.
If instead you want to have them saved no matter who calls the routine, you can do the following. Create the routine as follows
def my2dfunction(x,y,theta, local_debug={}):
'''x and y are 1-d arrays of len(x)==len(y)
theta is an array of 5 model parameters '''
f1=theta[0]+theta[1]*x+theta[2]*x**2
f2=theta[3]*y+theta[4]*y**2
local_debug["f1"] = f1
local_debug["f2"] = f2
return f1+f2
then you can access the local_debug dictionary from outside
python3: my2dfunction.__defaults__[0]
python2: my2dfunction.func_defaults[0]
Keep in mind that having mutables (e.g. lists, dictionaries etc.) as defaults is a python faux pas, but in this case has a motivation.