I need to run a loop where I use the iterator only for the bound and access the result of the loop afterwards.
import tensorflow as tf
def add(h_tl):
res = tf.add(h_tl, tf.constant(1, shape=[2,1]))
return res
x = tf.constant(5) # iterator for while loop
h_tl = tf.constant(0, shape=[2,1])
H_l = tf.constant(0, shape=[2,1])
def body(x): # Body of loop
global H_l
global h_tl # I need to use H_l and h_tl in the body of the loop
h_tl = add(h_tl) # Some simple modifications to h_tl which will be added to H_l
H_l = tf.concat([H_l, h_tl], 1)
# I want the result of H_l after the loop and use it for further calculations
return (x+1)
def condition(x): # Stop condition
return x < tf.constant(10)
y = tf.while_loop(condition, body, [x])
sess = tf.Session()
print(sess.run(y))
print(sess.run(H_l)) # Result of the loop
sess.close()
So, I want to do calculations on H_l which is done for every x between 5 and 10. How do I access the end result of H_l after the loop is completed? When I run x I get the correct value of 10, but when I try to run H_l I get the following error: "Operation 'while/concat' has been marked as not fetchable."
TensorFlow while loops are designed to allow different iterations to be run in parallel. This causes them to behave rather differently from what you'd expect coming from other programming languages.
One such oddity is that if you create a tensor inside the loop, you cannot access it from outside. In your case, you are creating a new tensor with every tf.concat op. Yes, even though that tensor is stored under the same Python variable name as the previous version, it is still a new tensor: indeed, the original H_l is a constant, so its value cannot be changed; you can only create a new tensor in its place. The same is true for h_tl.
To solve this problem for H_l, what you want is to store an array of tensors, and add another entry to this array at each iteration. The data structure for doing this is tf.TensorArray, which you should initialize outside the loop and then write into at each iteration.
For h_tl, in which you only want to examine the final value, you don't need a tf.TensorArray; you could use a tf.Variable instead, and use tf.assign to update its value.
In both cases, take care to ensure that the mutating ops are executed, by including them in a tf.control_dependencies block.
Related
...and a suggestion to Use a.any() or a.all().
I am new to python and i am trying to implement a sabr model. I have defined a function with the following parameters:
def haganimpliedvol(a,f,k,B,v,t,p):
if k != f:
z = v/a*math.pow(f*k,(1-B)/2)*math.log(f/k)
xz = math.log((math.sqrt(1-2*p*z+math.pow(z,2))+z-p)/(1-p))
sigma = a/math.pow(f*k,(1-B)/2)*(1 + math.pow(1-B,2)/24* math.pow(math.log(f/k),2)+\
math.pow(1-B,4)/1920* math.pow(math.log(f/k),4))*\
xz*\
(1+(math.pow(1-B,2)/24*math.pow(a,2)/math.pow(f/k,1-B)+1/4*(p*B*v*a)/math.pow(f/k,(1-B)/2)+\
(2-3*math.pow(p,2))/24*math.pow(v,2)))*t
else:
sigma = a/math.pow(f,1-B)*\
(1+(math.pow(1-B,2)/24*math.pow(a,2)/math.pow(f,(2-2*B))+\
1/4*(p*B*a*v)/math.pow(f,1-B)+(2-3*math.pow(p,2))/24*math.pow(v,2)))*t
return(sigma)
Now I define another function to and call the haganimpliedvol() function
params = [0.4,0.6,0.1,-0.4]
def objective(params):
global k,sigma_iv,t,f
a = params[0]
B = params[1]
v = params[2]
p = params[1]
for (i,j,k) in zip(k,t,f):
calc_vols = np.array([haganimpliedvol(a,f,k,B,v,t,p)])
return(calc_vols)
As can be seen, a few parameters in the functions are list. I want to get an array as an output. However, I keep getting the message in the subject line.
Pay attention to the variables in this call:
for (i,j,k) in zip(k,t,f):
calc_vols = np.array([haganimpliedvol(a,f,k,B,v,t,p)])
for the zip to work, k,t, f have to be lists or arrays of matching size;
Done use k for an iteration variable; it is already used in the zip. I think you are just being careless here; or confused.
And the arguments to the hagen... function. Are the f, k, t supposed to be variables used in the zip? It would make more sense to use the iteration variables (i,j,?). Again, this just looks like you are careless, or don't care what happens.
As for the ambiguity error, that most likely arises in the
if k != f:
If either k or f is an array (or both) the k!=f will be a boolean array. That can't be used in if, which requires a simple True or False value. It does not iterate on the conditions. It is a basic Python if - a switch.
This ambiguity error comes up frequently, in various contexts, but all with the same basic issue - using an array in a context that requires a scalar T/F. A simple web search should provide lots of examples.
#hpaulj thank you for leading me on the right path. I vectorized my function and made some edits and now it is working fine.
haganimpliedvol = np.vectorize(haganimpliedvol,excluded = ['a','B','v','p'])
params = [0.2,0.7,0.01,-0.4]
def objective(params):
global k,sigma_iv,t,f
a = params[0]
B = params[1]
v = params[2]
p = params[1]
calc_vols = haganimpliedvol(a,f,k,B,v,t,p)
return(calc_vols)
Are you sure you want to pass arrays into the haganimpliedvol() function?
The general convention is to write functions which take a single input type.
Maybe call it one per item in the array?
Or write the function in a way that, if it sees the input is a list it iterates and if it sees the inputs arent lists then it just calculates it one time.
See this thread for ideas
How to make a function that can handle single inputs or lists of inputs
I'm trying to dynamically add function calls to fill in array columns. I will be accessing the array millions of times so it needs to be quick.
I'm thinking to add the call of a function into a dictionary by using a string variable
numpy_array[row,column] = dict[key[index containing function call]]
The full scope of the code I'm working with is too large to post here is an equivalent simplistic example I've tried.
def hello(input):
return input
dict1 = {}
#another function returns the name and ID values
name = 'hello'
ID = 0
dict1["hi"] = globals()[name](ID)
print (dict1)
but it literally activates the function when using
globals()[name](ID)
instead of copy pasting hello(0) as a variable into the dictionary.
I'm a bit out of my depth here.
What is the proper way to implement this?
Is there a more efficient way to do this than reading into a dictionary on every call of
numpy_array[row,column] = dict[key[index containing function call]]
as I will be accessing and updating it millions of times.
I don't know if the dictionary is called every time the array is written to or if the location of the column is already saved into cache.
Would appreciate the help.
edit
Ultimately what I'm trying to do is initialize some arrays, dictionaries, and values with a function
def initialize(*args):
create arrays and dictionaries
assign values to global and local variables, arrays, dictionaries
Each time the initialize() function is used it creates a new set of variables (names, values, ect) that direct to a different function with a different set of variables.
I have an numpy array which I want to store information from the function and associated values created from the initialize() function.
So in other words, in the above example hello(0), the name of the function, it's value, and some other things as set up within initialize()
What I'm trying to do is add the function with these settings to the numpy array as a new column before I run the main program.
So as another example. If I was setting up hello() (and hello() was a complex function) and when I used initialize() it might give me a value of 1 for hello(1).
Then if I use initialize again it might give me a value of 2 for hello(2).
If I used it one more time it might give the value 0 for the function goodbye(0).
So in this scenaro let's say I have an array
array[row,0] = stuff()
array[row,1] = things()
array[row,2] = more_stuff()
array[row,3] = more_things()
Now I want it to look like
array[row,0] = stuff()
array[row,1] = things()
array[row,2] = more_stuff()
array[row,3] = more_things()
array[row,4] = hello(1)
array[row,5] = hello(2)
array[row,6] = goodbye(0)
As a third, example.
def function1():
do something
def function2():
do something
def function3():
do something
numpy_array(size)
initialize():
do some stuff
then add function1(23) to the next column in numpy_array
initialize():
do some stuff
then add function2(5) to the next column in numpy_array
initialize():
do some stuff
then add function3(50) to the next column in numpy_array
So as you can see. I need to permanently append new columns to the array and feed the new columns with the function/value as directed by the initialize() function without manual intervention.
So fundamentally I need to figure out how to assign syntax to an array column based upon a string value without activating the syntax on assignment.
edit #2
I guess my explanations weren't clear enough.
Here is another way to look at it.
I'm trying to dynamically assign functions to an additional column in a numpy array based upon the output of a function.
The functions added to the array column will be used to fill the array millions of times with data.
The functions added to the array can be various different function with various different input values and the amount of functions added can vary.
I've tried assigning the functions to a dictionary using exec(), eval(), and globals() but when using these during assignment it just instantly activates the functions instead of assigning them.
numpy_array = np.array((1,5))
def some_function():
do some stuff
return ('other_function(15)')
#somehow add 'other_function(15)' to the array column.
numpy_array([1,6] = other_function(15)
The functions returned by some_function() may or may not exist each time the program is run so the functions added to the array are also dynamic.
I'm not sure this is what the OP is after, but here is a way to make an indirection of functions by name:
def make_fun_dict():
magic = 17
def foo(x):
return x + magic
def bar(x):
return 2 * x + 1
def hello(x):
return x**2
return {k: f for k, f in locals().items() if hasattr(f, '__name__')}
mydict = make_fun_dict()
>>> mydict
{'foo': <function __main__.make_fun_dict.<locals>.foo(x)>,
'bar': <function __main__.make_fun_dict.<locals>.bar(x)>,
'hello': <function __main__.make_fun_dict.<locals>.hello(x)>}
>>> mydict['foo'](0)
17
Example usage:
x = np.arange(5, dtype=int)
names = ['foo', 'bar', 'hello', 'foo', 'hello']
>>> np.array([mydict[name](v) for name, v in zip(names, x)])
array([17, 3, 4, 20, 16])
I'm learning python, and I'm having trouble saving the return of a function to a specific variable. I have computed a simple function 'average' that is supposed to return the average value of a list. This works fine, however, when I try to store the result of average in a variable, I get told that x isn't defined.
def average(x):
return sum(x)/len(x)
var=average(x)
How do I store the return of the function in a variable?
Edit:
I misunderstood the task, which was simply to store the results of a specific computation in a variable.
x indeed is not defined
def average(x):
return sum(x)/len(x)
x = [1,2,3] # this was missing
var=average(x)
https://repl.it/join/eldrjqcr-datamafia
The function is a black box. You made the black box expect one mandatory input (x), therefore you have to provide it first i.e. var = average([1, 2, 3]).
Read the error message, x isn't defined. The variable x exists only in the average function and not in the program. You need to set x to something first.
e.g.
def average(x):
return sum(x)/len(x)
x=[1,2,3]
var=average(x)
this will not cause an error
I want to print a tensor in my program to see its internal values once it gets evaluated. The problem, however, is that the tensor being declared inside a function. To understand my problem better, here is some example code to better explain what it is I want to do:
a = tf.Variable([[2,3,4], [5,6,7]])
b = tf.Variable([[1,2,2], [3,3,3]])
def divide(a,b):
with tf.variable_scope('tfdiv', reuse=True):
c = tf.divide(a,b, name='c')
# Cannot print(c) here, as this will only yield tf info on c
return c
d = divide(a,b)
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
sess.run(d)
sess.run(tf.get_variable('tfdiv/c:0').eval(session=sess))
Previously, I have been able to do a print(c.eval(session=sess)), but as c is a local variable inside a function now, that does not work. As can be seen in the code above, I have tried to use tensorflow's variable scope in order to access the variable and then evaluate it. Unfortunately, this results in the error message:
ValueError: Shape of a new variable (tfdiv/c:0) must be fully defined, but
instead was <unknown>.
I tried to use the reuse=True flag, but I still get the same error. Any thoughts on how I can solve this problem? Best would be if there is a print(c) equivalent that can be put into the divide function, as written in the code above.
This will achieve what you want to do:
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
print(sess.run(d))
Alternatively, you could replace the last line with:
print(sess.run(tf.get_default_graph().get_tensor_by_name('tfdiv/c:0')))
It is important to understand the difference between Python side code and TensorFlow side code. In python, you only setup the graph: d = divide(a, b) creates something like:
You set up a node (square) that will divide the data in the nodes a and b. It doesn't divide them right away! Note that in black you have the python variable names, and in gray you have the TensorFlow node names 1. a and b also have some default names, if you didn't specify them. The gray "c" you specified with name='c'. And local variable c and global variable d (Python) both refer to that same operation (node).
This is why if you say print(d) merely prints the info about that node. Once you setup the graph, doing sess.run(d) runs all the nodes required by the node in d on TensorFlow side. Then it retrieves the result and makes it available on python side as a numpy array.
You can use tf.Print(input, data) to print tensors on TF side. Note this is an operation (a node in the graph) that does nothing to the input tensor, it merely passes it through, while also printing everything in data.
In your case, you can use Print on tensorflow side it like this:
def divide(a,b):
with tf.variable_scope('tfdiv', reuse=True):
c = tf.divide(a,b, name='c')
cp = tf.Print(c, [c], message='Value of c: ', name='P')
return cp
This effectively adds another node in the graph (named P on TF side):
Now the value of operation c will be printed every time it will be evaluated. Note it will also be printed every time one of its dependencies will be evaluated, for example if you later do e = d + 1, when you evaluate e, it needs d, which refers to the printing node (returned from the function divide).
Finally, note that if you do this in a Jupyter notebook, the print will appear in the terminal of the notebook server. The details of this are not important for now :).
1 the :0 is added by default so that you can retrieve any tensor with by using name_of_op:0. Distinction between name of operation (tfdiv/c) and name of tensor(tfdiv/c:0).
I was trying to build an autoincrementing graph in TensorFlow. I thought that the assign op might be suitable for that, but found no documentation for it.
I assumed that this op returns its value—like in C-like languages—and wrote the following code:
import tensorflow as tf
counter = tf.Variable(0, name="counter")
one = tf.constant(1)
ten = tf.constant(10)
new_counter = tf.add(counter, one)
assign = tf.assign(counter, new_counter)
result = tf.add(assign, ten)
init_op = tf.initialize_all_variables()
with tf.Session() as sess:
sess.run(init_op)
for _ in range(3):
print sess.run(result)
and this code works.
The question is: is this the expected behavior? Why is the assign op not documented here: https://www.tensorflow.org/versions/0.6.0/api_docs/index.html
Is it a non-recommended op?
The tf.assign() operator is the underlying mechanism that implements the Variable.assign() method. It takes a mutable tensor (with tf.*_ref type) and a new value, and returns a mutable tensor that has been updated with the new value. The return value is provided to make it easier to order an assignment before a subsequent read, but this feature is not well documented. An example will hopefully illustrate:
v = tf.Variable(0)
new_v = v.assign(10)
output = v + 5 # `v` is evaluated before or after the assignment.
sess.run(v.initializer)
result, _ = sess.run([output, new_v.op])
print result # ==> 10 or 15, depending on the order of execution.
v = tf.Variable(0)
new_v = v.assign(10)
output = new_v + 5 # `new_v` is evaluated after the assignment.
sess.run(v.initializer)
result = sess.run([output])
print result # ==> 15
In your code example the dataflow dependencies enforce the order of execution [read counter] -> new_counter = tf.add(...) -> tf.assign(...) -> [read output of assign] -> result = tf.add(...), which means that the semantics are unambiguous. However, the read-modify-write steps to update the counter are somewhat inefficient, and can have unexpected behavior when there are multiple steps running concurrently. For example, multiple threads accessing the same variable could observe the counter moving backwards (in the case that an older value was written back after a newer value).
I would recommend that you use Variable.assign_add() to update the counter, as follows:
counter = tf.Variable(0, name="counter")
one = tf.constant(1)
ten = tf.constant(10)
# assign_add ensures that the counter always moves forward.
updated_counter = counter.assign_add(one, use_locking=True)
result = tf.add(updated_counter, ten)
# ...
tf.assign() is nicely documented in the latest versions and it is used frequently in the projects.
This operation outputs "ref" after the assignment is done. This makes
it easier to chain operations that need to use the reset value.
In easier words it takes your original tensor and a new tensor. It updates original value of your tensor with a new value and returns the reference of your original tensor.