I just installed TensorFlow using conda install tensorflowfrom the anaconda prompt. I'm using Python 3.6 on Windows 10.
I thought I'd try it out with something simple, like
rnd_ints = tf.random_normal([10], dtype=tf.float64)
When I call rnd_ints all I get is this:
<tf.Tensor 'random_normal:0' shape=(10,) dtype=float64>
I thought I was supposed to be getting an array object of some sort?
From the documentation:
A Tensor is a symbolic handle to one of the outputs of an Operation.
It does not hold the values of that operation's output, but instead
provides a means of computing those values in a TensorFlow tf.Session.
This class has two primary purposes:
A Tensor can be passed as an input to another Operation. This builds a
dataflow connection between operations, which enables TensorFlow to
execute an entire Graph that represents a large, multi-step
computation.
After the graph has been launched in a session, the value of the
Tensor can be computed by passing it to tf.Session.run. t.eval() is a
shortcut for calling tf.get_default_session().run(t).
The answer to your question: when you call tf.random_normal() you create a Tensor object, which does not have actual values stored. In order to get an output you'll need to run it inside a session. Here's how you can get an actual output:
import tensorflow as tf
rnd_ints = tf.random_normal([10], dtype=tf.float64)
with tf.Session() as sess:
rnd = sess.run(rnd_ints)
print(rnd)
# [-1.59628093 0.62648824 0.18566968 0.2274149 1.27171951 -0.18103614
# -2.05964716 0.37477217 0.3355942 -1.57350681]
Related
I'm finding a way to getting a tensor's value. In most case, the problem would be solved by calling "sess.run(target_op)". However, I want to know another way. I am editing the code downloaded from GitHub so there's already a session running code there. Without touching the session running part, is there any way to get some specific tensor value? In my case, the code is built for getting accuracy for image recognition. While session runs and doing the accuracy evaluation I also want to get "prediction" tensor value in the same session without creating another session. For example, an operation like tf.Print shows tensor value througha terminal window without running session directly(in the first figure we just have to do sess.run(e) to print out tensor from c)
example of tf.Print
a = tf.constant(5)
b = tf.constant(3)
c = tf.add(a,b)
#print tensor c (which is 8)
d = tf.Print(c,[c])
f = tf.constant(2)
e = tf.multiply(f,d)
sess = tf.Session()
#print operation can be executed without running the session directly
g = sess.run(e)`
Like the tf.Print is there any operation that gets tensor value without running session directly? (like the second figure)
example of operation I am looking for
More specifically, what I want is to get the value of tensor(with actual numbers and arrays, not just 'tensor' data structure)and pass it to the global variable to access the value freely even after the session closes. The session only executes the operator which is located at end of the graph while the tensor I want the value is located in the middle of the graph. With restriction that I cannot create more session than the original code has, is there any way to get the specific tensor value?( I can't use .eval() or .run() because either needs to access 'session'. the code I am editing runs the code by using slim.evaluate_once function and as session() is binded to the function, I cannot approach to session())
There is no reason why you can't just call any tensor from the graph, provided you feed in the appropriate feed_dict. For instance say you want a tensor called biasAdd:0 and your so called-end tensor is called prediction
Then you can just get this tensor and evaluate it:
tensor = graph.get_tensor_by_name("biasAdd:0")
tensor_value, prediction_value = ses.run([tensor, prediction],... )
In tensorflow you have to use run or eval to get a numerical value from the graph
I'm new at Tensorflow. I am having a litte trouble at understanding its constants. I have this simple code mentioned below:
import tensorflow as tf
vector = tf.constant([[1,2,3,4],[4,5,6,7],[8,9,1,2]],tf.int32,name="vector")
with tf.Session() as sess:
v = sess.run(vector)
argm = tf.argmax(v,1)
print(argm)
I expect this to return something like [4,7,8], as I understood from the documentation. Instead, I get this:
Tensor("ArgMax:0", shape=(3,), dtype=int64).
So, i don't know what am I doing wrong.
Alternatively to the answer of #James, you might want to use tensorflow's eager execution, which behaves more like "standard" python: operations are executed as you type them, no more graphs and Session.
import tensorflow as tf
tf.enable_eager_execution()
vector = tf.constant([[1,2,3,4],[4,5,6,7],[8,9,1,2]],tf.int32,name="vector")
argm = tf.argmax(vector,1)
print(argm)
Tensorflow operations like tf.argmax somewhat unintuitively don't perform the operation that they're stating, but add an operation to a graph that will be performed. When you run argm = tf.argmax(v,1), the return value is tensor that isn't evaluated yet.
If you want the result of the argmax operation, you can run something like this:
import tensorflow as tf
vector = tf.constant([[1,2,3,4],[4,5,6,7],[8,9,1,2]],tf.int32,name="vector")
argm = tf.argmax(vector,1)
with tf.Session() as sess:
a = sess.run(argm)
print(a)
With this code, we're explicitly asking Tensorflow to run the computations to compute the result of the tf.argmax operation. With your previous code, we ran the computation to compute v (which is a constant, so that's pretty quick), then define a new graph operation to compute argmax on that - but never actually do the computation.
import tensorflow as tf
def read_data(file_queue):
reader = tf.TextLineReader(skip_header_lines=1)
key, value = reader.read(file_queue)
defaults = [[0], [0]]
value1, value2 = tf.decode_csv(value, defaults)
return tf.stack(value1, value2)
But I would like to deal with some of the data, such as "3 wins 2 fails" converted to [3, 2]
Using tensorflow you're describing a computation. value1,value2 (and any other operation resulting from a tf.* call) are symbolic variables, that points to nodes in a graph.
That's why if you print "the data" you get Tensor("DecodeCSV:0", shape=(), dtype=int32), that's the python representation of "the data".
The actual data, instead, is present only once the graph is built and placed into a Session.
In short, if you want to extract the "real data" you have to exit from the tensorflow graph and fetch the values (thus forcing the execution of the operations described in the graph).
You have to do something like:
sess = tf.Session()
v1,v2 = sess.run([value1, value2])
return v2,v2
However, this is not the correct way of using tensorflow.
Instead, you have to describe as much as possible the computation, and then execute everything into the graph when needed.
(creating a session, allocate the memory, place the graph into it, execute the operations, data transfer, ... are heavy operations that should be done not so frequently)
Therefore, I suggest you of having a look at the control flow operation that tensorflow offers: https://www.tensorflow.org/api_guides/python/control_flow_ops#Comparison_Operators
You can control the flow of the values into the graph using them, avoiding to force useless exchanges of data between tensorflow and python
I'm after advice on how to debug what on Tensorflow is struggling with when it hangs.
I have a multi layer CNN which hangs upon global_variables_initializer() is run in the session. I am getting no errors or messages on the console output.
Is there an intelligent way of debugging what Tensorflow is struggling with when it hangs instead of repeatedly commenting out lines of code that makes the graph, and re-running to see where it hangs. Would TensorFlow debugger (tfdbg) help? What options do I have?
Ideally it would be great to just to break current execution and look at some stack or similar to see where the execution is hanging during the init.
I'm currently running Tensorflow 0.12.1 with Python 3 inside a Jupiter notebook.
I managed to solve the problem. The tip from #amo-ej1 to run in a regular file was a step in the correct direction. This uncovered that the tensor flow process was killing itself off with a SIGKILL and returning an error code of 137.
I tried Tensorflow Debugger tfdbg though this did not provide any further details as the problem was the graph did not initialize. I started to think the graph structure was incorrect, so I dumped out the graph structure using:
tf.summary.FileWriter('./logs/traing_graph', graph)
I then used up Tensorboard to inspect the resultant summary graph structure data dumped out the the directory and found that the tensor dimensions of the Fully Connected layer was wrong , having a width of 15million !!?! (wrong)
It turned out that one of the configurable parameters of the graph was incorrect. It was picking the dimension of the layer 2 tensor shape incorrectly from an incorrect addressing the previous tf.shape type property and it exploded the dimensions of the graph.
There were no OOM error messages in /var/log/system.log so I am unsure why the graph initialisation caused the python tensorflow script process to die.
I fixed the dimensions of the graph and graph initialization worked just fine!
My top tip is visualise your graph with Tensorboard before initialisation and training to do a quick check the resultant graph structure you coded it what you expected it to be. You probably will save yourself a lot of time! :-)
A common methodology to debug tensorflow is to replace the placeholders and/or variables with numpy arrays and put them inside tf.const. When you do so you can actually examine the logic of your code by setting a breakpoints and to see numbers in "pythoninc" and not just tensors. It will be much easier to help you if you would post your code here, but here is a dummy example:
with tf.name_scope('scope_name'):
### This block is for debug only
import numpy as np
batch_size = 20
sess = tf.Session()
sess.run(tf.tables_initializer())
init_op = tf.global_variables_initializer()
sess.run(init_op)
### End of first debug block
## Replacing Placeholders for debug - uncomment the placehlolders and comment the numpy arrays to producation mode
const_a = tf.constant((np.random.rand(batch_size, 26) > 0.85).astype(int), dtype=tf.float32)
const_b = tf.constant(np.random.randint(0, 20, batch_size * 26).reshape((batch_size, 26)), dtype=tf.float32)
# real_a_placeholder = tf.log(input_placeholder_dict[A_DATA])
# real_b_placeholder = tf.log(input_placeholder_dict[B_DATA])
# dummy opreation
c = a - b
# selecting top k - in the sanity check you can see here that you actullay get the top items and top values
top_k = 5
top_k_values, top_k_indices = tf.nn.top_k(c,
k=top_k, sorted=True,
name="top_k")
## Replacing Variable for debug - uncomment the variables and comment the numpy arrays to producation mode
Now, run your code with breakpoints and you have 2 options to see the values in the debugger:
1.sess.run(palceholder_name)
2.you can use eval - varaible_name.eval(sessnio=sess)
I'm trying to run open code of A3C reinforcement learning algorithm to learn A3C in A3C code
However,I got several errors and I could fix except one.
In the code, ref() which is a member function of tf.Variable is used (1,2), but in recent tensorflow version 0.12rc, that function seems to be deprecated.
So I don't know what is the best way to replace it (I don't understand exactly why the author used ref()). When I just changed it to the variable itself (for example v.ref() to v), there was no error, but reward is not changed. It seems it cannot learn and I guess it is because the variables are not properly updated.
Please advise me what is the proper way to modify the code to work.
The new method tf.Variable.read_value() is the replacement for tf.Variable.ref() in TensorFlow 0.12 and later.
The use case for this method is slightly tricky to explain, and is motivated by some caching behavior that causes multiple uses of a remote variable on a different device to use a cached value. Let's say you have the following code:
with tf.device("/cpu:0")
v = tf.Variable([[1.]])
with tf.device("/gpu:0")
# The value of `v` will be captured at this point and cached until `m2`
# is computed.
m1 = tf.matmul(v, ...)
with tf.control_dependencies([m1])
# The assign happens (on the GPU) after `m1`, but before `m2` is computed.
assign_op = v.assign([[2.]])
with tf.control_dependencies([assign_op]):
with tf.device("/gpu:0"):
# The initially read value of `v` (i.e. [[1.]]) will be used here,
# even though `m2` is computed after the assign.
m2 = tf.matmul(v, ...)
sess.run(m2)
You can use tf.Variable.read_value() to force TensorFlow to read the variable again later, and it will be subject to whatever control dependencies are in place. So if you wanted to see the result of the assign when computing m2, you'd modify the last block of the program as follows:
with tf.control_dependencies([assign_op]):
with tf.device("/gpu:0"):
# The `read_value()` call will cause TensorFlow to transfer the
# new value of `v` from the CPU to the GPU before computing `m2`.
m2 = tf.matmul(v.read_value(), ...)
(Note that, currently, if all of the ops were on the same device, you wouldn't need to use read_value(), because TensorFlow doesn't make a copy of the variable when it is used as the input to an op on the same device. This can cause a lot of confusion—for example when you enqueue a variable to a queue!—and it's one of the reasons that we're working on enhancing the memory model for variables.)