Develop Reference

Why match_filenames_once function returns a local variable

I was trying to understand the mechanism of tensorflow for reading images using queues. I was using the code found here, whom basic parts are:
filename_queue = tf.train.string_input_producer(tf.train.match_filenames_once('D:/Dataset/*.jpg'))
image_reader = tf.WholeFileReader()
image_name, image_file = image_reader.read(filename_queue)
image = tf.image.decode_jpeg(image_file)
with tf.Session() as sess:
tf.global_variables_initializer().run()
coord = tf.train.Coordinator()
threads = tf.train.start_queue_runners(coord=coord)
image_tensor = sess.run([image])
print(image_tensor)
which in reality does nothing special. I was getting an error:
OutOfRangeError (see above for traceback): FIFOQueue
'_0_input_producer' is closed and has insufficient elements (requested
1, current size 0)
which lead to search for missing images, wrong folder, wrong glob pattern etc until I discovered that tensorflow basically meant this:
"You need to initialize local variables also"!
Besides the fact that the code seemed to work in the original gist with just this substitution:
tf.initialize_all_variables().run()
instead of
tf.global_variables_initializer().run()
in my code it does not work. It produces the same error. I guess it has changed the implementation of initialize_all_variables() with tensorflow development (I am using 1.3.0), since in here it mentions that it initialize local variables also.
So, the final conclusion I came with was that I should initialize local variables also. And my code worked. The error message is awfully misleading (which did not help at all) but anyway to the main part I am a bit confused why am I getting a local variable by match_filenames_once. In documentation there is no reference about this (I am not sure it should though).
Am I always going to get local from this match_filenames_once? Can I control it somehow?

Value Errors on tensorflow

I write a simple code with the tensorflow toolkit as follow:
import tensorflow as tf
import numpy as np
if __name__=="__main__":
inp = np.random.randint(1,3,(1,20,300,1))
inputs = tf.convert_to_tensor(inp,dtype=tf.float32)
with tf.variable_scope('convpool1') as scope:
kernel = tf.get_variable('weights',[1,300,1,1],initializer=tf.truncated_normal_initializer(stddev=5e-2,dtype=tf.float32))
conv1 = tf.nn.conv2d(inputs,kernel,[1,1,1,1],padding='VALID')
init = tf.initialize_all_variables()
with tf.Session() as sess:
sess.run(init)
with tf.variable_scope('convpool1'):
k_ = sess.run(kernel)
c1_ =sess.run(conv1)
It works when I run this code first time,but when I run the same again,it raises an　Error:
Variable convpool1/weights already exists, disallowed. Did you mean to set reuse=True in VarScope?
And I restart the IDE(spyder), run this code,it works again.What could be the origin of this error and how i can resolve it? Thanks for your time!

Using
tf.reset_default_graph()
at the beginning should resolve your problem. I am not 100% sure but suspect spyder still have the Graph object (and thus the var scope is already created).

TensorFlow: Node not found

I am training a neural network and have been running this code without any problems but sometimes (twice) I get an error Not Found: FetchOutputs node not found at the line y_1 = sess.run(get_labels(step)) (See below).
get_labels(step) is a function to return the correct labels of my training images which is in a text file.
def get_labels(step):
with open('labels.txt','r') as fin:
reader = csv.reader(fin)
c = [[int(s) for s in row] for i,row in enumerate(reader) if i==step]
label_numbers = np.array(c)
# Convert to one-hot vectors
numpy_label = np.zeros((BATCH_SIZE,5))
for i in range(BATCH_SIZE):
numpy_label[i,label_numbers[0][i]-1] = 1
# Convert to tensor
y_label = tf.convert_to_tensor(numpy_label,dtype=tf.float32)
return y_label
This is my main function:
def main():
# Placeholder for correct labels
y_label = tf.placeholder(tf.float32,shape=[BATCH_SIZE,5])
< Other functions etc. >
sess.run(tf.initialize_all_variables())
tf.train.start_queue_runners(sess=sess)
for step in range(1000):
# Get labels for current batch
y_1 = sess.run(get_labels(step))
# Train
sess.run([train_step],feed_dict={y_label:y_1})
< Other stuff like writing summaries, saving variables etc. >
sess.close()
From reading some of the issues on GitHub, I know this is to do with the fact that I call y_1 = sess.run(get_labels(step)) after tf.train.start_queue_runners(sess=sess) but I don't understand:
why it works most of the time, but occasionally doesn't?
Is y_1 = sess.run(get_labels(step)) adding or modifying nodes in the graph? I thought I was just running a node get_labels(step) that was already defined in the graph. I tried finalizing the graph before starting the queue runners but that gave me the error that finalized graphs cannot be modified.
What would be the proper way to write the code? Usually I just restart my program and it is fine - but clearly I am not doing it the proper way.
Thank you!
EDIT:
I think it might be important to mention that this happens when I am trying to run a TensorFlow script in a separate screen on a server i.e. I have one screen running a TensorFlow script and now I create a new screen to run a different TensorFlow script. I just started using screens so I might be missing something fundamental about how they work.

Display image of graph in TensorFlow?

I wrote a simple script to calculate the golden ratio from 1,2,5. Is there a way to actually produce a visual through tensorflow (possibly with the aid of matplotlib or networkx) of the actual graph structure? The doc of tensorflow is pretty similar to a factor graph so I was wondering:
How can an image of the graph structure be generated through tensorflow?
In this example below, it would be C_1, C_2, C_3 as individual nodes, and then C_1 would have the tf.sqrt operation followed by the operation that brings them together. Maybe the graph structure (nodes,edges) can be imported into networkx? I see that the tensor objects have a graph attribute but I haven't found out how to actually use this for imaging purposes.
#!/usr/bin/python
import tensorflow as tf
C_1 = tf.constant(5.0)
C_2 = tf.constant(1.0)
C_3 = tf.constant(2.0)
golden_ratio = (tf.sqrt(C_1) + C_2)/C_3
sess = tf.Session()
print sess.run(golden_ratio) #1.61803
sess.close()

This is exactly what tensorboard was created for. You need to slightly modify your code to store the information about your graph.
import tensorflow as tf
C_1 = tf.constant(5.0)
C_2 = tf.constant(1.0)
C_3 = tf.constant(2.0)
golden_ratio = (tf.sqrt(C_1) + C_2)/C_3
with tf.Session() as sess:
writer = tf.summary.FileWriter('logs', sess.graph)
print sess.run(golden_ratio)
writer.close()
This will create a logs folder with event files in your working directory. After this you should run tensorboard from your command line tensorboard --logdir="logs" and navigate to the url it gives you (http://127.0.0.1:6006). In your browser go to GRAPHS tab and enjoy your graph.
You will use TB a lot if you are going to do anything with TF. So it makes sense to learn about it more from official tutorials and from this video.

You can get an image of the graph using Tensorboard. You need to edit your code to output the graph, and then you can launch tensorboard and see it. See, in particular, TensorBoard: Graph Visualization. You create a SummaryWriter and include the sess.graph_def in it. The graph def will be output to the log directory.

How to create a Tensorflow Tensorboard Empty Graph

launch tensorboard with tensorboard --logdir=/home/vagrant/notebook
at tensorboard:6006 > graph, it says No graph definition files were found.
To store a graph, create a tf.python.training.summary_io.SummaryWriter and pass the graph either via the constructor, or by calling its add_graph() method.
import tensorflow as tf
sess = tf.Session()
writer = tf.python.training.summary_io.SummaryWriter("/home/vagrant/notebook", sess.graph_def)
However the page is still empty, how can I start playing with tensorboard?
current tensorboard
result wanted
An empty graph that can add nodes, editable.
update
Seems like tensorboard is unable to create a graph to add nodes, drag and edit etc ( I am confused by the official video ).
running https://github.com/tensorflow/tensorflow/blob/master/tensorflow/g3doc/tutorials/mnist/fully_connected_feed.py and then tensorboard --logdir=/home/vagrant/notebook/data is able to view the graph
However seems like tensorflow only provide ability to view summary, nothing much different to make it standout

TensorBoard is a tool for visualizing the TensorFlow graph and analyzing recorded metrics during training and inference. The graph is created using the Python API, then written out using the tf.train.SummaryWriter.add_graph() method. When you load the file written by the SummaryWriter into TensorBoard, you can see the graph that was saved, and interactively explore it.
However, TensorBoard is not a tool for building the graph itself. It does not have any support for adding nodes to the graph.

Starting from the following Code Example, I can add one line as shown below:
import tensorflow as tf
import numpy as np
sess = tf.InteractiveSession() #define a session
# Create 100 phony x, y data points in NumPy, y = x * 0.1 + 0.3
x_data = np.random.rand(100).astype("float32")
y_data = x_data * 0.1 + 0.3
# Try to find values for W and b that compute y_data = W * x_data + b
# (We know that W should be 0.1 and b 0.3, but Tensorflow will
# figure that out for us.)
W = tf.Variable(tf.random_uniform([1], -1.0, 1.0))
b = tf.Variable(tf.zeros([1]))
y = W * x_data + b
# Minimize the mean squared errors.
loss = tf.reduce_mean(tf.square(y - y_data))
optimizer = tf.train.GradientDescentOptimizer(0.5)
train = optimizer.minimize(loss)
# Before starting, initialize the variables. We will 'run' this first.
init = tf.initialize_all_variables()
# Launch the graph.
sess = tf.Session()
sess.run(init)
#### ----> ADD THIS LINE <---- ####
writer = tf.train.SummaryWriter("/tmp/test", sess.graph)
# Fit the line.
for step in xrange(201):
sess.run(train)
if step % 20 == 0:
print(step, sess.run(W), sess.run(b))
# Learns best fit is W: [0.1], b: [0.3]
And then run tensorboard from the command line, pointing to the appropriate directory. This shows a complete call for the SummaryWriter. It is important to note the following things:
SummaryWriter is passed a Session, and so must happen after the Session (or InteractiveSession) is created
That Session may be created early in the program, but when the Session is passed to the SummaryWriter, the graph as it exists at that point is written to the file that the TensorBoard will use.

In this page, there is a very simple code that you can use to test your installation: http://tensorflow.org/get_started
I included this line
tf.train.write_graph(sess.graph_def, '/home/daniel/Documents/Projetos/Prorum/ProgramasEmPython/TestingTensorFlow/fileGraph', 'graph.pbtxt')
After this "sess.run(init)"
This will generate a file that you have to upload to the "TensorBoard".
In order to open the TensorBoard, supposing that it is installed in your computer (it must be if you use pip to install), I used the terminal of Ubuntu and wrote:
"tensorboard --logdir nameOfDirectory"
Then, you should open your browser in Port 6006:
http://localhost:6006/
This will open the TensorBoard. I went to the "Graph Menu" and uploaded the file. It generated this figure below:
So, what I have done is to transfer the model I created in Python to TensorBoard. I believe that it is possible to create an empty one, if no model is created (only the session is initiated). However, I am not sure if you are able to change this directly in the TensorBoard.
I have answered before this question here in Portuguese with more details for Brazilian users. Maybe it can be useful for other people: http://prorum.com/index.php/1843/recentemente-plataforma-aprendizagem-primeira-impressao

i solved by on windows:
file_writer = tf.summary.FileWriter("output", sess.graph)
for that directory "output". I opened command on windows.
typed
tensorboard --logdir="C:\Users\kiran\machine Learning\output"
my mistake was on that line..

The graphs in TensorBoard do not show up if you are using Firefox. You have to install Chrome.

result wanted
An empty graph that can add nodes, editable.
I think you will find the Orange tool useful. It allows you to drag and drop various nodes and implement algorithms via GUI.

I had to use
python -m tensorflow.tensorboard --logdir="C:\tmp\tensorflow\.."
somehow tensorboard --logdir didn't work.
My environment
OS: Windows 7, Python 3.5, and Tensorflow 1.1.0