I have three networks, call them V, V_target, and Actor, and I'm trying to achieve the following setup:
V and Actor share certain layers.
V_target is an exact duplicate of V.
For those familiar with deep RL, I'm using this within an actor-critic algorithm with shared layers between the value and policy networks, plus a target network V_target. I tried the following:
def shared(...):
# define some variables, e.g.
W = get_variable('W', ...)
def Actor(...):
with tf.variable_scope("shared"):
shared_out = shared(...)
... actor-specific layers ...
def V(...):
with tf.variable_scope("shared", reuse=True):
shared_out = shared(...)
... V-specific layers...
with tf.variable_scope("Policy"):
actor_out = Actor(...)
with tf.variable_scope("V_main"):
V_out = V(...)
with tf.variable_scope("V_target"):
V_target = V(...)
As expected, this doesn't work because the use of the outermost variable_scope prevents sharing between Policy and V_main: the Variable W has name Policy/shared/W in one scope but has name V_main/shared/W under the second scope.
Why not use tf.name_scope("Policy") and tf.name_scope("V_main")? If I do that, the shared variables can be defined, but then I don't have a good way to get the variables under V_main and V_target. Specifically, because tf.name_scope does not append anything to names created by tf.get_variable, I cannot use tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES ,'V_main') and tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES ,'V_target') to get both sets of objects for the so-called "target updates".
Is there any clever way around this?
I suggest you do the trick described in this question: How to create variable outside of current scope in Tensorflow?
You can clear the current variable scope by providing an instance of an existing scope.
So you simply need to define tf.variable_scope("shared") once, remember the reference to this instance and use it inside all other variable scopes (with reuse=True). W variable will be created in shared scope, no matter what the outer scope is.
Related
It can someone be convenient to group variables under a given object.
My use case is tensorflow, where you often have to define a graph first and then feed it with actual data. To avoid getting the names of the graph variables jumbled up with those of the data variables, it's useful to group them all under an object. What I've been doing is:
g = lambda: None
g.iterator = tf.data.Dataset.from_tensor_slices((x_train, y_train)).batch(minibatch_size).make_initializable_iterator()
g.x_next, g.y_next = g.iterator.get_next()
g.data_updates = g.x_data.assign(g.x_next), g.y_data.assign(g.y_next)
Except that when you use lambda: None your coworkers tend to get angry and confused.
Is there an alternative that provides equally clean syntax but uses something that is more obviously a container than lambda: None?
I first tried making them all static members of a class, but the problem is that static members cannot reference other static members. g=object() would be nice but doesn't allow you to assign attributes.
If it's not worth defining a dedicated class, you can use types.SimpleNamespace, which is a class specifically designed to do nothing but hold attributes.
g = types.SimpleNamespace()
g.iterator = tf.data.Dataset.from_tensor_slices((x_train, y_train)).batch(minibatch_size).make_initializable_iterator()
g.x_next, g.y_next = g.iterator.get_next()
g.data_updates = g.x_data.assign(g.x_next), g.y_data.assign(g.y_next)
I would like to use a function's parameter to create dynamic names of dataframes and/or objects in Python. I have about 40 different names so it would be really elegant to do this in a function. Is there a way to do this or do I need to do this via 'dict'? I read that 'exec' is dangerous (not that I could get this to work). SAS has this feature for their macros which is where I am coming from. Here is an example of what I am trying to do (using '#' for illustrative purposes):
def TrainModels (mtype):
model_#mtype = ExtraTreesClassifier()
model_#mtype.fit(X_#mtype, Y_#mtype)
TrainModels ('FirstModel')
TrainModels ('SecondModel')
You could use a dictionary for this:
models = {}
def TrainModels (mtype):
models[mtype] = ExtraTreesClassifier()
models[mtype].fit()
First of all, any name you define within your TrainModels function will be local to that function, so won't be accessible in the rest of your program. So you have to define a global name.
Everything in Python is a dictionary, including the global namespace. You can define a new global name dynamically as follows:
my_name = 'foo'
globals()[my_name] = 'bar'
This is terrible and you should never do it. It adds too much indirection to your code. When someone else (or yourself in 3 months when the code is no longer fresh in your mind) reads the code and see 'foo' used elsewhere, they'll have a hard time figuring out where it came from. Code analysis tools will not be able to help you.
I would use a dict as Milkboat suggested.
I'm using the high level tf.contrib.learn.Experiment object to interleave training and evaluation. However, I'm facing an issue with the local variables from the evaluation and metrics modules that are reported as non initialized:
Variables not initialized: mean/total, mean/count, eval_step
I provide a custom local_init_op to tf.train.Scaffold which basically looks like this:
scaffold = tf.train.Scaffold(
local_init_op=tf.group(
iterator.initializer,
tf.tables_initializer(),
tf.local_variables_initializer()))
(where iterator is a tf.contrib.data.Iterator.)
which is then stored in a tf.estimator.EstimatorSpec to be returned by the tf.estimator.Estimator's model_fn function.
As I don't think tf.local_variables_initializer() operates lazily, it means these variables are not yet created.
So how to initialize them?
The only solution I found is to not use a custom local_init_op but rely on the default one which is built in Scaffold.finalize, when all variables are created.
To initialize my iterator I simply added it in the TABLE_INITIALIZERS collection:
tf.add_to_collection(tf.GraphKeys.TABLE_INITIALIZERS, iterator.initializer)
I have a setup where I need to initialize an LSTM after the main initialization which uses tf.initialize_all_variables(). I.e. I want to call tf.initialize_variables([var_list])
Is there way to collect all the internal trainable variables for both:
rnn_cell.BasicLSTM
rnn_cell.MultiRNNCell
so that I can initialize JUST these parameters?
The main reason I want this is because I do not want to re-initialize some trained values from earlier.
The easiest way to solve your problem is to use variable scope. The names of the variables within a scope will be prefixed with its name. Here is a short snippet:
cell = rnn_cell.BasicLSTMCell(num_nodes)
with tf.variable_scope("LSTM") as vs:
# Execute the LSTM cell here in any way, for example:
for i in range(num_steps):
output[i], state = cell(input_data[i], state)
# Retrieve just the LSTM variables.
lstm_variables = [v for v in tf.all_variables()
if v.name.startswith(vs.name)]
# [..]
# Initialize the LSTM variables.
tf.initialize_variables(lstm_variables)
It would work the same way with MultiRNNCell.
EDIT: changed tf.trainable_variables to tf.all_variables()
You can also use tf.get_collection():
cell = rnn_cell.BasicLSTMCell(num_nodes)
with tf.variable_scope("LSTM") as vs:
# Execute the LSTM cell here in any way, for example:
for i in range(num_steps):
output[i], state = cell(input_data[i], state)
lstm_variables = tf.get_collection(tf.GraphKeys.VARIABLES, scope=vs.name)
(partly copied from Rafal's answer)
Note that the last line is equivalent to the list comprehension in Rafal's code.
Basically, tensorflow stores a global collection of variables, which can be fetched by either tf.all_variables() or tf.get_collection(tf.GraphKeys.VARIABLES). If you specify scope (scope name) in the tf.get_collection() function, then you only fetch tensors (variables in this case) in the collection whose scopes are under the specified scope.
EDIT:
You can also use tf.GraphKeys.TRAINABLE_VARIABLES to get trainable variables only. But since vanilla BasicLSTMCell does not initialize any non-trainable variable, both will be functionally equivalent. For a complete list of default graph collections, check this out.
This question already has answers here:
What's the difference of name scope and a variable scope in tensorflow?
(8 answers)
Closed 3 years ago.
What is the difference between variable_scope and name_scope? The variable scope tutorial talks about variable_scope implicitly opening name_scope. I also noticed that creating a variable in a name_scope automatically expands its name with the scope name as well. So, what is the difference?
I had problems understanding the difference between variable_scope and name_scope (they looked almost the same) before I tried to visualize everything by creating a simple example:
import tensorflow as tf
def scoping(fn, scope1, scope2, vals):
with fn(scope1):
a = tf.Variable(vals[0], name='a')
b = tf.get_variable('b', initializer=vals[1])
c = tf.constant(vals[2], name='c')
with fn(scope2):
d = tf.add(a * b, c, name='res')
print '\n '.join([scope1, a.name, b.name, c.name, d.name]), '\n'
return d
d1 = scoping(tf.variable_scope, 'scope_vars', 'res', [1, 2, 3])
d2 = scoping(tf.name_scope, 'scope_name', 'res', [1, 2, 3])
with tf.Session() as sess:
writer = tf.summary.FileWriter('logs', sess.graph)
sess.run(tf.global_variables_initializer())
print sess.run([d1, d2])
writer.close()
Here I create a function that creates some variables and constants and groups them in scopes (depending by the type I provided). In this function I also print the names of all the variables. After that I executes the graph to get values of the resulting values and save event-files to investigate them in tensorboard. If you run this, you will get the following:
scope_vars
scope_vars/a:0
scope_vars/b:0
scope_vars/c:0
scope_vars/res/res:0
scope_name
scope_name/a:0
b:0
scope_name/c:0
scope_name/res/res:0
You see the similar pattern if you open TB (as you see b is outside of scope_name rectangular):
This gives you the answer:
Now you see that tf.variable_scope() adds a prefix to the names of all variables (no matter how you create them), ops, constants. On the other hand tf.name_scope() ignores variables created with tf.get_variable() because it assumes that you know which variable and in which scope you wanted to use.
A good documentation on Sharing variables tells you that
tf.variable_scope(): Manages namespaces for names passed to tf.get_variable().
The same documentation provides a more details how does Variable Scope work and when it is useful.
When you create a variable with tf.get_variable instead of tf.Variable, Tensorflow will start checking the names of the vars created with the same method to see if they collide. If they do, an exception will be raised. If you created a var with tf.get_variable and you try to change the prefix of your variable names by using the tf.name_scope context manager, this won't prevent the Tensorflow of raising an exception. Only tf.variable_scope context manager will effectively change the name of your var in this case. Or if you want to reuse the variable you should call scope.reuse_variables() before creating the var the second time.
In summary, tf.name_scope just add a prefix to all tensor created in that scope (except the vars created with tf.get_variable), and tf.variable_scope add a prefix to the variables created with tf.get_variable.
tf.variable_scope is an evolution of tf.name_scope to handle Variable reuse. As you noticed, it does more than tf.name_scope, so there is no real reason to use tf.name_scope: not surprisingly, a TF developper advises to just use tf.variable_scope.
My understanding for having tf.name_scope still lying around is that there are subtle incompatibilities in the behavior of those two, which invalidates tf.variable_scope as a drop-in replacement for tf.name_scope.