I am trying to build a RLearning agent using a old tutorial (https://www.youtube.com/watch?v=OYhFoMySoVs&t=2444s). I fixed a earlier bug about the old gym step function, but I can't seem to figure out how to fix the reshaping issue. If anyone has any advice it would be much appreciated.
I looked through the Numpy documentation to see if I could solve the bug but cound'nt fix it. Also I looked at the YouTube comments but there doesn't appear to be any fixes. I looked at a similar tutorial and ran into the same issue.
My Code (I didn't include the code for the DQNAgent if that is needed please ask)
env = gym.make('CartPole-v1')
state_size = env.observation_space.shape[0]
action_size = env.action_space.n
batch_size = 32
n_episodes = 1001
# Stores the model
output_dir = './model_output/cartpole'
if not os.path.exists(output_dir):
os.makedirs(output_dir)```
agent = DQNAgent(state_size, action_size)
done = False
for e in range(n_episodes):
state = env.reset() # reset the state
print(state)
print([1, state_size])
state = np.reshape(state, [1, state_size])
for time in range(5000): # If the game lasts 5000 time steps
# env.render()
action = agent.act(state) # Get an action (0 - left, 1 - right)
# Early this will be random, but as time passes it will be more logical (Exploiting)
next_state, reward, done, truncated, _ = env.step(action)
reward = reward if not done else -10 # Penalize poor actions
next_state = np.reshape(next_state, [1, state_size])
agent.remember(state, action, reward, next_state, done)
state = next_state
if done: # To see how the agent performs if completed
print(f"Episode: {e}/{n_episodes}, score: {time}, e: {agent.epsilon:.2}")
break
if len(agent.memory) > batch_size: # Gives a chance for the agent to update
agent.replay(batch_size)
Also if anyone has any other good resources for learning about reinforcement learning please share. Thank you in advance.
Edit -- The error message is
Traceback (most recent call last):
File "C:\Users\josep\VS Code Files\Python Projects\RL_Gym\main.py", line 120, in <module>
state = np.reshape(state, [1, state_size])
File "<__array_function__ internals>", line 200, in reshape
File "C:\Users\josep\VS Code Files\Python Projects\RL_Gym\venv\lib\site-packages\numpy\core\fromnumeric.py", line 298, in reshape
return _wrapfunc(a, 'reshape', newshape, order=order)
File "C:\Users\josep\VS Code Files\Python Projects\RL_Gym\venv\lib\site-packages\numpy\core\fromnumeric.py", line 54, in _wrapfunc
return _wrapit(obj, method, *args, **kwds)
File "C:\Users\josep\VS Code Files\Python Projects\RL_Gym\venv\lib\site-packages\numpy\core\fromnumeric.py", line 43, in _wrapit
result = getattr(asarray(obj), method)(*args, **kwds)
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.
Related
I design a CNN model which has two stages. First stage is generating proposals like RPN in Faster RCNN and the second feeds these proposals into the following part.
It causes error in the second step.
Accroding the below error information, it seems like the second input is not correctly assigned to the multi GPU.
However, The model works file with single gpu.
File "/home/f523/guazai/sdb/rsy/cornerPoject/myCornerNet6/exp/train.py", line 212, in run_epoch
cls, rgr = self.model([proposal, fm], stage='two')
File "/home/f523/anaconda3/envs/rsy/lib/python3.6/site-packages/torch/nn/modules/module.py", line 722, in _call_impl
result = self.forward(*input, **kwargs)
File "/home/f523/anaconda3/envs/rsy/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 156, in forward
return self.gather(outputs, self.output_device)
File "/home/f523/anaconda3/envs/rsy/lib/python3.6/site-packages/torch/nn/parallel/data_parallel.py", line 168, in gather
return gather(outputs, output_device, dim=self.dim)
File "/home/f523/anaconda3/envs/rsy/lib/python3.6/site-packages/torch/nn/parallel/scatter_gather.py", line 68, in gather
res = gather_map(outputs)
File "/home/f523/anaconda3/envs/rsy/lib/python3.6/site-packages/torch/nn/parallel/scatter_gather.py", line 63, in gather_map
return type(out)(map(gather_map, zip(*outputs)))
File "/home/f523/anaconda3/envs/rsy/lib/python3.6/site-packages/torch/nn/parallel/scatter_gather.py", line 55, in gather_map
return Gather.apply(target_device, dim, *outputs)
File "/home/f523/anaconda3/envs/rsy/lib/python3.6/site-packages/torch/nn/parallel/_functions.py", line 68, in forward
return comm.gather(inputs, ctx.dim, ctx.target_device)
File "/home/f523/anaconda3/envs/rsy/lib/python3.6/site-packages/torch/cuda/comm.py", line 166, in gather
return torch._C._gather(tensors, dim, destination)
RuntimeError: CUDA error: an illegal memory access was encountered
PS
my model sciprt shows below. I want my two-stage model can support for multi batch. e.g. the batch size is 4 and every img output 128 proposal, so the proposal size in here is (4*128, 5)
def _stage2(self, xs):
proposal, fm = xs
if proposal.dim()==2 and proposal.size(1) == 5:
# train mode
roi = roi_align(fm, proposal, output_size=[15, 15])
elif proposal.dim()==3 and proposal.size(2) == 4:
# eval mode
roi = roi_align(fm, [proposal[0]], output_size=[15, 15])
else:
assert AssertionError(" The boxes tensor shape should be Tensor[K, 5] in train or Tensor[N, 4] in eval")
x = self.big_kernel(roi)
cls = self.cls_fm(x)
rgr = self.rgr_fm(x)
return cls, rgr
I know where I am wrong. Here’s my second stage to feed input
cls, offset = self.model([proposal, fm], stage='two')
proposal is the ROI whose shape is [N, 5], the 1th dim is the batch index. e.g. The batch size is 4, the range of index is [0,1,2,3]. And fm is the feature map.
When I use the mult-gpu like 2 gpu. the proposal and fm will be split into two branch and feed into two gpu. However the batch index range still be [0,1,2,3], then cause a index error and raise gpu error.
What I do is add a line before roi_align like below:
from torchvision.ops import roi_align
proposal[:, 0] = proposal[:, 0] % fm.size(0) # this make multi-gpu work
roi = roi_align(fm, proposal, output_size=[15, 15])
I was in the middle of training my gan when a very unexpected error came up. I have no idea how to fix it. The error doesn't come right away it happens about 2-3 minutes into my training. Here is the Error
Traceback (most recent call last):
File "gan.py", line 103, in <module>
train(X_train_dataset,200)
File "gan.py", line 80, in train
train_step(images) # takes images and improves both the generator and the discriminator
File "gan.py", line 91, in train_step
discriminator_loss = get_discriminator_loss(real_output,fake_output)
File "gan.py", line 48, in get_discriminator_loss
return fake_loss+real_loss
File "/home/jake/.local/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py", line 1125, in binary_op_wrapper
return func(x, y, name=name)
File "/home/jake/.local/lib/python3.6/site-packages/tensorflow/python/util/dispatch.py", line 201, in wrapper
return target(*args, **kwargs)
File "/home/jake/.local/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py", line 1447, in _add_dispatch
return gen_math_ops.add_v2(x, y, name=name)
File "/home/jake/.local/lib/python3.6/site-packages/tensorflow/python/ops/gen_math_ops.py", line 486, in add_v2
_ops.raise_from_not_ok_status(e, name)
File "/home/jake/.local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 6843, in raise_from_not_ok_status
six.raise_from(core._status_to_exception(e.code, message), None)
File "<string>", line 3, in raise_from
tensorflow.python.framework.errors_impl.InvalidArgumentError: Incompatible shapes: [100] vs. [13] [Op:AddV2]
So from I can tell from this call back my error occures during my get_discriminator_loss() so here is that code.
def get_discriminator_loss(real_predictions,fake_predictions):
real_predictions = tf.sigmoid(real_predictions)
fake_predictions = tf.sigmoid(fake_predictions)
real_loss=tf.losses.binary_crossentropy(tf.ones_like(real_predictions),real_predictions)
fake_loss=tf.losses.binary_crossentropy(tf.zeros_like(fake_predictions),fake_predictions)
return fake_loss+real_loss
Does anyone have any ideas? And remember this is after running successfully for about 2-3 minutes. The error doesn't occur in the first many passes.
I've found the source of my error but I don't know why it's occuring.
My real loss at one of the passes has only 13 values instead of the normal 100
How can this be?
Here is my full code.
import tensorflow as tf
import matplotlib.pyplot as plt
import numpy as np
import time
import pickle
pickle_in_X = open("X.pickle","rb")
pickle_in_y = open("y.pickle","rb")
X=pickle.load(pickle_in_X)
y = pickle.load(pickle_in_y)
y = np.array(y)
X_train = X[ int(len(X)*.3): ]
y_train = y[ int(len(y)*.3 ): ]
X_test = X[ :int(len(X)*.3) ]
y_test = X[ :int(len(y)*.3) ]
X_train = (X_train-127.5)/127.5
BATCH_SIZE = 100
X_train_dataset = tf.data.Dataset.from_tensor_slices(X_train).batch(BATCH_SIZE)
#creates a discriminator model.
#discriminator will ouput 0-1 which represents the probability that the image is real
def make_discriminator():
model = tf.keras.Sequential()
model.add(tf.keras.layers.Conv2D(7,(3,3),padding="same",input_shape=(40,40,1)))
model.add(tf.keras.layers.Flatten())
model.add(tf.keras.layers.LeakyReLU())
model.add(tf.keras.layers.Dense(50,activation="relu"))
model.add(tf.keras.layers.Dense(1))
return model
model_discriminator = make_discriminator()
discriminator_optimizer = tf.optimizers.Adam(1e-3)
#real_loss is the amount of error when trying to guess that the real images are in fact real. i.e loss will be if our discriminator guesses that there is a 100% chance that this real image is real
#fake_loss is the amount of error when trying to guess that the fake images are in fact fake. i.e loss will be zero if our discriminator guesses there is a 0% chance that this fake image is fake
#returns the total of our loss
def get_discriminator_loss(real_predictions,fake_predictions):
real_predictions = tf.sigmoid(real_predictions)
fake_predictions = tf.sigmoid(fake_predictions)
real_loss=tf.losses.binary_crossentropy(tf.ones_like(real_predictions),real_predictions)
fake_loss=tf.losses.binary_crossentropy(tf.zeros_like(fake_predictions),fake_predictions)
return fake_loss+real_loss
#take an input of a random string of numbers. and output either a dog or a cat
def make_generator():
model = tf.keras.Sequential()
model.add(tf.keras.layers.Dense(10*10*256,input_shape = (100,)))
model.add(tf.keras.layers.BatchNormalization())
model.add(tf.keras.layers.Reshape((10,10,256)))
model.add(tf.keras.layers.Conv2DTranspose(128,(3,3),padding="same"))
model.add(tf.keras.layers.BatchNormalization())
model.add(tf.keras.layers.Conv2DTranspose(64,(3,3),strides=(2,2),padding="same"))
model.add(tf.keras.layers.BatchNormalization())
model.add(tf.keras.layers.Conv2DTranspose(1,(3,3),strides=(2,2),padding="same"))
return model
model_generator = make_generator()
#generator gets rewarded when it fools the discriminator
def get_generator_loss(fake_predictions):
fake_predictions = tf.sigmoid(fake_predictions)
fake_loss=tf.losses.binary_crossentropy(tf.ones_like(fake_predictions),fake_predictions)
return fake_loss
generator_optimizer = tf.optimizers.Adam(1e-3)
#training
def train(X_train_dataset,epochs):
for _ in range(epochs):
for images in X_train_dataset:
images = tf.cast(images,tf.dtypes.float32)
train_step(images) # takes images and improves both the generator and the discriminator
def train_step(images):
fake_image_noise = np.random.randn(BATCH_SIZE,100)#produces 100 random numbers that wll be converted to images
with tf.GradientTape() as generator_gradient, tf.GradientTape() as discriminator_gradient:
generated_images = model_generator(fake_image_noise)
real_output = model_discriminator(images)
fake_output = model_discriminator(generated_images)
generator_loss = get_generator_loss(fake_output)
discriminator_loss = get_discriminator_loss(real_output,fake_output)
gradients_of_generator = generator_gradient.gradient(generator_loss,model_generator.trainable_variables)#gradient of gen loss with respect to trainable variables
gradients_of_discriminator = discriminator_gradient.gradient(discriminator_loss,model_discriminator.trainable_variables)
discriminator_optimizer.apply_gradients(zip(gradients_of_discriminator,model_discriminator.trainable_variables))
generator_optimizer.apply_gradients(zip(gradients_of_generator,model_generator.trainable_variables))
print("generator loss: ", np.mean(generator_loss))
print("discriminator loss: ",np.mean(discriminator_loss))
train(X_train_dataset,200)
model_generator.save('genModel')
model_discriminator.save('discModel')
If the size of your dataset is not a multiple of your batch size, then your last batch will have a smaller number of samples than the other batches. To avoid this, you can force a tf.data.Dataset to drop the last batch if it is smaller than the batch size. See the documentation for more information.
tf.data.Dataset.from_tensor_slices(X_train).batch(BATCH_SIZE, drop_remainder=True)
Traceback (most recent call last):
File "c:/Users/levin/Desktop/programming/nn.py", line 208, in <module>
agent.train(BATCHSIZE)
File "c:/Users/levin/Desktop/programming/nn.py", line 147, in train
output = F.mse_loss(prediction, target)
File "C:\Users\levin\Anaconda3\lib\site-packages\torch\nn\functional.py", line 2203, in mse_loss
if not (target.size() == input.size()):
AttributeError: 'NoneType' object has no attribute 'size'
This above is the Error that I'm constantly getting and I really don't know how to fix it.
This some code that might be important
def train(self, BATCHSIZE):
trainsample = random.sample(self.memory, BATCHSIZE)
for state, action, reward, new_state, gameovertemp in trainsample:
if gameovertemp:
target = torch.tensor(reward).grad_fn
else:
target = reward + self.gamma * torch.max(self.dqn.forward(new_state))
self.dqn.zero_grad()
prediction = torch.max(self.dqn.forward(state))
#print(prediction, "prediction")
#print(target, "target")
output = F.mse_loss(prediction, target)
output.backward()
self.optimizer.step()
As stated in a comment the error due to either target of input to be None and is not related to the size() attribute.
The problem is probably at this line:
target = torch.tensor(reward).grad_fn
Here you convert reward to a new Tensor. However, a Tensor created by the user always has a grad_fn equal to None (as explained in Pytorch Autograd).
To have a grad_fn a Tensor must be the result of some computation, not a static value.
The thing is that mse_loss does not expect target to be differentiable, as the name suggest it is just the value to be compared.
Try to remove the grad_fn from this line the raw Tensor should be sufficient.
I'm trying to use tf.contrib.training.stratified_sample in Tensorflow to balance classes. I made a quick example below to test it, drawing samples from two unbalanced classes in a balanced way and verifying it, but I'm getting an error.
import tensorflow as tf
from tensorflow.python.framework import ops
from tensorflow.python.framework import dtypes
batch_size = 10
data = ['a']*9990+['b']*10
labels = [1]*9990+[0]*10
data_tensor = ops.convert_to_tensor(data, dtype=dtypes.string)
label_tensor = ops.convert_to_tensor(labels)
target_probs = [0.5,0.5]
data_batch, label_batch = tf.contrib.training.stratified_sample(
data_tensor, label_tensor, target_probs, batch_size,
queue_capacity=2*batch_size)
with tf.Session() as sess:
d,l = sess.run(data_batch,label_batch)
print('percentage "a" = %.3f' % (np.sum(l)/len(l)))
The error I'm getting is:
Traceback (most recent call last):
File "/home/jason/code/scrap.py", line 56, in <module>
test_stratified_sample()
File "/home/jason/code/scrap.py", line 47, in test_stratified_sample
queue_capacity=2*batch_size)
File "/usr/local/lib/python3.4/dist-packages/tensorflow/contrib/training/python/training/sampling_ops.py", line 191, in stratified_sample
with ops.name_scope(name, 'stratified_sample', tensors + [labels]):
File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/ops/math_ops.py", line 829, in binary_op_wrapper
y = ops.convert_to_tensor(y, dtype=x.dtype.base_dtype, name="y")
File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/framework/ops.py", line 676, in convert_to_tensor
as_ref=False) File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/framework/ops.py", line 741, in internal_convert_to_tensor
ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/framework/constant_op.py", line 113, in _constant_tensor_conversion_function
return constant(v, dtype=dtype, name=name)
File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/framework/constant_op.py", line 102, in constant
tensor_util.make_tensor_proto(value, dtype=dtype, shape=shape, verify_shape=verify_shape))
File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/framework/tensor_util.py", line 374, in make_tensor_proto
_AssertCompatible(values, dtype)
File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/framework/tensor_util.py", line 302, in _AssertCompatible
(dtype.name, repr(mismatch), type(mismatch).__name__)) TypeError: Expected string, got list containing Tensors of type '_Message' instead.
The error doesn't explain what I'm doing wrong. I also tried putting the raw data and labels in (without converting to a tensor), as well as tried using tf.train.slice_input_producer to create an initial queue of the data and label tensors.
Has anyone gotten stratified_sample to work? I haven't been able to find any examples.
I've modified the code into something that works for me. Summary of the changes:
Use enqueue_many=True to enqueue a batch of examples with different labels. Otherwise it's expecting a single scalar label Tensor (which can be stochastic when evaluated by the queue runners).
The first argument is expected to be a list of Tensors. It should have a better error message (I think this is what you ran into). Please do send a pull request or open an issue on Github for a better error message.
Start queue runners. Otherwise code that uses queues will deadlock. Or use Estimators or MonitoredSession so you don't need to worry about this.
(Edit based on comments) stratified_sample does not shuffle the data, it just accepts/rejects! So if your data is not randomized, consider putting it through slice_input_producer (enqueue_many=False) or shuffle_batch (enqueue_many=True) before sampling if you want it to come out in a random order.
Modified code (improved based on Jason's comments):
import numpy
import tensorflow as tf
from tensorflow.python.framework import ops
from tensorflow.python.framework import dtypes
with tf.Graph().as_default():
batch_size = 100
data = ['a']*9000+['b']*1000
labels = [1]*9000+[0]*1000
data_tensor = ops.convert_to_tensor(data, dtype=dtypes.string)
label_tensor = ops.convert_to_tensor(labels, dtype=dtypes.int32)
shuffled_data, shuffled_labels = tf.train.slice_input_producer(
[data_tensor, label_tensor], shuffle=True, capacity=3*batch_size)
target_probs = numpy.array([0.5,0.5])
data_batch, label_batch = tf.contrib.training.stratified_sample(
[shuffled_data], shuffled_labels, target_probs, batch_size,
queue_capacity=2*batch_size)
with tf.Session() as session:
tf.local_variables_initializer().run()
tf.global_variables_initializer().run()
coordinator = tf.train.Coordinator()
tf.train.start_queue_runners(session, coord=coordinator)
num_iter = 10
sum_ones = 0.
for _ in range(num_iter):
d, l = session.run([data_batch, label_batch])
count_ones = l.sum()
sum_ones += float(count_ones)
print('percentage "a" = %.3f' % (float(count_ones) / len(l)))
print('Overall: {}'.format(sum_ones / (num_iter * batch_size)))
coordinator.request_stop()
coordinator.join()
Outputs:
percentage "a" = 0.480
percentage "a" = 0.440
percentage "a" = 0.580
percentage "a" = 0.570
percentage "a" = 0.580
percentage "a" = 0.520
percentage "a" = 0.480
percentage "a" = 0.460
percentage "a" = 0.390
percentage "a" = 0.530
Overall: 0.503
I'm building an encoder-decoder model in TensorFlow 1.0.1 using the legacy sequence-to-sequence framework. Everything works as it should when I have one layer of LSTMs in the encoder and decoder. However, when I try with >1 layers of LSTMs wrapped in a MultiRNNCell, I get an error when calling tf.contrib.legacy_seq2seq.rnn_decoder.
The full error is at the end up this post, but in brief, it's caused by a line
(c_prev, m_prev) = state
in TensorFlow that throws TypeError: 'Tensor' object is not iterable.. I'm confused by this, since the initial state I'm passing to rnn_decoder is indeed a tuple as it should be. As far as I can tell, the only difference between using 1 or >1 layers is that the latter involves using MultiRNNCell. Are there some API quirks that I should know about when using this?
This is my code (based on the example in this GitHub repo). Apologies for its length; this is as minimal I could make it while still being complete and verifiable.
import tensorflow as tf
import tensorflow.contrib.legacy_seq2seq as seq2seq
import tensorflow.contrib.rnn as rnn
seq_len = 50
input_dim = 300
output_dim = 12
num_layers = 2
hidden_units = 100
sess = tf.Session()
encoder_inputs = []
decoder_inputs = []
for i in range(seq_len):
encoder_inputs.append(tf.placeholder(tf.float32, shape=(None, input_dim),
name="encoder_{0}".format(i)))
for i in range(seq_len + 1):
decoder_inputs.append(tf.placeholder(tf.float32, shape=(None, output_dim),
name="decoder_{0}".format(i)))
if num_layers > 1:
# Encoder cells (bidirectional)
# Forward
enc_cells_fw = [rnn.LSTMCell(hidden_units)
for _ in range(num_layers)]
enc_cell_fw = rnn.MultiRNNCell(enc_cells_fw)
# Backward
enc_cells_bw = [rnn.LSTMCell(hidden_units)
for _ in range(num_layers)]
enc_cell_bw = rnn.MultiRNNCell(enc_cells_bw)
# Decoder cell
dec_cells = [rnn.LSTMCell(2*hidden_units)
for _ in range(num_layers)]
dec_cell = rnn.MultiRNNCell(dec_cells)
else:
# Encoder
enc_cell_fw = rnn.LSTMCell(hidden_units)
enc_cell_bw = rnn.LSTMCell(hidden_units)
# Decoder
dec_cell = rnn.LSTMCell(2*hidden_units)
# Make sure input and output are the correct dimensions
enc_cell_fw = rnn.InputProjectionWrapper(enc_cell_fw, input_dim)
enc_cell_bw = rnn.InputProjectionWrapper(enc_cell_bw, input_dim)
dec_cell = rnn.OutputProjectionWrapper(dec_cell, output_dim)
_, final_fw_state, final_bw_state = \
rnn.static_bidirectional_rnn(enc_cell_fw,
enc_cell_bw,
encoder_inputs,
dtype=tf.float32)
# Concatenate forward and backward cell states
# (The state is a tuple of previous output and cell state)
if num_layers == 1:
initial_dec_state = tuple([tf.concat([final_fw_state[i],
final_bw_state[i]], 1)
for i in range(2)])
else:
initial_dec_state = tuple([tf.concat([final_fw_state[-1][i],
final_bw_state[-1][i]], 1)
for i in range(2)])
decoder = seq2seq.rnn_decoder(decoder_inputs, initial_dec_state, dec_cell)
tf.global_variables_initializer().run(session=sess)
And this is the error:
Traceback (most recent call last):
File "example.py", line 67, in <module>
decoder = seq2seq.rnn_decoder(decoder_inputs, initial_dec_state, dec_cell)
File "/home/tao/.virtualenvs/example/lib/python2.7/site-packages/tensorflow/contrib/legacy_seq2seq/python/ops/seq2seq.py", line 150, in rnn_decoder
output, state = cell(inp, state)
File "/home/tao/.virtualenvs/example/lib/python2.7/site-packages/tensorflow/contrib/rnn/python/ops/core_rnn_cell_impl.py", line 426, in __call__
output, res_state = self._cell(inputs, state)
File "/home/tao/.virtualenvs/example/lib/python2.7/site-packages/tensorflow/contrib/rnn/python/ops/core_rnn_cell_impl.py", line 655, in __call__
cur_inp, new_state = cell(cur_inp, cur_state)
File "/home/tao/.virtualenvs/example/lib/python2.7/site-packages/tensorflow/contrib/rnn/python/ops/core_rnn_cell_impl.py", line 321, in __call__
(c_prev, m_prev) = state
File "/home/tao/.virtualenvs/example/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 502, in __iter__
raise TypeError("'Tensor' object is not iterable.")
TypeError: 'Tensor' object is not iterable.
Thank you!
The problem is in the format of the initial state (initial_dec_state) passed to seq2seq.rnn_decoder.
When you use rnn.MultiRNNCell, you're building a multilayer recurrent network, so you need to provide an initial state for each of these layers.
Hence, you should provide a list of tuples as the initial state, where each element of the list is the previous state coming from the corresponding layer of the recurrent network.
So your initial_dec_state, initialized like this:
initial_dec_state = tuple([tf.concat([final_fw_state[-1][i],
final_bw_state[-1][i]], 1)
for i in range(2)])
instead should be like this:
initial_dec_state = [
tuple([tf.concat([final_fw_state[j][i],final_bw_state[j][i]], 1)
for i in range(2)]) for j in range(len(final_fw_state))
]
which creates a list of tuples in the format:
[(state_c1, state_m1), (state_c2, state_m2) ...]
In more detail, the 'Tensor' object is not iterable. error, happens because seq2seq.rnn_decoder internally calls your rnn.MultiRNNCell (dec_cell) passing the initial state (initial_dec_state) to it.
rnn.MultiRNNCell.__call__ iterates through the list of initial states and for each one of them extracts the tuple (c_prev, m_prev) (in the statement (c_prev, m_prev) = state).
So if you pass just a tuple, rnn.MultiRNNCell.__call__ will iterate over it, and as soon as it reaches the (c_prev, m_prev) = state it will find a tensor (which should be a tuple) as state and will throw the 'Tensor' object is not iterable. error.
A good way to know which format of initial state a seq2seq.rnn_decoder expects, is to call dec_cell.zero_state(batch_size, dtype=tf.float32). This method returns zero-filled state tensor(s) in the exact format needed to initialize the recurrent module that you're using.