Develop Reference

get_flat_tensor_specs nest.flatten(element_spec), []) results in AttributeError: 'list' object has no attribute '_flat_tensor_specs'

AttributeError: 'list' object has no attribute '_flatten_tensor_specs'
problem:
Currently, when creating an iterative process with a build federated averaging process, I am able to pass in a functools. partial successful.
however, there must be an error in the above code that produces a structure needed for the model_fn to go through
correctly.
have tried:
looking at the input data. This custom data set is built for a autoencoder and the original TF solution never used labels. As this solution is built to take in the training data, train the model, then work on the test data, and validate on the validation set to produce a threshold.
There might be a issue with the underlying production.
sources:
df, y_train = get_train_data(sysarg)
x_train, x_opt, x_test = np.split(df.sample(frac=1,
random_state=17),
[int(1 / 3 * len(df)), int(2 / 3 * len(df))])
x_train, x_opt, x_test = create_scalar(x_opt, x_test, x_train)
input_spec = tf.nest.map_structure(tf.TensorSpec.from_tensor,
[tf.convert_to_tensor(x_train),
tf.convert_to_tensor(y_train)])
assign_weights_fn = compression_process_adapter.CompressionServerState.assign_weights_to_keras_model
iterative_process = tff.learning.build_federated_averaging_process(
model_fn=functools.partial(model_builder,
input_dim=sysarg, input_spec=input_spec),
client_optimizer_fn=client_optimizer_fn,
server_optimizer_fn=server_optimizer_fn,
)
iterative_process = compression_process_adapter.CompressionProcessAdapter(iterative_process)
trackback error
Traceback (most recent call last):
File "B:\tools and software\Anaconda\envs\bookProjects\lib\site-packages\IPython\core\interactiveshell.py", line 3331, in run_code
exec(code_obj, self.user_global_ns, self.user_ns)
File "<ipython-input-2-47998fd56829>", line 1, in <module>
runfile('B:/projects/openProjects/githubprojects/BotnetTrafficAnalysisFederaedLearning/anomaly-detection/train_v04.py', args=['--experiment_name=temp', '--client_batch_size=20', '--client_optimizer=sgd', '--client_learning_rate=0.2', '--server_optimizer=sgd', '--server_learning_rate=1.0', '--total_rounds=200', '--rounds_per_eval=1', '--rounds_per_checkpoint=50', '--rounds_per_profile=0', '--root_output_dir=B:/projects/openProjects/githubprojects/BotnetTrafficAnalysisFederaedLearning/anomaly-detection/logs/fed_out/'], wdir='B:/projects/openProjects/githubprojects/BotnetTrafficAnalysisFederaedLearning/anomaly-detection')
File "B:\tools and software\PyCharm 2020.1\plugins\python\helpers\pydev\_pydev_bundle\pydev_umd.py", line 197, in runfile
pydev_imports.execfile(filename, global_vars, local_vars) # execute the script
File "B:\tools and software\PyCharm 2020.1\plugins\python\helpers\pydev\_pydev_imps\_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "B:/projects/openProjects/githubprojects/BotnetTrafficAnalysisFederaedLearning/anomaly-detection/train_v04.py", line 306, in <module>
app.run(main)
File "B:\tools and software\Anaconda\envs\bookProjects\lib\site-packages\absl\app.py", line 299, in run
_run_main(main, args)
File "B:\tools and software\Anaconda\envs\bookProjects\lib\site-packages\absl\app.py", line 250, in _run_main
sys.exit(main(argv))
File "B:/projects/openProjects/githubprojects/BotnetTrafficAnalysisFederaedLearning/anomaly-detection/train_v04.py", line 299, in main
train_main()
File "B:/projects/openProjects/githubprojects/BotnetTrafficAnalysisFederaedLearning/anomaly-detection/train_v04.py", line 262, in train_main
server_optimizer_fn=server_optimizer_fn,
File "B:\tools and software\Anaconda\envs\bookProjects\lib\site-packages\tensorflow_federated\python\learning\federated_averaging.py", line 211, in build_federated_averaging_process
stateful_delta_aggregate_fn, stateful_model_broadcast_fn)
File "B:\tools and software\Anaconda\envs\bookProjects\lib\site-packages\tensorflow_federated\python\learning\framework\optimizer_utils.py", line 521, in build_model_delta_optimizer_process
model_broadcast_state_type=model_broadcast_state_type)
File "B:\tools and software\Anaconda\envs\bookProjects\lib\site-packages\tensorflow_federated\python\learning\framework\optimizer_utils.py", line 368, in _build_one_round_computation
#tff.tf_computation(dataset_type, model_weights_type)
File "B:\tools and software\Anaconda\envs\bookProjects\lib\site-packages\tensorflow_federated\python\core\impl\wrappers\computation_wrapper.py", line 337, in <lambda>
return lambda fn: _wrap(fn, arg_type, self._wrapper_fn)
File "B:\tools and software\Anaconda\envs\bookProjects\lib\site-packages\tensorflow_federated\python\core\impl\wrappers\computation_wrapper.py", line 89, in _wrap
concrete_fn = wrapper_fn(fn, parameter_type, unpack=None)
File "B:\tools and software\Anaconda\envs\bookProjects\lib\site-packages\tensorflow_federated\python\core\impl\wrappers\computation_wrapper_instances.py", line 51, in _tf_wrapper_fn
target_fn, parameter_type, ctx_stack)
File "B:\tools and software\Anaconda\envs\bookProjects\lib\site-packages\tensorflow_federated\python\core\impl\tensorflow_serialization.py", line 274, in serialize_py_fn_as_tf_computation
result = target(*args)
File "B:\tools and software\Anaconda\envs\bookProjects\lib\site-packages\tensorflow_federated\python\core\impl\utils\function_utils.py", line 517, in <lambda>
return lambda arg: _unpack_and_call(fn, arg_types, kwarg_types, arg)
File "B:\tools and software\Anaconda\envs\bookProjects\lib\site-packages\tensorflow_federated\python\core\impl\utils\function_utils.py", line 510, in _unpack_and_call
return fn(*args, **kwargs)
File "B:\tools and software\Anaconda\envs\bookProjects\lib\site-packages\tensorflow_federated\python\learning\framework\optimizer_utils.py", line 381, in _compute_local_training_and_client_delta
client_output = client_delta_fn(dataset, initial_model_weights)
File "B:\tools and software\Anaconda\envs\bookProjects\lib\site-packages\tensorflow\python\eager\def_function.py", line 580, in __call__
result = self._call(*args, **kwds)
File "B:\tools and software\Anaconda\envs\bookProjects\lib\site-packages\tensorflow\python\eager\def_function.py", line 627, in _call
self._initialize(args, kwds, add_initializers_to=initializers)
File "B:\tools and software\Anaconda\envs\bookProjects\lib\site-packages\tensorflow\python\eager\def_function.py", line 506, in _initialize
*args, **kwds))
File "B:\tools and software\Anaconda\envs\bookProjects\lib\site-packages\tensorflow\python\eager\function.py", line 2446, in _get_concrete_function_internal_garbage_collected
graph_function, _, _ = self._maybe_define_function(args, kwargs)
File "B:\tools and software\Anaconda\envs\bookProjects\lib\site-packages\tensorflow\python\eager\function.py", line 2777, in _maybe_define_function
graph_function = self._create_graph_function(args, kwargs)
File "B:\tools and software\Anaconda\envs\bookProjects\lib\site-packages\tensorflow\python\eager\function.py", line 2667, in _create_graph_function
capture_by_value=self._capture_by_value),
File "B:\tools and software\Anaconda\envs\bookProjects\lib\site-packages\tensorflow\python\framework\func_graph.py", line 981, in func_graph_from_py_func
func_outputs = python_func(*func_args, **func_kwargs)
File "B:\tools and software\Anaconda\envs\bookProjects\lib\site-packages\tensorflow\python\eager\def_function.py", line 441, in wrapped_fn
return weak_wrapped_fn().__wrapped__(*args, **kwds)
File "B:\tools and software\Anaconda\envs\bookProjects\lib\site-packages\tensorflow\python\eager\function.py", line 3299, in bound_method_wrapper
return wrapped_fn(*args, **kwargs)
File "B:\tools and software\Anaconda\envs\bookProjects\lib\site-packages\tensorflow\python\framework\func_graph.py", line 968, in wrapper
raise e.ag_error_metadata.to_exception(e)
Additional Error list:
AttributeError: in user code:
B:\tools and software\Anaconda\envs\bookProjects\lib\site-packages\tensorflow_federated\python\learning\federated_averaging.py:90 __call__ *
num_examples_sum = dataset.reduce(
B:\tools and software\Anaconda\envs\bookProjects\lib\site-packages\tensorflow\python\data\ops\dataset_ops.py:1932 reduce **
add_to_graph=False)
B:\tools and software\Anaconda\envs\bookProjects\lib\site-packages\tensorflow\python\data\ops\dataset_ops.py:3210 __init__
self._input_structure),
B:\tools and software\Anaconda\envs\bookProjects\lib\site-packages\tensorflow\python\data\util\structure.py:270 get_flat_tensor_specs
nest.flatten(element_spec), [])
B:\tools and software\Anaconda\envs\bookProjects\lib\site-packages\tensorflow\python\data\util\structure.py:269 <lambda>
return functools.reduce(lambda state, value: state + value._flat_tensor_specs,
AttributeError: 'list' object has no attribute '_flat_tensor_specs'
GitHub link

How to access/manipulate elements of tensor in keras model?

I want to test a new network structure which requires changing some of the elements of a tensor in a keras model. If I could find a way to convert/copy the tensor to a numpy array and then later transform it back into a tensor, then I should be able to make the model work.
I tried using the .eval() method to convert the tensor to a numpy array, but it gives me errors. I am also using this model with a DQN Agent from keras-rl, so it is possible the error is from how keras-rl uses the model. Here is my code:
def Filter_Features(F):
sess = Session()
with sess.as_default():
F_np = F.eval()
min_pos = np.argmin(F_np)
F_np[min_pos] = 0
return convert_to_tensor(F_np)
def create_model(nb_actions, num_frames=4):
inputs = Input(shape = (num_frames,84,84))
F = Conv2D(16,(8,8), activation='relu', strides=(4,4), data_format = "channels_first")(inputs)
...
F_k = Lambda(Filter_Features)(F)
actions = Dense(nb_actions, activation = 'linear')(F_k)
nnf_model = Model(inputs = inputs, outputs = actions)
return nnf_model
Note that the code runs if I remove the lambda layer, so the issue must be originating there. I received the error:
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1356, in _do_call
return fn(*args)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1341, in _run_fn
options, feed_dict, fetch_list, target_list, run_metadata)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1429, in _call_tf_sessionrun
run_metadata)
tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value conv2d_1/bias
[[{{node conv2d_1/bias/read}}]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "Atari_Test.py", line 32, in <module>
model = Model_StackExchange.create_model(nb_actions = nb_actions)
File "/Users/j/deep-rl/Model_StackExchange.py", line 26, in create_model
F_k = Lambda(Filter_Features)(F)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/keras/engine/base_layer.py", line 457, in __call__
output = self.call(inputs, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/keras/layers/core.py", line 687, in call
return self.function(inputs, **arguments)
File "/Users/j/deep-rl/Model_StackExchange.py", line 11, in Filter_Features
F_np = F.eval()
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 731, in eval
return _eval_using_default_session(self, feed_dict, self.graph, session)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 5579, in _eval_using_default_session
return session.run(tensors, feed_dict)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 950, in run
run_metadata_ptr)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1173, in _run
feed_dict_tensor, options, run_metadata)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1350, in _do_run
run_metadata)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow/python/client/session.py", line 1370, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value conv2d_1/bias
[[node conv2d_1/bias/read (defined at /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:402) ]]
Original stack trace for 'conv2d_1/bias/read':
File "Atari_Test.py", line 32, in <module>
model = Model_StackExchange.create_model(nb_actions = nb_actions)
File "/Users/j/deep-rl/Model_StackExchange.py", line 21, in create_model
F = Conv2D(16,(8,8), activation='relu', strides=(4,4), data_format = "channels_first")(inputs)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/keras/engine/base_layer.py", line 431, in __call__
self.build(unpack_singleton(input_shapes))
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/keras/layers/convolutional.py", line 147, in build
constraint=self.bias_constraint)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/keras/legacy/interfaces.py", line 91, in wrapper
return func(*args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/keras/engine/base_layer.py", line 252, in add_weight
constraint=constraint)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py", line 402, in variable
v = tf.Variable(value, dtype=tf.as_dtype(dtype), name=name)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow/python/ops/variables.py", line 259, in __call__
return cls._variable_v1_call(*args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow/python/ops/variables.py", line 220, in _variable_v1_call
shape=shape)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow/python/ops/variables.py", line 198, in <lambda>
previous_getter = lambda **kwargs: default_variable_creator(None, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow/python/ops/variable_scope.py", line 2511, in default_variable_creator
shape=shape)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow/python/ops/variables.py", line 263, in __call__
return super(VariableMetaclass, cls).__call__(*args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow/python/ops/variables.py", line 1568, in __init__
shape=shape)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow/python/ops/variables.py", line 1755, in _init_from_args
self._snapshot = array_ops.identity(self._variable, name="read")
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow/python/util/dispatch.py", line 180, in wrapper
return target(*args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow/python/ops/array_ops.py", line 86, in identity
ret = gen_array_ops.identity(input, name=name)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow/python/ops/gen_array_ops.py", line 4253, in identity
"Identity", input=input, name=name)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py", line 788, in _apply_op_helper
op_def=op_def)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
return func(*args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 3616, in create_op
op_def=op_def)
File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 2005, in __init__
self._traceback = tf_stack.extract_stack()
Please let me know if you know how to access and change the elements of a tensor within a keras model. Thank you.

<Tensorflow object detection>TypeError: resize_images() got an unexpected keyword argument 'preserve_aspect_ratio'

cuda 9.0
cudnn 7.5
python 3.5.2
tensorflow-gpu 1.8
I don't know where the error occurred, I also tried python 3.6.3. This error will also occur. Please help.
I am training model_main.py file, but I get the following error.
python model_main.py --model_dir=F:/cindy/cindybackup/tensorflow1/test/training -pipeline_config_path=F:/cindy/cindybackup/tensorflow1/test/data/faster_rcnn_inception_v2_pets.config --alsologtostderr --num_train_steps=1000 --num_eval_steps=10
It shows the following:
WARNING:tensorflow:Forced number of epochs for all eval validations to
be 1.
WARNING:tensorflow:Expected number of evaluation epochs is 1, but instead encountered eval_on_train_input_config.num_epochs = 0.
Overwriting num_epochs to 1.
WARNING:tensorflow:Using temporary folder as model directory: C:\Users\wyh\AppData\Local\Temp\tmplh3q4jn2
WARNING:tensorflow:Estimator's model_fn (.model_fn at 0x00000256FF7F1400>) includes
params argument, but params are not passed to Estimator.
WARNING:tensorflow:num_readers has been reduced to 1 to match input file shards.
Traceback (most recent call last):
File "model_main.py", line 109, in
tf.app.run()
File "C:\Users\wyh\AppData\Local\conda\conda\envs\py352\lib\site-packages\tensorflow\python\platform\app.py",
line 126, in run
_sys.exit(main(argv))
File "model_main.py", line 105, in main
tf.estimator.train_and_evaluate(estimator, train_spec, eval_specs[0])
File "C:\Users\wyh\AppData\Local\conda\conda\envs\py352\lib\site-packages\tensorflow\python\estimator\training.py",
line 439, in train_and_evaluate
executor.run()
File "C:\Users\wyh\AppData\Local\conda\conda\envs\py352\lib\site-packages\tensorflow\python\estimator\training.py",
line 518, in run
self.run_local()
File "C:\Users\wyh\AppData\Local\conda\conda\envs\py352\lib\site-packages\tensorflow\python\estimator\training.py",
line 650, in run_local
hooks=train_hooks)
File "C:\Users\wyh\AppData\Local\conda\conda\envs\py352\lib\site-packages\tensorflow\python\estimator\estimator.py",
line 363, in train
loss = self._train_model(input_fn, hooks, saving_listeners)
File "C:\Users\wyh\AppData\Local\conda\conda\envs\py352\lib\site-packages\tensorflow\python\estimator\estimator.py",
line 843, in _train_model
return self._train_model_default(input_fn, hooks, saving_listeners)
File "C:\Users\wyh\AppData\Local\conda\conda\envs\py352\lib\site-packages\tensorflow\python\estimator\estimator.py",
line 853, in _train_model_default
input_fn, model_fn_lib.ModeKeys.TRAIN))
File "C:\Users\wyh\AppData\Local\conda\conda\envs\py352\lib\site-packages\tensorflow\python\estimator\estimator.py",
line 691, in _get_features_and_labels_from_input_fn
result = self._call_input_fn(input_fn, mode)
File "C:\Users\wyh\AppData\Local\conda\conda\envs\py352\lib\site-packages\tensorflow\python\estimator\estimator.py",
line 798, in _call_input_fn
return input_fn(**kwargs)
File "F:\cindy\cindybackup\tensorflow1\models\research\object_detection\inputs.py",
line 525, in _train_input_fn
batch_size=params['batch_size'] if params else train_config.batch_size)
File "F:\cindy\cindybackup\tensorflow1\models\research\object_detection\builders\dataset_builder.py",
line 149, in build
dataset = data_map_fn(process_fn, num_parallel_calls=num_parallel_calls)
File "C:\Users\wyh\AppData\Local\conda\conda\envs\py352\lib\site-packages\tensorflow\python\data\ops\dataset_ops.py",
line 853, in map
return ParallelMapDataset(self, map_func, num_parallel_calls)
File "C:\Users\wyh\AppData\Local\conda\conda\envs\py352\lib\site-packages\tensorflow\python\data\ops\dataset_ops.py",
line 1870, in init
super(ParallelMapDataset, self).init(input_dataset, map_func)
File "C:\Users\wyh\AppData\Local\conda\conda\envs\py352\lib\site-packages\tensorflow\python\data\ops\dataset_ops.py",
line 1839, in init
self._map_func.add_to_graph(ops.get_default_graph())
File "C:\Users\wyh\AppData\Local\conda\conda\envs\py352\lib\site-packages\tensorflow\python\framework\function.py",
line 484, in add_to_graph
self._create_definition_if_needed()
File "C:\Users\wyh\AppData\Local\conda\conda\envs\py352\lib\site-packages\tensorflow\python\framework\function.py",
line 319, in _create_definition_if_needed
self._create_definition_if_needed_impl()
File "C:\Users\wyh\AppData\Local\conda\conda\envs\py352\lib\site-packages\tensorflow\python\framework\function.py",
line 336, in _create_definition_if_needed_impl
outputs = self._func(*inputs)
File "C:\Users\wyh\AppData\Local\conda\conda\envs\py352\lib\site-packages\tensorflow\python\data\ops\dataset_ops.py",
line 1804, in tf_map_func
ret = map_func(nested_args)
File "F:\cindy\cindybackup\tensorflow1\models\research\object_detection\builders\dataset_builder.py",
line 130, in process_fn
processed_tensors = transform_input_data_fn(processed_tensors)
File "F:\cindy\cindybackup\tensorflow1\models\research\object_detection\inputs.py",
line 515, in transform_and_pad_input_data_fn
tensor_dict=transform_data_fn(tensor_dict),
File "F:\cindy\cindybackup\tensorflow1\models\research\object_detection\inputs.py",
line 129, in transform_input_data
tf.expand_dims(tf.to_float(image), axis=0))
File "F:\cindy\cindybackup\tensorflow1\models\research\object_detection\meta_architectures\faster_rcnn_meta_arch.py",
line 543, in preprocess
parallel_iterations=self._parallel_iterations)
File "F:\cindy\cindybackup\tensorflow1\models\research\object_detection\utils\shape_utils.py",
line 237, in static_or_dynamic_map_fn
outputs = [fn(arg) for arg in tf.unstack(elems)]
File "F:\cindy\cindybackup\tensorflow1\models\research\object_detection\utils\shape_utils.py",
line 237, in
outputs = [fn(arg) for arg in tf.unstack(elems)]
File "F:\cindy\cindybackup\tensorflow1\models\research\object_detection\core\preprocessor.py",
line 2264, in resize_to_range
lambda: _resize_portrait_image(image))
File "C:\Users\wyh\AppData\Local\conda\conda\envs\py352\lib\site-packages\tensorflow\python\util\deprecation.py",
line 432, in new_func
return func(*args, **kwargs)
File "C:\Users\wyh\AppData\Local\conda\conda\envs\py352\lib\site-packages\tensorflow\python\ops\control_flow_ops.py",
line 2063, in cond
orig_res_t, res_t = context_t.BuildCondBranch(true_fn)
File "C:\Users\wyh\AppData\Local\conda\conda\envs\py352\lib\site-packages\tensorflow\python\ops\control_flow_ops.py",
line 1913, in BuildCondBranch
original_result = fn()
File "F:\cindy\cindybackup\tensorflow1\models\research\object_detection\core\preprocessor.py",
line 2263, in
lambda: _resize_landscape_image(image),
File "F:\cindy\cindybackup\tensorflow1\models\research\object_detection\core\preprocessor.py",
line 2245, in _resize_landscape_image
align_corners=align_corners, preserve_aspect_ratio=True)
TypeError: resize_images() got an unexpected keyword argument 'preserve_aspect_ratio'
Thanks~

The problem is not resolved yet in tensorflow models(https://github.com/tensorflow/models/)
I just removed the preserve_aspect_ratio in object_detection/core/preprocessor.py
align_corners=align_corners, preserve_aspect_ratio=True)
align_corners=align_corners)

tensorflow.python.framework.errors_impl.InvalidArgumentError: indices[27,4,0] = -1 is not in [0, 31488)

When I run the shell of walkthrough training a good English-to-German translation model using the Transformer model, but I encountered the problem.
My problem is:
INFO:tensorflow:Total trainable variables size: 60276736
INFO:tensorflow:Total embedding variables size: 16384
INFO:tensorflow:Total non-embedding variables size: 60260352
INFO:tensorflow:Computing gradients for global model_fn.
INFO:tensorflow:Global model_fn finished.
INFO:tensorflow:Create CheckpointSaverHook.
2017-06-30 15:05:58.562782: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2017-06-30 15:05:58.562814: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2017-06-30 15:05:58.562820: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2017-06-30 15:05:58.562824: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2017-06-30 15:05:58.562829: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
Traceback (most recent call last):
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.4/site-packages/tensorflow/python/client/session.py", line 1139, in _do_call
return fn(*args)
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.4/site-packages/tensorflow/python/client/session.py", line 1121, in _run_fn
status, run_metadata)
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.4/contextlib.py", line 66, in exit
next(self.gen)
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.4/site-packages/tensorflow/python/framework/errors_impl.py", line 466, in raise_exception_on_not_ok_status
pywrap_tensorflow.TF_GetCode(status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: indices[27,4,0] = -1 is not in [0, 31488)
[[Node: symbol_modality_31488_512/parallel_0/symbol_modality_31488_512/shared/Gather = Gather[Tindices=DT_INT32, Tparams=DT_FLOAT, validate_indices=true, _device="/job:localhost/replica:0/task:0/cpu:0"](symbol_modality_31488_512/parallel_0/symbol_modality_31488_512/shared/ConvertGradientToTensor_cc661786, symbol_modality_31488_512/parallel_0/symbol_modality_31488_512/shared/Squeeze)]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/sycmss/tc/anaconda3/envs/tensorflow/bin/t2t-trainer", line 83, in
tf.app.run()
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.4/site-packages/tensorflow/python/platform/app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "/home/sycmss/tc/anaconda3/envs/tensorflow/bin/t2t-trainer", line 79, in main
schedule=FLAGS.schedule)
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.4/site-packages/tensor2tensor/utils/trainer_utils.py", line 240, in run
run_locally(exp_fn(output_dir))
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.4/site-packages/tensor2tensor/utils/trainer_utils.py", line 532, in run_locally
exp.train()
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.4/site-packages/tensorflow/contrib/learn/python/learn/experiment.py", line 275, in train
hooks=self._train_monitors + extra_hooks)
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.4/site-packages/tensorflow/contrib/learn/python/learn/experiment.py", line 665, in _call_train
monitors=hooks)
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.4/site-packages/tensorflow/python/util/deprecation.py", line 289, in new_func
return func(*args, **kwargs)
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.4/site-packages/tensorflow/contrib/learn/python/learn/estimators/estimator.py", line 455, in fit
loss = self._train_model(input_fn=input_fn, hooks=hooks)
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.4/site-packages/tensorflow/contrib/learn/python/learn/estimators/estimator.py", line 1007, in _train_model
_, loss = mon_sess.run([model_fn_ops.train_op, model_fn_ops.loss])
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.4/site-packages/tensorflow/python/training/monitored_session.py", line 505, in run
run_metadata=run_metadata)
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.4/site-packages/tensorflow/python/training/monitored_session.py", line 842, in run
run_metadata=run_metadata)
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.4/site-packages/tensorflow/python/training/monitored_session.py", line 798, in run
return self._sess.run(*args, **kwargs)
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.4/site-packages/tensorflow/python/training/monitored_session.py", line 952, in run
run_metadata=run_metadata)
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.4/site-packages/tensorflow/python/training/monitored_session.py", line 798, in run
return self._sess.run(*args, **kwargs)
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.4/site-packages/tensorflow/python/client/session.py", line 789, in run
run_metadata_ptr)
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.4/site-packages/tensorflow/python/client/session.py", line 997, in _run
feed_dict_string, options, run_metadata)
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.4/site-packages/tensorflow/python/client/session.py", line 1132, in _do_run
target_list, options, run_metadata)
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.4/site-packages/tensorflow/python/client/session.py", line 1152, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: indices[27,4,0] = -1 is not in [0, 31488)
[[Node: symbol_modality_31488_512/parallel_0/symbol_modality_31488_512/shared/Gather = Gather[Tindices=DT_INT32, Tparams=DT_FLOAT, validate_indices=true, _device="/job:localhost/replica:0/task:0/cpu:0"](symbol_modality_31488_512/parallel_0/symbol_modality_31488_512/shared/ConvertGradientToTensor_cc661786, symbol_modality_31488_512/parallel_0/symbol_modality_31488_512/shared/Squeeze)]]
Caused by op 'symbol_modality_31488_512/parallel_0/symbol_modality_31488_512/shared/Gather', defined at:
File "/home/sycmss/tc/anaconda3/envs/tensorflow/bin/t2t-trainer", line 83, in
tf.app.run()
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.4/site-packages/tensorflow/python/platform/app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "/home/sycmss/tc/anaconda3/envs/tensorflow/bin/t2t-trainer", line 79, in main
schedule=FLAGS.schedule)
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.4/site-packages/tensor2tensor/utils/trainer_utils.py", line 240, in run
run_locally(exp_fn(output_dir))
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.4/site-packages/tensor2tensor/utils/trainer_utils.py", line 532, in run_locally
exp.train()
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.4/site-packages/tensorflow/contrib/learn/python/learn/experiment.py", line 275, in train
hooks=self._train_monitors + extra_hooks)
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.4/site-packages/tensorflow/contrib/learn/python/learn/experiment.py", line 665, in _call_train
monitors=hooks)
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.4/site-packages/tensorflow/python/util/deprecation.py", line 289, in new_func
return func(*args, **kwargs)
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.4/site-packages/tensorflow/contrib/learn/python/learn/estimators/estimator.py", line 455, in fit
loss = self._train_model(input_fn=input_fn, hooks=hooks)
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.4/site-packages/tensorflow/contrib/learn/python/learn/estimators/estimator.py", line 955, in _train_model
model_fn_ops = self._get_train_ops(features, labels)
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.4/site-packages/tensorflow/contrib/learn/python/learn/estimators/estimator.py", line 1162, in _get_train_ops
return self._call_model_fn(features, labels, model_fn_lib.ModeKeys.TRAIN)
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.4/site-packages/tensorflow/contrib/learn/python/learn/estimators/estimator.py", line 1133, in _call_model_fn
model_fn_results = self._model_fn(features, labels, **kwargs)
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.4/site-packages/tensor2tensor/utils/trainer_utils.py", line 424, in model_fn
len(hparams.problems) - 1)
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.4/site-packages/tensor2tensor/utils/trainer_utils.py", line 751, in _cond_on_index
return fn(cur_idx)
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.4/site-packages/tensor2tensor/utils/trainer_utils.py", line 406, in nth_model
features, skip=(skipping_is_on and skip_this_one))
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.4/site-packages/tensor2tensor/utils/t2t_model.py", line 377, in model_fn
sharded_features[key], dp)
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.4/site-packages/tensor2tensor/utils/modality.py", line 91, in bottom_sharded
return data_parallelism(self.bottom, xs)
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.4/site-packages/tensor2tensor/utils/expert_utils.py", line 294, in call
outputs.append(fns[i](*my_args[i], **my_kwargs[i]))
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.4/site-packages/tensor2tensor/models/modalities.py", line 88, in bottom
return self.bottom_simple(x, "shared", reuse=None)
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.4/site-packages/tensor2tensor/models/modalities.py", line 80, in bottom_simple
ret = tf.gather(var, x)
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.4/site-packages/tensorflow/python/ops/gen_array_ops.py", line 1179, in gather
validate_indices=validate_indices, name=name)
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.4/site-packages/tensorflow/python/framework/op_def_library.py", line 767, in apply_op
op_def=op_def)
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.4/site-packages/tensorflow/python/framework/ops.py", line 2506, in create_op
original_op=self._default_original_op, op_def=op_def)
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.4/site-packages/tensorflow/python/framework/ops.py", line 1269, in init
self._traceback = _extract_stack()
InvalidArgumentError (see above for traceback): indices[27,4,0] = -1 is not in [0, 31488)
[[Node: symbol_modality_31488_512/parallel_0/symbol_modality_31488_512/shared/Gather = Gather[Tindices=DT_INT32, Tparams=DT_FLOAT, validate_indices=true, _device="/job:localhost/replica:0/task:0/cpu:0"](symbol_modality_31488_512/parallel_0/symbol_modality_31488_512/shared/ConvertGradientToTensor_cc661786, symbol_modality_31488_512/parallel_0/symbol_modality_31488_512/shared/Squeeze)]]
INFO:tensorflow:Creating experiment, storing model files in /root/t2t_train/wmt_ende_tokens_32k/transformer-transformer_base
INFO:tensorflow:datashard_devices: ['gpu:0']
INFO:tensorflow:caching_devices: None
INFO:tensorflow:Using config: {'_task_type': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x7f884bf21630>, '_model_dir': '/root/t2t_train/wmt_ende_tokens_32k/transformer-transformer_base', '_save_checkpoints_secs': 600, '_save_summary_steps': 100, '_session_config': allow_soft_placement: true
graph_options {
optimizer_options {
}
}
, '_tf_config': gpu_options {
per_process_gpu_memory_fraction: 1.0
}
, '_task_id': 0, '_tf_random_seed': None, '_num_ps_replicas': 0, '_evaluation_master': '', '_keep_checkpoint_max': 20, '_keep_checkpoint_every_n_hours': 10000, '_master': '', '_is_chief': True, '_num_worker_replicas': 0, '_save_checkpoints_steps': None, '_environment': 'local'}
INFO:tensorflow:Performing Decoding from a file.
INFO:tensorflow:Getting sorted inputs
Traceback (most recent call last):
File "/home/sycmss/tc/anaconda3/envs/tensorflow/bin/t2t-trainer", line 83, in
tf.app.run()
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.4/site-packages/tensorflow/python/platform/app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "/home/sycmss/tc/anaconda3/envs/tensorflow/bin/t2t-trainer", line 79, in main
schedule=FLAGS.schedule)
File "/home/sycmss/tc/anaconda3/envs/tensorflow/l`enter code here`ib/python3.4/site-packages/tensor2tensor/utils/trainer_utils.py", line 240, in run
run_locally(exp_fn(output_dir))
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.4/site-packages/tensor2tensor/utils/trainer_utils.py", line 544, in run_locally
decode_from_file(estimator, FLAGS.decode_from_file)
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.`enter code here`4/site-packages/tensor2tensor/utils/trainer_utils.py", line`enter code here` 648, in decode_from_file
as_iterable=True)
File "/home/sycmss/tc/anaconda3/envs/tensorflow/lib/python3.4/site-packages/tensorflow/python/util/deprecation.py", line 289, in new_func
return func(*args, **kwargs)
How should I solve this problem?
Thank you

Race Condition when reading from generator

I am running a python program on Python2.7. Wherein I am trying to print the value of a generator using the following approach.
y = m.predict(input_fn=lambda:input_fn(df_predict), as_iterable=True)
#output of this is a generator
print(type(y))
print(list(y))
The following error is displayed:
Traceback (most recent call last):
File "tensorflow/examples/wide_n_deep_model_predictions_v3.0.py", line 176, in <module>
tf.app.run()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 43, in run
sys.exit(main(sys.argv[:1] + flags_passthrough))
File "tensorflow/examples/wide_n_deep_model_predictions_v3.0.py", line 171, in main
train_and_eval()
File "tensorflow/examples/wide_n_deep_model_predictions_v3.0.py", line 167, in train_and_eval
print (list(y))
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/estimators/dnn_linear_combined.py", line 328, in _as_iterable
for pred in preds:
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/estimators/estimator.py", line 909, in _infer_model_as_iterable
restore_checkpoint_path=checkpoint_path):
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/graph_actions.py", line 867, in run_feeds_iter
yield session.run(output_dict, f)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 766, in run
run_metadata_ptr)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 964, in _run
feed_dict_string, options, run_metadata)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1014, in _do_run
target_list, options, run_metadata)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1034, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: WhereOp: Race condition between counting the number of true elements and writing them. When counting, saw 2645 elements; but when writing their indices, saw 19 elements.
[[Node: linear/linear/ip/ip_weights/Where = Where[_device="/job:localhost/replica:0/task:0/cpu:0"](linear/linear/ip/ip_weights/GreaterEqual/_409)]]
Caused by op u'linear/linear/ip/ip_weights/Where', defined at:
File "tensorflow/examples/wide_n_deep_model_predictions_v3.0.py", line 176, in <module>
tf.app.run()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 43, in run
sys.exit(main(sys.argv[:1] + flags_passthrough))
File "tensorflow/examples/wide_n_deep_model_predictions_v3.0.py", line 171, in main
train_and_eval()
File "tensorflow/examples/wide_n_deep_model_predictions_v3.0.py", line 163, in train_and_eval
y = m.predict(input_fn=lambda:input_fn(df_predict), as_iterable=True)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/util/deprecation.py", line 245, in new_func
return func(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/estimators/dnn_linear_combined.py", line 747, in predict
as_iterable=as_iterable)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/util/deprecation.py", line 191, in new_func
return func(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/estimators/estimator.py", line 477, in predict
as_iterable=as_iterable)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/estimators/estimator.py", line 847, in _infer_model
infer_ops = self._get_predict_ops(features)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/estimators/estimator.py", line 1113, in _get_predict_ops
return self._call_model_fn(features, labels, ModeKeys.INFER)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/estimators/estimator.py", line 1026, in _call_model_fn
params=self.params)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/estimators/dnn_linear_combined.py", line 504, in _dnn_linear_combined_model_fn
scope=scope)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/layers/python/layers/feature_column_ops.py", line 535, in weighted_sum_from_feature_columns
weight_collections)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/layers/python/layers/feature_column_ops.py", line 332, in _create_embedding_lookup
name=column.name + '_weights')
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/layers/python/layers/embedding_ops.py", line 123, in safe_embedding_lookup_sparse
sparse_ids, sparse_weights = _prune_invalid_ids(sparse_ids, sparse_weights)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/layers/python/layers/embedding_ops.py", line 169, in _prune_invalid_ids
sparse_ids = sparse_ops.sparse_retain(sparse_ids, is_id_valid)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/sparse_ops.py", line 880, in sparse_retain
where_true = array_ops.reshape(array_ops.where(to_retain), [-1])
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/array_ops.py", line 2501, in where
return gen_array_ops.where(input=condition, name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gen_array_ops.py", line 3821, in where
result = _op_def_lib.apply_op("Where", input=input, name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py", line 759, in apply_op
op_def=op_def)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 2259, in create_op
original_op=self._default_original_op, op_def=op_def)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1130, in __init__
self._traceback = _extract_stack()
InvalidArgumentError (see above for traceback): WhereOp: Race condition between counting the number of true elements and writing them. When counting, saw 2645 elements; but when writing their indices, saw 19 elements.
[[Node: linear/linear/ip/ip_weights/Where = Where[_device="/job:localhost/replica:0/task:0/cpu:0"](linear/linear/ip/ip_weights/GreaterEqual/_409)]]
On running print ('Predictions: {}'.format(str(y))) the output is Predictions: <generator object _as_iterable at 0x7f1883235550>
Can someone guide me how can I print the generator?

I think you need to do something like this
# Print out predictions
y = regressor.predict(input_fn=lambda: input_fn(prediction_set))
# .predict() returns an iterator; convert to a list and print predictions
predictions = list(itertools.islice(y, 6))
print("Predictions: {}".format(str(predictions)))
I found it from the boston tutorial: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/examples/tutorials/input_fn/boston.py