I've designed a custom op and it works as expected in python. I call it with the usual lines of code:
import tensorflow as tf
custom_mod = tf.load_op_library('/path/to/.so')
with tf.device("/gpu:0"):
with tf.Session as sess:
#(note: function name .exec() depends on kernel code)
custom_mod_in_graph = custom_mod.exec()
sess.run(custom_mod_in_graph)
However I need to run the same op with the TF C++ API, unfortunately I can't find any respective calling procedures (i.e.: no equivalent for tf.load_op_library).
Are they missing or is there a different way of integrating the ops there?
Related
I have upgraded with tf_upgrade_v2 TF1 code to TF2. I'm a noob with both. I got the next error:
RuntimeError: tf.placeholder() is not compatible with eager execution.
I have some tf.compat.v1.placeholder().
self.temperature = tf.compat.v1.placeholder_with_default(1., shape=())
self.edges_labels = tf.compat.v1.placeholder(dtype=tf.int64, shape=(None, vertexes, vertexes))
self.nodes_labels = tf.compat.v1.placeholder(dtype=tf.int64, shape=(None, vertexes))
self.embeddings = tf.compat.v1.placeholder(dtype=tf.float32, shape=(None, embedding_dim))
Could you give me any advice about how to proceed? Any "fast" solutions? or should I to recode this?
I found an easy solution here: disable Tensorflow eager execution
Basicaly it is:
tf.compat.v1.disable_eager_execution()
With this, you disable the default activate eager execution and you don't need to touch the code much more.
tf.placeholder() is meant to be fed to the session that when run receive the values from feed dict and perform the required operation.
Generally, you would create a Session() with 'with' keyword and run it. But this might not favour all situations due to which you would require immediate execution. This is called eager execution.
Example:
generally, this is the procedure to run a Session:
import tensorflow as tf
def square(num):
return tf.square(num)
p = tf.placeholder(tf.float32)
q = square(num)
with tf.Session() as sess:
print(sess.run(q, feed_dict={num: 10})
But when we run with eager execution we run it as:
import tensorflow as tf
tf.enable_eager_execution()
def square(num):
return tf.square(num)
print(square(10))
Therefore we need not run it inside a session explicitly and can be more intuitive in most of the cases. This provides more of an interactive execution.
For further details visit:
https://www.tensorflow.org/guide/eager
If you are converting the code from tensorflow v1 to tensorflow v2, You must implement tf.compat.v1 and Placeholder is present at tf.compat.v1.placeholder but this can only be executed in eager mode off.
tf.compat.v1.disable_eager_execution()
TensorFlow released the eager execution mode, for which each node is immediately executed after definition. Statements using tf.placeholder are thus no longer valid.
In TensorFlow 1.X, placeholders are created and meant to be fed with actual values when a tf.Session is instantiated. However, from TensorFlow2.0 onwards, Eager Execution has been enabled by default, so the notion of a "placeholder" does not make sense as operations are computed immediately (rather than being differed with the old paradigm).
Also see Functions, not Sessions,
# TensorFlow 1.X
outputs = session.run(f(placeholder), feed_dict={placeholder: input})
# TensorFlow 2.0
outputs = f(input)
If you are getting this error while doing object detection using TensorFlow model then use exporter_main_v2.py instead of export_inference_graph.py for exporting the model. This is right method to do. If you just off eager_execution then it will solve this error but generate other.
Also note that there are some parameter change like hear you will specify the path to directory of checkpoint instead of path to checkpoint. refer to this document for how to do object detection with TensorFlow V2
To solve this, you have to disable the default activate eager execution. So add the following code line.
tf.compat.v1.disable_eager_execution() #<--- Disable eager execution
Before the error fixing
After the error fixing
I am training my model using model.train_on_batch(). I dig into Keras code how to use train_ob_batch properly. I notice before calling this function, Keras calls several routines, including model._make_train_function().
But, I found many Github sources, they use model.train_on_batch without calling _make_train_function() first.
So, my question is whether calling _make_train_function() has to be called first before train_on_batch? I am using Keras 2.2.4 and Tensorflow 1.12.0
Thanks.
I have a tf.nn.rnn_cell.BasicLSTMCell as part of my NN architecture. I use a for loop because it is recursing over input a fixed number of time steps. Something like this:
lstm_cell = tf.nn.rnn_cell.BasicLSTMCell(num_units=lstm_dimensionality, name="forward_lstm")
_, (lstm_memory, lstm_hidden) = lstm_cell(input_m, state=[lstm_memory, lstm_hidden])
for i in range(3):
# HERE is where the error is thrown
_, (lstm_memory, lstm_hidden) = lstm_cell(input_m, state=[lstm_memory, lstm_hidden])
It works quite well locally on a single device. It also works fine in Google ML Engine on a single GPU. However, when I try distributing to 4 GPUs using tf.distribute.MirroredStrategy, it throws an exception
ValueError: At least one of name (None) and default_name (None) must be provided.
The lstm_cell callable doesn't even take a name parameter so it's confusing.
There isn't much room for details here, so I've created a toy example in this Github repo to reproduce the bug in ML Engine. It is specifically on this line where the error is thrown.
Tensorflow: 1.13.1
ML Engine: --runtime-version 1.13
In your code here you use a scope in the function compute_initial_lstm_state.
The you reuse the 2 returned values here
You use a scope to generate values and you assign them without scope.
This should be your root error. With a single GPU, the scope can be deduce automatically. But with multi gpu, it's not possible and it fail.
I am using WALS method in order to perform matrix factorization. Initially in tensorflow 1.13 I can import factorization_ops using
from tensorflow.contrib.factorization.python.ops import factorization_ops
As described in the documentation
Wals model can be called from factorization_ops by using
factorization_ops.WALSModel
Using same command in tensorflow 2.0 giving me following error
ModuleNotFoundError: No module named 'tensorflow.contrib.factorization
Going through the issue there appears to be no way out to use WALSModel in tensorflow 2.0+.
Also it has been mentioned here in tensorflow release updates that tf.contrib has been deprecated, and functionality has been either migrated to the core TensorFlow API, to an ecosystem project such as tensorflow/addons or tensorflow/io, or removed entirely.
How can I use WALS model in tensorflow 2.0 (Currently I am using 2.0.0-rc0 on windows machine) ? Is WALSModel has been removed or I am missing out some information ?
I believe WALS is not supported in TF 2.0 ...The official recommendation model is Neural Collaborative Filter (NCF)
I hope this helps.
M
I have the same issue, but I don't really have time to write a library myself unfortunately. There are several potential options that I am considering:
Stick with TF1.X until someone creates a library
Switch to using lightfm to continue using WALS
Switch to neural collaborative filtering using embedding layers with keras and a dot product layer. See this paper https://arxiv.org/abs/1708.05031, and this code implementation:
from tensorflow.keras.layers import Input, Embedding, Flatten, Dot, Dense
from tensorflow.keras.models import Model
#import tensorflow.distribute
def get_compiled_model(n_users, n_items, embedding_dims=20):
# Product embedding
prod_input = Input(shape=[1], name="Item-Input")
prod_embedding = Embedding(n_items+1, embedding_dims, name="Item-Embedding")(prod_input)
prod_vec = Flatten(name="Flatten-Product")(prod_embedding)
# User embedding
user_input = Input(shape=[1], name="User-Input")
user_embedding = Embedding(n_users+1, embedding_dims, name="User-Embedding")(user_input)
user_vec = Flatten(name="Flatten-Users")(user_embedding)
# The output is the dot product of the two, i.e. a one-hot vector
dot_product = Dot(name="Dot-Product", axes=1)([prod_vec, user_vec])
# compile - uncomment these two lines to make training distributed
# dist_strat = distribute.Strategy()
# with dist_strat.scope():
model = Model(inputs = [user_input, prod_input], outputs = dot_product)
model.compile(
optimizer='adam',
loss='mean_squared_error'
)
return model
I have compared the Tensorflow implementation of WALS to other implementations with respect to compute resources and accuracy (https://github.com/gtsoukas/cfzoo). The comparison suggests that the implicit Python package (https://github.com/benfred/implicit) is a good replacement that delivers superior performance.
I think that WALS has been removed. As part of tf.contrib it is not supported by TF2 and I do not think it fits in any of core or sub-projects.
Your best bet is probably to make it available as a third-party library.
I expect to use it for my project, but the need to re-write it (mainly copy what was in TF1 and make it work as a separate library compatible with TF2) reduce the priority of this task...
Let us know if you start to code something. Thanks.
Alexis.
With NO keras can you do eager execution in tensorflow? I have a non-neural network model in TensorFlow graph code to move to eager. This is a low rank matrix factorization for recommender system.
Python language.
Thank you
Request that answerers please demonstrate working code. If answer includes speculation then please state explicitly.
Yes, you can certainly use eager execution without Keras. Keras is built on top of the lower level operations that support eager execution.
For example:
import tensorflow as tf
import numpy as np
tf.enable_eager_execution()
W = tf.contrib.eager.Variable(tf.random_normal((10, 10)))
def model(x):
return tf.matmul(x, W)
data = np.random.randn(3, 10).astype(np.float32)
print(model(data))
You can see some more detailed tutorials at https://www.tensorflow.org/tutorials/eager/
That said, there are various corner cases/errors you might hit if trying to run arbitrary code written to construct a graph with eager execution enabled, and slight refactoring may be needed. Those would depend on the details of how the code is structured.
The reverse (i.e., writing code that works with eager execution enabled) generally works out well to construct the equivalent graph when eager execution is not enabled.
Hope that helps.