I face problems when trying to get a numpy array from a tensorflow tensor. I use a tensorflow hub module but I don't want to use tensorflow in downstream tasks but rather need a numpy array.
I know that I have to call the 'eval()' method on the tensor from within a tensorflow session. But unfortuantely I cannot get it to work... :( It tells me that the "tables are not initialized". I tried to add 'sess.run(tf.tables_initializer())' but then I get the error: 'NotFoundError: Resource localhost/module_1/embeddings_morph_specialized/class tensorflow::Var does not exist'. I am not sure what to try next. I have also tried 'sess.run()' but have also been unsuccessful.
import numpy as np
import tensorflow as tf
import tensorflow_hub as hub
embed = hub.Module("https://public.ukp.informatik.tu-darmstadt.de/arxiv2018-xling-sentence-embeddings/tf-hub/monolingual/1")
X = embed(["This is a test."])
# I tried:
#with tf.Session() as sess:
# sess.run(tf.tables_initializer())
# X.eval()
'X' is the tensor which I would like to convert to a numpy array.
Any help is appreciated. :)
Thank you.
Unfortunately, tf_hub modules are not yet supported in eager mode except in tf 2 (which is still in beta and I think needs slightly different hub modules anyway).
Therefore you'll need to run this in a session.
Something like:
embed = hub.Module("https://public.ukp.informatik.tu-darmstadt.de/arxiv2018-xling-sentence-embeddings/tf-hub/monolingual/1")
X = embed(["This is a test."])
with tf.Session() as session:
session.run([tf.global_variables_initializer(), tf.tables_initializer()])
numpy_arr = session.run(X)
Related
I'm trying to get reproducible results with Keras, however every time I run the program I get different results.
I've set the python hash seed, the Numpy random seed, the random seed, the TensorFlow seed, and the kernel_initializer glorot_uniform seed, but I still don't get reproducible results. Are there any other things I can do to get reproducible results?
I expect the predictions to be the same, however they are not. I get different results every single time.
with TENSORFLOW 2
import tensorflow as tf
tf.random.set_seed(33)
os.environ['PYTHONHASHSEED'] = str(33)
np.random.seed(33)
random.seed(33)
session_conf = tf.compat.v1.ConfigProto(
intra_op_parallelism_threads=1,
inter_op_parallelism_threads=1
)
sess = tf.compat.v1.Session(
graph=tf.compat.v1.get_default_graph(),
config=session_conf
)
tf.compat.v1.keras.backend.set_session(sess)
Because you're using Keras with Tensorflow as backend, you will find it is pretty hard to get reproducible result especially when GPU is enable. However, there is still a method to achieve this.
First, do not use GPU.
import os
os.environ["CUDA_DEVICE_ORDER"] = "PCI_BUS_ID"
os.environ["CUDA_VISIBLE_DEVICES"] = ""
Second, as you've did in code, set seed for Numpy, Random, TensorFlow and so on.
import tensorflow as tf
import numpy as np
import random as rn
sd = 1 # Here sd means seed.
np.random.seed(sd)
rn.seed(sd)
os.environ['PYTHONHASHSEED']=str(sd)
from keras import backend as K
config = tf.ConfigProto(intra_op_parallelism_threads=1,inter_op_parallelism_threads=1)
tf.set_random_seed(sd)
sess = tf.Session(graph=tf.get_default_graph(), config=config)
K.set_session(sess)
One final word, both two pieces of code should be placed at the begining of your code.
I created a rule to achieve reproducibility:
Works for python 3.6, not 3.7
First install Keras 2.2.4
After install tensorflow 1.9
And finally in the code:
import numpy as np
import random as rn
import tensorflow as tf
import keras
from keras import backend as K
#-----------------------------Keras reproducible------------------#
SEED = 1234
tf.set_random_seed(SEED)
os.environ['PYTHONHASHSEED'] = str(SEED)
np.random.seed(SEED)
rn.seed(SEED)
session_conf = tf.ConfigProto(
intra_op_parallelism_threads=1,
inter_op_parallelism_threads=1
)
sess = tf.Session(
graph=tf.get_default_graph(),
config=session_conf
)
K.set_session(sess)
#-----------------------------------------------------------------#
(Only tested for Tensorflow 2)
Besides setting the random seeds, I found that my RTX 3080 GPU would only give deterministic results if I used tf.float64 instead of the default of tf.float32. This appears to be due to rounding errors on the GPU, which leads to differences in the path taken during gradient descent. Note that this does not guarantee reproducibility across different GPUs. Different GPU architectures are not guaranteed to perform operations in exactly the same way. Such differences in implementation may cause differences in rounding, which can in turn affect the convergence of your model.
I'm currently running in a problem where tensorflow doesn't produce the results I intended to get. When I tried to debug the problem I noticed that up to a division everything seems to work out fine.
https://imgur.com/a/DT4RWiS
Can someone enlighten me what might be happening here?
There might be some stochasticity involved. Consider the following example:
import tensorflow as tf
a = tf.random_uniform(shape=(4,), minval=1, maxval=2)
b = tf.random_uniform(shape=(4,), minval=1, maxval=2)
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
print(a.eval()/b.eval())
print((a/b).eval())
When a.eval() and (a/b).eval() are called, tf.random_uniform is executed and tensor a is filled with different random numbers.
This is because the / operator in tensorflow follows the Python 2 syntax and performs an integer division (see https://www.tensorflow.org/api_docs/python/tf/div). You should use tf.divide instead. In general for any mathematical operation between tensors you should use tf.operation_name
In the following code, "a" works perfectly fine, and "c" also works. But "b" causes an error. Could someone explain the reason?
#!/usr/bin/python
import tensorflow as tf
import numpy as np
a = tf.Variable(tf.random_normal((10, 4)))
b = tf.constant(tf.random_normal((10, 4)))
c = tf.constant(np.random.randn(10, 4))
I am also a new one who start using tensorflow. I believe that there is something wrong with your variable type. According to the tensorflow API, you should feed a constant or list of value to 'tf.constant()'. However, in you code, before you initialize the variables and run this session, 'tf.random_normal()' is something like a placeholder without any real meaning. You can try to run this code. I am not sure if I understand this problem and I would like to discuss with you.
import tensorflow as tf
a = tf.random_normal((10, 4))
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
b = tf.constant(sess.run(a))
print(sess.run(b))
I'm new at Tensorflow. I am having a litte trouble at understanding its constants. I have this simple code mentioned below:
import tensorflow as tf
vector = tf.constant([[1,2,3,4],[4,5,6,7],[8,9,1,2]],tf.int32,name="vector")
with tf.Session() as sess:
v = sess.run(vector)
argm = tf.argmax(v,1)
print(argm)
I expect this to return something like [4,7,8], as I understood from the documentation. Instead, I get this:
Tensor("ArgMax:0", shape=(3,), dtype=int64).
So, i don't know what am I doing wrong.
Alternatively to the answer of #James, you might want to use tensorflow's eager execution, which behaves more like "standard" python: operations are executed as you type them, no more graphs and Session.
import tensorflow as tf
tf.enable_eager_execution()
vector = tf.constant([[1,2,3,4],[4,5,6,7],[8,9,1,2]],tf.int32,name="vector")
argm = tf.argmax(vector,1)
print(argm)
Tensorflow operations like tf.argmax somewhat unintuitively don't perform the operation that they're stating, but add an operation to a graph that will be performed. When you run argm = tf.argmax(v,1), the return value is tensor that isn't evaluated yet.
If you want the result of the argmax operation, you can run something like this:
import tensorflow as tf
vector = tf.constant([[1,2,3,4],[4,5,6,7],[8,9,1,2]],tf.int32,name="vector")
argm = tf.argmax(vector,1)
with tf.Session() as sess:
a = sess.run(argm)
print(a)
With this code, we're explicitly asking Tensorflow to run the computations to compute the result of the tf.argmax operation. With your previous code, we ran the computation to compute v (which is a constant, so that's pretty quick), then define a new graph operation to compute argmax on that - but never actually do the computation.
I am learning tensorflow and i have two questions first is , is it necessery to run each opeation in session ? like if i create a simple program in which there are three operations addition and substraction and matrix multiply then is it necessary to run all those operation in session ?
import tensorflow as tf
import numpy as np
import os
os.environ["TF_CPP_MIN_LOG_LEVEL"]="3"
a=np.array([[1,2,3],[4,5,6],[5,6,7]],dtype="float32")
b=np.array([[7,8,9],[9,6,5],[6,7,8]],dtype="float32")
ab=tf.Variable(a)
bb=tf.constant(b)
inn=tf.global_variables_initializer()
add=ab+bb
sub=(ab-bb)
mul=tf.matmul(a,b)
with tf.Session() as rr:
rr.run(inn)
rr.run(ab)
res=rr.run(mul)
print(res)
so now i have to run each operation (add,sub and matmul) in session ??
if "yes" i have to run then can i run all those operation together ? i tried but i got error :
first i tried this :
import tensorflow as tf
import numpy as np
import os
os.environ["TF_CPP_MIN_LOG_LEVEL"]="3"
a=np.array([[1,2,3],[4,5,6],[5,6,7]],dtype="float32")
b=np.array([[7,8,9],[9,6,5],[6,7,8]],dtype="float32")
ab=tf.Variable(a)
bb=tf.constant(b)
inn=tf.global_variables_initializer()
add=ab+bb
sub=(ab-bb)
mul=tf.matmul(a,b)
with tf.Session() as rr:
rr.run(inn,ab,add,sub,mul)
rr.run(ab)
res=rr.run(mul)
print(res)
i got this error :
TypeError: run() takes from 2 to 5 positional arguments but 6 were given
so i removed one argument (mul) from run then i got this error :
raise TypeError("Using a `tf.Tensor` as a Python `bool` is not allowed. "
TypeError: Using a `tf.Tensor` as a Python `bool` is not allowed. Use `if t is not None:` instead of `if t:` to test if a tensor is defined, and use TensorFlow ops such as tf.cond to execute subgraphs conditioned on the value of a tensor.
how i can run all operation once or i have to run each individually ?
second during writing this code i tried to multiply two matrix with shape [2,3] my program was :
import tensorflow as tf
import numpy as np
import os
os.environ["TF_CPP_MIN_LOG_LEVEL"]="3"
a=np.array([[1,2,3],[4,5,6]],dtype="float32")
b=np.array([[7,8,9],[9,6,5]],dtype="float32")
ab=tf.Variable(a)
bb=tf.constant(b)
inn=tf.global_variables_initializer()
add=ab+bb
sub=(ab-bb)
mul=tf.matmul(a,b)
with tf.Session() as rr:
rr.run(inn)
rr.run(ab)
res=rr.run(mul)
print(res)
i got this error :
ValueError: Dimensions must be equal, but are 3 and 2 for 'MatMul' (op: 'MatMul') with input shapes: [2,3], [2,3].
how multiply a=([1,2,3],[4,5,6]) b= ([9,8,7],[6,4,3]) ??
To answer the first question, yes, everything runs in a session. You define a graph and then the session is when the graph gets compiled, loaded into tensorflow internals and executed the state of all the variables is kept as long as the session is open.
Your code is a bit strange though. You define one of your inputs as a constant, but the other one as a variable but the variable one never changes. But at the very basic level, you can run multiple operations in a session by passing multiple operations as a list. Here is how:
a=np.array([[1,2,3],[4,5,6]],dtype="float32")
b=np.array([[7,8,9],[9,6,5]],dtype="float32")
tfa = tf.constant(a)
tfb = tf.constant(b)
add = tfa + tfb
sub = tfa - tfb
with tf.Session() as s:
res_add, res_sub = s.run([add, sub])
print(res_add)
print(res_sub)
and the output is
[[ 8. 10. 12.]
[ 13. 11. 11.]]
[[-6. -6. -6.]
[-5. -1. 1.]]
Your matrix multiplication is not going to work, because inner dimensions have to match. Does this answer your question, or were you trying to do something else?