I face a problem when I try to change the shape of tf.SparseTensor inside a tf.while_loop. Let's say I have this sparse tensor:
indices = np.array([[0, 0], [0, 1], [0, 2], [0, 3], [0, 4], [0, 5],
[1, 0], [1, 1], [1, 3], [1, 4], [1, 5],
[2, 1], [2, 2], [2, 3], [2, 4],
[3, 0], [3, 1], [3, 2], [3, 3], [3, 4], [3, 5],
[4, 0], [4, 2], [4, 3], [4, 4], [4, 5]], dtype=np.int64)
values = np.array([7, 6, 7, 4, 5, 4,
6, 7, 4, 3, 4,
3, 3, 1, 1,
1, 2, 2, 3, 3, 4,
1, 1, 2, 3, 3], dtype=np.float64)
dense_shape = np.array([5, 6], dtype=np.int64)
tRatings = tf.SparseTensor(indices, values, dense_shape)
So, I want to take a slice from the first 3 rows. I know for that purpose I can use tf.sparse_slice but this is an example. In my real code, I gather multiple rows from the sparse Tensor which they are not serial. The code I wrote is this:
subTensor = tf.sparse_slice(tRatings, [0, 0], [1, 6])
i = tf.constant(1)
def condition(i, sub):
return tf.less(i, 3)
def body(i, sub):
tempUser = tf.sparse_slice(tRatings, [i, 0], [1, 6])
sub = tf.sparse_concat(axis = 0, sp_inputs = [sub, tempUser])
return [tf.add(i, 1), sub]
subTensor = tf.while_loop(condition1, body1, [i, subTensor], shape_invariants=[i.get_shape(), tf.TensorShape([2])])[1]
which does't work for some reason when I run it. I get this:
ValueError: Dimensions 1 and 2 are not compatible
According to https://www.tensorflow.org/api_docs/python/tf/while_loop it says that:
The shape_invariants argument allows the caller to specify a less specific shape invariant for each loop variable, which is needed if the shape varies between iterations. The tf.Tensor.set_shape function may also be used in the body function to indicate that the output loop variable has a particular shape. The shape invariant for SparseTensor and IndexedSlices are treated specially as follows:
a) If a loop variable is a SparseTensor, the shape invariant must be TensorShape([r]) where r is the rank of the dense tensor represented by the sparse tensor. It means the shapes of the three tensors of the SparseTensor are ([None], [None, r], [r]). NOTE: The shape invariant here is the shape of the SparseTensor.dense_shape property. It must be the shape of a vector.
What am I missing here?
There are two problems.
First the problem in Tensorflow code. Change this line to:
var.indices.set_shape(tensor_shape.TensorShape([None, shape[0]]))
Another small problem in your code. You have to use int64 type for indexing variable:
i = tf.constant(1, dtype=tf.int64)
Related
Given an two arrays: an input array and a repeat array, I would like to receive an array which is repeated along a new dimension a specified amount of times for each row and padded until the ending.
to_repeat = np.array([1, 2, 3, 4, 5, 6])
repeats = np.array([1, 2, 2, 3, 3, 1])
# I want final array to look like the following:
#[[1, 0, 0],
# [2, 2, 0],
# [3, 3, 0],
# [4, 4, 4],
# [5, 5, 5],
# [6, 0, 0]]
The issue is that I'm operating with large datasets (10M or so) so a list comprehension is too slow - what is a fast way to achieve this?
Here's one with masking based on this idea -
m = repeats[:,None] > np.arange(repeats.max())
out = np.zeros(m.shape,dtype=to_repeat.dtype)
out[m] = np.repeat(to_repeat,repeats)
Sample output -
In [44]: out
Out[44]:
array([[1, 0, 0],
[2, 2, 0],
[3, 3, 0],
[4, 4, 4],
[5, 5, 5],
[6, 0, 0]])
Or with broadcasted-multiplication -
In [67]: m*to_repeat[:,None]
Out[67]:
array([[1, 0, 0],
[2, 2, 0],
[3, 3, 0],
[4, 4, 4],
[5, 5, 5],
[6, 0, 0]])
For large datasets/sizes, we can leverage multi-cores and be more efficient on memory with numexpr module on that broadcasting -
In [64]: import numexpr as ne
# Re-using mask `m` from previous method
In [65]: ne.evaluate('m*R',{'m':m,'R':to_repeat[:,None]})
Out[65]:
array([[1, 0, 0],
[2, 2, 0],
[3, 3, 0],
[4, 4, 4],
[5, 5, 5],
[6, 0, 0]])
a = np.zeros((5,4,3))
v = np.ones((5, 4), dtype=int)
data = a[v]
shp = data.shape
This code gives shp==(5,4,4,3)
I don't understand why. How can a larger array be output? makes no sense to me and would love an explanation.
This is known as advanced indexing. Advanced indexing allows you to select arbitrary elements in the input array based on an N-dimensional index.
Let's use another example to make it clearer:
a = np.random.randint(1, 5, (5,4,3))
v = np.ones((5, 4), dtype=int)
Say in this case a is:
array([[[2, 1, 1],
[3, 4, 4],
[4, 3, 2],
[2, 2, 2]],
[[4, 4, 1],
[3, 3, 4],
[3, 4, 2],
[1, 3, 1]],
[[3, 1, 3],
[4, 3, 1],
[2, 1, 4],
[1, 2, 2]],
...
By indexing with an array of np.ones:
print(v)
array([[1, 1, 1, 1],
[1, 1, 1, 1],
[1, 1, 1, 1],
[1, 1, 1, 1],
[1, 1, 1, 1]])
You will simply be indexing a with 1 along the first axis as many times as v. Putting it in another way, when you do:
a[1]
[[4, 4, 1],
[3, 3, 4],
[3, 4, 2],
[1, 3, 1]]
You're indexing along the first axis, as no indexing is specified along the additional axes. It is the same as doing a[1, ...], i.e taking a full slice along the remaining axes. Hence by indexing with a 2D array of ones, you will have the above 2D array (5, 4) times stacked together, resulting in an ndarray of shape (5, 4, 4, 3). Or in other words, a[1], of shape (4,3), stacked 5*4=20 times.
Hence, in this case you'd be getting:
array([[[[4, 4, 1],
[3, 3, 4],
[3, 4, 2],
[1, 3, 1]],
[[4, 4, 1],
[3, 3, 4],
[3, 4, 2],
[1, 3, 1]],
...
the value of v is:
[[1 1 1 1]
[1 1 1 1]
[1 1 1 1]
[1 1 1 1]
[1 1 1 1]]
every single 1 indexes a complete "row" in a, but every "element" in said "row" is a matrix. so every "row" in v indexes a "row" of "matrix"es in a.
(does this make any sense to you..?)
so you get 5 * 4 1s, each is a 4*3 "matrix".
if instead of zeroes you define a as a = np.arange(5*4*3).reshape((5, 4, 3))
it might be easier to understand, because you get to see which parts of a are being chosen:
import numpy as np
a = np.arange(5*4*3).reshape((5, 4, 3))
v = np.ones((5,4), dtype=int)
data = a[v]
print(data)
(output is pretty long, I don't want to paste it here)
I want to compute the mean for axis=0 for a tf.SparseTensor. I want something like tf.sparse_reduce_sum. TensorFlow doesn't provide a similar function for the mean calculation. Is there any way to count the values in each row in order to divide them with the sum?
indices = np.array([[0, 0], [0, 1], [0, 2], [0, 3], [0, 4], [0, 5],
[1, 0], [1, 1], [1, 3], [1, 4], [1, 5],
[2, 1], [2, 2], [2, 3], [2, 4],
[3, 0], [3, 1], [3, 2], [3, 3], [3, 4], [3, 5],
[4, 0], [4, 2], [4, 3], [4, 4], [4, 5]], dtype=np.int64)
values = np.array([7, 6, 7, 4, 5, 4,
6, 7, 4, 3, 4,
3, 3, 1, 1,
1, 2, 2, 3, 3, 4,
1, 1, 2, 3, 3], dtype=np.float64)
dense_shape = np.array([5, 6], dtype=np.int64)
tRatings = tf.SparseTensor(indices, values, dense_shape)
You can compute the reduced mean from the reduced sum by dividing by the size of the 0th dimension:
tRatings = tf.SparseTensor(indices, values, dense_shape)
reduced_sum = tf.sparse_reduce_sum(tRatings, 0) # Sum of each row
reduced_mean = reduced_sum / tf.cast(tRatings.dense_shape[0], tf.float64) # Mean of each row
Try to use get_shape() and then multiply shape[0] * shape[1] this is the total number of elements
For example, in Numpy I can get some values like this.
d = np.array([[0, 1, 2], [3, 4, 5], [6, 7, 8]])
# array([[0, 1, 2],
# [3, 4, 5],
# [6, 7, 8]])
d[[0, 1, 2], [2, 1, 0]]
# array([2, 4, 6])
So I can retrieve [2, 4, 6].
how can I do the same thing in TensorFlow?
x = tf.Variable([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
init_op = tf.initialize_all_variables()
with tf.Session() as sess:
sess.run(init_op)
print sess.run([x[[0, 1, 2], [2,1,0]]])[0]
it raises TypeError
TypeError: Bad slice index [0, 1, 2] of type <type 'list'>
My Question is how can I get the same value through TensorFlow?
print sess.run([x[[0, 1, 2], [2,1,0]]])[0]
One solution I found is to use gather_nd function.
sess.run([tf.gather_nd(x, [[0, 2], [1, 1], [2, 0]])])
# [3 5 7]
Is there any other functions? a function more similar to Numpy in TensorFlow?
I have 2 arrays. "A" is one of them with arbitrary length (let's assume 1000 entries for a start), where each point holds a n-dimensional vector, where each entry represents a scalar. "B" is the other one, with n entries that each hold a 3-dimensional vector. How can I do a scalar multiplication, so that the result is one array "C", where each entry is the scalar multiplication of each of the n scalars with each of the n 3-Dimensional Vectors?
As an example in 4-D:
a=[[1,2,3,4],[5,6,7,8],....]
b=[[1,0,0],[0,1,0],[0,0,1],[1,1,1]]
and a result
c=[[1*[1,0,0],2*[0,1,0],3*[0,0,1],4*[1,1,1]] , [5*[1,0,0],...],...]
The implementation should be in numpy without to large for loops, because there are expected to be way more than 1000 entries. n is expected to be 7 in our case.
If you start with:
a = np.array([[1,2,3,4],[5,6,7,8]])
b = np.array([[1,0,0],[0,1,0],[0,0,1],[1,1,1]])
Then we can add an extra axis to a, and repeating the array along it gives us...
>>> a[:,:,None].repeat(3, axis=2)
array([[[1, 1, 1],
[2, 2, 2],
[3, 3, 3],
[4, 4, 4]],
[[5, 5, 5],
[6, 6, 6],
[7, 7, 7],
[8, 8, 8]]])
Now, as #Jaime says, there is no need to use the repeat while operating, because NumPy's broadcasting takes care of it:
>>> a[:,:,None] * b
array([[[1, 0, 0],
[0, 2, 0],
[0, 0, 3],
[4, 4, 4]],
[[5, 0, 0],
[0, 6, 0],
[0, 0, 7],
[8, 8, 8]]])