Develop Reference

How to modify a single value of a 2D tensor programatically by index

I have a 2D tensor my_tensor size [50,50] and dtype int32 and I need to increment the value at one specific location. The indices of the location to be updated is given by 2 integer tensors, which give the location in axis 0 and axis 1, respectively:
idx_0 is:
tf.Tensor([27], shape=(1,), dtype=int32)
idx_1 is:
tf.Tensor([26], shape=(1,), dtype=int32)
Tensorflow's tensor_scatter_nd_add seems to be the solution. The code works if I define the indexes manually, but if I try to use idx_0 and idx_1, every implementation gives some index/dimension mis-match error.
This works, incrementing location (27,26):
tf.tensor_scatter_nd_add(reversals_count, [[27, 26]], [1])
but this raises an error:
tf.tensor_scatter_nd_add(reversals_count, [[idx_0, idx_1]], [1])
with the error message
{InvalidArgumentError}Outer dimensions of indices and update must match. Indices shape: [1,2,1], updates shape:[1] [Op:TensorScatterAdd]
How can I use the idx_0 and idx_1 tensors in place of [[27, 26]]? Other syntaxes I've tried similarly do not produce the correct dimensions:
[[idx_0], [idx_1]]
tf.concat([idx_0, idx_1], axis=0)

I have created an (5x5) unit matrix for sake of simplicity and updating them at indices (0,0) ,(1,1),(2,2),(3,3) (i.e at the first four diagonal elements) . First define the indices as tensors ,then values as tensors that will add up those at respective indices then update using "tf.tensor_scatter_nd_add" command . You can do similarly for a (50X50) matrix.Thanks!
import tensorflow as tf
indices = tf.constant([[0,0], [1,1], [2,2],[3,3]]) # updating at diagonal index elements, you can see the change
updates = tf.constant([9, 10, 11, 12])# values that will add up at respective indexes
print("original tensor is ")
tensor = tf.ones([5,5], dtype=tf.int32)
print(tensor)
print("updated tensor is ")
updated = tf.tensor_scatter_nd_add(tensor, indices, updates)
print(updated)

ValueError: cannot reshape array of size 74404 into shape (6764,1691,1)

My df shape is 2D (6764, 11).
I want to reshape it into 3D with 1691 time steps (i.e., 1/4 of 6764)
df = df.values.reshape((df.shape[0], 1691, df.shape[1]))
I get the error: ValueError: cannot reshape array of size 74404 into shape (6764,1691,11)
Why I get size 74404??? I get is 1674*11, but why is doing this multiplication?
edit
I actually want to reshape my data into [6764, 1691, 11], which is the dimension required for an LSTM model. This dimension stands for [Samples, TimeSteps, Features] where samples are the number of data points, time steps the number of data points I want to analyse/predict, and 11 the inputs (columns) I am using. Any advise on how to achieve this shape without getting the error ? my reference is this

From 2D dataframe you have an array of 6764 x 11 = 74404 values.
Multiplication indicates the number of values you have in the array/dataframe.
from your code (df.shape[0], 1691, df10.shape[1])) it would generate 6764 x 1691x 11 = 125817164 which is not matching to input array values thats why you are getting an error.
Considering you want 1691 series you can reshape your data into (1691 x 4 x 11)
df = df.values.reshape((1691,4, df.shape[1]))
If you need only 1st column that is 6764 values to reshape then use below code although it will generate 2D array with (1691,4) shape.
df = df['column_name'].values.reshape((1691,4))

Add a specifc value as a new dimension of a numpy array

I have a numpy array of shape (224,224,3) after reading a image. However I would like to convert this into a shape of (4,224,224,3).
I would like to kind of repeat the same values.
I am trying to append like shown below this but it doesn't work.
np.append(image,[[[4]]],axis=1)
Instead it throws the below error
ValueError: all the input arrays must have same number of dimensions
I expect my output shape to be (4,224,224,3)
Can you guide me on how to do this?

You could use np.repeat setting axis to 0:
out = np.repeat([image], 4, axis=0)
out.shape
# (4, 224, 224, 3)

Slice Variable via another Variable Tensorflow

I have numpy code for a project and wanted to convert it to tensorflow.
I have a 2D Tensor like x => [[0,1],[1,2],[2,3]] etc. and I want to slice a 3D tensor y using this. e.g. y[x[:,0], x[:,1], :] but it doesn't work. Following is error:
ValueError: Shape must be rank - but is rank - for 'strided_slice_?' (op: 'StridedSlice') with input shapes: [-], [-], [-], [-].
Can anyone please help!
Thanks

You need scalars to index into y, not tensors of rank 1+.
Try y[x[0, 0], x[0, 1], :] for a quick test.

Confusion about rank and shape in TensorFlow

I am confused about rank and shape concept of TensorFlow. I have read the details from here and did run some code to clear my concept about them. But I am still confused and need help to understand.
x = tf.placeholder(tf.float32, shape=[2, 12])
print(x.get_shape()) # ==> (2, 12)
print(x[0, :].get_shape()) # ==> (12,)
print(x[1, :].get_shape()) # ==> (12,)
print(x[2, :].get_shape()) # ==> (12,)
print(x[120, :].get_shape()) # ==> (12,)
I thought x is like a 2d matrix where 2 is number of rows and 12 is number of columns. Then why I am getting shape for x[120, :] as (12, )? How even x[120, :] is possible with the given shape?
Besides, since I thought x is a 2D tensor, its rank is also 2 because dimension and rank is the same thing for tensors (according to my understanding). But when I run:
print(x[0].get_shape())
I am getting this error:
Shape (2, 12) must have rank 1
It means my understanding is wrong about rank and dimension. What I am missing about rank and dimensions? Is rank and dimension two different things? How the rank of tensor x in the above example is 1? How can I set the rank of a tensor? Can anyone explain in details with some comprehensive examples?

I find the link you provide very clear.
The rank of a tensor is the number of dimensions it has
a matrix has 2 dimensions, so its rank is 2
a colored image has 3 dimensions [height, width, 3] so its rank is 3
The shape of a tensor is the detailed number of components in each dimension
a matrix has 2 dimensions, rank 2 and can have a shape like [6, 10], where 6 is the number of rows and 10 the number of columns
a 200x200 colored image (of rank 3) will have a shape [200, 200, 3]
For your examples, x[120, :] is possible to write because TensorFlow is not checking yet if 120 is a valid index. When you create your session and run the code, you will find an error:
res = x[120, :]
with tf.Session():
sess.run(res, feed_dict={x: np.zeros((2, 12))})
InvalidArgumentError: slice index 120 of dimension 0 out of bounds.
As said in my comment, x[0] should work with the latest version of TensorFlow and it should give a tensor of shape (12,), and rank 1.

After some analysis, I found that while declaring tensor the number of square braces used gives the rank of a tensor. And the number of values in a shape is equal to the rank. For example:
a1=tf.constant(
[
[
[1,2,3,4,5],
[1,2,3,4,5],
[1,2,3,4,5],
[1,2,3,4,5]
]
]
)
Here, I used 3 square braces. So the rank is 3. Also the shape would be in the format (x,y,z) since the rank is 3.
Now the values of x,y and z:
x= Number of commas in the first brace plus 1
so x=0+1=1
similarly
y=3+1=4
z=4+1=5
Finally the rank is 3 and shape is (1,4,5)