When I view the explanation of the function torch.split in PyTorch, I find it difficult for me to read as a non-English-speaker:
torch.split(tensor, split_size_or_sections, dim=0)
[...]
If split_size_or_sections is a list, then tensor will be split
into len(split_size_or_sections) chunks with sizes in dim according
to split_size_or_sections.
Does "with sizes in dim" mean "with sizes in split_size_or_sections along the dimension dim"?
Don't worry - your English is fine, that line is a bit confusing.
Yes you're correct. It means if you pass a list e.g. split_size_or_sections=[1,2,4,5] it will split the tensor into len([1,2,4,5]) chunks (with the splits happening across dim), and each chunk will be of length 1, 2, 4, 5 respectively.
This implicitly assumes that sum([1,2,4,5]) equals the size of dim, and will return an error if not.
Related
In order to make say it simply, I have a list of dimension [32, 31, 4] which I would like to reduce to shape [32, 31, 3] in order to replace every array in the last dimension by an array of size (3).
for a in range(len(liste)): #len(list) = 95
for b in range(len(liste[a])): #shape = [32, 31, 3], b travels in the 1st dim.
#print('frame : ', liste[a][b].shape) #[31, 4]
#print('b', b) #32 frames each time ok
for c in range(len(liste[a][b])):
#print('c', c) #31 each time ok
#print('norme du quaternion', np.abs(np.linalg.norm(liste[a][b][c]))) #norm = 1
r = quat2expmap(liste[a][b][c]) #convertion to expmap successful
#print('ExpMap : ', r)
quat = liste[a][b][c]
quat = r #this works
#print('quat', quat)
liste[a][b][c] = r #this doesn't work
To be more precise, I have a dataset of 95 different gestures each represented by 32 frames and quaternions. I converted the quaternions into ExpMap but due to the difference of shapes I am unable to replace the quaternions by their corresponding ExpMap. The error code I receive the most is the following:
ValueError: could not broadcast input array from shape (3) into shape (4)
It comes from the last line of the code.
The weirdest thing is that when I take the quaternion apart and replace it, it works parfectly, yet python would refuse that I do it inside my list. I don't really get why.
Could you lighten me about it? How could I get the proper dimension in my list? I tried all the tricks such as del, remove() but got no result...
You seem to be using numpy arrays (not Python lists). Numpy does not allow changing dimensions on assignment to an element of an array because it would become irregular (some entries with 4 and some with 3).
Also, iterating through numpy arrays using loops is the wrong way to use numpy. In this case you're probably looking at applying the quat2expmap function to the 4th dimension of your matrix to produce a new matrix of shape (95,32,31,3). This will make maximum use of numpy's parallelism and can be written in a couple of lines without any loops.
You could either modify the quat2expmap function so that it works directly on your 4d matrix (will be fastest approach) or use np.apply_along_axis (which is not much faster than loops).
I have a 2d tensor in tensorflow,
Let's say for example a 2*4 tensor [[1.,2.,3.,4.],[2.,4.,5.,6.]].
I have a function a() to let each row in the tensor to pass, and then sum over all the results of a(). How to do it (not doing it in the session)?
The output should be a([1.,2.,3.,4.]) + a([2.,4.,5.,6.]), in practice I have a very large tensor with many rows.
This is different from reduce_sum, because the a() function here is quite complex, which cannot be directly used through vectorization.
Many thanks!
Perhaps what you're looking for is the map_fn function in Tensorflow. map_fn(a, elems) unpacks a tensor, elems along its first dimension into a sequences of slices, and then applies the supplied function a to each slice, followed by combining the outputs into a single tensor again by concatenating along the first dimension.
It sounds like what you want is
Y = map_fn(a, X)
answer = reduce_sum(Y, axis=0)
where X is your supplied tensor.
I have a batch of input:
input = tf.placeholder(tf.float32, [NUM_SAMPLE, None, 15])
For each one in the batch, I have a dictionary that describes the relationship of rows. It looks like:
dic = {i:{j:rij,k:rik,...},j:{i:rij,l:rjl,...},...}
Now I wanna do this for each sample and corresponding dic:
updated_sample = sample
for i in range(len(sample)):
for j in dic[i]:
tmp = concanate(sample[j],rij)
updated_sample[i] += matmul(tmp,W)
in which W is the same for all samples and rows.
However, I cannot use len(sample) in tensorflow. It seems tf.while_loop may be the answer, but I don't know how to use it in this problem. Any suggestions?
Besides, can I use dictionary in this way in tensorflow?
There are 2 analogs in tensorflow for len(sample):
tf.shape(sample)[0]
sample.get_shape().as_list()[0]
The first one, tf.shape(sample) returns a tensor of integers of length equal to the rank of the tensor, doing tf.shape(sample)[0] is a tensor with shape () and should be used within the tenosrflow workflow.
The second one, sample.get_shape() returns a Tensor.shape object, doing sample.get_shape().as_list() transforms this into a list of integers.
In your case, you should to use the second of these.
Consider also the option of doing this computations at the numpy level, and then input them into the graph through placeholders.
I have a dataset of one-line sentences and each sentence belongs to a class based on the context. I have created a lexicon of important words and converted my input data into a list of features, where each feature is a vector of the length of the lexicon.
I want to input this data to a dynamic LSTM cell, but can't figure out how to reshape it.
Consider my batch_size = 100, length_lexicon = 64, nRows_Input = 1000
Why not use numpy.reshape?
Check out this documentation: https://docs.scipy.org/doc/numpy/reference/generated/numpy.reshape.html
For example:
>>> a = np.arange(6).reshape((3, 2))
>>> a
array([[0, 1],
[2, 3],
[4, 5]])
numpy.reshape¶
numpy.reshape(a, newshape, order='C')
Gives a new shape to an array without changing its data.
Parameters:
a : array_like
Array to be reshaped.
newshape : int or tuple of ints
The new shape should be compatible with the original shape. If an integer, then the result will be a 1-D array of that length. One
shape dimension can be -1. In this case, the value is inferred from
the length of the array and remaining dimensions.
order : {‘C’, ‘F’, ‘A’}, optional
Read the elements of a using this index order, and place the elements into the reshaped array using this index order. ‘C’ means to
read / write the elements using C-like index order, with the last axis
index changing fastest, back to the first axis index changing slowest.
‘F’ means to read / write the elements using Fortran-like index order,
with the first index changing fastest, and the last index changing
slowest. Note that the ‘C’ and ‘F’ options take no account of the
memory layout of the underlying array, and only refer to the order of
indexing. ‘A’ means to read / write the elements in Fortran-like index
order if a is Fortran contiguous in memory, C-like order otherwise.
Returns:
reshaped_array : ndarray
This will be a new view object if possible; otherwise, it will be a copy. Note there is no guarantee of the memory layout (C- or
Fortran- contiguous) of the returned array.
I am adapting the cifar10 convolution example to my problem. I'd like to change the data input from a design that reads images one-at-a-time from a file to a design that operates on an already-in-memory set of images. The original inputs() function looks like this:
read_input = cifar10_input.read_cifar10(filename_queue)
reshaped_image = tf.cast(read_input.uint8image, tf.float32)
# Crop the central [height, width] of the image.
resized_image = tf.image.resize_image_with_crop_or_pad(reshaped_image,
width, height)
In the original version, read_input is a tensor containing one image.
I keep all my images in RAM, so instead of using filename_queue, I have one huge images_tensor = tf.constant(images), where images_tensor.shape is (something, 32, 32, 3).
My question is very-very basic: what is the best way to apply some function (tf.image.resize_image_with_crop_or_pad in my case) to all elements of images_tensor?
Iterating is problematic in tensorflow, with limited slices(TensorFlow - numpy-like tensor indexing). Is there a solution to achieving this using just one command?
As of version 0.8 there is map_fn. From the documentation:
map_fn(fn, elems, dtype=None, parallel_iterations=10, back_prop=True,
swap_memory=False, name=None)
map on the list of tensors unpacked from elems on dimension 0.
This map operator repeatedly applies the callable fn to a sequence of elements from first to last. The elements are made of the tensors unpacked from elems. dtype is the data type of the return value of fn. Users must provide dtype if it is different from the data type of elems.
Suppose that elems is unpacked into values, a list of tensors. The shape of the result tensor is [len(values)] + fn(values[0]).shape.
Args:
fn: The callable to be performed.
elems: A tensor to be unpacked to apply fn.
dtype: (optional) The output type of fn.
parallel_iterations: (optional) The number of iterations allowed to run
in parallel.
back_prop: (optional) True enables back propagation.
swap_memory: (optional) True enables GPU-CPU memory swapping.
name: (optional) Name prefix for the returned tensors.
Returns:
A tensor that packs the results of applying fn to the list of tensors
unpacked from elems, from first to last.
Raises:
TypeError: if fn is not callable.
Example:
elems = [1, 2, 3, 4, 5, 6]
squares = map_fn(lambda x: x * x, elems)
# squares == [1, 4, 9, 16, 25, 36]
```
There are a few answers - none quite as elegant as a map function. Which is best depends a bit on your desire for memory efficiency.
(a) You can use enqueue_many to throw them into a tf.FIFOQueue and then dequeue and tf.image.resize_image_with_crop_or_pad an image at a time, and concat it all back into one big smoosh. This is probably slow. Requires N calls to run for N images.
(b) You could use a single placeholder feed and run to resize and crop them on their way in from your original datasource. This is possibly the best option from a memory perspective, because you never have to store the unresized data in memory.
(c) You could use the tf.control_flow_ops.While op to iterate through the full batch and build up the result in a tf.Variable. Particularly if you take advantage of the parallel execution permitted by while, this is likely to be the fastest approach.
I'd probably go for option (c) unless you want to minimize memory use, in which case filtering it on the way in (option b) would be a better choice.
Tensorflow provides a couple of higher-order functions and one of them is tf.map_fn. The usage is very easy: you define your mappping and apply it to the tensor:
variable = tf.Variable(...)
mapping = lambda x: f(x)
res = tf.map_fn(mapping, variable)