Develop Reference

Reshaping numpy arrays of differing dimensions inside an array

So the task is to optimise a Neural Network with a PSO. The PSO needs a one-dimensional list of all the weights and biases, like so [0.1 0.244 ... 0.214]. The NN needs an array of arrays with different dimensions, like so [[x,y], [m,n], ...(all the hidden layer matrices)... ,[p,q]] X and y are the dimensions for the input layer, then all the hidden layers and finally p and q - the dimensions of the output layer.
I can easily flatten the array to pass it to the PSO, but I need a method that takes the modified array and reshapes it back into the same array of arrays with the same dimensions as the starting one from the NN.
The dimensions depend on the amount of neurons in a layer, we have that information from the start.
I have tried to keep track of the shapes array and create an indices array to know when to stop but it doesn't seem to work. I am trying something with slicing now but no cigar yet. A modification to the NN is also possible but how to create it so it takes a predefined list of weights? There might be a very nice and efficient way to do it but I just haven't thought of it yet... Any suggestions?
Example:
a = np.array([1,2,3])
b = np.array([7,8,9,10])
c = np.array([12,13,14,15,16])
b.reshape(2,2)
arr = []
arr.append(a)
arr.append(b)
arr.append(c)
This is a very simple example of what the list of weights is as the NN works with it - a list of multi-dimensional array. Arr can be converted into a numpy array of objects if necessary with np.asarray(arr).
Flattening is easy, here is how I do it (there might be a better that doesn't need a loop, if you know, I'd be thankful if you shared).
Flattening:
new_arr = np.array([])
for i in range(len(arr)):
new_arr = np.append(arr, arr[i].flatten())
My question is how to take new_arr and put it back together to look like arr and is there a beautiful and fast way to do it.

You can save the shape in a variable (it's just a tuple). Try something like:
...
old_shape = arr.shape
# ... do flattening here
new_arr.reshape(old_shape)

new_arr = np.array([])
shapes=[]
for i in range(len(arr)):
new_arr = np.append(new_arr, arr[i].flatten())
shapes.append(arr[i].shape)
#do whatever
restoredArray =[]
offset=0
for i in range(len(shapes)):
s = shapes[i]
n = np.prod(s)
restoredArray.append(new_arr[offset:(offset+n)].reshape(s))
offset+=n

locations of maximum values of tensorflow tensor across an axis

I have a rank 3 tensor and another empty tensor of same shape. I am trying to find the maximum values across 3rd(Z) axis for all X,Y locations and insert 1 into the corresponding location in the empty tensor. I have achieved this in numpy in the following way
a = np.random.rand(5,5,3)>=0.5
empty_tensor = np.zeros((5,5,3))
max_z_indices = a.argmax(axis=-1)
empty_tensor[np.arange(a.shape[0])[:,None],np.arange(a.shape[1]),max_z_indices] = 1
In tensorflow I have
a_tf = tf.Variable(a)
empty_tensor_tf = tf.Variable(np.zeros((5,5,3)))
max_z_indices = sess.run(tf.argmax(a_tf,axis=-1))
I know that I can explicitly write the X,Y,Z indices of max values along 3rd dimension of tensor a_tf and use tf.scatter_nd_update to update empty_tensor_tf but I was hoping to find a better way (broadcasting) as in the last line of numpy code.

You can use tf.reduce_max to get the max value for each z-index, then use tf.where to convert it to 1 or 0 according to the cond.
import tensorflow as tf
# tf_a is (5,5,3) tensor
max_val = tf.reduce_max(tf_a, axis=-1,keepdims=True)
cond = tf.equal(tf_a, max_val)
res = tf.where(cond, tf.ones_like(tf_a), tf.zeros_like(tf_a))

Get the value at a position from all layers in python

I have 3 numpy arrays of shape (224, 224, 20). I want to go through each of (224, 224) values in all 20 layers (dimensions) and compare them to get the highest among them. For 3 Dimensional, I am able to come up with this:
arr1 = np.array([[[1,2,3],[4,5,6]],[[10,11,12],[15,16,17]]])
for x in range(0,2):
for y in range(0,2):
print(arr1[:,x,y])
But, I somehow couldn't understand how to convert it for (224,224,20) shaped arrays.
I also need the index of the layer which contains the maximum value.

To get max values along one dimension, you can use numpy.amax, checkout:
https://docs.scipy.org/doc/numpy/reference/generated/numpy.amax.html

You can do this with numpy.max instead of a for loop:
https://docs.scipy.org/doc/numpy/reference/generated/numpy.ndarray.max.html
np.max(arr1, axis=2)
To get the index, use numpy.argmax
https://docs.scipy.org/doc/numpy/reference/generated/numpy.argmax.html
np.argmax(arr1, axis=2)

Numpy [...,None]

I have found myself needing to add features to existing numpy arrays which has led to a question around what the last portion of the following code is actually doing:
np.ones(shape=feature_set.shape)[...,None]
Set-up
As an example, let's say I wish to solve for linear regression parameter estimates by using numpy and solving:
Assume I have a feature set shape (50,1), a target variable of shape (50,), and I wish to use the shape of my target variable to add a column for intercept values.
It would look something like this:
# Create random target & feature set
y_train = np.random.randint(0,100, size = (50,))
feature_set = np.random.randint(0,100,size=(50,1))
# Build a set of 1s after shape of target variable
int_train = np.ones(shape=y_train.shape)[...,None]
# Able to then add int_train to feature set
X = np.concatenate((int_train, feature_set),1)
What I Think I Know
I see the difference in output when I include [...,None] vs when I leave it off. Here it is:
The second version returns an error around input arrays needing the same number of dimensions, and eventually I stumbled on the solution to use [...,None].
Main Question
While I see the output of [...,None] gives me what I want, I am struggling to find any information on what it is actually supposed to do. Can anybody walk me through what this code actually means, what the None argument is doing, etc?
Thank you!

The slice of [..., None] consists of two "shortcuts":
The ellipsis literal component:
The dots (...) represent as many colons as needed to produce a complete indexing tuple. For example, if x is a rank 5 array (i.e., it has 5 axes), then
x[1,2,...] is equivalent to x[1,2,:,:,:],
x[...,3] to x[:,:,:,:,3] and
x[4,...,5,:] to x[4,:,:,5,:].
(Source)
The None component:
numpy.newaxis
The newaxis object can be used in all slicing operations to create an axis of length one. newaxis is an alias for ‘None’, and ‘None’ can be used in place of this with the same result.
(Source)
So, arr[..., None] takes an array of dimension N and "adds" a dimension "at the end" for a resulting array of dimension N+1.
Example:
import numpy as np
x = np.array([[1,2,3],[4,5,6]])
print(x.shape) # (2, 3)
y = x[...,None]
print(y.shape) # (2, 3, 1)
z = x[:,:,np.newaxis]
print(z.shape) # (2, 3, 1)
a = np.expand_dims(x, axis=-1)
print(a.shape) # (2, 3, 1)
print((y == z).all()) # True
print((y == a).all()) # True

Consider this code:
np.ones(shape=(2,3))[...,None].shape
As you see the 'None' phrase change the (2,3) matrix to a (2,3,1) tensor. As a matter of fact it put the matrix in the LAST index of the tensor.
If you use
np.ones(shape=(2,3))[None, ...].shape
it put the matrix in the FIRST‌ index of the tensor

Slice multiple slices at once with tensorflow

I am preparing the input tensor for the tensorflow RNN.
Currently I am doing the following
rnn_format = list()
for each in range(batch_size):
rnn_format.append(tf.slice(input2Dpadded,[each,0],[max_steps,10]))
lstm_input = tf.stack(rnn_format)
Would it be possible to do this at once, without loop, with some tensorflow function?

As suggested by Peter Hawkins, you can use gather_nd with the appropriate indices to get there.
Your uniform cropping on the inner dimension can simply be done before the call to gather_nd.
Example:
import tensorflow as tf
import numpy as np
sess = tf.InteractiveSession()
# integer image simply because it is more readable to me
im0 = np.random.randint(10, size=(20,20))
im = tf.constant(im0)
max_steps = 3
batch_size = 10
# create the appropriate indices here
indices = (np.arange(max_steps) +
np.arange(batch_size)[:,np.newaxis])[...,np.newaxis]
# crop then call gather_nd
res = tf.gather_nd(im[:,:10], indices).eval()
# check that the resulting tensors are equal to what you had previously
for each in range(batch_size):
assert(np.all(tf.slice(im, [each,0],[max_steps,10]).eval() == res[each]))
EDIT
If your slices indices are in a tensor, you simply replace numpy's operations with tensorflow's operations when creating indices:
# indices stored in a 1D array
my_indices = tf.constant([1, 8, 3, 0, 0])
indices = (np.arange(max_steps) +
my_indices[:,tf.newaxis])[...,tf.newaxis]
Further remarks:
indices is created by taking advantage of broadcasting during the addition: arrays are virtually tiled so that their dimensions match. Broadcasting is supported by numpy and by tensorflow in a similar fashion.
Ellipsis ... is part of the standard numpy slicing notation, it basically fills all remaining dimensions left by the other slicing indices. So [..., newaxis] is basically equivalent to expand_dims(·, -1).

Try tf.split or tf.split_v. See here:
https://www.tensorflow.org/api_docs/python/tf/split
Does that help?