I would like to create a numpy array by concatenating two or more numpy arrays with shape (1, x, 1) where x is variable.
Here is the problem in detail.
x1 = #numpy array with shape (x,)
x2 = #numpy array with shape (y,)
#create batch
x1 = np.expand_dims(x1, 0) #shape (1, x)
x2 = np.expand_dims(x2, 0) #shape (1, y)
#add channel dimension
x1 = np.expand_dims(x1, -1) #shape (1, x, 1)
x2 = np.expand_dims(x2, -1) #shape (1, y, 1)
#merge the two arrays
x = np.concatenate((x1, x2), axis=0)
#expected shape (2, ??, 1)
Note the expected shape (2, ??, 1). I am wondering if what I am trying to do is doable.
Executing this code raises a ValueError:
ValueError: all the input array dimensions for the concatenation axis must match exactly, but along dimension 1, the array at index 0 has size 138241 and the array at index 1 has size 104321
This is the entire code related to my question. You should be able to run this code and see the plots created - by just pasting and running it into your IDE.
import numpy as np
import matplotlib.pyplot as plt
np.random.seed(1)
x = np.random.randn(4, 3, 3, 2)
x_pad = np.pad(x, ((0,0), (2, 2), (2, 2), (0,0))\
, mode='constant', constant_values = (0,0))
print ("x.shape =\n", x.shape)
print ("x_pad.shape =\n", x_pad.shape)
print ("x[1,1] =\n", x[1,1])
print ("x_pad[1,1] =\n", x_pad[1,1])
fig, axarr = plt.subplots(1, 2)
axarr[0].set_title('x')
axarr[0].imshow(x[0,:,:,0])
axarr[1].set_title('x_pad')
axarr[1].imshow(x_pad[0,:,:,0])
Specifically, my question is related to these two lines of code:
x = np.random.randn(4, 3, 3, 2)
x_pad = np.pad(x, ((0,0), (2, 2), (2, 2), (0,0)), mode='constant', constant_values = (0,0))
I want to pad the 2nd and 3rd dimension in x. So, I want to pad x[1] which has a value of 3 and x[2] which also has the value of 3. Based on the problem that I am solving, x[0] and x[3], which contain '4' and '2' respectively, represent something else. x[0] represents the number of number of such 3*3 matrices and x[3] the channels.
My question is about around how python is representing this information and about how we are interpreting it. Are these the same?
The statement x = np.random.randn (4, 3, 3, 2) created a matrix 4 rows by 3 columns and each element in this 4*3 matrix is a 3 row by 2 column matrix. That is how Python is representing the x_pad. Is this understanding correct?
If so, then in the np.pad statement, we are padding the number of columns in the outer matrix (which is 3 in the 4*3). We are also padding the number of rows, which is 3, in the “3*2” - that is, the number of rows in the inner matrix).
The 3, 3 in (4, 3, 3, 2) was supposed to be part of just one matrix and not the columns of the outer matrix and the rows of the inner matrix? I am having trouble visualizing this? Can someone please clarify. Thank you!
These lines:
x = np.random.randn(4, 3, 3, 2)
x_pad = np.pad(x, ((0,0), (2, 2), (2, 2), (0,0)), mode='constant', constant_values = (0,0))
are equivalent to:
x = np.random.randn(4, 3, 3, 2)
x_pad = np.zeros((4, 3+2+2, 3+2+2, 2))
x_pad[:, 2:-2, 2:-2, :] = x
You could interpret a 4-D array as being a 2-D array of 2-D arrays if that fits whatever this data represents for you, but numpy internally stores arrays as a 1D array of data; with x[i,j,k,l] pointing to data[l+n3*(k + n2*(j + n1*i))] where n1, n2, n3 are the lengths of the corresponding axes.
Visualizing 4-D (and higher) arrays is very difficult for humans. You just have to keep track of the indices for the four axes when you deal with such arrays.
I have this function padding function.
It takes X (a list of numpy arrays of dimensions (13,n) where n varies between 0-99 with each array) and returns and returns X_new which should also be a list of numpy arrays that are all shape (13,99) after padding.
X_new = []
for x in X:
shp_1 = len(x[1])
if shp_1 != targetdim:
X_new.append(np.pad(x[1], (0, targetdim - shp_1), 'constant', constant_values=0))
else:
X_new.append(x)
Checking its output by checking the shape of the arrays in X_new
(13, 99) #correct dimensions
(13, 99)
(99,) #wrong
(13, 99)
(13, 99)
(13, 99)
(13, 99)
(99,)
(13, 99)
(99,)
(13, 99)
X_new.append(np.pad(x[1], (0, targetdim - shp_1), 'constant', constant_values=0)) works as intended as it pads column x[1] to 99 if needed.
The problem is that the function only appends the padded array in x[1] to new_list, x[0] is discarded.
The result is that where padding is applied the output shape is (99,) instead of the desired (13,99).
My question is how do I resolve this issue with append?
In short, my the goal is to reproduce list X with a list of padded arrays, any alternative methods of achieving this goal are also welcome.
I have the following section of code:
print X.shape
print Y.shape
X = X[np.where(X[:, 2] > 0.05)]
Y = Y[np.where(Y[:, 2] > 0.05)]
print X.shape
print Y.shape
print Y
The output of the code is:
(231, 3)
(231, 3)
(55, 3)
(1, 56)
[[-1.98769812 -1.93259862 -2.29080624 -0.83792838 -1.94892429 -1.58289539
-1.90247292 -1.4556473 -2.19070534 -1.73212961 -1.61395937 -1.14633543
-1.10683535 -1.02509949 -1.25594622 -1.4080105 -1.63474736 -1.95824227
-2.17106426 -1.91125435 -2.0718614 -1.89993348 -1.54472541 -0.87961375
-1.85216151 -2.121226 -1.88453394 -1.29166291 -1.78698561 -2.08983923
-2.02139995 -0.9349243 -0.90184218 -0.69882058 -1.05909316 -2.15477887
-1.43069109 -2.03090084 -0.5736147 -1.51153658 -2.22790561 -0.75759474
-1.76501869 -1.38221479 -1.17738909 -1.64007052 -1.88721801 -1.290115
-1.76288243 -2.08980588 -1.85872696 -1.79981182 -1.56103353 -1.41155315
-0.73043509 -1.27204526]]
I was expecting that the updated Y array would have the shape (56,3). How can the same operation on two numpy arrays of the same shape produce different shapes?
So I have 2 images, X and Y, as numpy arrays, each of shape (3, 30, 30): that is, 3 channels (RGB), each of height and width 30 pixels. I'd like to pair them up into a numpy array to get a specific output shape:
my_pair = pair_up_images(X, Y)
my_pair.shape = (2, 3, 30, 30)
Such that I can get the original images by slicing:
my_pair[0] == X
my_pair[1] == Y
After a few attempts, I keep getting either:
my_pair.shape = (2,) #By converting the images into lists and adding them.
This works as well, but the next step in the pipeline just requires a shape (2, 3, 30, 30)
my_pair.shape = (6, 30, 30) # using np.vstack
my_pair.shape = (3, 60, 30) # using np.hstack
Thanks!
Simply:
Z = np.array([X, Y])
Z.shape
Out[62]: (2, 3, 30, 30)