I can not understand what this reshaping actually do with an array of 28*28.
the code is:
x.reshape([1,28,28,1])
Reshape - as the name suggests - reshapes your array into an array of different shape.
>>> import numpy as np
>>> x = np.arange(28*28)
>>> x.shape
(784,)
>>> y = x.reshape(28,28)
>>> y.shape
(28, 28)
>>> z = y.reshape([1, 28, 28, 1])
>>> z.shape
(1, 28, 28, 1)
A shape of 1 implies that the respective dimension has a length of 1. This is most useful when working with broadcasting, as the array will be repeated along that dimension as needed.
>>> a = np.array([1, 2, 3]).reshape(3, 1)
>>> b = np.array([1, 2, 3]).reshape(1, 3)
>>> a * b
array([[1, 2, 3],
[2, 4, 6],
[3, 6, 9]])
Another use is to differentiate between row and column vectors, which you can understand as matrices of shape [1, X] or [X, 1] respectively.
>>> row_vector = np.array([1, 2, 3]).reshape(1,3)
>>> row_vector
array([[1, 2, 3]])
>>> column_vector = np.array([1,2,3]).reshape(3,1)
>>> column_vector
array([[1],
[2],
[3]])
Related
There are two numpy array a and w,
both of which have the same shape (d1,d2,..,dk,N).
We can think there are N sample with shape (d1,d2,...,dk).
Now, I want to sort a and w along a's last axis.
For example, a and w have shape (2,4):
a = [[3,2,4,1],
[2,3,1,4]]
w = [[10,20,30,40],
[80,70,60,50]]
sorted_index = a.argsort()
# array([[3, 1, 0, 2],
# [2, 0, 1, 3]])
I want:
a = a.sort() # default axis = -1
# a = [[1,2,3,4],
# [1,2,3,4]]
and w should be:
# w = [[40,20,10,30],
# [60,80,70,50]]
Of course, in that case, the following code work
x = a.argsort()
w[0,:] = w[0,x[0]]
w[1,:] = w[1,x[1]]
But when the sample have many dimension (>1), that code doesn't work.
Can anyone come up with solutions? Thanks!
There's a function for that, np.take_along_axis:
>>> a = np.array([[3,2,4,1], [2,3,1,4]])
>>> w = np.array([[10,20,30,40], [80,70,60,50]])
>>> sorted_index = a.argsort()
>>> sorted_index
array([[3, 1, 0, 2],
[2, 0, 1, 3]])
>>> np.take_along_axis(a, sorted_index, axis=-1)
array([[1, 2, 3, 4],
[1, 2, 3, 4]])
>>> np.take_along_axis(w, sorted_index, axis=-1)
array([[40, 20, 10, 30],
[60, 80, 70, 50]])
>>>
It will also work when a and w have arbitrary shape.
In python numpy package, I am having trouble understanding the situation where an ndarray has the 2nd dimension being empty. Here is an example:
In[1]: d2 = np.random.rand(10)
In[2]: d2.shape = (-1, 1)
In[3]: print d2.shape
In[4]: print(d2)
In[5]: print d2[::2, 0].shape
In[6]: print d2[::2, 0]
Out[3]:(10, 1)
Out[4]:
[[ 0.12362278]
[ 0.26365227]
[ 0.33939172]
[ 0.91501369]
[ 0.97008342]
[ 0.95294087]
[ 0.38906367]
[ 0.1012371 ]
[ 0.67842086]
[ 0.23711077]]
Out[5]: (5,)
Out[6]: [ 0.12362278 0.33939172 0.97008342 0.38906367 0.67842086]
My understanding is that d2 is a 10 rows by 1 column ndarray.
Out[6] is obviously a 1 by 5 array, how can the dimensions be (5,) ?
What does the empty 2nd dimension mean?
Let me just give you one example that illustrate one important difference.
d1 = np.array([1,2,3,4,5]) # array([1, 2, 3, 4, 5])
d1.shape -> (5,) # row array.
d1.size -> 5
# Note: d1.T is the same as d1.
d2 = d1[np.newaxis] # array([[1, 2, 3, 4, 5]]). Note extra []
d2.shape -> (1,5)
d2.size -> 5
# Note: d2.T will give a column array
array([[1],
[2],
[3],
[4],
[5]])
d2.T.shape -> (5,1)
I also thought ndarrays would represent even 1-d arrays as 2-d arrays with a thickness of 1. Maybe because of the name "ndarray" makes us think high dimensional, however, n can be 1, so ndarrays can just have one dimension.
Compare these
x = np.array([[1], [2], [3], [4]])
x.shape
# (4, 1)
x = np.array([[1, 2, 3, 4]])
x.shape
#(1, 4)
x = np.array([1, 2, 3, 4])
x.shape
#(4,)
and (4,) means (4).
If I reshape x and back to (4), it comes back to original
x.shape = (2,2)
x
# array([[1, 2],
# [3, 4]])
x.shape = (4)
x
# array([1, 2, 3, 4])
The main thing to understand here is that indexing with an integer is different than indexing with a slice. For example, when you index a 1d array or a list with an integer you get a scalar but when you index with a slice, you get an array or a list respectively. The same thing applies to 2d+ arrays. So for example:
# Make a 3d array:
import numpy as np
array = np.arange(60).reshape((3, 4, 5))
# Indexing with ints gives a scalar
print array[2, 3, 4] == 59
# True
# Indexing with slices gives a 3d array
print array[:2, :2, :2].shape
# (2, 2, 2)
# Indexing with a mix of slices and ints will give an array with < 3 dims
print array[0, :2, :3].shape
# (2, 3)
print array[:, 2, 0:1].shape
# (3, 1)
This can be really useful conceptually, because sometimes its great to think of an array as a collection of vectors, for example I can represent N points in space as an (N, 3) array:
n_points = np.random.random([10, 3])
point_2 = n_points[2]
print all(point_2 == n_points[2, :])
# True
What is the difference between a numpy array (lets say X) that has a shape of (N,1) and (N,). Aren't both of them Nx1 matrices ? The reason I ask is because sometimes computations return either one or the other.
This is a 1D array:
>>> np.array([1, 2, 3]).shape
(3,)
This array is a 2D but there is only one element in the first dimension:
>>> np.array([[1, 2, 3]]).shape
(1, 3)
Transposing gives the shape you are asking for:
>>> np.array([[1, 2, 3]]).T.shape
(3, 1)
Now, look at the array. Only the first column of this 2D array is filled.
>>> np.array([[1, 2, 3]]).T
array([[1],
[2],
[3]])
Given these two arrays:
>>> a = np.array([[1, 2, 3]])
>>> b = np.array([[1, 2, 3]]).T
>>> a
array([[1, 2, 3]])
>>> b
array([[1],
[2],
[3]])
You can take advantage of broadcasting:
>>> a * b
array([[1, 2, 3],
[2, 4, 6],
[3, 6, 9]])
The missing numbers are filled in. Think for rows and columns in table or spreadsheet.
>>> a + b
array([[2, 3, 4],
[3, 4, 5],
[4, 5, 6]])
Doing this with higher dimensions gets tougher on your imagination.
How do I select a row from a NxM numpy array as an array of size 1xM:
> import numpy
> a = numpy.array([[1,2], [3,4], [5,6]])
> a
array([[1, 2],
[3, 4],
[5, 6]])
> a.shape
(e, 2)
> a[0]
array([1, 2])
> a[0].shape
(2,)
I'd like
a[0].shape == (1,2)
I'm doing this because a library I want to use seems to require this.
If you have something of shape (2,) and you want to add a new axis so that the shape is (1,2), the easiest way is to use np.newaxis:
a = np.array([1,2])
a.shape
#(2,)
b = a[np.newaxis, :]
print b
#array([[1,2]])
b.shape
#(1,2)
If you have something of shape (N,2) and want to slice it with the same dimensionality to get a slice with shape (1,2), then you can use a range of length 1 as your slice instead of one index:
a = np.array([[1,2], [3,4], [5,6]])
a[0:1]
#array([[1, 2]])
a[0:1].shape
#(1,2)
Another trick is that some functions have a keepdims option, for example:
a
#array([[1, 2],
# [3, 4],
# [5, 6]])
a.sum(1)
#array([ 3, 7, 11])
a.sum(1, keepdims=True)
#array([[ 3],
# [ 7],
# [11]])
If you already have it, call .reshape():
>>> a = numpy.array([[1, 2], [3, 4]])
>>> b = a[0]
>>> c = b.reshape((1, -1))
>>> c
array([[1, 2]])
>>> c.shape
(1, 2)
You can also use a range to keep the array two-dimensional in the first place:
>>> b = a[0:1]
>>> b
array([[1, 2]])
>>> b.shape
(1, 2)
Note that all of these will have the same backing store.
I would like to implement a diagonal matrix apply function that is created by providing the diagonal d first, and then doing a bunch of matrix-vector multiplications with x. Of course I wouldn't want to create an actual matrix because all that's needed is a elementwise vector multiplication.
Now, some users are going to provide a diagonal d of shape (k,), some of shape (k,1). Also, x can have shapes (k,) and (k,1). I would like the apply() method to behave just like the * for numpy matrices in that the result has the same shape as the input x.
Hence the question: In Python/Numpy, is there a non-iffy way to elementwise-multiply two np.arrays x and y of shapes (k,) or (k,1) (in any combination) such that the resulting array has the shape of x?
I experimented a little with [:,None],
x = np.empty((4,1))
y = np.empty(4)
(x * y).shape # (4,4) -- nope
(y * y).shape # (4,) -- yes
(x * y[:,None]).shape # (4, 1) -- yes
(y * y[:,None]).shape # (4,4) -- nope
and I could certainly wrap my code in if len(x.shape)==...:, but that doesn't feel very pythonic.
Suggestions?
Now that I understand your question, my suggestion would be simply to reshape. Calling reshape returns a view, so it doesn't incur any big copying costs or anything like that. Simply reshape the arrays, multiply, and reshape again:
>>> def shape_preserving_mult(x, y):
... return (x.reshape((-1,)) * y.reshape((-1))).reshape(x.shape)
...
Or more concisely, as you and rroowwllaanndd pointed out:
>>> def shape_preserving_mult(x, y):
... return x * y.reshape(x.shape)
...
>>> shape_preserving_mult(x, y)
array([[ 0],
[ 1],
[ 4],
[ 9],
[16]])
>>> shape_preserving_mult(x, y.T)
array([[ 0],
[ 1],
[ 4],
[ 9],
[16]])
>>> shape_preserving_mult(x.T, y)
array([[ 0, 1, 4, 9, 16]])
>>> shape_preserving_mult(x.T, y.T)
array([[ 0, 1, 4, 9, 16]])
The substance of my previous suggestion remains below.
It's worth noting that if you multiply a numpy array of shape (1, 4) with an array of shape (4,) you get something close to what you want.
>>> x = numpy.arange(5).reshape((5, 1))
>>> y = numpy.arange(5)
>>> x.shape
(5, 1)
>>> x.T.shape
(1, 5)
>>> y * x.T
array([[ 0, 1, 4, 9, 16]])
This doesn't have the shape of a, but it does have the shape of a.T. You could always call T on the result again. This will work on arrays of shape (5,) too, because the transpose operation on a 1-d array causes no change. So perhaps you could do this:
>>> def transposed_mult(x, y):
... return (x.T * y).T
...
>>> transposed_mult(x, y)
array([[ 0],
[ 1],
[ 4],
[ 9],
[16]])
But of course this causes the opposite problem if you pass an array of shape (1, 5):
>>> transposed_mult(x.T, y)
array([[ 0, 0, 0, 0, 0],
[ 0, 1, 2, 3, 4],
[ 0, 2, 4, 6, 8],
[ 0, 3, 6, 9, 12],
[ 0, 4, 8, 12, 16]])
So transposed_mult does the exact thing you asked for in your original post, but if you need any further flexibility, it won't work as expected. And indeed, it seems you need additional flexibility.
If you can assume that x and y are both essentially 1D arrays then using numpy's .flatten() array method:
>>> result=x.flatten()*y.flatten()
Should return a 1D array:
>>> result.shape
(4,)
To preserve the shape of x you could do the following:
>>> result = x*y.reshape(x.shape)
>>> result.shape
(4,1)